JP4718501B2 - Call quality test server and call quality test method - Google Patents

Description

The present invention relates to a speech quality test server and call quality test methods relate to perform speech quality testing of the network by particular presenting an image to the user.

  As a method of evaluating the quality of video and audio, the method of evaluating the signal-to-noise ratio (signal-to-noise ratio) of video and the degree to which the audio signal has deteriorated by comparing the transmitted audio data with the original data before transmission A method such as PESQ (Perceptual Evaluation of Speech Quality) is known. These are signals to be evaluated, and do not consider transmission conditions in the network.

  On the other hand, since voice calls and videophones are based on two-way communication, not only signal deterioration but also evaluation of signal transmission delay and video and audio synchronization (lip sync) should be conducted. is important. In order to evaluate the effects of such delay and lip sync on the call quality, it is necessary to evaluate the call quality by actually communicating “calling” between the callers. Moreover, in order to perform an appropriate evaluation test, it is necessary to provide an appropriate topic in advance so that sufficient “contact” is performed between subjects. As a method of performing an evaluation test after providing a topic in advance, a method in which two subjects are shown a photo in advance and then the subjects discuss whether the photos seen by the video phone are the same or different ITU-T (International Telecommunication Union Telecommunication Standardization Sector) Proposed in 920.

Moreover, there exists a thing like the following patent document 1 as a related technique.
JP-A-2005-244321

  However, the above-mentioned ITU-TP. In a method such as 920, the photograph to be shown to the subject is selected without considering the age and ability of the subject. For example, the contents of the photograph cannot be memorized sufficiently depending on the memory ability of the subject and the call quality is evaluated. There is a problem of affecting the communication. Therefore, sufficient evaluation of the call quality could not be performed.

The present invention has been made in view of the above points, and an object of the present invention is to improve the reliability of a call quality evaluation result in a network call quality test performed by presenting an image to a user. to provide a for speech quality test server and call quality test methods.

  The present invention has been made to solve the above-described problem, and includes a storage unit that stores a material image, an image generation unit that generates a test image from the material image of the storage unit according to a user attribute, and a network. The image providing means for providing the test image generated by the image generating means to a plurality of communication terminals performing the call quality test as an image to be presented to the user in the call quality test of Provide a quality test server.

In this invention, the call quality test server generates a test image from the material image according to the user's attribute, and provides the test image to the communication terminal. Therefore, since the test image is generated according to the user attribute, the content of the test image becomes appropriate for the user who evaluates the call quality, and sufficient communication can be achieved in the call during the call quality test. As a result, it is possible to sufficiently evaluate the call quality, and the reliability of the obtained evaluation result is improved.
In the call quality test, the communication terminal presents (displays) the test image to the user, that is, the subject in advance, the user stores the content of the test image, and the user talks between the communication terminals regarding the content of the test image. And the communication, the user takes a procedure of evaluating whether or not the telephone call can be made without any problem in the video and audio.

  In the call quality test server, the image generation unit adjusts the number and / or size of the material images included in the test image according to a user attribute.

  According to the present invention, it is possible to generate a test image in which the number and size of material images are appropriately adjusted according to user attributes. In the call quality test, it is a condition for obtaining a sufficient evaluation result that the test image that is presented to the user in advance and matches the user's ability (sight and memory). Therefore, it is important to adjust the number of material images in the test image (related to the user's memory and the like) and the size (related to the user's visual acuity and the like).

  In the call quality test server, the image generation unit generates a test image including the material image that is different for each communication terminal.

  According to the present invention, the number of different material images and what the different material images are can be set as one of evaluation items by the user. That is, the user can search for a mistake in the test image of the other party during communication with the other party in the call in the call quality test, and can answer the number and contents as one of the evaluation items. If the test image does not match the user's ability, the correctness / incorrectness of the mistake the user has searched for may be due to a problem with the call quality or due to the user's own ability (sight, memory) The above items cannot be used as evaluation items. In the present invention, the test image corresponding to the user attribute is generated as described above. Therefore, the user's ability for the used test image is guaranteed. It can be judged that there is a problem in quality.

  In the call quality test server, the user attribute includes at least one of the user's age and visual acuity.

  According to the present invention, it is possible to generate an appropriate test image suitable for the user's visual acuity and memory (age).

  The call quality test server may further include a counting unit for counting the test results of the call quality test transmitted from the communication terminal.

  According to this invention, the evaluation results answered by the user can be aggregated to analyze the call quality of the network.

  In the call quality test server, the image generation means further generates the test image according to the tabulated test results.

  According to the present invention, a more appropriate test image can be generated by repeating the number of call quality tests.

  The present invention also includes a step of generating a test image from a material image according to a user attribute, and a plurality of tests for performing the call quality test on the generated test image as an image to be presented to a user in a network call quality test. A communication quality test method comprising: providing to the communication terminal; and counting the test results of the call quality test transmitted from the communication terminal.

  According to the present invention, since the test image to be presented to the user in the network call quality test is generated according to the user attribute, the reliability of the call quality evaluation result can be improved.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
First, the contents of a call quality test to which the present invention is applied will be described with reference to FIGS.
FIG. 1 shows a configuration example according to an embodiment of a network that performs a call quality test. The network shown in FIG. 1 includes a call quality test server 1, ten client PCs (personal computers) 2 A (communication terminals), and a router 3. Five client PCs 2A are arranged in each of the laboratory A and the laboratory B, and are connected to the call quality test server 1 via the router 3.

  In the call quality test using the network configuration of FIG. 1, 10 users (subjects) participate in the call quality test. Among the 10 users, 5 users use the client PC 2A of the laboratory A, and the remaining 5 users use the client PC 2A of the laboratory B. The user in the laboratory A and the user in the laboratory B make five pairs one by one, and make a videophone call between the pairs using the client PC 2A. In this configuration, in particular, a call quality test is performed for the purpose of evaluating the call quality of a network at a laboratory level and clarifying the call quality required in an actual commercial network. Therefore, the call quality test server 1 is provided with a function capable of setting transmission parameters such as delay and packet loss (see FIG. 4).

  FIG. 2 shows a configuration example according to another embodiment of a network that performs a call quality test. The network in FIG. 2 includes a call quality test server 1, a mobile phone terminal 2 </ b> B (communication terminal), and a base station 4. Each mobile phone terminal 2 </ b> B is connected to each other via the base station 4 and also connected to the call quality test server 1.

The network shown in FIG. 2 is a mobile phone network that is operating in actual commercial use, and a user participates in a call quality test by making a videophone call with the other user using the mobile phone terminal 2B. In this configuration, it is possible to measure call quality in a commercial mobile phone network by performing a call quality test.
The following description is common to both the network configurations of FIG. 1 and FIG. 2 unless otherwise specified.

FIG. 3 is a flowchart for explaining a procedure for conducting a call quality test.
First, the user inputs attribute information such as age and visual acuity to the communication terminal 2 (either the client PC 2A or the mobile phone terminal 2B, the same applies hereinafter) (step S1). The input attribute information is transmitted from the communication terminal 2 to the call quality test server 1 and stored in the call quality test server 1. Here, it is assumed that there are two users A and B who participate in the call quality test, and attribute information is collected from each user and stored. In this step, a series of operations such as image storage, videophone call, evaluation result input, etc. to be performed in the following production test are practiced in advance, and the practice result is used as one of user attribute information. It may be transmitted to the call quality test server 1 and stored.

  Next, the call quality test server 1 generates a test image to be presented to the user from the material image according to the attribute information of each user acquired in step S1 (step S2). Details of the test image generation will be described later.

  Next, the call quality test server 1 transmits the generated test image to the communication terminal 2 of each user. Each communication terminal 2 acquires a test image to be transmitted and displays the test image on the display for a predetermined time. As the predetermined time, for example, a time of about 1 minute is set. Since the test image is presented to each user in this way, the user views the test image for the predetermined time and stores the content (step S3).

  In the case of a call quality test in a laboratory (that is, when a call quality test is performed with the network configuration of FIG. 1), the call quality test server 1 sets transmission parameters for a videophone call in the next step S5 ( Step S4). Examples of transmission parameters to be set include a delay amount and an error rate. Note that this step is omitted when the call quality test is performed in the commercial mobile phone network of FIG.

Next, the users A and B make a videophone call between the communication terminals 2 (step S5). Assume that a predetermined time of about 3 minutes is set as the time for making a videophone call. In this videophone call, the user makes a call with the content of the test image presented in step S3 as a topic, and discusses the coincidence and difference between the test images of each other. For example, when a test image as shown in FIG. 7 to be described later is presented to the user, the contents of the discussion (call) are as follows as an example.
User A: “There was a house with a blue roof in the middle.”
User B: “The roof was red here. Was there a window?”

  By discussing the above topics through a videophone, each user can experience and evaluate the quality of video and audio in a videophone call. For example, it is evaluated whether there is any disturbance in the video or noise in the audio, or whether there is a delay or a synchronization shift between the video and the audio. Since the user stores the test image in step S3 and does not need to see anything other than the videophone screen during the test, the user can concentrate on the videophone video and audio during the call.

  When the predetermined time elapses, the videophone call is terminated, and each user inputs the evaluation result of the call quality test to the communication terminal 2 (step S6). The input evaluation result is transmitted from the communication terminal 2 to the call quality test server 1 and stored in the call quality test server 1. Here, the evaluation result items are (1) the number of mistakes between the test image of the other party found by the user in the videophone call and the content of the mistake, or (2) the videophone call. The MOS evaluation value for the quality of the experienced video and audio is used. Note that MOS (Mean Opinion Score) is a numerical evaluation of user's subjective evaluation in five levels (very good, good, normal, bad, very bad), and represents the evaluation result of the call quality test. Commonly used as a scale.

The call quality test server 1 analyzes the evaluation result of the call quality test collected from each user (step S7). For example, the average or variance of MOS evaluation values is calculated, or the correct answer rate for the number of test image errors is calculated. Since the call quality test server 1 has generated the test images of the users A and B, it knows the number of mistakes K included between the test images of each user, and the number of mistakes KA answered by the user _A and It is determined whether the number of mistakes K _B answered from the user B matches K. And if both K _A and K _B match K, the correct answer rate is 100%, if either is correct, the correct answer rate is 50%, and if both do not match, the correct answer rate is 0%. Become. The correct answer rate can be calculated for all users who participate in the call quality test. When a high correct answer rate is obtained, it is determined that the call quality of the videophone is good, and when a low correct answer rate is obtained, it is possible to determine that there is a problem with the call quality. The same determination may be made from the analysis result of the MOS evaluation value.

Next, a call quality test server 1 and a communication terminal 2 according to an embodiment of the present invention will be described.
FIG. 4 is a block diagram showing the configuration of the call quality test server 1 and the communication terminal 2. The communication terminal 2 shown in the figure shows a user A and a user B who make a videophone call, but the configuration is the same.

  In FIG. 4, the call quality test server 1 includes a material image DB (database) 11, a test image generation unit 12, and an evaluation result totaling unit 13. When used in the network configuration of FIG. 1 described above, the call quality test server 1 further includes a network setting unit 14.

  The material image DB 11 is a database that stores image data of material images that are materials for generating test images. FIG. 5 shows an example of a material image. As the material image, a picture such as a house, a person's face, a person's hand, a post box, a tree, a signal, a flag painted in three colors, a bent arrow, or the like is used. Further, these material images include a plurality of pictures with different details. For example, in the case of a house picture, a plurality of house pictures having different roof colors, window shapes and colors are prepared. In addition, in the case of a person's hand picture, a hand picture of multiple people with different hand shapes is prepared, such as a hand with a pointing hand, a hand with an extended palm, a hand with a choki shape, etc. . Similarly, in the case of a signal or flag picture, a plurality of pictures having different colors are prepared. A plurality of bent pictures with different shapes are also prepared. The material image is preferably a simplified picture composed of straight lines and simple curves so that the user can easily memorize the contents in the call quality test. The format of the image data may be any format such as a bitmap format or JPEG format.

  The test image generation unit 12 generates a test image by combining material images in the material image DB 11 according to user attribute information acquired from the communication terminal 2. There are two test images to be generated: a test image for user A and a test image for user B. The number and size of the material images included in these two test images are determined in accordance with the attribute of either user A or user B. When there are five pairs for videophone calls as shown in FIG. 1 described above, the number and size of material images are determined for each pair. FIG. 6 shows a table in which the relationship between the user attribute and the number and size of material images to be used is referred to when the test image generation unit 12 generates a test image.

In FIG. 6, when the user's visual acuity (the lower visual acuity among users A and B; the same applies hereinafter) is 0.7 or more, the test image generation unit 12 sets the material image to a normal size (the material image in the material image DB 11). To generate a test image. Further, when the visual acuity is 0.3 or more and less than 0.7, the material image is enlarged to 1.5 times the normal size and used to generate a test image. The same is true for other eyesights.
Further, when the result of the user's prior practice (the result of the worse of users A and B. The same applies hereinafter) is 95% or higher, the test image generation unit 12 generates a test image using six material images. To do. When the correct answer rate is 70% or more and less than 95%, a test image is generated using five material images. The same applies to other correct answer rates. The reason for changing the number of material images according to the result of prior practice is to take into account the user's memory.
Note that the number and size of the material images may be determined according to the age of the user.

  In this way, since the test image is generated according to the user's attributes (age, ability such as visual acuity and memory), the content of the test image can be appropriate for the user who evaluates the call quality. Therefore, for example, the user cannot sufficiently memorize the content of the test image, which causes troubles in communication for evaluating the call quality, or conversely, the content of the test image is too simple and the conversation ends briefly. Thus, the problem that the original call quality is not sufficiently evaluated does not occur, and the call quality test can proceed smoothly. This prevents inconvenience that the evaluation result of the call quality test is affected by the user's ability and the like, and it is possible to obtain an evaluation result with high reliability.

  The test image generation unit 12 also causes errors to be included in the test image for user A and the test image for user B to be generated. To include mistakes, for example, change the color, orientation, and size of the material image to be used, change the location where the material image is placed in the test image, and expand the hand and palm that are pointed The user A and the user B exchange material images having similar shapes such as their hands. The test image generation unit 12 selects these methods as appropriate (for example, randomly), and generates a test image including an error for each user.

  An example of the test image generated in this way is shown in FIG. In FIG. 7, one is a test image for user A, and the other is a test image for user B. Both are generated to include three errors: the size of the tree, the color of the house roof, and the shape of the human hand.

Returning to the block diagram of FIG.
The evaluation result totaling unit 13 acquires the evaluation result of the call quality test input by the user from the communication terminal 2, and totalizes / analyzes it. The specific contents of the analysis are, as described above, calculation of the average and variance of MOS evaluation values, calculation of the correct answer rate for the number of test image errors, and the like. In calculating the correct answer rate, the evaluation result totaling unit 13 obtains the correct number of correct answers from the test image generating unit 12, and calculates the correct answer rate from the answers collected from the communication terminal 2 and the correct answer values. The evaluation result totaling unit 13 also obtains a result of preliminary practice from the communication terminal 2 and provides the result to the test image generating unit 12.

  The network setting unit 14 sets transmission parameters such as transmission delay and error rate for a videophone call between the communication terminals 2. According to the transmission parameters set by the network setting unit 14, a videophone call in the call quality test is performed. The transmission parameters set by the network setting unit 14 are supplied to the evaluation result totaling unit 13 and are subjected to totalization / analysis together with the evaluation results from the communication terminal 2. As described above, when the call quality test is performed with the configuration of FIG. 2, the network setting unit 14 is omitted, and the transmission parameter is not set. In FIG. 4, the network setting unit 14 is shown as a device external to the call quality test server 1, but the call quality test server 1 may be configured to include this network setting unit 14.

  Next, in FIG. 4, the communication terminal 2 includes an attribute information input unit 21, a display 22, a videophone call unit 23, an evaluation result input unit 24, a data communication unit 25, and a control unit 26. Composed.

  The attribute information input unit 21 receives input of attribute information from the user, and the evaluation result input unit 24 receives input of the evaluation result from the user. The input attribute information and evaluation result are transmitted from the data communication unit 25 to the call quality test server 1. The attribute information input unit 21 and the evaluation result input unit 24 can be configured by an input interface such as a numeric keypad or a keyboard.

  The display 22 displays the test image acquired from the call quality test server 1. The display of the test image on the display 22 is performed for a predetermined time under the control of the control unit 26.

  The videophone call unit 23 performs videophone communication with the communication terminal 2 of the other user who is calling. Videophone calls are also made for a predetermined time under the control of the control unit 26. The videophone call unit 23 includes functions such as a camera, a microphone, an audio codec, and a video codec necessary for making a videophone call.

  The data communication unit 25 transmits / receives attribute information, evaluation results, and test image data to / from the call quality test server 1.

  The control unit 26 controls the above-described units of the communication terminal 2, and includes a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Read / Write Memory as needed), and the like. The ROM stores a computer program for operating the communication terminal 2, and the CPU reads the program and executes the program, thereby causing the communication terminal 2 to realize a predetermined function.

  That is, the CPU first receives attribute information input by the user from the attribute information input unit 21 and transmits the attribute information from the data communication unit 25 to the call quality test server 1. Then, the CPU receives the test image from the call quality test server 1 received by the data communication unit 2, passes the test image to the display 22, and displays it for a predetermined time. Thereafter, the CPU causes the videophone call unit 23 to execute videophone communication for a predetermined time. After the videophone call is finished, the CPU receives the evaluation result input by the user from the evaluation result input unit 24 and causes the data communication unit 25 to transmit the evaluation result to the call quality test server 1.

  As described above, the embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to the above, and various design changes and the like can be made without departing from the scope of the present invention. It is possible to

  For example, in the above embodiment, the description has been made on the assumption that a video phone call quality test is performed. However, the present invention can be applied to a voice call quality test using only voice.

  The specific contents of the material image and the test image generated from the material image are not particularly limited in the present invention. For example, in addition to the material image illustrated in FIG. 5, a simple map, graph, house floor plan, or the like may be used. In particular, in order to effectively evaluate the video in a videophone call, it is preferable to include a picture that induces body language in the call in the test image. Here, the “picture that induces the body language” refers to a picture of a bent arrow shown in FIG. 5, for example. That is, it is expected that the user inevitably moves the hand etc. to have a conversation when the user discusses the coincidence / difference in a videophone call using such a picture as a topic. Thus, the videophone call does not end with a voice-only conversation, and the user's operation can be transmitted as an image and evaluated.

  Further, the communication terminal 2 transmits the time information when the call quality test is performed together with the evaluation result and the position information of the communication terminal 2 to the call quality test server 1, and the evaluation result including the time information and the position information is included. It is also possible to perform aggregation and analysis. By conducting such call quality tests for a large number of users in a mobile phone network that is operating commercially, map information showing the call quality of the mobile phone network can be created, and the call quality changes with time It is possible to obtain information on the map, and it is also possible to provide services using such map information.

1 is a configuration diagram according to an embodiment of a network that performs a call quality test. FIG. It is a block diagram by one Embodiment of the network which performs a call quality test. It is the flowchart explaining the implementation procedure of a call quality test. It is a block diagram which shows the structure of the server for call quality tests, and a communication terminal. It is the figure which showed the example of the material image. It is an example of a table in which the relationship between user attributes and the number and size of material images to be used is set. It is the figure which showed the example of the test image produced | generated by the server for call quality tests.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Call quality test server 2 ... Communication terminal 2A ... Client PC 2B ... Mobile phone terminal 3 ... Router 4 ... Base station 11 ... Material image DB 12 ... Test image generation part 13 ... Evaluation result totaling part 14 ... Network setting part 21 ... Attribute information input unit 22 ... Display 23 ... Video phone call unit 24 ... Evaluation result input unit 25 ... Data communication unit 26 ... Control unit

Claims (7)

Storage means for storing material images;
Image generating means for generating a test image from the material image of the storage means according to user attributes;
Image providing means for providing a test image generated by the image generating means to a plurality of communication terminals performing the call quality test as an image to be presented to a user in a network call quality test;
A call quality test server characterized by comprising:   2. The call quality test server according to claim 1, wherein the image generation means adjusts the number and / or size of the material images included in the test image according to a user attribute. .   The call quality test server according to claim 1, wherein the image generation unit generates a test image including the material image that is different for each of the communication terminals.   The call quality test server according to any one of claims 1 to 3, wherein the user attribute includes at least one of a user's age and visual acuity.   5. The call quality test server according to claim 1, further comprising a counting unit that counts the test results of the call quality test transmitted from the communication terminal.   6. The call quality test server according to claim 5, wherein the image generation means further generates the test image according to the tabulated test results. Generating a test image from a material image according to user attributes;
Providing the generated test image to a plurality of communication terminals performing the call quality test as an image to be presented to a user in a call quality test of the network;
Totalizing test results of the call quality test transmitted from the communication terminal;
A call quality test method comprising:

Images