JP4867780B2 - Quality control system and quality control method in rich media communication - Google Patents

Description

  In the present invention, a stream type medium (hereinafter referred to as a stream type medium) in which a certain amount of data such as video and audio is constantly transmitted and received, and various amounts of data such as document sharing and file transfer are intermittently transmitted. Relates to a quality control system and a quality control method in a rich media communication system that combines media transmitted and received (hereinafter referred to as burst type media).

  An example of quality control of a conventional rich media communication system is described in Non-Patent Document 1. In Non-Patent Document 1, in rich media communication combining voice, video, and document sharing, the quality of video transmitted from the terminal is controlled according to the CPU usage rate and network bandwidth measured by the terminal. Therefore, it is described that quality deterioration caused by an overload of the terminal is prevented.

  Further, Patent Document 1 describes an example of a communication system in which a relay device provides comfortable communication quality even when a line condition changes. In the system described in Patent Document 1, a relay device to which a plurality of terminal devices are connected has a transmission error detection unit. The transmission error detector calculates a transmission error rate based on information from the terminal device, and the relay device controls the amount of error correction code added to the data transmitted by the media to each terminal device according to the result. . With such a configuration, it is possible to maintain the quality of media data reproduced by each terminal device even when the line status fluctuates.

JP 2006-345289 A "Communication door express functions / features", [online], NEC Corporation, [March 26, 2007 search], Internet <URL http://www.nec.co.jp/middle/commdoor/products/express /kinou.html>

  As a configuration of the terminal device that realizes the rich media communication described in Non-Patent Document 1, a configuration shown in the block diagram of FIG. 27 is conceivable. The terminal device shown in FIG. 27 includes a data processing device 600 and a storage device 610. The storage device 610 includes a communication information management unit 611 that stores information such as an address of a communication partner terminal device. In the data processing apparatus 600, the stream type media control unit 601 performs transmission / reception control of stream type media data (stream type media data). The burst type media control means 602 performs transmission / reception control of burst type media data (burst type media data). The stream type media transmission unit 603 performs processing for transmitting the stream type media data to the communication partner terminal device under the control of the stream type media control unit 601 and the stream type media quality control unit 607. The stream type media receiving means 605 receives stream type media data. The burst-type media transmission unit 604 performs processing for transmitting burst-type media data to a communication partner terminal device under the control of the burst-type media control unit 602. The burst type media receiving means 606 receives burst type media data.

  The terminal state measuring unit 608 measures the CPU load state and the bandwidth usage rate in the terminal device. The stream-type media quality control unit 607 indicates that the CPU load state measured by the terminal state measurement unit 608 indicates a high load or the bandwidth utilization rate measured by the terminal state measurement unit 608 indicates a high utilization rate. A process for reducing the data amount of the stream type media is executed.

  FIG. 28 is a block diagram illustrating a configuration example of a communication system to which the quality control method described in Patent Document 1 is applied. The relay device 722 includes a transmission error rate detection unit 731 that detects a transmission error and calculates a transmission error rate, a line usage rate detection unit 732 that detects a communication line usage rate and calculates a line usage rate, and a relay device 722. Hierarchy allocation unit 733 that determines the number of information source code layers included in the encoded information to be transmitted and outputs the layer allocation information, and inputs the priority for the terminal device and the layer information for the source communication terminal via transmission path 723 In addition, a terminal information transmission / reception unit 734 that outputs information on the number of layers to be transmitted to each terminal device, a voice information storage unit 735 that inputs and holds voice information, and a transmission path 723 are transmitted from the relay device group. Encoding information receiving unit 736 for receiving encoded information, reconfiguration processing unit 737 for reconstructing encoded information based on layer allocation information, and reconstructed encoding And a coded information transmission unit 738 for output via the transmission line 721 to distribution to the appropriate terminal.

  In the relay device 722, when the transmission error rate detection unit 731 detects that the transmission error rate has increased, the reconfiguration processing unit 737 increases the amount of encoded information transmitted to each terminal device in order to maintain call quality. The transmission error correction code is inserted. Further, when the line usage rate detection unit 732 detects that the line usage rate has increased, the information of the encoded information transmitted to the terminal device is reduced.

  However, in the method described in Non-Patent Document 1 (see FIG. 27), since the quality control of the stream type media is performed after the status measurement result by the terminal status measuring means 608 is recognized, a control delay occurs. However, communication quality degradation occurs until the control is executed. Moreover, since only the state of the own terminal is targeted, the possibility of quality degradation at the receiving terminal is not lost. In order to solve this, there is no burst media processing when video quality is determined in consideration of resource usage by burst media processing so that quality degradation does not occur even if both media are used simultaneously In some cases, resources are wasted.

  Further, in the method described in Non-Patent Document 1 and the method described in Patent Document 1 (see FIG. 28), quality control is performed when it is detected that “deterioration of terminal state” or “transmission error” has occurred. Therefore, at the time of starting the quality control, the communication quality is already deteriorated or an event that causes the deterioration has already occurred.

  That is, the conventional method or means has the following problems.

  The first problem is that if the quality of stream media such as video is determined assuming a resource shortage due to the processing load of burst media, the resources of terminals and networks cannot be fully utilized. The reason for this is that resource use by burst type media processing occurs only intermittently.

  The second problem is that communication quality deterioration or an event that causes deterioration has already occurred when communication quality control is performed. The reason is that the communication quality control is triggered by detecting the deterioration of quality or the cause of the deterioration in the terminal or the relay device.

  Accordingly, an object of the present invention is to provide a quality control system and a quality control method capable of satisfying both the sufficient use of resources of terminals and networks and the avoidance of quality degradation due to lack of resources in rich media communication.

  It is another object of the present invention to provide a quality control system and a quality control method capable of avoiding quality deterioration before an event that causes quality deterioration or deterioration occurs in rich media communication.

The quality control system according to the present invention relays rich media communication in a rich media communication quality control system that combines stream type media in which a predetermined amount of data is constantly transmitted and received and burst type media in which data is transmitted and received in bursts. The relay apparatus performs a burst type media receiving unit (for example, burst type media receiving unit 111) for notifying completion of data reception and transmission of burst type media, and a stream type according to the notification. Stream type media quality control means (for example, stream type media quality control means 114) for controlling the quality of the media, and the terminal at the time of receiving the data according to the received data amount of the burst type media and the state of the stream type media Estimating the load of the location, so that the terminal state estimating means for determining an amount of data reduction of the stream type media from (e.g., terminal state estimating means 316), characterized in that it comprises a.

  The relay device may include a control method storage unit (for example, control method storage unit 121) that provides a stream type media quality control method to the stream type media quality control means.

  The relay device includes a terminal capability information storage unit (for example, terminal capability information storage unit 323) that provides terminal capability information to the terminal state estimation unit, and a state estimation rule storage unit (for example, state estimation rule) that provides a state estimation rule. Storage unit 324).

The quality control system according to the present invention relays rich media communication in a rich media communication quality control system that combines stream type media in which a predetermined amount of data is constantly transmitted and received and burst type media in which data is transmitted and received in bursts. The relay device is connected to the burst type media receiving means (for example, burst type media receiving means 111) for notifying the completion of data reception and transmission of burst type media, in accordance with the notification. stream type media quality control means in accordance with the capabilities of each terminal device to control the quality of the stream type of media (e.g., stream type media quality control means 114) and wherein the including that.

  The terminal device includes control information transmission / reception means (for example, control information transmission / reception means 517) for notifying the relay device of terminal capability information, and the relay device receives control information of the terminal device from the terminal device. Means (for example, control information receiving means 417) and a terminal capability information storage unit (for example, terminal capability information storage unit 423) for providing the received terminal information to the stream type media quality control unit may be included.

  The terminal device has terminal state measuring means (for example, terminal state measuring means 718) for measuring the resource state of the terminal device, and control information transmitting / receiving means (for example, control information transmitting / receiving means 517) for notifying the measured resource state of the terminal device. The relay apparatus may include control information receiving means (for example, control information receiving means 617) that receives the resource status of the terminal apparatus from the terminal apparatus and notifies the stream type media quality control means.

  The relay apparatus includes quality control content determination means (for example, quality control content determination means 818) that selects an appropriate medium as a quality control target from a plurality of stream-type media and also determines a quality control method for the media. But you can.

  The relay device has a terminal capability information storage unit (for example, a terminal capability information storage unit 323) that provides terminal capability information to a quality control content determination unit, and a control content determination rule storage unit (for example, a control content determination rule). And a control content determination rule storage unit 824).

Quality control method according to the present invention is a quality control method for a rich media communication that combines a burst type of media for sending and receiving data media and intermittently varying amounts of stream type a predetermined amount of data is always transmitted and received, Rich The media communication relay device starts the quality control of the stream type media triggered by the reception of the burst type media data, and ends the quality control of the stream type media triggered by the completion of the transmission of the burst type media data. The relay device estimates the resource status of the terminal device at the time of receiving the data according to the data amount of the burst type media and the format and data amount of the stream type media, and determines the quality control content of the stream type media from the estimation result It is characterized by doing.

  In the quality control method, the condition of the resource state of the terminal device to be maintained may be given by the participant during the communication.

The quality control method according to the present invention is a rich media communication quality control method that combines a stream type medium that constantly transmits and receives a predetermined amount of data and a burst type medium that intermittently transmits and receives various amounts of data. The media communication relay device starts the quality control of the stream type media triggered by the reception of the burst type media data, and ends the quality control of the stream type media triggered by the completion of the transmission of the burst type media data. The relay apparatus determines the quality control content of the stream type media in accordance with individual capabilities and attribute values notified from the connected terminal apparatus .

  In the quality control method, the terminal device measures the resource state of the terminal device, and notifies the relay device when there is a certain fluctuation, and the relay device reports the quality of the stream type media according to the contents of the fluctuation. The control content may be changed.

  In the quality control method, when the terminal device exceeds or falls below a predetermined threshold value for a predetermined time, or when the measured value exceeds a predetermined difference value by comparing a plurality of measured values If there is, it may be judged as a large fluctuation.

  In the quality control method, the relay apparatus may select an appropriate medium as a quality control target from a plurality of stream type media.

  In the quality control method, when the relay device determines the quality control target medium and determines the quality control method for the medium, the quality control result is evaluated for all the quality control methods of a plurality of media, and the result Based on the above, an appropriate medium and its control method may be determined.

A preferable aspect of the quality control system according to the present invention is, for example, in rich media communication in which stream type media and burst type media are combined, rich media communication is performed by a plurality of terminal devices via a relay device, Burst type media receiving means for notifying completion of data reception and transmission of burst type media (for example, burst type media receiving means 111 shown in FIG. 2), stream type for implementing quality control of stream type media and accompanying data amount reduction According to the media quality control means (for example, the stream type media quality control means 114 shown in FIG. 2), the load of the terminal device at the time of data reception is estimated according to the received data amount of the burst type media and the state of the stream type media, Stroke from results Terminal state estimating means for determining an amount of data reduction of over beam-type medium (for example, terminal state estimating means 316) and having a. Adopting such a configuration, starting the quality control and data volume reduction of the stream type media by the data reception notification of the burst type media, and ending the quality control by the transmission completion notification achieve the two objects of the present invention can do.

  ADVANTAGE OF THE INVENTION According to this invention, resource shortage can be avoided, fully utilizing the resource of a terminal and a network in rich media communication. As a result, high-quality rich media communication can be realized. The reason is that the quality control of the stream type media is performed only when the data processing of the burst type media that occurs intermittently occurs.

  Further, according to the present invention, it is possible to avoid a shortage of network and terminal resources that cause quality degradation in rich media communication before actual occurrence. The reason is that the quality control can be performed before data communication of burst type media causing a shortage of resources of the network or the terminal or processing at the terminal.

Embodiment 1 FIG.
Hereinafter, a first embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram illustrating a configuration example of a communication system (quality control system) to which a quality control method is applied. In the communication system shown in FIG. 1, the terminal device 2-1 and the terminal device 2-2 communicate via the relay device 1-1. Furthermore, the terminal devices 2-1 and 2-2 can communicate with the terminal device 2-3 via the relay devices 1-1 and 1-2.

  Note that the relay device 1-1 and the relay device 1-2 are devices having the same configuration, and the capabilities of the components are also the same. Moreover, the terminal devices 2-1, 2-2, and 2-3 are devices having the same configuration, and the capabilities of the components are the same. Hereinafter, when referring to the relay apparatuses 1-1 and 2 generically, or when referring to either of the relay apparatuses 1-1 and 2, the relay apparatus 1 may be described. In addition, the terminal device 2-1, 2, 3 may be referred to as the terminal device 2 when it is comprehensively indicated, or when any of the terminal devices 2-1, 2, 3 is indicated.

  The relay device 1 is connected to one or more terminal devices 2. Further, the relay devices 1 may be connected to each other. When communication is performed, two or more terminal devices 2 communicate with each other via one or more relay devices 1.

  FIG. 2 is a block diagram illustrating a configuration example of the relay device 1 and the terminal device 2. FIG. 2 illustrates a case where the relay device 1-1 is connected to the terminal devices 2-1, 2-2.

  The relay device 1 includes a data processing device 11 and a storage device 12.

  The data processing apparatus 11 includes a burst type media receiving unit 111, a burst type media transmitting unit 112, a stream type media receiving unit 113, a stream type media quality control unit 114, and a stream type media transmitting unit 115. The storage device 12 includes a control method storage unit 121 and a communication information management unit 122.

  The terminal device 2 includes a data processing device 21 and a storage device 22.

  The data processing device 21 includes a stream type media control unit 211, a burst type media control unit 212, a stream type media transmission unit 213, a burst type media transmission unit 214, a stream type media reception unit 215, and a burst type media reception. Means 216. The storage device 22 includes a communication information management unit 221.

  The relay device 1 relays data transmitted by the burst-type media transmission unit 214 regarding burst-type media data such as file transfer and document sharing. Similarly, the relay device 1 relays data transmitted by the stream-type media transmission unit 213 with respect to stream-type media data to which a predetermined amount of data such as audio and video is always transmitted.

  The burst-type media receiving unit 111 receives communication contents from all the terminal devices 2 connected to the relay device 1 and other relay devices. The burst-type media transmission unit 112 transmits the communication content received by the burst-type media reception unit 111 to the target connected terminal device 2 and the relay device 1 based on the stored information of the communication information management unit 122.

  Similarly, the stream type media receiving unit 113 receives stream type media data from the stream type media transmitting unit 213, for example. The stream type media transmission unit 115 transmits the data received by the stream type media reception unit 113 to, for example, the stream type media reception unit 216.

  In the burst type media receiving unit 111 and the stream type media receiving unit 113, at the time of data reception, the received data is data obtained by communication with the terminal device 2 of the other party performing communication (information transmission) as a transmission source. Is determined based on the information of the communication information management unit 122. If the communication is not from the terminal device 2 that is executing communication, the burst-type media receiving unit 111 and the stream-type media receiving unit 113 may behave as not relaying as unauthorized communication.

  The stream type media quality control means 114 changes the quality and data amount of the stream type media to be relayed according to the information indicating the conversion method stored in the control method storage unit 121. Examples of processing performed by the stream-type media quality control unit 114 include changes in resolution and frame rate by transcoding, and partial reduction of high-resolution data and high frame rate data of a bitstream by scalable coding.

  The communication information management unit 122 stores address information of all terminal devices 2 participating in communication. For example, for the terminal device 2-3 (see FIG. 1) that performs communication via the relay device 1-2, the address information of the terminal device 2-3 and the relay device to which the terminal device 2-3 is directly connected. The information associated with the address information 1-2 is stored. The communication information management unit 122 stores the address information of all the terminal devices 2 participating in the communication, thereby enabling communication with all the terminal devices 2 participating in the communication.

  In the terminal device 2, the stream type media control unit 211 and the burst type media control unit 212 control the display of contents of each media and the input by the user, respectively.

  For example, the stream type media control unit 211 of the terminal device 2-1 causes the stream type media transmission unit 213 to transmit the input content according to the address of the connection destination relay device 1 stored in the communication information management unit 221.

  For example, the stream type media receiving unit 215 of the terminal device 2-2 receives the stream type media data transmitted by the connection destination relay device 1. The stream type media control unit 211 displays the received stream type media data.

  For burst type media, burst type media control means 212, burst type media transmission means 214 and burst type media reception means 216 are stream type media control means 211, stream type media transmission means 213 and stream type for stream type media data. The same behavior as the media receiving means 215 is performed.

  FIG. 2 illustrates a state in which the terminal device 2-1 serves as a data transmission source in communication and communicates with the terminal device 2-2 via the relay device 1-1. In communication, the terminal devices 2 communicate with each other. For example, when the terminal device 2-2 is a transmission source and performs communication with the terminal device 2-1, the state illustrated in FIG. 3 is obtained. FIG. 3 is an explanatory diagram showing a data flow when the terminal device 2-2 transmits data to the terminal device 2-1. The configurations of the relay device 1 and the terminal device 2 shown in FIG. 3 are the same as the configuration example shown in FIG. 2, and only the data flow is different.

  When a plurality of terminal devices 2 are connected to the relay device 1, as shown in FIG. 4, the burst type media receiving unit 111, the burst type media transmitting unit 112, and the stream type media receiving unit 113 of the relay device 1 are used. A plurality of terminal devices 2 are connected to the stream type media transmission means 115, respectively. FIG. 4 is an explanatory diagram illustrating a connection example when a plurality of terminal devices 2 are connected to the relay device 1. FIG. 4 shows only a part of the configurations of the relay device 1 and the terminal device 2.

  Next, the operation of the first embodiment will be described with reference to FIG. 2 and FIG. FIG. 5 is a flowchart showing the operation of the quality control system in the first embodiment.

  When the burst type media receiving unit 111 receives data from the burst type media transmitting unit 214 during execution of the communication combining the burst type media and the stream type media (step S101), the burst type media receiving unit 111 sends the data to the stream type media quality control unit 114. It is notified that the burst media data has arrived (step S102).

  Receiving the notification, the stream-type media quality control means 114 starts conversion processing for reducing the quality and reducing the data amount of the stream-type media being relayed (step S103).

  The stream type media quality control unit 114 selects the conversion processing method from the control method storage unit 121 so that the condition of the terminal device 2 satisfies a certain condition according to the data amount and data format of the stream type media data being relayed. May be. For example, when the amount of data is large, it may be possible to select a conversion processing method with a larger amount of data reduction so that the amount of data received by the terminal device 2 falls within a certain range. The converted stream type media data is relayed in the same manner as before the conversion process starts, but the data amount is reduced as a result of the conversion.

  After the notification to the stream type media quality control unit 114, the burst type media reception unit 111 transmits the received data to the burst type media transmission unit 112 (step S104). The burst-type media transmission unit 112 transmits data to the terminal device 2-2 based on the transmission destination information of the communication information management unit 122, thereby relaying the burst-type media.

  If new data does not arrive from the burst-type media transmission unit 214 even after a predetermined time has elapsed after transmitting the data, the burst-type media reception unit 111 sends the burst-type media data to the stream-type media quality control unit 114. Is transmitted (step S105). Receiving the notification, the stream type media quality control means 114 ends the conversion process for reducing the data amount (step S106). As a result, the stream type media to be relayed returns to the quality prior to the start of the conversion process.

  As described above, according to the first embodiment, when burst-type media data and stream-type media data are communicated simultaneously, the data amount of stream-type media data is reduced, and the network bandwidth is reduced. The usage amount and the processing load on the terminal device 2 are reduced. Therefore, when the stream type media performs high quality communication, even if the network bandwidth between the relay device 1 and the terminal device 2 that can be used in this communication and the processing capability of the terminal device 2 are almost used up, the burst type media is used. By adding the data communication and the processing at the terminal, it is possible to avoid the deterioration of the quality of the stream type media due to insufficient processing capability. That is, the stream-type media can perform the highest quality communication as much as possible from time to time.

Embodiment 2. FIG.
Next, a second embodiment of the present invention will be described with reference to the drawings. FIG. 6 is a block diagram illustrating a configuration example of the relay device 3 according to the second embodiment. The relay device 3 illustrated in FIG. 6 includes a data processing device 31 and a storage device 32. FIG. 6 illustrates the case where the relay device 3 is connected to the terminal devices 2-1 and 2-2. The configuration of the terminal devices 2-1 and 2-2 is the same as the configuration shown in FIG.

  Referring to FIG. 6, in the second embodiment, a data processing device 31 is provided instead of the data processing device 11 in the first embodiment shown in FIG. The data processing device 31 includes terminal state estimation means 316 that is not provided in the data processing device 11. Further, the burst type media receiving unit 111, the stream type media receiving unit 113, and the stream type media quality control unit 114 shown in FIG. 2 are different from each other in the same name (burst type media receiving unit 311, stream type media receiving unit 313). And stream type media quality control means 314). Further, in the storage device 32 in this embodiment, a terminal capability information storage unit 323 and a state estimation rule storage unit 324 are added to the storage device 12 shown in FIG.

  The burst media receiving means 311 notifies the terminal state estimating means 316 of the size of the received data when receiving data. Upon receiving the notification, the terminal state estimation unit 316 receives information indicating the state of the stream type media being relayed from the stream type media reception unit 313 and data processing of the terminal device 2 in communication stored in the terminal capability information storage unit 323. From the capability information of the device 21, using the rules stored in the state estimation rule storage unit 324, it is estimated what state the terminal device 2 will be in when data of the burst type media to be relayed is received. .

  Further, the terminal state estimation means 316 determines, based on the estimation result, how much the data amount of the stream-type media being relayed can be reduced to meet the condition for which the state of the terminal device 2 is given. For example, the terminal state estimation means 316 calculates the data amount of the stream type media for the state of the terminal device 2 to satisfy a predetermined condition, and calculates the difference from the data amount of the stream type media being relayed as the reduction amount. . The predetermined condition is to avoid deterioration of the quality of the stream type media. The terminal state estimation unit 316 instructs the stream type media quality control unit 314 about the calculated reduction amount.

  The stream-type media quality control means 314 selects an optimum conversion method from the control method storage unit 121 to realize the instruction content, and controls the quality of the stream-type media being relayed according to the method, thereby controlling the amount of data. Realize reductions. The conversion method, like the stream type media quality control means 114, is a partial reduction of transcoded or scalable encoded bit stream data.

  Next, the operation of the second embodiment will be described with reference to the drawings. FIG. 7 is a flowchart illustrating the operation of the quality control system according to the second embodiment. Since the processing in steps S101 and S103 to S106 in FIG. 7 is the same as the processing in the first embodiment, description thereof is omitted.

  When the burst media receiving means 311 receives the data (step S101), it notifies the terminal state estimating means 316 of the data received and the size of the received data (step S201).

  Upon receiving the notification, the terminal state estimation unit 316 receives the current stream type media data amount acquired from the stream type media reception unit 313 at regular intervals, the notified burst type media data amount, The capability information of the data processing device 21 of the terminal device 2 is applied to the rules of the state estimation rule storage unit 324. Then, the terminal state estimation means 316 estimates the state of the terminal device 2 when data of the burst type media to be relayed is received in addition to the stream type media currently being relayed. As the content of the estimation, for example, a CPU load state and a bandwidth usage rate can be considered.

  Then, based on the estimation result, the terminal state estimation means 316 determines how much the data amount of the stream type media being relayed can be reduced so that the terminal device 2 can follow the given condition regarding the state. A reduction amount is instructed to the media quality control means 314 (step S202).

  For example, in order to reduce the bandwidth usage rate to 80% or less, an instruction to reduce the data amount of the stream type media to 1 Mbps can be considered. The condition regarding the state of the terminal device 2 may be given in advance to the terminal state estimation means 316, or may be given from the terminal device 2 by the user during communication.

  The stream type media quality control means 314 determines whether or not quality control is performed at the time of instruction (step S203). If not executed (No), an optimal conversion method for realizing the instruction content is selected from the control method storage unit 121 (step S205), and the process proceeds to step S103.

  If quality control is being performed in step S203 (Yes), it is determined whether the current control method is optimal for the instruction content (step S204). If there is another optimum method (No), the stream type media quality control means 314 moves to step S205 and selects another method. If a new quality control method is selected, the quality control of the stream type media by the selected method and the accompanying data amount reduction are started (step S103).

  In step S204, when the current control method is optimal (Yes), the process proceeds to step S104. In step S <b> 104, the burst type media reception unit 111 transmits the received data to the burst type media transmission unit 112. The burst type media transmission means 112 transmits data to the terminal device 2-2. The burst type media receiving means 311 determines whether or not the burst type media data arrives again within a predetermined time (step S206). If it has arrived (Yes), the process proceeds to step S101 and the process is repeated. When it does not arrive (No), the process proceeds to step S105.

  As described above, according to the second embodiment, by changing the quality control method of the stream type media according to the data amount of the burst type media, the stream type media by the quality control for reducing the data amount is used. It is possible to minimize the quality degradation of the product.

Embodiment 3 FIG.
Next, a third embodiment of the present invention will be described with reference to the drawings.

  FIG. 8 is a block diagram illustrating a configuration example of the relay device 4 according to the third embodiment. The relay device 4 illustrated in FIG. 8 includes a data processing device 41 and a storage device 42. FIG. 8 illustrates a case where the relay device 4 is connected to the terminal device 5.

  FIG. 9 is a block diagram illustrating a configuration example of the terminal device 5 according to the third embodiment. The terminal device 5 illustrated in FIG. 9 includes a data processing device 51 and a storage device 52. FIG. 9 shows a part of the configuration of the relay device 4.

  Referring to FIG. 8, in the third embodiment, a data processing device 41 is provided instead of the data processing device 11 of the relay device 1 in the first embodiment shown in FIG. The data processing device 41 is different from the data processing device 11 in that the stream type media quality control unit 114 and the stream type media transmission unit 115 exist in the same number as the connected terminal device 5.

  Further, in the data processing apparatus 41 in this embodiment, control information receiving means 417 is added to the data processing apparatus 11 of the relay apparatus 1. In the storage device 42, a terminal capability information storage unit 423 is added to the storage device 12 of the relay device 1.

  Referring to FIG. 9, a control information transmitting / receiving unit 517 is added to the data processing device 51 with respect to the data processing device 21 of the terminal device 2. In the storage device 52, a terminal information storage unit 522 is added to the storage device 22 of the terminal device 2.

  The stream type media quality control means 114 and the stream type media transmission means 115 adjust the quality of the stream type media and the transmission of the controlled media in accordance with the processing capability of the data processing device 51 of each connected terminal device 5. I do.

  The control information transmitting / receiving unit 517 transmits the terminal address information stored in the terminal information storage unit 522 and the capability information of the data processing device 51 to the control information receiving unit 417.

  Upon receiving the terminal address information and the capability information, the control information receiving unit 417 stores the terminal address information in the communication information management unit 122 and stores the capability information in the terminal capability information storage unit 423.

  Next, the operation of the third embodiment will be described with reference to the drawings. In the operation of the third embodiment, the connection time information transmission processing (see FIG. 10) and the quality control method selection processing (see FIG. 11) are compared to the processing in the operation of the first embodiment. And have been added.

  FIG. 10 is a flowchart showing the connection information transmission operation in the third embodiment.

  In order for the terminal device 5 to participate in the communication, after connecting to the appropriate relay device 4, the control information transmitting / receiving unit 517 acquires the identifier of the participating communication from the communication information management unit 221 and sends it to the control information receiving unit 417. Send.

  The control information receiving unit 417 collates the received communication identifier with the identifier of the communication being executed in the relay device 4 stored in the communication information management unit 122. If participation is possible, the control information receiving unit 417 stores the address information of the terminal device 5 in the communication information management unit 122 and transmits information indicating that participation is possible to the terminal device 5 (step S301).

  Also, the control information receiving unit 417 newly adds a stream type media quality control unit 114 and a stream type media transmission unit 115 for the terminal device 5 newly participating in communication (step S302).

  The control information transmission / reception unit 517 that has received the information indicating that participation is possible acquires the capability information (terminal capability information) of the terminal device 5 from the terminal information storage unit 522 and transmits it to the control information reception unit 417 (step S303). The control information receiving unit 417 stores the received capability information in the terminal capability information storage unit 423 (step S304).

  Next, the operation of quality control method selection will be described with reference to FIG. FIG. 11 is a flowchart showing an operation of quality control method selection in the third embodiment.

  When communication is actually started following the connection-time information transmission operation, the stream-type media reception unit 113 receives, for example, the stream-type media data transmitted by the stream-type media transmission unit 213 (step S401). The data is passed to all the stream type media quality control means 114.

  The stream type media quality control unit 114 acquires the capability information of the terminal device 5 that the stream type media quality control unit 114 is in charge of from the terminal capability information storage unit 423 (step S402). The stream type media quality control means 114 observes a given condition (for example, keeps the CPU usage rate at 80% or less) regarding the state of the data amount of the stream type media from the acquired terminal capability information. Two values are estimated: the maximum processable amount and the maximum processable amount that allows the terminal device 5 to comply with a given condition regarding the state even when burst type media data is simultaneously processed.

  The stream type media quality control means 114 compares the estimated result with the received data amount of the current stream type media. The stream type media quality control means 114 determines whether or not the data amount reduction by the quality control is necessary. If necessary, the stream type media quality control means 114 determines an appropriate quality control method based on the information indicating the conversion method stored in the control method storage unit 121. select. Also, a quality control method to be used when simultaneous burst media is relayed is selected (step S403).

  The stream-type media quality control means 114 determines whether or not quality control is necessary even for the stream-type media alone (step S404). If it is determined that it is necessary (Yes), the stream-type media quality control means 114 starts reducing the data amount by the selected quality control method. (Step S405). The stream type media transmission unit 115 transmits the quality controlled stream type media data to the terminal device 5 (step S406).

  If it is determined in step S404 that quality control is unnecessary (No), the stream-type media transmission unit 115 transmits the stream-type media data as it is to the terminal device 5 (step S406).

  The subsequent quality control operation accompanying burst type data reception is the same as the processing in steps S101 to S106 shown in FIG. The stream-type media quality control unit 114 uses the method selected in step S403 (see FIG. 11) as the quality control method used in the data amount reduction in step S103 (see FIG. 5).

  According to the third embodiment, by changing the quality control method of the stream type media according to the capability of the terminal, the quality degradation of the stream type media due to the quality control for reducing the data amount is matched to the capability of each terminal. Can be minimized.

Embodiment 4 FIG.
Next, a fourth embodiment of the present invention will be described with reference to the drawings.

  FIG. 12 is a block diagram illustrating a configuration example of the relay device 6 according to the fourth embodiment. The relay device 6 illustrated in FIG. 12 includes a data processing device 61 and a storage device 42. FIG. 12 illustrates a case where the relay device 6 is connected to the terminal device 7.

  FIG. 13 is a block diagram illustrating a configuration example of the terminal device 7 according to the fourth embodiment. A terminal device 7 illustrated in FIG. 13 includes a data processing device 71 and a storage device 52. FIG. 13 shows a part of the configuration of the relay device 6.

  Referring to FIG. 12, in the fourth embodiment, a data processing device 61 is provided instead of the data processing device 41 in the third embodiment shown in FIG. In the data processing device 61, as compared with the data processing device 41, the control information receiving means 417 is a means (control information receiving means 617) having the same name and a different content. Referring to FIG. 13, terminal state measuring means 718 is newly added to the data processing device 71 with respect to the data processing device 51 of the terminal device 5 shown in FIG. 9.

  The terminal state measuring unit 718 measures the usage state of the terminal resource at regular intervals, and notifies the control information transmitting / receiving unit 517 when there is a certain fluctuation or more. Examples of usage status of terminal resources include CPU and memory usage rates.

  When the control information receiving unit 617 receives the terminal resource usage status change notification from the terminal device 7, the control information receiving unit 617 notifies the stream type media quality control unit 114 of the change contents.

  Next, the operation of the fourth embodiment will be described with reference to the drawings. In the operation of the fourth embodiment, a process (see FIG. 14) for performing quality control associated with terminal state fluctuation is added to the process in the operation of the third embodiment.

  FIG. 14 is a flowchart illustrating an operation of quality control according to the terminal state variation in the fourth embodiment.

  The connection time information transmission shown in steps S301 to S304 in FIG. 10, the quality control method selection shown in steps S401 to S406 in FIG. 11, and the burst type data reception shown in steps S101 to S106 in FIG. Since the accompanying quality control operation is the same, description thereof is omitted.

  After the connection information transmission and quality control method selection operations are completed, during communication execution, the terminal state measurement means 718 measures the resource usage status of the terminal at regular intervals (step S501).

  When the measured value of the resource usage situation has changed greatly, the terminal state measuring unit 718 notifies the control information transmitting / receiving unit 517 of the change and the actual changed value (step S502). As a method for judging a large fluctuation of a measured value, for example, when a predetermined threshold is exceeded or below a predetermined time, it is judged that the fluctuation is large, or a plurality of measured values are compared and given in advance. A method may be considered in which when there is a fluctuation exceeding the difference value, it is judged as a big fluctuation.

  Upon receiving the notification, the control information transmitting / receiving unit 517 notifies the control information receiving unit 617 that the terminal resource usage status has changed and the actual change contents.

  The control information receiving unit 617 determines the corresponding stream-type media quality control unit 114 from the address information of the transmission source terminal device 7 and the stored information of the communication information management unit 122, and transmits the received content.

  The received stream type media quality control means 114 uses the received information, the capability information of the terminal device 7 stored in the terminal capability information storage unit 423, and the state and format of the stream type media currently being relayed as shown in FIG. Similar to the processing in step S403, both the quality control method for relaying only stream type media and the quality control method for relaying burst type media at the same time are determined again (step S503).

  The stream type media quality control means 114 determines whether or not the quality control method needs to be changed as a result of the processing in step S503 (step S504). If it is determined that it is necessary (Yes), the stream type media quality control means 114 depends on the determined quality control method. Reduction of the data amount of the stream type media is started (step S505). The quality control method for relaying the re-determined burst type media at the same time is used in the quality control operation (step S103 in FIG. 5) accompanying the burst type data reception. If it is determined in step S504 that it is not necessary (No), the series of processing ends.

  According to the fourth embodiment, by changing the quality control method of the stream type media according to the change of the resource state of the terminal, the terminal device 7 becomes an unexpected overload state due to a load other than communication, It is possible to avoid a significant deterioration in communication quality. In addition, since the quality of the stream-type media can be controlled more finely, when the resources of the terminal device 7 are sufficient, the degradation of the quality of the stream-type media due to quality control for reducing the amount of data is minimized. It becomes possible.

Embodiment 5 FIG.
Next, a fifth embodiment of the present invention will be described with reference to the drawings.

  FIG. 15 is a block diagram illustrating a configuration example of the relay device 8 according to the fifth embodiment. A relay device 8 shown in FIG. 15 includes a data processing device 81 and a storage device 82. FIG. 15 illustrates a case where the relay device 8 is connected to the terminal device 9.

  FIG. 16 is a block diagram illustrating a configuration example of the terminal device 9 according to the fifth embodiment. The terminal device 9 illustrated in FIG. 16 includes a data processing device 91 and a storage device 22. FIG. 16 shows a part of the configuration of the relay device 8.

  Referring to FIG. 15, in the fifth embodiment, unlike the data processing device 31 in the second embodiment shown in FIG. 6, communication using two types of media, burst type media and stream type media, is performed. In order to do this, in the data processing device 81, burst type media receiving means 311, burst type media transmitting means 112, stream type media receiving means 313, stream type media quality control means 314, stream type media transmitting means 115, There are two each.

  For the same reason, referring to FIG. 16, unlike the data processing device 21 of the terminal device 2 shown in FIG. 2, the data processing device 91 also has a burst type media reception unit 216, a burst type media transmission unit 214, and There are two burst-type media control means 212, two stream-type media reception means 215, two stream-type media control means 211, and two stream-type media transmission means 213.

  In FIG. 15, the data processing device 81 of the relay device 8 includes burst type media receiving units 311x and 311y, burst type media transmitting units 112x and 112y, stream type media receiving units 313a and 313b, and stream type media quality control. A case is shown in which means 314a and 314b and stream type media transmission means 115a and 115b are provided. In FIG. 16, the data processing device 91 of the terminal device 9 includes burst type media receiving means 216x and 216y, burst type media transmission means 214x and 214y, burst type media control means 212x and 212y, and stream type media. The case where receiving means 215a and 215b, stream type media control means 211a and 211b, and stream type media sending means 213a and 213b is shown.

  Referring to FIG. 15, in the data processing apparatus 81, the terminal state estimation means 316 in the data processing apparatus 31 of the relay apparatus 3 shown in FIG. 6 is replaced with the quality control content determination means 818. In the storage device 82, the state estimation rule storage unit 324 in the storage device 32 is replaced with a control content determination rule storage unit 824.

  Similar to the terminal state estimation unit 316, the quality control content determination unit 818 receives the data reception notification from the burst type media reception unit 311x or the burst type media reception unit 311y, and then receives the stream type media reception unit 313a or the stream type media. The data of the burst type media to be relayed is received from the state and format of the stream type media being relayed obtained from the receiving means 313b and the capability information of the communicating terminal device 9 stored in the terminal capability information storage unit 323. In this case, it is estimated what state the terminal device 9 is in.

  Further, the quality control content determination means 818 performs quality control to prevent an overload state of the terminal device 9 from the rules stored in the control content determination rule storage unit 824 and the control method stored in the control method storage unit 121. Determine target stream media and quality control method. A plurality of media may be selected in the selection of the quality control target stream type media.

  Next, the operation of the fifth embodiment will be described with reference to the drawings. FIG. 17 is a flowchart showing the operation of the quality control system in the fifth embodiment. The operations in steps S101, S201, S206, and S103 to S106 in FIG. 17 are the same as the operations in the second embodiment (see FIG. 7), and thus description thereof is omitted.

  The quality control content determination unit 818 notified of the burst type media data reception, the media type, and the amount of the data is the stream type media being relayed obtained from the stream type media reception unit 313a or the stream type media reception unit 313b at regular intervals. When the data of the burst type media to be relayed is received from the state and format of the data and the capability information of the communicating terminal device 2 stored in the terminal capability information storage unit 323, what state the terminal device 2 is in It is estimated whether it becomes (step S601).

  Subsequently, the quality control content determination unit 818 selects an appropriate control method from the control method stored in the control method storage unit 121 according to the rules stored in the control content determination rule storage unit 824 so that the state satisfies the given condition. A control method and a target stream type medium are selected (step S602). The selected control method is notified to the target stream type media quality control means 314.

  Receiving the notification, the stream type media quality control means 314 determines whether or not the designated quality control means is being executed (step S603). If it is not in execution (No), the stream type media quality control means 314 starts quality control (step S103), and proceeds to step S104. If it is being executed (Yes), the process proceeds to step S104.

  According to the fifth embodiment, when there are a plurality of stream-type media, an effective media can be selected as a quality control target, so that the quality of the media as a whole can be kept higher. become.

Embodiment 6 FIG.
Next, a sixth embodiment of the present invention will be described with reference to the drawings.

  FIG. 18 is a block diagram illustrating a configuration example of the relay device 10 according to the sixth embodiment. The relay device 10 illustrated in FIG. 18 includes a data processing device 101 and a storage device 102. FIG. 18 illustrates a case where the relay device 10 is connected to the terminal device 1A.

  FIG. 19 is a block diagram illustrating a configuration example of the terminal device 1A according to the sixth embodiment. A terminal device 1A shown in FIG. 19 includes a data processing device 1A1 and a storage device 1A2. FIG. 19 illustrates a case where the terminal device 1A is connected to the terminal device 1A-2 via the relay device 10.

  Referring to FIG. 18, in the sixth embodiment, unlike the first embodiment shown in FIG. 2, the data processing device 101 does not have the stream type media quality control means 114 in the data processing device 11. . Further, the storage device 102 does not include the control method storage unit 121 in the storage device 12.

  Referring to FIG. 19, in the data processing apparatus 1A1, a stream type media quality control means 1A18 is added to the data processing apparatus 21, and the burst type media control means 212 has the same name and different contents (burst type media control means). Means 1A12). In the storage device 1A2, a control method storage unit 1A22 is added to the storage device 22.

  The connection relationship between the terminal device 2 and the relay device 1 may be a direct connection between the terminal devices 1A as shown in FIG. 20 in addition to the first embodiment shown in FIG. FIG. 20 is an explanatory diagram illustrating a connection example of the terminal device 1A.

  The stream-type media quality control means 1A18 corresponds to the stream-type media quality control means 114 in the best mode for carrying out the first invention. According to the conversion method stored in the control method storage unit 1A22, the stream-type media quality control means 1A18 Change quality and data volume. In addition to the function of the burst type media control unit 212, the burst type media control unit 1A12 has a function of giving a quality control instruction to the stream type media quality control unit 1A18 when transmitting media data.

  Next, the operation of the sixth embodiment will be described with reference to the drawings. FIG. 21 is a flowchart showing the operation of the quality control system in the sixth embodiment. Compared with the operation of the first embodiment (see FIG. 5), the contents are almost the same, but in the first embodiment, the relay device 1 performs the processing shown in FIG. The sixth embodiment is different in that it is performed in the terminal device 1A.

  Referring to FIG. 21, when the burst type media control means 1A12 tries to transmit data of the burst type media by the user's operation or the like while executing the communication combining the burst type media and the stream type media (step S701), The burst type media control unit 1A12 notifies the stream type media quality control unit 1A18 that the burst type media data is to be transmitted (step S702).

  Upon receiving the notification, the stream-type media quality control unit 1A18 starts conversion processing for reducing the quality and reducing the amount of data for the stream-type media being transmitted (step S703). The stream-type media quality control means 1A18 controls the conversion method so that the condition of the destination terminal device 1A satisfies certain conditions depending on the data amount and data format of the stream-type media being relayed. You may choose from. For example, when the amount of data is large, it may be possible to select a conversion processing method with a larger amount of data reduction so that the amount of data received by the terminal device 1A falls within a certain range. The converted stream type media is transmitted in the same manner as before the conversion process starts, but the data amount is reduced as a result of the conversion.

  After the notification to the stream type media quality control unit 1A18, the burst type media control unit 1A12 transmits the data to be transmitted to the burst type media transmission unit 214 (step S704). The burst-type media transmission unit 214 transmits data to the terminal device 1A-2 based on the transmission destination information of the communication information management unit 221.

  If no new data is transmitted after the burst-type media control unit 1A12 transmits data after a certain period of time, the stream-type media quality control unit 1A18 is notified that the transmission of the burst-type media data has been completed. (Step S705). The stream-type media quality control unit 1A18 that has received the notification ends the conversion process for reducing the data amount (step S706). As a result, the stream type media to be transmitted returns to the quality before the start of the conversion process.

  According to the sixth embodiment, it is possible to obtain the same effect as that of the first embodiment even in communication without a relay device.

  Next, the operation of the quality control system of the present invention will be described using a specific embodiment. This example corresponds to the fifth embodiment of the present invention (see FIGS. 15 to 17).

  In this embodiment, a central processing unit and a memory are considered as data processing devices, and a hard disk is considered as a storage device. The computer program on the memory functions as each means by being executed by the central processing unit.

  Further, in this embodiment, it is assumed that material sharing is handled as the burst media X, file transfer is handled as the burst media Y, video is handled as the stream media A, and audio is handled as the stream media B.

  FIG. 22 is an explanatory diagram of an example of terminal capability information. FIG. 23 is an explanatory diagram illustrating an example of a control method. FIG. 24 is an explanatory diagram illustrating an example of a control content determination rule. FIG. 22 shows how much terminal load each communication medium gives in the case of a specified terminal capability. FIG. 23 shows how much the load on the terminal is reduced when the quality is lowered based on the communication at the normal quality for both audio and video. In the determination rule of FIG. 24, in order to handle two different resource amounts on the same scale, the combined effect is calculated based on the ratio to the reduction target.

  FIG. 25 is an explanatory diagram illustrating a connection example of the relay device and the terminal device in the quality control system. The quality control system shown in FIG. 25 includes a relay device and terminal devices A, B, and C. Hereinafter, it is assumed that communication is being performed between the terminal apparatuses A, B, and C in the configuration shown in FIG.

  With reference to FIG. 17, the operation of the quality control system in the present embodiment will be described. During communication with video and audio, an 8 Mbyte file is transferred from the terminal device B to the terminal devices A and C using a file transfer tool. When the relay device receives the file data transmitted from the terminal device C (step S101), the following calculation is performed from the data amount of 8 Mbytes in the file transfer (step S201).

  First, in accordance with the terminal capability information illustrated in FIG. 22, the load exerted on the terminal by the file transfer data is calculated. As a result, it is calculated that 8% is consumed in the CPU usage rate and 8 Mbps is additionally consumed in the NW band (step S601).

  In order to keep the CPU usage rate and NW bandwidth usage of the terminal apparatuses A and C constant, the quality control method is subsequently selected from the control method illustrated in FIG. 23 and the control content determination rule illustrated in FIG.

  If each control method is evaluated according to the control content determination rule illustrated in FIG. 24, a result as shown in FIG. 26 is obtained, and a quality control method for reducing the resolution of the video is selected (step S602). FIG. 26 is an explanatory diagram illustrating an example of a result of evaluating a control method.

  Since quality control is not currently performed (No in step S603), when the process of reducing the resolution of the selected video and reducing the data amount is started (step S103), the file transfer data is also transmitted to the terminal device A, Is transmitted to C (step S104).

  If the communication of only audio and video continues for a certain period of time after the file transfer is completed (step S206), it is determined that the video quality control is no longer necessary (step S105), the quality control ends, and the original video quality is relayed. (Step S106).

  The present invention can be applied to quality control of a rich media communication system that combines stream-type media and burst-type media.

It is a block diagram which shows the structural example of the communication system with which the quality control method is applied. It is a block diagram which shows the structural example of a relay apparatus and a terminal device. It is explanatory drawing which shows the flow of data when a terminal device transmits data to a terminal device. It is explanatory drawing which shows the example of a connection in case a some terminal device is connected to a relay apparatus. It is a flowchart which shows operation | movement of the communication system of 1st Embodiment. It is a block diagram which shows the structural example of the relay apparatus in 2nd Embodiment. It is a flowchart which shows operation | movement of the communication system of 2nd Embodiment. It is a block diagram which shows the structural example of the relay apparatus in 3rd Embodiment. It is a block diagram which shows the structural example of the terminal device in 3rd Embodiment. It is a flowchart which shows the operation | movement of the information transmission at the time of 3rd Embodiment. It is a flowchart which shows the operation | movement of the quality control method selection in 3rd Embodiment. It is a block diagram which shows the structural example of the relay apparatus in 4th Embodiment. It is a block diagram which shows the structural example of the terminal device in 4th Embodiment. It is a flowchart which shows the operation | movement of the quality control accompanying the terminal state fluctuation | variation in 4th Embodiment. It is a block diagram which shows the structural example in the relay apparatus of 5th Embodiment. It is a block diagram which shows the structural example in the terminal device of 5th Embodiment. It is a flowchart which shows operation | movement of the communication system of 5th Embodiment. It is a block diagram which shows the structural example of the relay apparatus in 6th Embodiment. It is a block diagram which shows the structural example of the terminal device in 6th Embodiment. It is explanatory drawing which shows the example of a connection of a terminal device. It is a flowchart which shows operation | movement of the communication system of 6th Embodiment. It is explanatory drawing which shows the example of terminal capability information. It is explanatory drawing which shows the example of a control method. It is explanatory drawing which shows the example of the control content determination rule. It is explanatory drawing which shows the example of a connection of the relay apparatus with which the communication system is provided, and a terminal device. It is explanatory drawing which shows the example of the result of having evaluated the control method. It is a block diagram which shows the structural example of the terminal device which implement | achieves rich media communication. FIG. 10 is a block diagram illustrating a configuration example of a communication system described in Patent Literature 1.

Explanation of symbols

1, 3, 4, 6, 8, 9, 10 Relay device 2, 2 ', 5, 7, 1A Terminal device 11, 21, 31, 41, 51, 61, 71, 81, 91, 101, 1A1 Data processing Device 12, 22, 32, 42, 52, 82, 102, 1A2 Storage device 111, 216, 311, 311x, 311y, 216x, 216y Burst type media receiving means 112, 214, 112x, 112y, 214x, 214y Burst type media Transmission means 113, 215, 313, 313a, 313b, 215a, 215b Stream type media reception means 114, 314, 314a, 314b, 1A18 Stream type media quality control means 115, 213, 115a, 115b, 213a, 213b Stream type media transmission Means 121, 1A22 Control method storage unit 1 2,221 Communication information management unit 211, 211a, 211b Stream type media control unit 212, 212x, 212y, 1A12 Burst type media control unit 316 Terminal state estimation unit 323, 423 Terminal capability information storage unit 324 State estimation rule storage unit 417, 617 Control information receiving unit 517 Control information transmitting / receiving unit 522 Terminal information storage unit 718 Terminal state measurement unit 818 Quality control content determination unit 824 Control content determination rule storage unit

Claims (15)

In a rich media communication quality control system that combines a stream-type medium that constantly transmits and receives a predetermined amount of data and a burst-type medium that transmits and receives data in bursts,
Rich media communication is performed by a plurality of terminal devices via a relay device,
The relay device is
Burst type media receiving means for notifying completion of data reception and transmission of burst type media;
Stream type media quality control means for controlling the quality of the stream type media according to the notification;
Including terminal state estimation means for estimating the load of the terminal device at the time of receiving the data according to the received data amount of the burst type media and the state of the stream type media, and determining the data reduction amount of the stream type media from the result Feature quality control system. Relay device, the quality control system as recited in claim 1, including a control method storage unit to provide a control method of the stream-based media to stream-based media quality control means. The relay device
A terminal capability information storage unit that provides terminal capability information to the terminal state estimation means;
The quality control system according to claim 1 , further comprising a state estimation rule storage unit that provides a state estimation rule. In a rich media communication quality control system that combines a stream-type medium that constantly transmits and receives a predetermined amount of data and a burst-type medium that transmits and receives data in bursts,
Rich media communication is performed by a plurality of terminal devices via a relay device,
The relay device is
Burst type media receiving means for notifying completion of data reception and transmission of burst type media;
A stream type media quality control means for controlling the quality of the stream type media in accordance with the capability of each connected terminal device in accordance with the notification.
A quality control system characterized by that . The terminal device includes control information transmitting / receiving means for notifying the relay device of terminal capability information,
The relay device is
Control information receiving means for receiving capability information of the terminal device from the terminal device;
The quality control system according to claim 4 , further comprising: a terminal capability information storage unit that provides the received terminal information to the stream type media quality control means. The terminal device
Terminal state measuring means for measuring the resource state of the terminal device;
Control information transmitting / receiving means for notifying the measured resource state of the terminal device,
The quality control system according to claim 4 , wherein the relay device includes control information receiving means for receiving the resource status of the terminal device from the terminal device and notifying the stream type media quality control means. Relay device, the quality control system of claim 1, including a plurality of select media as a quality control target from the stream type of the media, together quality control content determining means for determining a quality control method for the media. The relay device
A terminal capability information storage unit that provides terminal capability information to the quality control content determination means;
The quality control system according to claim 7 , further comprising: a control content determination rule storage unit that provides a control content determination rule. In a quality control method of rich media communication that combines a stream type medium in which a predetermined amount of data is constantly transmitted and received and a burst type medium in which various amounts of data are intermittently transmitted and received,
Rich media communication repeater starts quality control of stream type media triggered by reception of burst type media data, and ends quality control of stream type media after completion of transmission of burst type media data ,
The relay device estimates the resource state of the terminal device at the time of receiving the data according to the data amount of the burst type media and the format and data amount of the stream type media, and the quality control content of the stream type media is determined from the estimation result. A quality control method characterized by determining . In a quality control method of rich media communication that combines a stream type medium in which a predetermined amount of data is constantly transmitted and received and a burst type medium in which various amounts of data are intermittently transmitted and received,
Rich media communication repeater starts quality control of stream type media triggered by reception of burst type media data, and ends quality control of stream type media after completion of transmission of burst type media data ,
The relay device determines the quality control content of the stream type media according to the individual capabilities and attribute values notified from the connected terminal device.
A quality control method characterized by that . The terminal device measures the resource state of the terminal device, and notifies the relay device when there is a certain variation, and the relay device changes the quality control content of the stream type media according to the variation content. quality control method according to claim 1 0, wherein. When the measurement result of the resource state exceeds or falls below a predetermined threshold value for a certain period of time, or when there is a fluctuation that exceeds a predetermined difference value by comparing a plurality of measured values quality control method according to claim 1 1, wherein it is determined that a large variation. The quality control method according to any one of claims 9 to 12, wherein a condition of a resource state of a terminal device to be maintained is given by a participant during communication. The quality control method according to claim 9 , wherein the relay device selects a medium as a quality control target from a plurality of stream-type media. When the relay device determines the quality control target media and the quality control method for the media, the relay device evaluates the quality control results for all the quality control methods of a plurality of media, and based on the results, The quality control method according to claim 14, wherein the control method is determined.

Images