WO2022201569A1

WO2022201569A1 - Control device, control method, and program

Info

Publication number: WO2022201569A1
Application number: PCT/JP2021/026085
Authority: WO
Inventors: 将裕横田; 和久山岸
Original assignee: 日本電信電話株式会社
Priority date: 2021-03-22
Filing date: 2021-07-12
Publication date: 2022-09-29
Also published as: JPWO2022201569A1; JP7448091B2; WO2022201225A1

Abstract

This control device for controlling on-line real-time communication performed by a plurality of terminals comprises: a selection unit that selects, from a bit rate ladder, an upload bit rate where a QoE value becomes largest within the range of a target value or lower, or an upload bit rate where each QoE value of all the terminals becomes smallest within the range of over the target value, with constraint conditions that an upload bit rate of a first terminal among the plurality of terminals be equal to or lower than an estimated value of an uploadable band after the lapse of prescribed time, and an estimated value of a downloadable band of the first terminal be equal to or larger than the total of upload bit rates of the other terminals, on a QoE model which uses bit rates as input and outputs the QoE values; and an instruction unit that transmits, to the first terminal, an encoding instruction at the upload bit rate selected by the selection unit. The control device thus suppresses the data communication amount while maintaining user's body sensory quality.

Description

Control device, control method and program

The present invention relates to a control device, control method and program.

In recent years, the use of online real-time communication services such as web conferencing is increasing as telework is promoted. Technologies such as WebRTC are available as technologies for realizing such services. WebRTC is a technology standardized by the World Wide Web Consortium and Internet Engineering Task Force.

In order to continuously provide end-users with online real-time communication services, it is necessary to improve the user experience quality (QoE (Quality of Experience)) when using the service, as well as the operating costs (NW equipment costs, etc.) should be reduced.

Until now, in WebRTC, as a bit rate control method for increasing QoE, there is a control method using QoS (Quality of Service) information such as bandwidth and packet loss (Non-Patent Document 1, Non-Patent Document 2 ).

Non-Patent Document 1 defines a message for estimating and notifying the available bandwidth for a session. When this message is received, by reducing the transmission bit rate to the notified bandwidth or less, distribution at a bit rate equal to or less than the available bandwidth is realized.

The control method of Non-Patent Document 2 controls the transmission bit rate based on packet loss and delay. Specifically, when a certain number of packet losses or more occur, it is determined that there is congestion, and the transmission rate is adjusted. In addition, the jitter caused by congestion is estimated from the observed jitter, and if the estimated value is above a certain value and continues for a certain time or longer, it is determined that congestion has occurred, and the transmission rate is adjusted. .

In addition, with the spread of mobile lines, online real-time communication is also being used on various lines. Especially in the case of mobile lines, there are many plans that set an upper limit on the amount of data communication, and such users have the intention of suppressing the amount of communication as much as possible, and there are cases where high-quality communication is unnecessary.

In the conventional technology, since the control is based on QoS information, a large amount of data may flow even though the QoE when actually using the service does not improve much (for example, when the throughput is high) , the distribution bit rate is also higher, but the quality may not improve as much as the improvement in the distribution bit rate.) As a result, if the user has a contract with a plan in which the upper limit of the data communication amount is set, the upper limit may be reached and the speed may be restricted, resulting in a decrease in QoE. In addition, an increase in the amount of data communication increases the operating costs of NW equipment and server equipment required by service providers.

In this way, conventional technology cannot select a delivery rate for online real-time communication based on operational costs and QoE.

When communicating with a large number of people, QoE must be calculated based on the video quality delivered by each user.

Also, when communicating with multiple points, bottlenecks may occur at each point, so it is necessary to implement control based on each band.

Furthermore, when implementing such control, there is the issue of how to set the target QoE.

The present invention has been made in view of the above points, and aims to suppress the amount of data communication while maintaining the user's quality of experience.

Therefore, in order to solve the above problem, a control device for controlling online real-time communication by a plurality of terminals is configured so that the upload bit rate of a first terminal among the plurality of terminals is equal to or less than the estimated upload available bandwidth after a predetermined time has elapsed. and that the estimated value of the available download bandwidth of the first terminal is equal to or greater than the sum of the upload bit rates of the other terminals as a constraint condition for the QoE model that outputs the QoE value with the bit rate as input , an upload bit rate that maximizes the QoE value within a range equal to or less than the target value, or an upload bit rate that minimizes the QoE value of all of the plurality of terminals exceeding the target value from the bit rate ladder. and an instruction unit configured to transmit an instruction for encoding at the upload bit rate selected by the selection unit to the first terminal.

It is possible to reduce the amount of data communication while maintaining the user's quality of experience.

1 is a diagram showing a configuration example of an online real-time communication system according to a first embodiment; FIG. 2 is a diagram illustrating an example hardware configuration of a control server 10 according to the first embodiment; FIG. 2 is a diagram illustrating functional configuration examples of a client terminal 20 and a control server 10 according to the first embodiment; FIG. FIG. 4 is a diagram for explaining selection of an upload bit rate; FIG. FIG. 10 is a diagram showing an example of functional configurations of a client terminal 20 and a control server 10 according to a second embodiment; FIG. 4 is a diagram for explaining calculation of the amount of available data for each day; FIG. 4 is a diagram showing an example of a relational expression between a daily average data amount and a daily average QoE;

The first embodiment will be described below based on the drawings. FIG. 1 is a diagram showing a configuration example of an online real-time communication system according to the first embodiment. In FIG. 1, a plurality of client terminals 20 are connected to a distribution server 30 and a control server 10 via a network such as the Internet.

Each of the plurality of client terminals 20 is a communication terminal such as a PC (Personal Computer), a smartphone, or a tablet terminal used by a user who participates in online real-time communication such as a web conference. The client terminal 20 transmits various logs to the control server 10, and encodes and transmits data related to media (video, audio, etc.) (hereinafter referred to as "media data") according to instructions from the control server 10. , receive and play media data from other participants.

The control server 10 is one or more computers that instruct encoding parameters to each client terminal 20 . The encoding parameter is the upload bitrate in one example. Alternatively, the upload bitrate, resolution and frame rate may be encoding parameters dictated by the control server 10 .

The distribution server 30 is one or more computers that distribute media data transmitted (uploaded) from each client terminal 20 to other client terminals 20 .

FIG. 2 is a diagram showing a hardware configuration example of the control server 10 according to the first embodiment. The control server 10 of FIG. 2 has a drive device 100, an auxiliary storage device 102, a memory device 103, a CPU 104, an interface device 105, and the like, which are connected to each other via a bus B, respectively.

A program that implements processing in the control server 10 is provided by a recording medium 101 such as a CD-ROM. When the recording medium 101 storing the program is set in the drive device 100 , the program is installed from the recording medium 101 to the auxiliary storage device 102 via the drive device 100 . However, the program does not necessarily need to be installed from the recording medium 101, and may be downloaded from another computer via the network. The auxiliary storage device 102 stores installed programs, as well as necessary files and data.

The memory device 103 reads and stores the program from the auxiliary storage device 102 when a program activation instruction is received. The CPU 104 executes functions related to the control server 10 according to programs stored in the memory device 103 . The interface device 105 is used as an interface for connecting to a network.

Note that the client terminal 20 and the distribution server 30 may also have the same hardware configuration as in FIG. However, the client terminal 20 has a display device and a speaker for outputting media data, a camera and a microphone for inputting media data, and the like.

FIG. 3 is a diagram showing a functional configuration example of the client terminal 20 and the control server 10 in the first embodiment.

　In FIG. 3, the client terminal 20 has a client log transmission unit 21, an encoding parameter control unit 22, a transmission data encoding unit 23, and a reception data decoding unit 24. Each of these units is realized by processing that one or more programs installed in the client terminal 20 cause the CPU of the client terminal 20 to execute.

The control server 10 has a client log collection unit 11 , a server log collection unit 12 , a target QoE input unit 13 , an encoding parameter selection unit 14 and an encoding parameter instruction unit 15 . These units are implemented by one or more programs installed in the control server 10 that cause the CPU 104 to execute.

A processing procedure executed in the present embodiment will be described with reference to FIG. It should be noted that the following processing is executed during online real-time communication such as a web conference (hereinafter simply referred to as "web conference") unless otherwise specified. Also, the following processing is executed for each of the plurality of client terminals 20 participating in the web conference, but for convenience, the description will focus on one client terminal 20 (hereinafter referred to as "target client terminal 20"). .

The client log transmission unit 21 of the target client terminal 20 transmits a log (hereinafter referred to as "client log") containing information necessary for estimating encoding parameters of media data such as an upload bit rate after the start of the Web conference. Logs are periodically collected from the target client terminal 20 (for example, at intervals of 1 second), and the collected client logs are transmitted to the control server 10 . Specifically, the client log transmission unit 21 transmits information about the quality of each stream (implemented upload bit rate, playback stop information, frame rate, resolution), information about the target client terminal 20 (used terminal, display size), Playback status (display screen size, mute status, display screen (camera screen, screen sharing, etc.), camera screen ON/OFF status), and information on QoS (Quality of Service) (download/upload throughput, RTT (Round Trip Time) , jitter, packet loss rate, etc.) and sends a client log containing this information to the control server 10 . Information about QoS (hereinafter referred to as “QoS information”) may be acquired by either the target client terminal 20 or the control server 10 .

It should be noted that a stream unit refers to a communication unit for audio communication, video communication, and screen sharing communication. If multiple streams (audio, video, screen sharing) are included in one transfer cycle of the client log, the client log includes information on the quality of the multiple streams.

The client log collection unit 11 of the control server 10 receives the client log transmitted from the client log transmission unit 21. The control server 10 basically executes the following process each time a client log is received.

The server log collection unit 12 collects a log (hereinafter referred to as "server log") containing information necessary for estimating encoding parameters (upload bit rate, etc.) each time a client log is received. Specifically, the server log collection unit 12 collects information (download/upload throughput, RTT, jitter, packet loss rate, etc.) related to the quality of service (QoS) for each stream as information necessary for estimating the encoding parameters. ) information. Note that when the QoS information is included in the client log, the server log collection unit 12 does not need to collect the QoS information.

The target QoE input unit 13 receives an input of a QoE (Quality of Experience) target value (hereinafter referred to as "target QoE") from the operator, and inputs the target QoE to the encoding parameter selection unit 14. Note that the target QoE is input in advance as a common value for all Web conferences. However, the target QoE may be changed at any timing.

The encoding parameter selection unit 14 executes processing each time a server log is collected. Based on the QoS information included in the latest client log or server log, the encoding parameter selection unit 14 calculates an estimated value of the available bandwidth from the current time until T2 seconds have passed. Specifically, the encoding parameter selection unit 14 multiplies the maximum upload throughput and download throughput achievable by the target client terminal 20 by a ratio α (0<α<1) specified by the operator. Estimated values of available bandwidth (available upload bandwidth, available download bandwidth) for T2 seconds (hereinafter simply referred to as "available upload bandwidth" and "available download bandwidth"). If the maximum upload throughput and the maximum download throughput that can be achieved by the target client terminal 20 cannot be obtained, and only the transfer data speed can be obtained, and no packet loss occurs, a sufficiently large value is the available bandwidth. should be an estimated value of On the other hand, when a certain number or more of packet losses have occurred, the result of multiplying the transfer data rate by α may be used as the estimated value of the available bandwidth. It is assumed that the packet loss threshold is given in advance. Since the QoE deteriorates significantly due to the occurrence of playback stop, it is necessary that the upload bit rate of the target client terminal 20 is equal to or lower than the available upload bandwidth, as shown in Equation 1 below, based on the available bandwidth information. In this embodiment, it is assumed that three people are holding a web conference (three users A, B, and C are conducting online real-time communication). Note that T2 seconds is a parameter indicating how much time in the future the quality is to be considered, and is set in advance.

In addition, as a condition for not causing playback stop, as shown in Equation 2 below, the available download bandwidth of the target client terminal 20 is equal to the upload bit rate of the client terminals 20 of other members participating in communication using the same service. must be greater than or equal to the sum of

Based on this condition (with Equations 1 and 2 as constraints), the encoding parameter selection unit 14 uses a QoE model that is an extension of Non-Patent Document 3, from the client log until T1 seconds before the current time The acquired bit rate, frame rate, and resolution are input, and from the current time to T2 seconds later, the bit rate, frame rate, and resolution of the bit rate ladder are input to calculate the QoE value, and from the target QoE input unit 13 Select the bit rate, frame rate, and resolution at which the maximum QoE value within the range below the input target QoE is calculated as the coding parameter to be indicated (Equation 3), or the QoE value of all users is the target The bit rate, frame rate, and resolution that minimize the bit rate in the range exceeding QoE are selected as the coding parameters to be indicated (equation 4) (FIG. 4). As a method of setting the target QoE, it may be possible to select such that the former is adopted when the amount of communication is desired to be reduced as much as possible, and the latter is adopted when the control is performed with the user's quality in mind. However, the bitrate ladder may not include frame rate and resolution. In this case, the encoding parameter selection unit 14 may select only the upload bit rate for each user using the most recent values for the resolution and frame rate. At this time, when the number of users is large, the amount of calculation for calculating the QoE is large, and the processing load may become large. In such a case, as a device to reduce the QoE calculation amount, the bit rate is increased by increasing the bit rate or the bit rate of a stream with a low video quality value for a short time. Alternatively, the bit rate may be increased until the target QoE is exceeded by increasing the rate of streams with low video quality values for a short period of time. When the processing load is large, a method of reducing the number of times of QoE calculation and reducing the amount of calculation may be adopted. The selection of the encoding parameter as described above by the encoding parameter selection unit 14 corresponds to the selection of the encoding parameter that brings the QoE after T2 seconds closer to the target QoE. It should be noted that T1 second is a parameter indicating how much time in the past is taken into account, and is set in advance.

When using the QoE model extended from Non-Patent Document 3, the video quality (QoE value) of the target client terminal 20 is determined by the upload bit rate of another user participating in the same communication, and the size of the screen on which each user is displayed. , and the display screen (screen sharing or camera image). A QoE model is described in Expression 5, taking as an example only a case of sharing camera images of three participating users. When the number of users increases or decreases, the terms of each stream are added or deleted. Also, the denominator for calculating the weight of the screen size is changed according to the increase or decrease in the number of users. Equation 5 shows a QoE model that calculates each participating user's QoE based on the other participating users' encoding parameters (upload bitrate, frame rate, and resolution).

　If the camera image is turned off, the delivery rate may be low and the QoE may be underestimated. Therefore, when the camera image is OFF, the stream need not be used for QoE calculation. If the stream is not used for QoE calculation, the existing control shall be applied. In addition, it is assumed that the target QoE is satisfied because the QoE cannot be calculated when the camera image is turned off in all streams.

Also, if screen sharing is implemented, add a screen sharing section. The screen sharing term utilizes the QoE formula for a single stream of screen sharing. The screen size is calculated according to the screen size ratio of the camera image and the entire screen sharing. Note that in screen sharing, the estimated QoE may be smaller than the actual QoE if the motion of the video is smaller than that of the camera and encoded by Variable Bitrate (VBR). Therefore, the same measures as in the case of turning off the camera image may be taken.

Note that the QoE model to be referred to is not limited to Non-Patent Document 3 as long as it is a model that can evaluate QoE with media parameters equivalent to those of Non-Patent Document 3 as inputs.

The encoding parameter instruction unit 15 of the control server 10 transmits an instruction to the target client terminal 20 to execute encoding and transmission of media data using the encoding parameters selected by the encoding parameter selection unit 14. do. When the encoding parameter selection unit 14 selects upload bitrate, resolution and frame rate, the encoding parameters include upload bitrate, resolution and frame rate. When the encoding parameter selection unit 14 selects only the upload bitrate, the encoding parameters include only the upload bitrate.

The encoding parameter control unit 22 of the target client terminal 20 sets the encoding parameter specified in the instruction received from the control server 10 to the transmission data encoding unit 23 .

The transmission data encoding unit 23 encodes the media data based on the set encoding parameters (that is, the encoding parameters instructed by the control server 10), and transmits the encoded media data.

As described above, according to the first embodiment, by performing control according to the target QoE, the amount of data communication is suppressed without greatly reducing the QoE (while maintaining the user's QoE). can do. As a result, a user who has a contract with a plan in which an upper limit is set for the amount of data communication is less likely to reach the speed limit and lower QoE. In addition, it is possible to reduce the cost of the equipment of the service provider.

Next, a second embodiment will be described. 2nd Embodiment demonstrates a different point from 1st Embodiment. Points not specifically mentioned in the second embodiment may be the same as in the first embodiment.

FIG. 5 is a diagram showing a functional configuration example of the client terminal 20 and the control server 10 in the second embodiment. In FIG. 5, the same parts as those in FIG. 3 are denoted by the same reference numerals, and the description thereof will be omitted. 5, the server log collection unit 12 is replaced with a server log collection unit 12a, and the target QoE input unit 13 is replaced with a target QoE calculation unit 16. In FIG.

In the second embodiment, every time a client log is received, the server log collection unit 12a collects the information necessary for estimating the target QoE from the server in addition to the information necessary for estimating the encoding parameters. Collect as logs. The information necessary for estimating the target QoE is, for example, performance information (data usage (data transfer volume), meeting time, number of meeting participants, number of meetings, etc.) for each past web conference (online real-time communication). be. The performance information is stored in the control server 10, for example.

The target QoE calculator 16 estimates the target QoE. Here, it is assumed that a web conference service provider uses a cloud service that provides network equipment (hereinafter simply referred to as "cloud service") to provide the web conference service. Web conferencing service providers have entered into a contract to pay a fee (operation cost) for the amount of data transferred with regard to the use of the cloud service.

In such a situation, the target QoE calculation unit 16 calculates the estimated execution date (estimation of the target QoE The daily average data volume is calculated by aggregating the daily cumulative data volume of the service provider and dividing the cumulative data volume by the number of days in the period.

The target QoE calculation unit 16 calculates the available remaining data volume from the operating cost budget (the fee to be paid for the cloud service) and the cost per band input by the operator. The target QoE calculation unit 16 calculates the daily available data volume (daily available data volume) from the available remaining data volume from the current time and the remaining number of days until the collection closing date (for example, the end of the month) as follows ( Figure 6).
Daily available data volume = Remaining available data volume / (Aggregation closing date - Estimated execution date)
In addition, in FIG. 6, the solid line indicates the amount of accumulated data for each day. A dashed line indicates a cumulative value based on the daily average data amount obtained by dividing the cumulative data by the number of days. The one-dot chain line indicates the daily available data amount.

The target QoE calculation unit 16 calculates the daily average QoE corresponding to the daily available data volume in the relational expression (FIG. 7) between the daily average data volume and the daily average QoE, and calculates the daily target average QoE (target QoE for the day). ). Since the average QoE is a daily value, the target QoE estimation by the target QoE calculator 16 may be performed once a day. That is, the target QoE should be updated each time one day passes. The target QoE calculator 16 passes the calculated target QoE to the encoding parameter selector 14 . Note that the relational expression between the daily average data amount and the daily average QoE is constructed (learned) by using the information because the same information is accumulated in the conference record information.

It should be noted that in each of the above embodiments, the control server 10 is an example of a control device. The encoding parameter selection unit 14 is an example of a selection unit. The encoding parameter instruction section 15 is an example of an instruction section. The target QoE calculator 16 is an example of a calculator. The target client terminal 20 is an example of a first terminal. After T2 seconds is an example after a predetermined time has elapsed.

Although the embodiments of the present invention have been described in detail above, the present invention is not limited to such specific embodiments, and various modifications can be made within the scope of the gist of the present invention described in the claims.・Changes are possible.

This application claims priority from International Patent Application No. PCT/JP2021/011625 filed on March 22, 2021, the entire contents of which are incorporated herein by reference. do.

10 control server 11 client log collection unit 12 server log collection unit 12a server log collection unit 13 target QoE input unit 14 encoding parameter selection unit 15 encoding parameter instruction unit 16 target QoE calculation unit 20 client terminal 21 client log transmission unit 22 code encoding parameter control unit 23 transmission data encoding unit 24 reception data decoding unit 30 distribution server 100 drive device 101 recording medium 102 auxiliary storage device 103 memory device 104 CPU
105 interface device B bus

Claims

A control device for controlling online real-time communication by a plurality of terminals,
An upload bit rate obtained or estimated for a first terminal among the plurality of terminals is equal to or less than an estimated available upload bandwidth after a predetermined time has elapsed, and an estimated available download bandwidth of the first terminal. is equal to or greater than the sum of the upload bit rates of the other terminals, as a constraint condition for the QoE model that outputs the QoE value with the bit rate as input, the upload bit that maximizes the QoE value within the range of the target value or less a selection unit that selects, from a bit rate ladder, a rate or an upload bit rate that is the smallest within a range in which the QoE values of all of the plurality of terminals exceed the target value;
an instruction unit that transmits an instruction for encoding at the upload bit rate selected by the selection unit to the first terminal;
A control device comprising:
Calculate the amount of usable data on a daily basis based on the actual information of the amount of data used in past online real-time communication, and calculate the daily average QoE corresponding to the usable data amount in the relational expression between the daily average data amount and the daily average QoE as the target value,
The control device according to claim 1, characterized by comprising:
The QoE model outputs a QoE value at the first terminal based on an upload bit rate, frame rate and resolution at a terminal different from the first terminal among the plurality of terminals.
3. The control device according to claim 1 or 2, characterized in that:
A control device that controls online real-time communication by multiple terminals,
The upload bit rate of a first terminal among the plurality of terminals is equal to or less than the estimated available upload bandwidth after a predetermined time has elapsed, and the estimated available download bandwidth of the first terminal is the other terminal. The upload bit rate that maximizes the QoE value within the range of the target value or less, or the plurality of a selection procedure for selecting from the bit rate ladder the upload bit rate that minimizes the QoE values of all the terminals in the range exceeding the target value;
an instruction procedure for transmitting an instruction for encoding at the upload bit rate selected by the selection procedure to the first terminal;
A control method characterized by executing
Calculate the amount of usable data on a daily basis based on the actual information of the amount of data used in past online real-time communication, and calculate the daily average QoE corresponding to the usable data amount in the relational expression between the daily average data amount and the daily average QoE A calculation procedure for calculating as the target value,
5. The control method according to claim 4, wherein said control device executes:
The QoE model outputs a QoE value at the first terminal based on an upload bit rate, frame rate and resolution at a terminal different from the first terminal among the plurality of terminals.
6. The control method according to claim 4 or 5, characterized in that:
A program characterized by causing a computer to function as the control device according to any one of claims 1 to 3.