JP2005101873A - Stream control method, communication equipment capable of using the same and terminal - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve the problem that it is difficult to fully guarantee service quality in stream control using quality information on a network layer. <P>SOLUTION: The first communication part 61 of an intermediate node 20 receives stream data from a server via a first network 41. A second communication part 62 receives reproduction output situation information on stream data from a terminal via a second network 42. A packet processor 64 performs protocol processing on transmission/reception packets and controls input/output of the packet to a buffer 60. A control unit 68 gives quality control instructions, based on a reproduction output situation in the terminal to a transcoder 66. The transcoder 66 transcodes the stream data and changes transmission quality. The packet processor 64 transfers the packet of stream data which is trancsoded via the second network 42. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a stream control technique, and more particularly to a stream control method for controlling transmission quality of a continuous media stream, and a communication apparatus and a terminal that can use the method.

  Broadband networks have spread to general users, and continuous media services that make heavy use of network resources, such as real-time communication by peer-to-peer using images and audio, and streaming distribution using video, have become popular. As a result, users are becoming more aware of the quality of network services, and users are becoming more sensitive to quality degradation. Quality of service (QoS) guarantees are also available for the Internet, which has provided best-effort services. Is strongly demanded. Also, the use of wireless communication is spreading, and QoS control is also required for wireless LAN access points.

In this situation, intermediate nodes in the network such as routers, switches, and wireless access points have not only relatively simple packet transfer control in the network layer, but also some sort of intelligence that considers the application layer. It has come to be required. For example, Patent Document 1 discloses a data transmission apparatus that estimates the transmission state of a transmission path and changes and controls the bit rate or error resilience level of transmission data.
JP 2002-204278 A

  In order to maintain the quality of service at the user level in continuous media data transmission such as voice and video in an IP (Internet Protocol) network, it is necessary for the receiving terminal to respond to the congestion state of the receiving network and the processing performance of the receiving terminal. It is necessary to send media data of suitable quality to the terminal. In an IP network, there are RTP (Real-Time Transport Protocol) / RTCP (Real-Time Transport Control Protocol) as a mechanism for notifying the reception state of continuous media data at a receiving terminal. This is a packet delay in the network layer. Information on fluctuation and loss rate is notified, and information on the actual reproduction quality of continuous media in the application layer is not necessarily reflected. With stream control that relies solely on quality information that can be detected at the network layer, it is difficult to sufficiently guarantee user-level QoS.

  The present invention has been made in view of such circumstances, and an object thereof is to provide a stream control technique capable of controlling the transmission quality of a stream in accordance with the reproduction state of stream data at a receiving terminal.

  One embodiment of the present invention relates to a communication device. In accordance with the output status of the stream data in the terminal, the transmission section that transmits the stream data to the terminal, the reception section that receives information on the output status at the time of reproduction of the stream data in the terminal from the terminal And a transcoding unit for transcoding stream data to be transmitted to the terminal.

  Another aspect of the present invention also relates to a communication device. In accordance with the output status of the stream data in the terminal, the transmission section that transmits the stream data to the terminal, the reception section that receives information on the output status at the time of reproduction of the stream data in the terminal from the terminal And a control unit for controlling packet transmission quality of a stream to be transmitted to the terminal.

  Yet another embodiment of the present invention relates to a terminal. The terminal includes: a reception buffer that buffers received stream data and absorbs packet delay fluctuation; an output unit that decodes and reproduces the stream data in the reception buffer; and an output at the time of reproduction of the stream data A control unit that transmits information on the situation to the transmission node of the stream data. The “transmission node” includes not only a server that transmits stream data but also an intermediate network node that transfers stream data.

  Yet another embodiment of the present invention relates to a stream control method. In this method, information regarding an output state at the time of reproduction of stream data in the terminal is received from the terminal, and transmission quality of the stream data transmitted to the terminal is feedback-controlled.

  It should be noted that any combination of the above-described constituent elements and a conversion of the expression of the present invention between a method, an apparatus, a system, a recording medium, a computer program, etc. are also effective as an aspect of the present invention.

  ADVANTAGE OF THE INVENTION According to this invention, the stream transmission control according to the reproduction | regeneration condition of the stream data in a terminal can be performed.

Embodiment 1
FIG. 1 is a configuration diagram of a stream transmission system according to the first embodiment. The server 10 and the intermediate node 20 are connected by a first network 41, and the intermediate node 20 and the terminal 30 are connected by a second network 42. The server 10 provides stream data of continuous media such as voice and video to the terminal 30 and performs stream control described later. The terminal 30 receives stream data from the server 10 via the intermediate node 20. The terminal 30 is, for example, a personal computer, a portable electronic device such as a PDA (Personal Data Assistant), a mobile phone having a data communication function, or the like. The intermediate node 20 is a network node such as a router or a switch, and performs standard packet transfer processing and performs later-described stream control. In this configuration diagram, the stream transmission side is a stream data server, but a real-time media generator such as a video camera may be used.

  FIG. 2 is a diagram for explaining a stream transmission mechanism in the stream transmission system of FIG. In the application layer, the server 10 executes a server application that controls transmission of continuous media data, the intermediate node 20 transcodes a stream transferred from the server 10 as necessary, and the terminal 30 Run the application. In the network layer, the server 10, the intermediate node 20, and the terminal 30 perform packet communication by IP (Internet Protocol) using UDP (User Datagram Protocol) or RTP.

  The stream data 200 generated in the application layer of the server 10 is put on a UDP packet or an RTP packet in the network layer and reaches the network layer of the intermediate node 20. The intermediate node 20 basically transfers the UDP or RTP packet as it is, but may transfer the packet after transcoding the stream data 200 in the application layer as will be described later. The packet thus transferred reaches the network layer of the terminal 30 and is supplied as stream data 200 to the application layer.

  The terminal 30 transmits to the intermediate node 20 the first control data 210 in which information related to the output status at the time of reproduction of the stream data 200 (hereinafter simply referred to as output status information) is stored in the payload of the IP packet.

  The intermediate node 20 grasps the output status of the stream data 200 in the terminal 30 based on the first control data 210 transmitted from the terminal 30, and sets the coding method and coding rate suitable for the processing performance and communication state of the terminal 30. Switching is performed, and the stream data 200 is transcoded and transferred. Further, when the intermediate node 20 receives the first control data 210 from the terminal 30, the intermediate node 20 controls the transmission quality of the stream data 200 in accordance with the output status at the terminal 30 by performing packet traffic control in the network layer. You can also

  Similarly, the terminal 30 transmits the second control data 220 including the output status information to the server 10. The server 10 grasps the output status of the stream data 200 in the terminal 30 based on the second control data 220, switches to an encoding method and encoding rate suitable for the processing performance and communication status of the terminal 30, and converts the stream data 200 into the stream data 200. Encode and send.

  FIG. 3 is a functional configuration diagram of the terminal 30. These configurations can be realized in terms of hardware by a CPU, memory, or other LSI of an arbitrary computer, and in terms of software, they are realized by a program having a stream control function loaded into the memory. So, functional blocks that are realized by their cooperation are drawn. Therefore, those skilled in the art will understand that these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof.

  The communication processing unit 50 has a packet processing function in the network layer, and receives the stream data 200 from the second network 42. The fluctuation absorbing reception buffer 52 is a reception buffer for the stream data 200 in the application layer, and is provided to absorb delay fluctuation of packets. The decoder 54 reads the stream data 200 from the fluctuation absorbing reception buffer 52 and performs a decoding process. When the stream data 200 is video data, the display unit 56 displays the decoded video frame on the display. When the stream data 200 is audio data, it is output to a speaker.

  The control unit 58 monitors the reproduction output status of the stream data 200 in the decoder 54 to obtain output status information, and gives the output status information to the communication processing unit 50. The communication processing unit 50 stores the output status information in the payload of the IP packet, First and second control data 210 and 220 are created and transmitted to the intermediate node 20 and the server 10.

  Output status information includes network quality information such as packet delay, delay fluctuation, loss rate, bit rate, throughput, media frame quality information such as video frame rate, frame output time, frame output time interval fluctuation, A time stamp at the time when the stream data is extracted from the buffer, a reproduction time delay at that time, reproduction availability, information on a missing frame in inter-frame prediction, and the like are included.

  FIG. 4 is a functional configuration diagram of the intermediate node 20 according to the first embodiment. The intermediate node 20 is a communication device such as a router or a switch. These functional blocks can also be realized in various forms by hardware only, software only, or a combination thereof.

  The first communication unit 61 performs packet communication with the first network 41, and the second communication unit 62 performs packet communication with the second network 42. The buffer 60 buffers packets transmitted and received by the first communication unit 61 and the second communication unit 62. The packet processing unit 64 performs protocol processing of transmission / reception packets and controls input / output of packets to / from the buffer 60.

  The control unit 68 receives the first control data 210 transmitted from the terminal 30 from the packet processing unit 64, extracts the output status information of the stream data 200 in the terminal 30 from the first control data 210, and outputs the output status information to the output status information. The original quality control instruction is given to the transcoding unit 66 and the packet processing unit 64.

  The transcoding unit 66 receives the stream data 200 from the packet processing unit 64 and changes the encoding method and the encoding rate of the stream data 200 in accordance with the quality control command given from the control unit 68. The transcoding unit 66 once decodes the stream data 200, changes the encoding parameters, thins the data, or changes the compression encoding algorithm to re-encode the stream data 200. The coding rate and coding method of 200 are changed.

  The transcoded stream data 200 is supplied to the packet processing unit 64 by the transcoding unit 66. The packet processing unit 64 stores the transcoded stream data 200 in the buffer 60, and the second communication unit 62 transfers the stream data 200 stored in the buffer 60 to the terminal 30. The packet processing unit 64 performs bandwidth control such as traffic shaping as necessary in accordance with a quality control command given from the control unit 68.

  FIG. 5 is a functional configuration diagram of the server 10 according to the first embodiment. These functional blocks can also be realized in various forms by hardware only, software only, or a combination thereof.

  The media transmission unit 76 reads the continuous media data requested by the terminal 30 from the media data storage unit 80 and gives it to the trunk coding unit 74. The trunk coding unit 74 generates the stream data 200 by a predetermined encoding method and writes it in the transmission buffer 72. The communication processing unit 70 transmits the stream data 200 stored in the transmission buffer 72 toward the terminal 30 via the first network 41.

  The control unit 78 receives the second control data 220 transmitted from the terminal 30 from the communication processing unit 70, extracts the output status information of the stream data 200 in the terminal 30 from the second control data 220, and outputs the output status information to the output status information. The original quality control instruction is given to the trunk coding unit 74. The trunk coding unit 74 changes the coding method and coding rate according to the quality control command given from the control unit 78, and generates the stream data 200 based on the new coding method or coding rate.

  FIG. 6 is a flowchart for explaining an example of a stream control procedure by the intermediate node 20. Here, the case of a video stream will be described.

  The intermediate node 20 receives the frame rate as output status information from the terminal 30 (S10). When the frame rate in the terminal 30 is smaller than the predetermined threshold of 15 frames / second (Y in S12), the control unit 68 gives an instruction to lower the service quality to the transcoding unit 66, and the transcoding unit 66 Based on the quality control command from the unit 68, the encoding method or the encoding rate is switched so as to lower the image quality and lower the bit rate of the stream data 200 (S14).

  When the frame rate is 15 frames / second or more (N in S12), the control unit 68 gives an instruction to increase the service quality to the transcoding unit 66. The transcoding unit 66 receives the quality control command from the control unit 68. Based on the above, the coding method or coding rate is switched so as to increase the image quality and increase the bit rate of the stream data 200 (S16).

  The stream control by the control unit 78 and the trunk coding unit 74 in the server 10 can also be performed in the same procedure as in FIG.

  According to the stream transmission system of the present embodiment, information on the actual stream reproduction quality at the application layer in the terminal 30 is fed back to the intermediate node 20 and the server 10 to control the transmission quality of the stream transmitted to the terminal 30. Can do. In the network layer, since it is not possible to observe until the data is discarded in the reception buffer in the application layer of the terminal 30, it is impossible to accurately grasp the reproduction quality of the stream. Since the reproduction quality in the layer is grasped, the service quality can be guaranteed at the user level.

Embodiment 2
FIG. 7 is a configuration diagram of the stream transmission system according to the second embodiment. The stream transmission system of the present embodiment is constructed in a network including a wireless LAN. The wireless LAN access point 90 communicates with the terminal 32 wirelessly. Here, a Bluetooth (trademark or registered trademark) standard will be described as an example of the wireless communication method, but the wireless communication method is not limited. The access point 90 is connected to the server 10 via the LAN 81, the stream data 200 transmitted from the server 10 reaches the access point 90 via the LAN 81, and the access point 90 sends the stream data 200 to the terminal 32. Transfer wirelessly.

  FIG. 8 is a functional configuration diagram of the terminal 32. The same components as those of the terminal 30 of the first embodiment shown in FIG. The control unit 58 monitors the reproduction output status of the stream data 200 in the decoder 54 and provides the output status information to the output status data adding unit 59. The output status data adding unit 59 adds this output status information to a reply signal in the wireless two-way communication by Bluetooth, and transmits it to the access point 90 via the communication processing unit 50.

  FIG. 9 is a functional configuration diagram of the access point 90. The communication unit 91 performs packet communication with the LAN 81, and the wireless communication unit 93 performs wireless packet communication with the terminal 32. The buffer 92 buffers communication packets from the communication unit 91 and the wireless communication unit 93. The packet processing unit 94 performs protocol processing of transmission / reception packets and controls input / output of packets to / from the buffer 92.

  The output status data extraction unit 96 receives a return signal from the terminal 32 in wireless bidirectional communication from the packet processing unit 94 and extracts output status information of the stream data 200 in the terminal 32 from the return signal. The control unit 98 gives a quality control command based on the extracted output status information to the packet processing unit 94.

  The stream data 200 transmitted by the server 10 is received by the buffer 92 via the LAN 81, put on a Bluetooth standard packet by the packet processing unit 94, and transmitted to the terminal 32 by the wireless communication unit 93. The packet processing unit 94 selects an ACL (Asynchronous Connection Less) packet type according to the quality control command given from the control unit 98 and controls the transmission quality of the stream data 200.

  FIG. 10 is a diagram for explaining a procedure of wireless communication by Bluetooth between the access point 90 and the terminal 32. The horizontal axis shows time, and the state of wireless two-way communication is illustrated by dividing every slot.

  In wireless bidirectional communication of time division duplex transmission method (TDD, Time Division Duplex), data packets are normally exchanged in both directions, but in this embodiment, since continuous media stream transmission, A data packet is transmitted in one direction, and a packet having only a header is returned as an ACK (Acknowledge) from the data receiving side. This reply packet is originally a bi-directional data packet, but it is not necessary to send data. Therefore, the bit indicating ACK in the header of the packet is set to 1, and the reply packet is sent as a header-only packet. In this embodiment, output status information is added to the data portion of the reply packet having only this header.

  As shown in the figure, packets 100 a to 100 c of stream data 200 (hereinafter simply referred to as data packets 100 a to 100 c) are transmitted to the terminal 32 from the access point 90 on the data transmission side. When receiving the data packet 100a to 100c from the access point 90, the terminal 32 on the data receiving side returns a reply packet 130a to 130c to the access point 90 in the next slot. The bit indicating ACK of the headers 110a to 110c of the reply packets 130a to 130c is set to 1. The output status data adding unit 59 of the terminal 32 stores the output status information in the data units 120a to 120c of the reply packets 130a to 130c. However, the bit length of the output status information is limited so that the bit lengths of the reply packets 130a to 130c fit within one slot.

  Since the reply packets 130a to 130c are originally transmitted by Bluetooth communication as long as they are within one slot, even if the data parts 120a to 120c are added to the reply packets 130a to 130c, the available wireless communication is possible. There is no change in the communication band. Therefore, even if output status information is added to reply packets 130a to 130c (hereinafter referred to as reply packets 130) in the time division bidirectional transmission method and fed back to the access point 90, stream data from the server 10 toward the terminal 32 is sent. There is no pressure on the 200 communication band.

  FIG. 11 is a flowchart for explaining an example of a stream control procedure by the access point 90. The access point 90 receives the reply packet 130 from the terminal 32, and the output status data extraction unit 96 acquires the fluctuation of the frame display interval as output status information from the received reply packet 130 (S20).

  When the fluctuation of the frame display interval is smaller than the predetermined threshold (Y in S22), the control unit 98 gives an instruction to lower the service quality to the packet processing unit 94. The packet processing unit 94 receives the quality control from the control unit 98. Based on the command, the packet is switched to the low-speed packet type in the Bluetooth communication, and the packet of the stream data 200 is wirelessly transmitted to the terminal 32 (S24).

  When the fluctuation of the frame display interval is equal to or greater than the predetermined threshold (N in S22), the control unit 98 gives an instruction to increase the service quality to the packet processing unit 94, and the packet processing unit 94 receives the quality from the control unit 98. Based on the control command, the packet is switched to the high-speed packet type in Bluetooth communication, and the packet of the stream data 200 is wirelessly transmitted to the terminal 32 (S26).

  Also in the stream transmission system according to the second embodiment, the second control data 220 is transmitted from the terminal 32 to the server 10 as in the first embodiment, and the server 10 responds to the output status of the stream data 200 at the terminal 32. Thus, the transmission quality of the stream data 200 may be controlled. Also in the access point 90 of the second embodiment, the transmission quality of the stream data 200 may be controlled by transcoding the stream data 200 as in the intermediate node 20 of the first embodiment.

  The present invention has been described based on the embodiments. The embodiments are exemplifications, and it will be understood by those skilled in the art that various modifications can be made to combinations of the respective constituent elements and processing processes, and such modifications are within the scope of the present invention. .

  As such a modification, the second embodiment has described the configuration in which packet types having different transmission rates are changed in Bluetooth wireless communication as transmission quality control of wireless communication in accordance with the stream output status in the receiving terminal. In wireless LAN communication, the transmission speed defined in the wireless LAN standard may be switched. Further, as a method for controlling transmission quality, a method for controlling error resilience by adjusting parameters of an error correction method in addition to a method for controlling transmission speed may be employed.

  In a network node, there are various variations in a method of executing a combination of stream quality control by transcoding at the application layer and packet quality control by bandwidth control or wireless communication parameter control at the network layer. For example, when stream data has been transferred at a bit rate that exceeds the available bandwidth at the receiving terminal, the bit rate is lowered by transcoding, and there is still no improvement in the output status of the stream data at the receiving terminal. There are methods such as lowering the bit rate by transcoding and controlling the transmission rate of packets in the network layer. As quality control in the application layer, in addition to transcoding, an error correction method such as FEC (forward error correction) may be introduced in the application layer so that error tolerance is performed at the application level.

1 is a configuration diagram of a stream transmission system according to Embodiment 1. FIG. It is a figure explaining the stream transmission mechanism in the stream transmission system of FIG. It is a function block diagram of the terminal of FIG. It is a function block diagram of the intermediate node of FIG. It is a function block diagram of the server of FIG. 5 is a flowchart illustrating a stream control procedure by the intermediate node in FIG. 4. 6 is a configuration diagram of a stream transmission system according to Embodiment 2. FIG. It is a function block diagram of the terminal of FIG. It is a function block diagram of the access point of FIG. FIG. 10 is a diagram illustrating a wireless communication procedure between the access point in FIG. 9 and the terminal in FIG. 8. 10 is a flowchart illustrating a stream control procedure by the access point of FIG. 9.

Explanation of symbols

  10 server, 20 intermediate node, 30, 32 terminal, 50 communication processing unit, 52 fluctuation receiving buffer, 54 decoder, 56 display unit, 58 control unit, 59 output status data adding unit, 60 buffer, 64 packet processing unit, 66 transcoding unit, 68 control unit, 70 communication processing unit, 72 transmission buffer, 74 trunk coding unit, 76 media transmission unit, 78 control unit, 80 media data storage unit, 90 access point, 92 buffer, 93 wireless communication unit, 94 packet processing unit, 96 output status data extraction unit, 98 control unit, 200 stream data, 210 first control data, 220 second control data.

Claims (6)

A transmission unit for transmitting stream data to the terminal;
A receiving unit for receiving, from the terminal, information related to an output status at the time of reproduction of stream data in the terminal;
A communication apparatus comprising: a transcoding unit that transcodes stream data to be transmitted to the terminal in accordance with the output status of stream data in the terminal. A transmission unit for transmitting stream data to the terminal;
A receiving unit for receiving, from the terminal, information related to an output status at the time of reproduction of stream data in the terminal;
And a control unit that controls packet transmission quality of a stream to be transmitted to the terminal in accordance with the output status of stream data in the terminal.   The communication apparatus according to claim 1 or 2, wherein the information on the output status is acquired from a reply packet from the terminal in a time division bidirectional transmission method. A receive buffer that buffers received stream data and absorbs packet delay fluctuations;
An output unit for decoding and reproducing the stream data in the reception buffer;
And a control unit that transmits information related to an output status during reproduction of the stream data to a transmission node of the stream data.   5. The terminal according to claim 4, wherein the control unit stores information related to the output status in a reply packet in a time-division bidirectional transmission method, and returns the information to the transmission node. A stream control method, comprising: receiving from the terminal information related to an output state at the time of reproduction of stream data in the terminal, and performing feedback control on transmission quality of the stream data transmitted to the terminal.

Images