JP2003125374A - Image distribution system, its transmitter, transmitting method, receiver and receiving method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image distribution system capable of distributing smooth images to audiences, without dropping out frames. SOLUTION: An image distribution controller 103 in a transmitter 104 transmits test data stored in a data memory 10 to a receiver 113 over a communication network 107. A receiving condition detector 112 in the receiver 113 detects the display condition such as frame drop out, etc., at reproducing the received test data and forms and sends information about the receiver, including the display condition, the processing power of the CPU, etc., to the transmitter 104. A distributing condition generator 105 generates image distributing conditions ensuring the reproduction in real time at the receiver, without frame drop out, based on the information about the receiver sent from the receiver 113, and presents the conditions to the receiver.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a system for delivering digital data such as video and audio from a transmitter to a receiver in real time for a fee using a network such as the Internet or a wireless network.

[0002]

2. Description of the Related Art RFC-1889 "RTP: A Transpor" is a standard document regarding a data transfer protocol for transmitting a data stream such as audio / video in real time.
t Protocol for Real-time Application ”. In this standard, as a protocol for controlling traffic on the transmission line and guaranteeing real-time transmission to some extent,
There is RTCP (Real-Time Transport Control Protocol). The RTCP periodically exchanges packet transmission / reception statistical information such as packet loss, transmission delay jitter, and round trip time (the reception side periodically feeds back information on the reception status), and transmits based on this information. The amount of data is adaptively controlled.

This protocol is for detecting whether or not the packet data that is actually transmitted can be reliably received, and the system capacity (CPU processing capacity) on the receiving device side when an image is actually displayed. It does not feed back to the display state which depends on the RAM capacity). As packet data, even if it can be received normally on the transmission path,
If the CPU processing capability is low, it is difficult to normally reproduce the image, and the image becomes a frame in which a frame drop (hereinafter, simply referred to as a frame drop) occurs.

Further, in Japanese Unexamined Patent Publication No. 9-46677 (title of the invention: image transmission system), a frame (I,
(P, B) is set, and the charging process is performed with a charge system according to the set frame. In this document, the traffic (congestion state) of the image data network (transmission path) 107 on the transmission side is detected by the traffic detection unit 712 as in the conventional example of FIG. 7, and I
It is intended to change the transfer rate by changing the combination of the frame, the P frame and the B frame to transmit the image.

In the current video distribution system via the Internet, video distributors such as concerts and movies present the contents to be distributed, the transfer rate of the distribution data, and the charging fee on the homepage. The viewer looks at the transfer rate and the charging fee, decides which contract to select if there are a plurality of distribution conditions, and concludes a contract. At this time, if the distribution is not actually started, it is not possible to determine whether the video can be displayed without properly dropping frames.

Depending on the video distribution system and application, data is transmitted until the buffer becomes full, and when the buffer becomes full and packet loss or frame drop is detected during video distribution, all the data in the buffer is The distribution video is temporarily stopped until the end of playback, the buffer is emptied, and then the lost data is buffered and the distribution is restarted.In this case, the video is discontinuous. Become.

[0007]

As described above, the above R
In the TP and the publicly known example, since only the traffic state of the transmission path is detected, there is a problem that when the image is actually reproduced on the receiving side, the transmitting side does not know whether or not the image can be reproduced without dropping frames. there were. The frame drop rate often depends on the system capacity of the receiving device, that is, the processing capacity of the CPU and the memory capacity such as RAM. In addition, there is a demand to view a smooth video without dropping frames even if the screen size is small.

Therefore, the present invention has been made in order to solve the above problem, and it is possible to deliver a smooth video to a receiving device, that is, a viewer without dropping frames and fixing the frame rate. The purpose is to provide a distribution system.

[0009]

In order to achieve the above object, the present invention relates to a transmitting device in a system for delivering video / audio data from a transmitting device to a receiving device using a communication network. A network processing unit that performs a connection process with the receiving device, a data storage unit that stores data including test data to be distributed to the receiving device, and data in the data storage unit via the network processing unit. When a delivery control unit that controls delivery to the receiving device and the test data in the data storage unit is delivered to the receiving device via the delivery control unit,
And a delivery condition creating unit that receives the reception information reported from the receiving device, creates a video delivery condition that allows the receiving device to reproduce in real time without dropping frames, and presents it to the receiving device.

The received information is the CPU of the receiving device.
It includes processing capability and frame drop rate when video data is reproduced on the receiving device side. The video distribution conditions include screen size, transfer rate, compression rate, and the like.

Further, the present invention is a receiving device in a system for delivering video / audio data from a transmitting device to a receiving device using a communication network, the network processing unit performing connection processing with the network and the transmitting device. And receiving the test data transmitted from the transmitting device via the network processing unit, detecting the reception and display state when the test data is reproduced in real time, and receiving the reception information including the reception and display state. A reception state detection unit that creates and sends to the transmission device, and a control for displaying the delivery condition returned from the transmission device as a result of the reception state detection unit sending the reception information to the transmission device. A display control unit to perform and a user interface for the user to select one of the delivery conditions displayed by the display control unit. ; And a scan processing unit.

Before the main part distribution, the display status of the receiving device side is fed back to the transmitting device side by using the test data, so that the real-time moving image distribution according to the processing capacity of the receiving device side becomes possible. In addition, by presenting one or more conditions such as a transfer rate, a screen size, a charge, etc., as a video distribution condition, a real-time moving image distribution without dropping frames can be provided, so that the viewer's options can be expanded.

As the test data, data equivalent to full-spec video data to be distributed is used and reproduced by the receiving device side, so that it is possible to know how much processing capacity the receiving device has. By using the received information as feedback information, it is possible to detect whether full-spec data can be reproduced or how much other parameters (screen size and transfer rate) should be changed while fixing the frame rate. Become.

Further, by providing the distribution condition in which the other parameters are changed without changing the frame rate, it becomes possible to view the smooth video on the receiving device side. Further, by presenting a plurality of distribution conditions together with the charging fee, the selection range of the viewer is expanded.

By including the display screen size in the reception information, the optimum transfer rate can be set on the receiving device side, and unnecessary data need not be transferred. In addition, it is possible to present appropriate selection conditions to the user.

[0016]

BEST MODE FOR CARRYING OUT THE INVENTION The present embodiment will be described below with reference to the drawings.

FIG. 1 is a block diagram showing the configuration of an embodiment of a video distribution system (transmitting device and receiving device) according to the present invention. In this embodiment, as the network 107,
The case of the Internet is shown. FIG. 2 is a flowchart for explaining the operation on the receiving device 113 side, and FIG.
It is a flowchart explaining operation | movement on the 4th side.

First, each component of the video distribution system shown in FIG. 1 will be described.

The control units 101 and 110 are a CPU and a RO for controlling the transmitting device 104 and the receiving device 113, respectively.
A controller composed of M and RAM. The network processing units 106 and 109 connect each device to the communication network 107 such as the Internet and perform TCP / TCP
It controls data transmission protocols such as IP and UDP.

The data storage unit 102 of the transmission device 104 is
It stores homepage data for guiding distribution, test data described below for distribution according to a request from the receiving device side, and a main part of distribution video. Video distribution control unit 103
Controls the transmission of the video data stored in the data storage unit 102 according to the video real-time transmission protocol. As will be described later, the distribution condition creation unit 105 receives the reception information on the side of the receiving device 113 that has received the test data (CPU processing capacity, display capacity, bandwidth of the transmission path, and frame drop rate on the receiving device side when reproducing the test data). Etc.) as feedback information, and conditions (transfer rate, screen size, charge) of distribution data that can be reproduced on the receiving device side without dropping frames are created.

The UI processing unit 108 of the receiving device 113 responds to the contents presented by the viewer from the transmitting device 104.
It is a user interface such as a keyboard and a remote controller for inputting characters and Yes / No. The display control unit 111 performs control for displaying the distribution video from the transmission device 104 and the homepage and displaying the video on a monitor. The reception state detection unit 112 detects the reception information from the test data transmitted from the transmission device 104, as described later.

Next, the operation of each device in FIG. 1 will be described with reference to the flow charts in FIGS.

The viewer obtains information regarding video distribution via the home page on the Internet 107. Using the UI processing unit 108 such as the keyboard of the receiving device 113, the transmitting device 10 indicates that the viewer wants to view (deliver).
4 (S101 in FIG. 2).

When the transmitting device 104 receives the desired viewing data from the viewer (S201 in FIG. 3), the transmitting device 104 transmits the information for the distribution contract to the receiving device 113 on the viewer side, and at the same time, the data storage section 102 stores the information. Compressed video data that is test data is transmitted (S202). At this time, the test video data to be distributed is video compressed data equivalent to the full specifications (maximum frame rate, maximum transfer rate, maximum screen size) of the video that the viewer originally desires to distribute. For example, it may be data obtained by extracting a few seconds of a full-spec distribution video, or any data that is much shorter than the main video, such as an advertisement video.

In the receiving device 113 which has received the test video, the received data is decompressed, and the reproduction display is displayed.
Then, the viewer views the reproduced data on a monitor (not shown) (S102). During reproduction, the reception device 113 creates reception information (CPU processing capacity of the reception device, frame drop rate on the reception side when reproducing video data, bandwidth of the transmission path, etc.) in the reception state detection unit 112, and this information is generated. The data is transmitted to the transmission device 104 (S103). This frame drop rate is an error check such as CRC for received data,
Alternatively, it can be calculated from packet drop detection data or the like.

FIG. 4 shows the monitor image P1 during the reception of the test image by the receiving device 113. As shown in the figure, the monitor image P1 displays the specifications (full-spec data) of the test image currently being received, such as the transfer rate, screen size, and frame rate. Further monitor image P1
Detects and displays frame drop information when a full-spec test video is being played. This frame drop information is the information detected by the reception state detection unit 112 as described above.

When the transmission device 104 receives the reception information of the reception device (S203), the distribution condition creating unit 105 analyzes this information. On the transmitting device 104 side, the frame drop rate (%) and the transfer rate (bps: bit /
Seconds), the display screen size, and the mutual relation of the traffic (transfer rate) of the transmission path, by changing each parameter, and the conditions under which no frame drop occurs are investigated in advance by an experiment.
Be prepared so that it can be calculated in light of such data. FIG. 5 shows the relationship between the frame drop rate and an appropriate transfer rate using the screen size as a parameter.

For example, as shown in FIG.
If the information indicating that there was% is received, the transmitting device 104
Presents an appropriate transfer rate and screen size, and one or more sets of billing charges (viewing charges) corresponding to them, to the receiving device as in the distribution condition screen P2 of FIG. 6 (S204).
The distribution conditions [1], [2], shown on the distribution condition screen P2,
Both [3] indicate conditions under which the receiving device 113 side that has transmitted the above-mentioned reception information can receive, reproduce, and display video data without dropping frames. In this example, the delivery condition [1],
[2] and [3] show distribution conditions when only the transfer rate and the display screen size are changed without changing the frame rate of the video. However, when the reception information includes information about the display screen size of the receiving device, the delivery condition of the screen size or less is presented.

On the side of the receiving device, the viewer confirms the presented distribution conditions on a monitor or the like via the display control means 111 (S104), and the UI processing unit 108 uses the user interface such as a keyboard or a remote controller. Select whether or not you want to subscribe (contract), and if you want to subscribe (contract), select the delivery conditions and the data (contract details)
Is transmitted to the transmitting device 104 (S105).

The transmitting device 104 which has received the above contract contents
Sends confirmation data to the receiving device side, and after confirmation (S20
6) Distribute the main part, which is the video data of the distribution condition (S
207). After receiving a response to the confirmation (S106), the receiving device 113 receives the main part, which is the distribution data of the desired distribution condition, and reproduces / views it (S107).

When the delivery conditions purchased with the above-described charging system are received and traffic is further generated on the transmission path, the delivery is performed while finely adjusting the compression rate by a real-time transmission protocol or the like.

In the embodiment described above, based on the frame drop rate when the test video is reproduced / displayed, FIG.
The transfer rate was calculated from the correlation shown in. But,
A file describing the CPU processing capacity and the display screen size is stored in the receiving device, and the data and the frame drop rate when the test video is previously reproduced / displayed are transmitted as the receiving information of the receiving device. The transmission rate may be sent to the device and the transmission device may calculate the transfer rate from the information. In addition, the CPU processing capacity and R unique to the receiving device of the viewer once distributed
Data related to AM size and display screen size may be stored on the transmitting device side, and when the same viewer requests distribution next time, the stored information may be presented at the time of test video distribution to request confirmation. It is possible.

[0033]

By transmitting the test video to the receiving device before the main part distribution, and feeding back the receiving side system capability such as the traffic state of the transmission path and the CPU processing capability from the reproducing state of the receiving device to the transmitting side, It is possible to perform distribution without dropping frames. Furthermore, by presenting at least one distribution condition that does not drop frames to the viewer, it is possible to expand the viewer's options.

[Brief description of drawings]

FIG. 1 is a diagram showing a configuration of a video distribution system according to an embodiment of the present invention.

FIG. 2 is a flowchart illustrating an embodiment of an operation on the receiving device side shown in FIG.

FIG. 3 is a flowchart illustrating an embodiment of an operation on the side of the transmission device illustrated in FIG.

FIG. 4 is a diagram showing a display example of a test image of the receiving apparatus according to the present invention.

FIG. 5 is a diagram illustrating an example of a transfer rate calculation method by the transmission device of the present invention.

FIG. 6 is a diagram showing a display example of distribution conditions and a charging fee for a receiving device of the present invention.

FIG. 7 is a diagram showing a configuration of a conventional video distribution system.

[Explanation of symbols]

101 transmitter control unit 102 data storage unit 103 Video distribution control unit 104 transmitter 105 Delivery condition creation section 106 network processing unit 107 Internet network 108 UI processing unit 109 Network processing unit 110 Receiver side controller 111 Display control unit 112 Reception state detector

Claims (10)

[Claims] 1. The transmission device in a system for delivering video / audio data from a transmission device to a reception device using a communication network, the network processing unit performing connection processing between the network and the reception device. A data storage unit that stores data including test data to be distributed to the reception device; a distribution control unit that controls distribution of the data in the data storage unit to the reception device via the network processing unit; and the data storage When the test data in the unit is distributed to the receiving device via the distribution control unit, the reception information reported from the receiving device in response to the test data is received, and the receiving device side does not drop the frame in real time. A transmission device comprising: a delivery condition creation unit that creates a replayable video delivery condition and presents it to the receiving device. 2. The transmission device according to claim 1, wherein the test data is video compression data having a video full spec that is to be distributed by the transmission device side. 3. The distribution condition creating unit distributes the test data to the receiving device, and based on the received information reported from the receiving device, a transfer rate without changing a frame rate of a video to be distributed. The transmitting device according to claim 1, wherein at least one distribution condition in which only the display screen size is changed is presented to the receiving device side. 4. The reception information reported from the receiving device includes a display screen size of the receiving device, and the delivery condition creating unit sets the screen size to be equal to or less than the display screen size in the presentation of the delivery condition. The transmission device according to claim 1, wherein the condition is presented. 5. The receiving device in a system for delivering video / audio data from a transmitting device to a receiving device using a communication network, the network processing unit performing connection processing between the network and the transmitting device, The test data transmitted from the transmission device is received via the network processing unit, the reception and display state when the test data is reproduced in real time is detected, and reception information including the reception and display state is created. And a display for performing control for displaying a delivery condition returned from the transmission device as a result of the reception state detection unit transmitting the reception information to the transmission device. A control unit, and one of the delivery conditions displayed by the display control unit
And a user interface processing unit for the user to select one of them. 6. The receiving apparatus according to claim 5, wherein the reception information includes a frame drop rate at the time of reproducing video data on the receiving apparatus side. 7. The reception information includes a CPU of the reception device.
The transmission device according to claim 5, further comprising processing capability. 8. A system for delivering video / audio data from a transmission device to a reception device using a communication network, wherein the transmission device includes a network processing unit that performs connection processing with the network and the reception device, A data storage unit that stores data including test data to be delivered to the receiving device; a delivery control unit that controls delivery of the data in the data storage unit to the receiving device via the network processing unit; and in the data storage unit When the test data of the above is distributed to the receiving device via the distribution control unit, the reception information reported from the receiving device in response to the test data is received, and the receiving device reproduces it in real time without dropping frames. And a delivery condition creating unit that calculates available video delivery conditions and presents them to the receiving device. And a network processing unit that performs connection processing with the transmission device, and a reception and display state when the test data transmitted from the transmission device is received via the network processing unit and the test data is reproduced in real time. Detect
A reception state detection unit that creates reception information including the reception and display states and sends the reception information to the transmission device, and a result that the reception state detection unit sends the reception information to the transmission device and is returned from the transmission device. A display control unit that performs control for displaying a delivery condition, and one of the delivery conditions displayed by the display control unit.
And a user interface processing unit for the user to select one of them. 9. A receiving method for receiving distribution conditions in a system for distributing video / audio data from a transmitting device to a receiving device for a fee using a communication network, wherein the desired viewing data is transmitted to the transmitting device side. A step of receiving and displaying a test image from a transmitting device after transmitting the desired viewing data, and detecting a reception and display state of the test image when the test image is displayed and including the reception and display state. Creating reception information and transmitting it to the transmitting device; after transmitting the reception information, checking whether or not a distribution condition is presented by the transmitting device; and when the distribution condition is transmitted, a distribution contract is signed. And a step of receiving confirmation data from the transmitting device after transmitting the contract content data when the distribution contract is made. And a step of receiving and reproducing the main part which is the video data of the distribution condition. 10. A transmission method for transmitting distribution conditions in a system for distributing video / audio data from a transmission device to a reception device for a fee using a communication network, the step of receiving desired viewing data from the reception device. And, after receiving the desired data, delivering a test video to the receiving device, checking whether or not reception information corresponding to the test video on the receiving device side is received, and when the reception information is received , A step of calculating from the received information the delivery conditions that can be reproduced in real time without dropping frames, further presenting at least one of the charging charges and presenting to the receiving device whether or not to contract, and the receiving device side contracted Check if the contract is successful, and if the contract is successful, send confirmation data to the receiving device. Transmission method characterized by a step of distributing the main is the data.