JPH09214936A - Network moving image distribution system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a moving image distribution system with which moving image distribution service can be operated without being interrupted adaptively for conditions in the environment to be affected by any external/internal factor. SOLUTION: A client 2 performs the centralized grasp concerning the conditions of a server 1 and a network 3 by watching the arrival conditions of data through a transmission processing delay detection part 25 and concerning the conditions of the client 2 by monitoring the progress of decoding through a decode processing delay detection part 24. The client 2 successively requests moving image data in a size required for decoding to the server 1, and the server 1 dividedly transfers big data as well. When delaying the decode processing of moving image data corresponding to the conditions of a system, delay up to a specified value is allowed. When the delay is increased over the specified value, the client 2 clears the non-processed moving image data received from the server 1 and stored in a buffer, requests new image data from delay cancel enabled time to the server 1 and starts decoding so that delay can be canceled.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a network moving image distribution system capable of distributing high quality moving images in a system composed of distributedly managed servers, clients and networks.

[0002]

2. Description of the Related Art Local area networks (hereinafter LA
The server / client type database system has become widespread due to the popularization of N), the improvement of computer performance, the large capacity and low price of disk devices. In this database system, a database is built on the server side, and the client side inquires of the server for information as necessary.

In recent database systems, not only text-based information but also data such as voice information and moving image information must be handled. In particular, moving image data is compressed and encoded in advance and stored, but the size of the data is larger than other media, so there is no way to transfer all the data to the client once and start playback, such as file transfer. Not very realistic. For this reason, a method is generally used in which data necessary for the client to decode and reproduce the video in real time is continuously supplied from the server. The server side controls the data supply rate, and the client only needs to receive the data and play it back.

[0004]

However, in order to provide a stable moving image distribution service by the distribution service method in the conventional database system that distributes moving images, a stable data supply to a plurality of clients is guaranteed. A high-performance server, a network with a guaranteed transmission bandwidth that does not affect other traffic when transferring data, and a system configuration with a client that has a uniform ability to reproduce supplied data without omission Is required. It is assumed that these are realized in an ideal system environment without considering other external factors. However, from the viewpoint of efficiency and economical efficiency, it is desired to realize the moving image distribution service in the existing environment.

The existing environment is defined for each of a server, a network, and a client. The server is assumed to be a general-purpose computer connected to an existing network and can be used for purposes other than the moving image distribution service. The network is also used for purposes other than moving image distribution, and bandwidth is not guaranteed during moving image transfer. In addition, in an environment in which multiple networks are interconnected, a situation is assumed in which a route from a server to a client passes through multiple networks. On the client side,
In an existing PC (personal computer) / WS (workstation), etc., from a client having dedicated hardware for decoding moving image coded data, resources such as CPU and memory sometimes change depending on internal and external processing conditions. Assume even sensitive software-based clients.

[0006] The present invention realizes a network moving image distribution system which is operated in the environment affected by external and internal factors as described above without adjusting the moving image distribution service while adapting to the situation. With the goal.

[0007]

SUMMARY OF THE INVENTION The present invention is a network moving image distribution system for distributing compression-encoded moving image data accumulated on a server side to a client in real time via a network. Has means for selecting a program of moving image data designated by a request from the client and sending it to the client, and the client has means for requesting a program of desired moving image data to the server. In the network moving image distribution system, means for dividing the moving image data into data packets when sending the moving image data, adding the time index of the data packet in the data packet, and sending the means, and notifying from the client If the time index of the received data packet is judged and the condition of the judgment is met, it is sent. Means for selecting a data packet to be transmitted and transmitting it to the client, and for the client side, means for receiving the data packet from the server and calculating the transmission delay from the time index included in the data packet. A means for calculating a processing delay until the data packet received from the server is processed inside the client, and an optimum data packet time index for delay adjustment by judging the calculated transmission delay and processing delay. A network moving image distribution system comprising: means for calculating and notifying the server is means for achieving the above object.

In the above network moving image distribution system, at the server side, the means for adding the time index of the data packet to the data packet and transmitting the data packet, when transmitting the data packet, includes the time index in the data packet. An identifier indicating whether or not the intra-frame coded data is included is sent, and the time index of the data packet notified from the client is judged, and if the condition of the judgment is met, the data packet to be transmitted is selected. The means for selecting and sending to the client judges the identifier notified from the client together with the time index, and when the condition of the judgment is met, selects the data packet to be sent and sends it to the client. Calculate the time index on the client side The means for notifying the server sends the identifier for requesting the intra-frame encoded data together with the time index. The insertion of the data encoded in the intra-frame encoding mode causes a disorder in the decoded moving image. It is preferable in that it eliminates this and speeds up the response.

Further, in the above network moving image distribution system, the server side has means for continuously transmitting to the client a plurality of packets designated by the request from the client in response to the request from the client. , On the client side, means for calculating and predicting at the time of decoding the number of data packets in which the moving image data of the video frame encoded by the intra-frame coding is divided and arranged, and transmitting a plurality of data packets to the server And a transmission delay, a processing delay, and a decoder for moving image data, which identifies the decoding mode of intraframe-coded moving image data or interframe-coded moving image data to stop the decoder and adjust the delay. A means for making a determination is provided for the case where the transmission band of the network is low and the transmission delay is large. It is preferred in order to eliminate the processing delay.

The present invention has means for detecting the status of each system, and executes the processing while adapting the processing according to the status of the server, the network, and the client. In most of the existing systems, the transfer rate of moving image data is controlled by the server side, but in the present invention, the control of the transfer rate and the detection of the situation are performed by the client. The status of the server and the network is grasped by the client in a concentrated manner by observing the arrival status of the data, and the status of the client is monitored by monitoring the progress of decoding. The client sequentially requests the server for moving image data of a size necessary for decoding, and the server also divides large data into packets and transfers them. Regarding decoding of video data, the processing may be delayed depending on each situation of the system, but in order to eliminate the delay, for example, a delay up to the default value is allowed, and the delay exceeds the default value. If the number increases, the client clears the moving image data that has been received from the server but has not been processed and is stored in the temporary buffer, and requests the server for new moving image data from the time when the delay can be resolved and starts decoding. . Here, when the method of performing inter-frame coding is used as the moving picture coding method, by inserting the data coded in the intra-frame coding mode periodically, the decoded moving image is disturbed. It is possible to eliminate it and make the response faster. As described above, by detecting the status of each system and executing the processing while adapting the processing according to the status, a system consisting of distributedly managed servers, networks, and clients, which is currently widely used, Realize a good video distribution service.

[0011]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Next, embodiments of the present invention will be described in detail with reference to the drawings.

FIG. 1 is a system configuration diagram illustrating a first embodiment of the present invention. In the present embodiment example, ISD
H.V. used in videophones, etc. An example of a system using the H.261 image coding system is shown. This system is realized by a server 1 and a plurality of clients 2 on a network 3 such as the Internet constructed as an aggregate of LANs as shown in FIG. The server 1 and the client 2 are interconnected via the network 3 to which they are connected.

FIG. 2 shows a data transmission mode in this embodiment. H. The H.261 coding method is a coding method using inter-frame differences. In the moving image data distributed to the client 2, as shown in FIG. 2A, an instruction is given to the encoder at the time of encoding, and video frames encoded in the intraframe encoding mode are periodically inserted. Hereinafter, a video frame coded in this intraframe coding mode is referred to as refresh processing data. Reference numeral 4 in FIG. 2A indicates the beginning of the refresh processing data periodically inserted in the moving image data. H. The coded data according to 261 is
Since it is represented as a bit string, in order to facilitate the transfer process of data on the network 3, the bit stream of data is divided into packets 5 of an appropriate size to be transferred on the network as shown in FIG. As shown in FIG. 2C, the header 7 is added to the packet 6 and the server 1 holds the aggregate of the packet data 6 as a single moving image data.

In the case of the storage service, an aggregate of the packet data 6 is held as a file, and in the case of real-time delivery, the packet data 6 in the range traced back from the latest data by a preset time. Holds a collection of.

The header 7 of each packet data is represented by the decoding process start time when the bitstream included in the packet is normally decoded without being affected by delay or the like until immediately before. A time index 71 to be displayed and an identifier 72 indicating whether or not intraframe coded data is included in the packet are included.

FIG. 3 is a block diagram showing the configurations of the server and the client in this embodiment. On the server 1 side,
It has a moving image data accumulation processing unit 11, a refresh processing data detection unit 12, and a server side packet transmission processing unit 13. On the other hand, the client 2 side has a moving image display processing unit 21, a decoding processing unit 22, a packet request and reception processing unit 2.
3, a decoding process delay detection unit 24, and a transmission process delay detection unit 25. The server-side packet transmission processing unit 13 on the server 1 side and the packet request / reception processing unit 23 on the client 2 side are connected to the network 3.

The operation of this embodiment will be described based on the configuration shown in FIG.

First, the packet request / reception processing unit 23 causes the server-side packet transmission processing unit 1 via the network 3.
3 is connected to specify the content name of the moving image data to be requested first. Next, the packet request / reception processing unit 23 on the client 2 side requests the server-side packet transmission processing unit 13 to transfer data in packet units from the refresh processing data that appears first, and starts decoding the data. The packet received by the packet request / reception processing unit 23 is
The data is stored in the internal reception buffer, and the decoding processing unit 22 sequentially reads and processes it. Further, on the client 2 side, at the same time as the first packet of the moving image data sent by the server side packet sending processing unit 13 is received, the decoding processing delay detecting unit 24 and the transmission processing delay detecting unit 25 transmit and decode processing elapsed time Start measuring.

At the time of receiving a new packet, the transmission processing delay detector 25 measures the processing elapsed time from the first arrival time of the first packet at the start of transfer. The processing elapsed time of the packet measured by the transmission processing delay detection unit 25 is compared with the time index 71 of the data stream in the packet recorded in the arrival packet.

The decoding process delay detection unit 24 starts the transfer from the reception buffer of the packet request and reception processing unit 23 when the decoding process of the video data included in the packet received from the packet request and reception processing unit 23 is completed. The processing elapsed time from the time when the first first packet of is read is measured. The processing elapsed time measured by the decoding processing delay detection unit 24 is compared with the time index 71 of the data stream in the next packet read from the packet request and reception processing unit 23.

The processing delay detection performed by the decoding processing delay detection unit 24 and the transmission processing delay detection unit 25 is performed by the same procedure as described below. FIG. 5 shows a conceptual diagram thereof.

When the processing elapsed time is smaller than the time index 71, the processing of the decoding processing unit 22 and the packet request / reception processing unit 23 is suspended until the processing start time. As shown in FIG. 5A, this is the case where the transmission and decoding processing time 111 per packet is less than the processing time 110 per packet.

As shown in FIG. 5B, the transmission and decoding processing time 112 per packet is the processing time 110.
When the processing elapsed time is longer than the processing start time by exceeding the above, two operations are performed depending on the size of the difference 113.

When the processing delay allowable value is set on the client side and the difference 113 is smaller than the allowable value, the decoding processing unit 22 and the packet request / reception processing unit 2 are used as they are.
The process of 3 is continued.

When the difference 113 is larger than the allowable value, the processing of the decoding processing unit 22 is interrupted, the packet which has already arrived at the packet processing and reception processing unit 23 is cleared, and then the server side packet transmission processing unit. 12, requesting refresh processing data appearing after the next packet waiting to be transmitted. At the time of requesting refresh processing data, the value of the difference 113 calculated by the decoding processing delay detecting unit 24 and the transmission processing delay detecting unit 25 is also notified to the server 1. On the server 1 side, in response to a refresh processing data request from the client 2, the refresh processing data detection unit 1
Time index 7 of the next packet to be transmitted in 3
A packet including refresh processing data that appears at a time later than the difference value sent from 1 is detected, and the packet is transmitted. When the client 2 side receives the refresh processing data in response to the request, the packet request / reception processing unit 23 continues to request continuous packets, and the decoding processing unit 22 continues until the start time recorded in the packet. The delay is eliminated by interrupting the decoding process, and the decoding process is restarted when the start time comes.

Next, a second embodiment of the present invention will be described.

This embodiment is an embodiment which is effective when the network between the server and the client is significantly lower than the transmission band for requesting the moving image data or when the transmission delay is large via a plurality of networks. . FIG. 4 shows a conceptual diagram of the transmission process in this embodiment.

In this system, refresh processing data is used as means for eliminating processing delay. H. 26
In the moving image data using the encoding method of No. 1, the refresh processing data itself has a larger size than other data, and compared with the maximum size of the packet 104 transmitted on the network as shown in FIG. However, there are cases in which the packet becomes large and the packet 105 extends over a plurality of packets. In order to eliminate the processing delay, all the refresh processing data must be received, decoded and displayed. However, when a plurality of packets are required, one request packet 101 is created as shown in FIG. On the other hand, transferring one data packet 102 is inefficient. In particular, when the transmission band is low or the transmission delay is large, the processing delay included in the transmission time 103 of the request packet cannot be ignored. Therefore, the client uses means for requesting a plurality of packets with one request to the server.

First, on the client side, when connecting to the server and receiving the first refresh processing data, a request is made in packet units. At this time, the time until the packet arrives for one request is measured, and it is confirmed whether the transmission time per packet is longer than the processing time of the moving image data included in one packet. Also, check how many packets the refresh processing data spanned.

When the transmission time of one packet is sufficiently smaller than the processing time of one packet, it is determined that the transmission band on the network is secured, and in order to distribute the traffic, a plurality of packet requests are required. Without, Figure 4
The request packet 106 and the data packet 107 in (b) are transmitted every time.

In a situation where the transmission time of one packet exceeds the processing time of one packet, when newly requesting refresh processing data for resynchronization, it is confirmed at the first refresh processing data request. Based on the number of packets, a request for a plurality of packets is set to 10 in FIG.
Do as in 8. In this case, the server continuously transmits the number of packets designated by one instruction as indicated by 109. The client side accumulates the moving image packets that are continuously sent in the reception buffer. The accumulated moving image packets in the reception buffer are sent to the decoding process from the one that arrives.

If the transmission delay cannot be secured, the refresh processing data may not be processed within the range of the processing delay allowable value set in the first embodiment. Therefore, even if the refresh processing data exceeds the allowable value, the refresh processing data may not be processed. If the process of is not completed, it has a function to suppress resynchronization. When the refresh processing data is processed in excess of the allowable value, resynchronization is performed again when the processing is completed. For other processing, the procedure of the first embodiment is followed.

[0033]

As described above, according to the present invention,
When distributing a moving image via a network, it operates adaptively according to the load situation occurring at each location of the server, network, and client that are distributed and managed, and the best moving image reproduction within the range of the capability can be performed. In the present invention, since the control is on the client side, the load on the server side can be reduced, the upper limit of the number of clients that can be accessed simultaneously can be increased, and unnecessary packet transmission on the network can also be suppressed. This is particularly effective when the client side uses software for decoding and on a network in which the transmission band required for distribution of moving image data is not guaranteed.

[Brief description of drawings]

FIG. 1 is a system configuration diagram showing an embodiment of the present invention.

2A, 2B, and 2C are views for explaining a data packet transmission mode in the above-described embodiment.

FIG. 3 is a block configuration diagram of a server and a client in the above embodiment example.

FIG. 4A is a diagram explaining a transmission process when one data packet is transferred for one request packet;
(B) is a conceptual diagram of a transmission process when a request packet and a data packet are alternately sent, and (c) is a conceptual diagram of a transmission process when a plurality of packets are collectively sent after the request packet

5A is a conceptual diagram of transmission processing when the transmission and decoding processing time per packet is shorter than the processing time per packet, and FIG. 5B is the transmission and decoding processing time per packet per packet Conceptual diagram of transmission processing when it is longer than processing time

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 ... Server 11 ... Moving image data storage processing unit 12 ... Refresh processing data detection unit 13 ... Server side packet transmission processing unit 2 ... Client 21 ... Moving image display processing unit 22 ... Decoding processing unit 23 ... Packet request and reception processing unit 24 ... decoding processing delay detecting section 25 ... transmission processing delay detecting section 3 ... network 4 ... start of refresh data 5,6 ... packet 7 ... header 71 ... time index 72 ... identifier

Claims (3)

[Claims] 1. A network moving image distribution system for distributing compression-encoded moving image data accumulated on a server side to a client in real time via a network, wherein the server side is requested by the client. In the network moving image distribution system, there is provided a means for selecting a program of specified moving image data and transmitting it to the client, and a client side having a means for requesting a program of desired moving image data to the server. On the side, when transmitting the moving image data, the means for dividing the moving image data into data packets, adding the time index of the data packet in the data packet and transmitting the data packet, and the time index of the data packet notified from the client Data packet to be sent when it is judged and the conditions of the judgment are met Means for selecting and transmitting to the client, the client side receiving means for receiving the data packet from the server and calculating the transmission delay from the time index included in the data packet, and receiving from the server Means for calculating a processing delay until the processed data packet is processed inside the client, determining the calculated transmission delay and processing delay, calculating an optimum data packet time index for delay adjustment, A means for notifying, and a network moving image distribution system comprising: 2. On the server side, when transmitting a data packet, the means for adding the time index of the data packet to the data packet and transmitting the data packet includes the intraframe coded data in the data packet together with the time index. The data packet to be transmitted is selected and sent to the client when the time index of the data packet notified from the client is judged and the condition of the judgment is met. The means determines the identifier notified from the client together with the time index, selects the data packet to be transmitted and sends it to the client when the condition of the determination is met, and calculates the time index on the client side. The means for notifying the server is
The network moving image distribution system according to claim 1, wherein an identifier for requesting intra-frame encoded data is transmitted together with the time index. 3. The server side has means for continuously transmitting a plurality of packets designated by a request from the client to the client in response to a request from the client, and the client side has an intraframe code. A method for calculating and predicting the number of data packets in which the moving image data of the video frame encoded by encoding is divided and arranged, a means for requesting the server to transmit a plurality of data packets, a transmission delay, A decoder for processing delay and moving image data, a means for identifying a decoding mode of intraframe-coded moving image data or interframe-coded moving image data, and determining whether to stop the decoder or make a delay adjustment. The network moving image distribution system according to claim 1 or 2.