JP2003179906A - Network moving picture distribution system and client apparatus in the same - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To achieve a moving picture distribution system that can be used without interrupting a moving picture distribution service while adapting to the condition under environment that is affected by external/internal factors. <P>SOLUTION: A client 2 intensively controls the condition of a server 1 and a network 3 by monitoring the arrival condition of data by a transmission processing delay detection section 25 and monitoring the progress of decoding by a decoding processing delay detection section 24. The client 2 successively requests the server 1 to send moving image data with a size required for decoding, and the server 1 divides and sends even large data for transmission. When the decoding processing of the moving image data is delayed according to the system condition, the delay up to a prescribed value is allowed. The client 2 clears unprocessed moving picture data that are received from the server 1 and are accumulated in a buffer when the delay exceeds a prescribed value, and requests the server 1 to send new moving picture data from time when delay can be solved for starting decoding, thus eliminating the delay. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a network moving image distribution system which enables distribution of high quality moving images in a system composed of distributedly managed servers, clients and networks, and a client in this system. It relates to the device.

[0002]

2. Description of the Related Art Local area networks (hereinafter LA
The server / client type database system has become widespread due to the popularization of N), the improvement of computer performance, the large capacity and low price of disk devices. In this database system, a database is built on the server side, and the client side inquires of the server for information as necessary.

In recent database systems, not only text-based information but also data such as voice information and moving image information must be handled. In particular, moving image data is compressed and encoded in advance and stored, but the size of the data is larger than other media, so there is no way to transfer all the data to the client once and start playback, such as file transfer. Not very realistic. For this reason, a method is generally used in which data necessary for the client to decode and reproduce the video in real time is continuously supplied from the server. The server side controls the data supply rate, and the client only needs to receive the data and play it back.

[0004]

However, in order to provide a stable moving image distribution service by the distribution service method in the conventional database system that distributes moving images, a stable data supply to a plurality of clients is guaranteed. A high-performance server, a network with a guaranteed transmission bandwidth that does not affect other traffic when transferring data, and a system configuration with a client that has a uniform ability to reproduce supplied data without omission Is required. It is assumed that these are realized in an ideal system environment without considering other external factors. However, from the viewpoint of efficiency and economical efficiency, it is desired to realize the moving image distribution service in the existing environment.

The existing environment is defined for each of a server, a network, and a client. The server is assumed to be a general-purpose computer connected to an existing network and can be used for purposes other than the moving image distribution service. The network is also used for purposes other than moving image distribution, and bandwidth is not guaranteed during moving image transfer. In addition, in an environment in which multiple networks are interconnected, a situation is assumed in which a route from a server to a client passes through multiple networks. On the client side,
In an existing PC (personal computer) / WS (workstation), etc., from a client having dedicated hardware for decoding moving image coded data, resources such as CPU and memory sometimes change depending on internal and external processing conditions. Assume even sensitive software-based clients.

[0006] The present invention realizes a network moving image distribution system which is operated in the environment affected by external and internal factors as described above without adjusting the moving image distribution service while adapting to the situation. For the purpose of
Moreover, it aims at providing the client apparatus for implement | achieving such a system.

[0007]

SUMMARY OF THE INVENTION The present invention is a network moving image distribution system for distributing compression-encoded moving image data accumulated on a server side to a client in real time via a network. Has means for selecting a program of moving image data designated by a request from the client and sending it to the client, and the client has means for requesting a program of desired moving image data to the server. In the network moving image distribution system, when receiving the first refresh processing data from the server, the client side makes a request in packet units, measures the time until the packet arrives for one request, and refreshes. A means for confirming the number of packets including the processed data and a transmission time per packet are included in one packet. If the processing time of moving image data exceeds the processing time of the moving image data, when requesting new refresh processing data for resynchronization, requesting a plurality of packets based on the number of packets confirmed at the time of the first refresh processing data request. And a means for transmitting to the server, and the server side has means for continuously transmitting to the client the number of moving image packets designated by the request from the client. System
Means for achieving the above object.

Further, the present invention is characterized in that, in the above-mentioned network moving image distribution system, the client side has resynchronization suppressing means for suppressing a request for refresh processing data during processing of refresh processing data. The network moving image distribution system as follows is used as means for achieving the above object.

In addition, in the network moving image distribution system according to the present invention, at the time of arrival of a new packet, at the client side, a first processing elapsed time from the first arrival time of the first packet at the start of transfer. And a first difference value obtained by subtracting the time index included in the arrival packet from the first processing elapsed time, and the first difference value is larger than a preset processing delay allowable value. In this case, the decoding process is interrupted, the packet request and the packet that has already arrived are cleared, the refresh process data appearing after the next packet waiting for transmission is requested to the server, and the first difference value is sent to the server. The server side divides the moving image data into packets and adds a time index to the header of the packets. In response to the refresh processing data request from the sending means and the client, a packet including refresh processing data whose time index of the next scheduled packet appears at a time later than the received first difference value is detected. However, a network moving image distribution system having means for transmitting from the packet is a means for achieving the above object.

Further, in the above-mentioned network moving image distribution system according to the present invention, at the time of completion of decoding processing of video data included in a packet, the first side of the client at the time of starting transfer from the receiving buffer. A means for measuring the second processing elapsed time from the time when the packet is read, and a second difference value obtained by subtracting the time index included in the next packet read from the reception buffer from the processing elapsed time of the measure 2. If the second difference value is larger than the preset processing delay allowance value, the decoding process is interrupted, the packet request and the packet that has already arrived are cleared, and after the next transmission waiting packet to the server. Requesting emerging refresh processing data and notifying the server of the second difference value,
On the server side, the moving image data is divided into packets,
A means for adding a time index to the header of the packet and transmitting the packet, and a time index of the next scheduled transmission packet at a time later than the received second difference value in response to the refresh processing data request from the client. A network moving image distribution system characterized by having a means for detecting a packet containing refresh processing data that appears and transmitting it from the packet is a means for achieving the above object.

Further, the present invention is a client device of a network moving image distribution system for distributing compressed and encoded moving image data accumulated on a server side to a client in real time via a network in response to a request from the client. , When receiving the first refresh processing data from the server, make a request in packet units and measure the time until the packet arrives for one request,
Means for confirming the number of packets including refresh processing data and requesting new refresh processing data for resynchronization when the transmission time per packet exceeds the processing time for moving image data included in one packet At this time, the client device in the network moving image distribution system is characterized in that it has a means for requesting a plurality of packets to the server based on the number of packets confirmed at the time of the first refresh processing data request. Use it as a means to achieve the purpose.

Further, according to the present invention, in the client device in the above-mentioned network moving image distribution system, there is provided a resynchronization suppressing means for suppressing a request for refresh processing data during processing of refresh processing data. A client device in the moving image distribution system is a means for achieving the above object.

Further, according to the present invention, in the client device in the network moving image distribution system, the first processing elapsed time from the arrival time of the first first packet at the start of transfer is measured at the arrival time of a new packet. And a first difference value obtained by subtracting a time index included in the arrival packet from the first processing elapsed time, and when the first difference value is larger than a preset processing delay allowable value, Means for interrupting the decoding process, clearing the packet request and already arrived packets, requesting the server for refresh process data appearing after the next packet waiting for transmission, and notifying the server of the first difference value A client device in a network moving image distribution system having: The means of the eye.

Further, according to the present invention, in the client device in the network moving image distribution system, when the decoding process of the video data included in the packet is completed, the first first packet at the start of transfer is read from the reception buffer. Means for measuring the second processing elapsed time from the time, and a second difference value obtained by subtracting the time index included in the next packet read from the reception buffer from the second processing elapsed time, When the second difference value is larger than the preset processing delay allowance value, the decoding process is interrupted, the packet request and the packet that has already arrived are cleared, and the refresh that appears after the next packet waiting to be sent to the server is refreshed. Means for requesting the processed data and notifying the server of the second difference value. The client devices in Ttowaku moving image distribution system, the means for achieving the above object.

The present invention has means for detecting the status of each system and executes the processing while adapting the processing according to the status of the server, the network and the client. In most of the existing systems, the transfer rate of moving image data is controlled by the server side, but in the present invention, the control of the transfer rate and the detection of the situation are performed by the client. The status of the server and the network is grasped by the client in a concentrated manner by observing the arrival status of the data, and the status of the client is monitored by monitoring the progress of decoding. The client sequentially requests the server for moving image data of a size necessary for decoding, and the server also divides large data into packets and transfers them. Regarding decoding of video data, the processing may be delayed depending on each situation of the system, but in order to eliminate the delay, for example, a delay up to the default value is allowed, and the delay exceeds the default value. If the number increases, the client clears the moving image data that has been received from the server and has not been processed in the temporary buffer, and requests the refresh processing data as new moving image data from the time when the delay can be resolved to the server. Start decoding. Here, when requesting the refresh processing data, when the first refresh processing data is received, the number of packets required for transmission of the refresh processing data determined by measuring the packet arrival time for one packet request is also included. In addition to requesting a plurality of packets, the resynchronization is suppressed during the processing of the refresh processing data. As described above, by detecting the status of each system and executing the processing while adapting the processing according to the status, a system consisting of distributedly managed servers, networks, and clients, which is currently widely used, Realize a good video distribution service.

[0016]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Next, embodiments of the present invention will be described in detail with reference to the drawings.

FIG. 1 is a system configuration diagram illustrating a first embodiment of the present invention. In the present embodiment example, ISD
H.V. used in videophones, etc. An example of a system using the H.261 image coding system is shown. This system is realized by a server 1 and a plurality of clients 2 on a network 3 such as the Internet constructed as an aggregate of LANs as shown in FIG. The server 1 and the client 2 are interconnected via the network 3 to which they are connected.

FIG. 2 shows a data transmission mode in this embodiment. H. The H.261 coding method is a coding method using inter-frame differences. In the moving image data distributed to the client 2, as shown in FIG. 2A, an instruction is given to the encoder at the time of encoding, and video frames encoded in the intraframe encoding mode are periodically inserted. Hereinafter, a video frame coded in this intraframe coding mode is referred to as refresh processing data. Reference numeral 4 in FIG. 2A indicates the beginning of the refresh processing data periodically inserted in the moving image data. H. The coded data according to 261 is
Since it is represented as a bit string, in order to facilitate the transfer process of data on the network 3, the bit stream of data is divided into packets 5 of an appropriate size to be transferred on the network as shown in FIG. As shown in FIG. 2C, the header 7 is added to the packet 6 and the server 1 holds the aggregate of the packet data 6 as a single moving image data.

In the case of a storage service, an aggregate of packet data 6 is held as a file, and in the case of real-time delivery, the packet data 6 in a range traced from the latest data to a preset time in the past. Holds a collection of.

The header 7 of each packet data is represented by the decoding process start time in a situation in which the bitstream contained in the packet is normally decoded up to the immediately preceding bitstream without being affected by delay or the like. A time index 71 to be displayed and an identifier 72 indicating whether or not intraframe coded data is included in the packet are included.

FIG. 3 is a block diagram showing the configurations of the server and the client in this embodiment. On the server 1 side,
It has a moving image data accumulation processing unit 11, a refresh processing data detection unit 12, and a server side packet transmission processing unit 13. On the other hand, the client 2 side has a moving image display processing unit 21, a decoding processing unit 22, a packet request and reception processing unit 2.
3, a decoding process delay detection unit 24, and a transmission process delay detection unit 25. The server-side packet transmission processing unit 13 on the server 1 side and the packet request / reception processing unit 23 on the client 2 side are connected to the network 3.

The operation of this embodiment will be described based on the configuration shown in FIG.

First, the packet request / reception processing unit 23 causes the server-side packet transmission processing unit 1 via the network 3.
3 is connected to specify the content name of the moving image data to be requested first. Next, the packet request / reception processing unit 23 on the client 2 side requests the server-side packet transmission processing unit 13 to transfer data in packet units from the refresh processing data that appears first, and starts decoding the data. The packet received by the packet request / reception processing unit 23 is
The data is stored in the internal reception buffer, and the decoding processing unit 22 sequentially reads and processes it. Further, on the client 2 side, at the same time as the first packet of the moving image data sent by the server side packet sending processing unit 13 is received, the decoding processing delay detecting unit 24 and the transmission processing delay detecting unit 25 transmit and decode processing elapsed time. Start measuring.

At the time of receiving a new packet, the transmission processing delay detector 25 measures the processing elapsed time from the first arrival time of the first packet at the start of transfer. The processing elapsed time of the packet measured by the transmission processing delay detection unit 25 is compared with the time index 71 of the data stream in the packet recorded in the arrival packet.

The decoding process delay detection unit 24 starts the transfer from the reception buffer of the packet request and reception processing unit 23 at the time when the decoding process of the video data included in the packet received from the packet request and reception processing unit 23 is completed. The processing elapsed time from the time when the first first packet of is read is measured. The processing elapsed time measured by the decoding processing delay detection unit 24 is compared with the time index 71 of the data stream in the next packet read from the packet request and reception processing unit 23.

The processing delay detection performed by the decoding processing delay detection unit 24 and the transmission processing delay detection unit 25 is performed by the same procedure as described below. FIG. 5 shows a conceptual diagram thereof.

If the processing elapsed time is smaller than the time index 71, the processing of the decoding processing unit 22 and the packet request / reception processing unit 23 is suspended until the processing start time. As shown in FIG. 5A, this is the case where the transmission and decoding processing time 111 per packet is less than the processing time 110 per packet.

As shown in FIG. 5B, the transmission and decoding processing time 112 per packet is the processing time 110.
When the processing elapsed time is longer than the processing start time by exceeding the above, two operations are performed depending on the size of the difference 113.

When the processing delay allowable value is set on the client side and the difference 113 is smaller than the allowable value, the decoding processing unit 22 and the packet request / reception processing unit 2 are used as they are.
The process of 3 is continued.

When the difference 113 is larger than the allowable value, the processing of the decoding processing unit 22 is interrupted, the packet which has already arrived at the packet processing and reception processing unit 23 is cleared, and then the server side packet transmission processing unit. 12, requesting refresh processing data appearing after the next packet waiting to be transmitted. At the time of requesting refresh processing data, the value of the difference 113 calculated by the decoding processing delay detecting unit 24 and the transmission processing delay detecting unit 25 is also notified to the server 1. On the server 1 side, in response to a refresh processing data request from the client 2, the refresh processing data detection unit 1
Time index 7 of the next packet to be transmitted in 3
A packet including refresh processing data that appears at a time later than the difference value sent from 1 is detected, and the packet is transmitted. When the client 2 side receives the refresh processing data in response to the request, the packet request / reception processing unit 23 continues to request continuous packets, and the decoding processing unit 22 continues until the start time recorded in the packet. The delay is eliminated by interrupting the decoding process, and the decoding process is restarted when the start time comes.

Next, a second embodiment of the present invention will be described.

The present embodiment is an embodiment that is effective when the network between the server and the client is significantly lower than the transmission band for requesting moving image data, or when the transmission delay through a plurality of networks is large. . FIG. 4 shows a conceptual diagram of the transmission process in this embodiment.

In this system, refresh processing data is used as means for eliminating processing delay. H. 26
In the moving image data using the encoding method of No. 1, the refresh processing data itself has a larger size than other data, and compared with the maximum size of the packet 104 transmitted on the network as shown in FIG. However, there are cases in which the packet becomes large and the packet 105 extends over a plurality of packets. In order to eliminate the processing delay, all the refresh processing data must be received, decoded and displayed. However, when a plurality of packets are required, one request packet 101 is created as shown in FIG. On the other hand, transferring one data packet 102 is inefficient. In particular, when the transmission band is low or the transmission delay is large, the processing delay included in the transmission time 103 of the request packet cannot be ignored. Therefore, the client uses means for requesting a plurality of packets with one request to the server.

First, on the client side, when connecting to the server and receiving the first refresh processing data, a request is made in packet units. At this time, the time until the packet arrives for one request is measured, and it is confirmed whether the transmission time per packet is longer than the processing time of the moving image data included in one packet. Also, check how many packets the refresh processing data spanned.

When the transmission time of one packet is sufficiently smaller than the processing time of one packet, it is judged that the transmission band on the network is secured, and a plurality of requests for packets are made in order to disperse the traffic. Without, Figure 4
The request packet 106 and the data packet 107 in (b) are transmitted every time.

In a situation in which the transmission time of one packet exceeds the processing time of one packet, when newly requesting refresh processing data for resynchronization, it is confirmed at the time of the first refresh processing data request. Based on the number of packets, a request for a plurality of packets is set to 10 in FIG.
Do as in 8. In this case, the server continuously transmits the number of packets designated by one instruction as indicated by 109. The client side accumulates the moving image packets that are continuously sent in the reception buffer. The accumulated moving image packets in the reception buffer are sent to the decoding process from the one that arrives.

If the transmission delay cannot be secured, the refresh processing data may not be processed within the processing delay allowable value range set in the first embodiment. Therefore, even if the refresh processing data exceeds the allowable value, the refresh processing data cannot be processed. If the process of is not completed, it has a function to suppress resynchronization. When the refresh processing data is processed in excess of the allowable value, resynchronization is performed again when the processing is completed. For other processing, the procedure of the first embodiment is followed.

[0038]

As described above, according to the present invention,
When distributing a moving image via a network, it operates adaptively according to the load situation occurring at each location of the server, network, and client that are distributed and managed, and the best moving image reproduction within the range of the capability can be performed. In the present invention, since the control is on the client side, the load on the server side can be reduced, the upper limit of the number of clients that can be accessed simultaneously can be increased, and unnecessary packet transmission on the network can also be suppressed. This is particularly effective when the client side uses software for decoding and on a network in which the transmission band required for distribution of moving image data is not guaranteed.

[Brief description of drawings]

FIG. 1 is a system configuration diagram showing an embodiment of the present invention.

2A, 2B, and 2C are views for explaining a data packet transmission mode in the above-described embodiment.

FIG. 3 is a block configuration diagram of a server and a client in the above embodiment example.

FIG. 4A is a diagram explaining a transmission process when one data packet is transferred for one request packet;
(B) is a conceptual diagram of a transmission process when a request packet and a data packet are alternately sent, and (c) is a conceptual diagram of a transmission process when a plurality of packets are collectively sent after the request packet

5A is a conceptual diagram of transmission processing when the transmission and decoding processing time per packet is shorter than the processing time per packet, and FIG. 5B is the transmission and decoding processing time per packet per packet Conceptual diagram of transmission processing when it is longer than processing time

[Explanation of symbols]

1 ... server 11 ... Moving image data storage processing unit 12 ... Refresh processing data detector 13 ... Server side packet transmission processing unit 2 ... Client 21 ... Moving image display processing unit 22 ... Decoding processing unit 23 ... Packet request and reception processing unit 24 ... Decoding processing delay detection unit 25 ... Transmission processing delay detection unit 3 ... Network 4 ... Start of refresh data 5, 6 ... Packet 7 ... Header 71 ... Time index 72 ... Identifier

Front page continued (51) Int.Cl. ⁷ Identification code FI theme code (reference) H04N 7/081 F term (reference) 5C053 FA23 GA11 GB06 GB21 LA11 LA15 5C063 AA01 AB03 AB07 AC01 AC05 AC10 CA23 CA36 DA07 DA13 DB10 5C064 BA07 BB05 BC10 BD01 BD08 BD09 5K030 GA02 GA13 HA08 HB02 HB13 HC01 JA07 JT02 JT04 KA01 KA03 KX11 KX30 LE03 MA12 MB09

Claims (8)

[Claims] 1. A network moving image distribution system for distributing compression-encoded moving image data accumulated on a server side to a client in real time via a network, wherein the server side is requested by the client. In a network moving image distribution system, which has means for selecting a program of specified moving image data and transmitting it to a client, and having a means for requesting a program of desired moving image data to the server on the client side, the client When receiving the first refresh processing data from the server, the request is made in packet units, the time until the packet arrives for one request is measured, and the number of packets including the refresh processing data is calculated. A means for confirming and a processing of moving image data including transmission time per packet included in one packet. When the time is exceeded, when requesting new refresh processing data for resynchronization, a means for requesting a plurality of packets to the server based on the number of packets confirmed at the time of the first refresh processing data request. A network moving image distribution system, characterized in that the server side has means for continuously transmitting to the client a number of moving image packets designated by a request from the client. 2. The network moving image distribution system according to claim 1, wherein the client side has resynchronization suppressing means for suppressing a request for refresh processing data during processing of refresh processing data. 3. The client side is provided with a first first packet at the start of transfer when a new packet arrives.
Means for measuring a first processing elapsed time from the packet arrival time, and a first difference value obtained by subtracting a time index included in the arrival packet from the first processing elapsed time, and the first difference value Is larger than the preset processing delay allowance, the decoding process is interrupted, the packet request and the packet that has already arrived are cleared, and the server is requested for refresh processing data that appears after the next packet waiting to be transmitted. And means for notifying the server of the first difference value, and means for dividing the moving image data into packets, adding a time index to a header of the packet, and transmitting the packet on the server side, In response to the refresh processing data request from the client, the time index of the next scheduled packet is after the received first difference value. 3. The network moving image distribution system according to claim 2, further comprising means for detecting a packet containing refresh processing data that appears at time and transmitting from the packet. 4. A second processing elapsed time from the time when the first first packet at the start of transfer is read from the reception buffer at the time when the decoding processing of the video data included in the packet ends, on the client side. And a second difference value obtained by subtracting a time index included in the next packet read from the reception buffer from the processing elapsed time of the measure 2, and the second difference value is set in advance. When it is larger than the processing delay allowable value, the decoding process is interrupted, the packet request and the packet that has already arrived are cleared, the server is requested for the refresh processing data appearing after the next packet waiting for transmission, and the second Means for notifying the server of the difference value, wherein the server side divides the moving image data into packets, and stores the time in the header of the packets. And a refresh processing data that the time index of the next scheduled transmission packet appears at a time after the received second difference value in response to the refresh processing data request from the client. 3. The network moving image distribution system according to claim 2, further comprising means for detecting the included packet and transmitting the packet. 5. A first refresh in a client device of a network moving image distribution system, which distributes compression-encoded moving image data accumulated on a server side to a client in real time via a network in response to a request from the client. When receiving processed data from the server, a means for requesting in packet units, measuring the time until a packet arrives for one request, and confirming the number of packets including refresh processing data, and If the transmission time of the data exceeds the processing time of the moving image data included in one packet, the number of packets confirmed at the time of the first refresh processing data request when newly requesting the refresh processing data for resynchronization. And means for requesting multiple packets to the server based on Client devices in the network moving image distribution system. 6. During the processing of the refresh processing data,
The client device in the network moving image distribution system according to claim 5, further comprising resynchronization suppressing means for suppressing a request for refresh processing data. 7. A means for measuring the first processing elapsed time from the first arrival time of the first packet at the start of transfer at the arrival of a new packet, and the arrival packet including the first processing elapsed time from the first processing elapsed time. The first difference value obtained by subtracting the time index is calculated, and when the first difference value is larger than the preset processing delay allowable value, the decoding process is interrupted, and the packet request and the already arrived packet are detected. 7. The network moving image according to claim 6, further comprising means for clearing and requesting the server to perform refresh processing data that appears after the next packet waiting for transmission, and notifying the server of the first difference value. Client device in distribution system. 8. A means for measuring a second processing elapsed time from the time when the first first packet at the start of transfer is read from the reception buffer at the time when the decoding processing of the video data included in the packet ends, A second difference value obtained by subtracting a time index included in the next packet read from the reception buffer from the second processing elapsed time is obtained, and the second difference value is larger than a preset processing delay allowable value. In this case, the decoding process is interrupted, the packet request and the packet that has already arrived are cleared, the server is requested to perform the refresh process data that appears after the next packet waiting to be transmitted, and the second difference value is sent to the server. 7. The client device in the network moving image distribution system according to claim 6, further comprising means for notifying.