JP2928881B1 - Video transfer method - Google Patents

Abstract

Abstract: PROBLEM TO BE SOLVED: To transfer a moving image without delay even when a load on a network increases. Each time a difference frame is received from a server in gateways (relay stations) 4A and 4B provided in a network 3 from a server 1 to clients 2A to 2C, coordinates of each vertex are calculated from the difference value. The change amount is calculated, and only the difference value having the change amount equal to or larger than the difference accuracy value is transmitted to the client as a new difference frame.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving image transfer method, and more particularly to a moving image transfer method in which a series of moving images composed of a plurality of time-sequential still images is converted into a reference frame and a difference frame and transferred. It is about the method.

[0002]

2. Description of the Related Art In general, when a series of moving images composed of a plurality of still images that are continuous in time series is transferred from a transmission source server to a transmission destination client via a network,
In order to reduce the amount of data to be transferred, a method is used in which a series of moving images is converted into a reference frame and a difference frame and transferred. On the other hand, in order to respond to a request from a client while receiving moving image data, a method of processing and transmitting moving image data has been proposed (for example, Japanese Patent Application Laid-Open No. 6-1496).
No. 49).

In this case, in a typical polygon format as a representation format of three-dimensional moving image data, a three-dimensional object is represented by vertex coordinates,
Still images are represented by ridge and plane data, and three-dimensional moving image data is configured by arranging these still images in chronological order. Here, when the processing capability of the client is reduced and the received frame cannot be displayed at predetermined time intervals, the client instructs the server to reduce the image quality.

In response to this, the server reduces the amount of moving image data to be transmitted by reducing the number of polygons from each frame in the moving image data to be transmitted to lower the image quality, or by thinning out reference frames. Therefore, the amount of video data that must be processed by the client decreases,
Even when the processing capacity is reduced, the received frame can be displayed at predetermined time intervals.

[0005]

However, in such a conventional moving image transfer method, the moving image data to be transmitted is reduced in response to a request from a client for reproducing and displaying the received moving image data. When the load on the network connecting the client and the client increases, it is impossible to cope with such a problem. In such a case, there is a problem that the moving image data stagnates in the network, and each frame cannot be displayed at predetermined time intervals. The present invention has been made to solve the above problems, and has as its object to provide a moving image transfer method capable of transferring a moving image without delay even when the load on the network increases.

[0006]

In order to achieve the above object, a moving image transfer method according to the present invention is directed to a relay station provided in a network from a server to a client, which receives a difference frame from a server. In this case, the difference values of the individual graphic data changed between the two still images at different times obtained from the difference frame are compared with the difference accuracy values indicating the accuracy of the image generated from the difference frame, respectively. In accordance with the comparison result, the necessity of transfer of each difference value is determined, and a new difference frame is generated from the difference value determined to be necessary for transfer and transferred to the client side. Therefore, by controlling the magnitude of the difference accuracy value according to the network load or the client request, the data capacity of the difference frame transmitted from the relay station to the client side is adjusted.

[0007]

Next, the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a moving image transfer system according to an embodiment of the present invention.
1 is a server for transmitting moving images, 2A to 2C are clients for displaying received moving images, and 3 is a communication network (hereinafter, referred to as a network) connecting the server 1 and clients 2A to 2C, which is composed of the Internet or an intranet. , 4B are gateways (relay stations) constituting the network 3.

Here, the client 2A is connected to the server 1 via the gateway 4A. The clients 2B and 2C are connected to the gateway 4B, and the gateway 4B is connected to the server 1 via the gateway 4A.

In the following, as described above, a method of transmitting a series of moving images composed of a plurality of time-sequential still images from the transmission source server 1 to the transmission destination clients 2A to 2C via the network 3 will be described. A case where a series of moving images is converted into a reference frame and a difference frame and transmitted will be described as an example.

FIG. 2 is a block diagram showing the configuration of the gateway. In the figure, reference numeral 11 denotes a data receiving unit for receiving moving image data from the server 1, and 12 denotes a data receiving unit 1.
A difference frame conversion unit that converts the accuracy of the difference frame in the moving image data received in 1 and outputs the converted data, and a data transmission unit 13 that transmits the moving image data from the difference frame conversion unit 12 to the client side.

Reference numeral 14 denotes a network monitoring unit for monitoring the load on the network on the output side to which the moving image data is transmitted from the data transmission unit 13. Reference numeral 15 denotes a difference precision value corresponding to the network load obtained by the network monitoring unit 14. A difference accuracy setting unit 17 to be set receives a difference accuracy setting request from a client, and a command receiving unit that outputs a difference accuracy value included in the request, and 16 is a command from the difference accuracy setting unit 15 and the command receiving unit 17 The difference precision value of
A difference accuracy management unit that manages each client.

In particular, the network monitoring unit 14 obtains, for each client, the current effective bandwidth of the network on the output side from the length of the frame transmitted from the network transmission unit 13 to the client and the time required for transmission. This monitors the network load. Further, the difference accuracy setting unit 15 lowers the difference accuracy when the execution bandwidth decreases and the load on the network increases.

FIG. 3 is a block diagram showing the configuration of the client. In the figure, reference numeral 21 denotes a data receiving unit for receiving moving image data from the server 1 side (gateway 4A, 4B side); and 22, a frame buffer comprising a memory for storing moving image data received by the data receiving unit 21, that is, a reference frame and a difference frame. An image development processing unit that develops a still image on 23 is a display unit that displays a still image on the screen by scanning the frame buffer 23.

Reference numeral 25 denotes a command input unit for receiving a request for setting a difference accuracy value from a client user, and reference numeral 26 denotes a command transmission unit for transmitting the difference accuracy setting request input from the command input unit 25 to the gateway.

FIGS. 4A and 4B are explanatory diagrams showing the structure of moving image data. FIG. 4A shows changes in polygons forming a three-dimensional still image, and FIG. 4B shows frames generated corresponding to each polygon. . In FIG. 4A, a triangular polygon 31 having vertices A, B, and C is transformed into polygons 32-34 over time.

These changes in polygons are captured as still images (frames) at predetermined time intervals. These still images are composed of a reference frame consisting of all graphic data of a still image at an arbitrary time and two still images at different times. This is converted into a difference frame composed of difference values of individual graphic data changed between images.

Therefore, the reference frame 41 includes the coordinate values of the vertices A to C constituting the polygon 31 and the vertices A to C.
It is generated from the graphic data indicating the connection information between Cs. Further, the difference frame 42 compares the polygon 32 with the immediately preceding polygon 31 and calculates the difference between the vertices A to C.
Is generated from the amount of change in the graphic data indicating the coordinate value of. In this manner, difference frames 43, 44,... Are sequentially generated from the difference from the immediately preceding polygon, and a reference frame is generated each time a predetermined number of difference frames are generated.

Next, as the operation of the present invention, the operation of the difference frame converter will be described. FIG. 5 is a block diagram showing the configuration of the difference frame conversion unit, FIG. 6 is a flowchart showing the difference frame conversion process, and FIG. 7 is an explanatory diagram showing the converted difference frame. In the present invention, each time a difference frame is received from the server side, the coordinate change amount of each vertex is calculated from the difference value, and only the difference value whose change amount is equal to or greater than the difference accuracy value is set as a new difference frame. Is transmitted to the client side.

The difference value of each vertex that has not been transmitted is sequentially accumulated and stored as an accumulated difference value. Each time a new difference frame is received, the difference value of each vertex included in the difference frame is stored in the corresponding accumulated value. This is added to the difference value, and the coordinate change amount of each vertex is calculated from the accumulated difference value after the addition.

In FIG. 5, reference numeral 51 denotes a data receiving unit 11
An input register to which the difference frame received in (see FIG. 2) is input, and 52 is an accumulator for adding the difference value included in the difference frame in the input register 51 to the accumulated difference value held internally. 53 is an accumulator 5
The arithmetic processing unit 54 calculates the amount of change in each vertex coordinate from the accumulated difference value obtained in step 2, and the amount of change in each vertex coordinate obtained in the operation processing unit 53 and the client from the difference accuracy management unit 16 The determination unit compares the difference accuracy value with the difference value, and determines whether or not the transfer of the accumulated difference value is necessary.

Reference numeral 55 denotes a gate for outputting only the cumulative difference value determined to be necessary to be transferred by the determining unit 54 among the cumulative difference values held in the accumulator 52, and 56 denotes a determining unit 5
By delaying the output indicating that the transfer is required from the gate 4, and outputting the reset signal after the output of the corresponding cumulative difference value from the gate 55 is completed, the transmitted cumulative difference value held in the accumulator 52 is initialized. (D).

FIG. 7A shows the change of the still images 101 to 104 shown in each frame, in this case, the change of the triangular polygon on the three-dimensional coordinates having vertices A to C, and FIG. Each vertex A to C in each of the still images 101 to 104
Are shown. In this case, since the still image 101 is transmitted by the reference frame, the coordinate value 111 of each vertex is transmitted from the server 1 (see FIG. 1) as it is. In addition, actually, connection information of each vertex is also transmitted in addition to these vertex coordinate values, but is omitted here.

FIG. 7C shows a difference frame from the server constituted by the differences between the vertex coordinate values, FIG. 7D shows the judgment result by the judgment unit 54, and FIG. 7E shows the judgment result according to the judgment result. This shows a new difference frame transmitted from the gate 55 of the gateway. At the timing of transmitting the frame indicating the still image 102, a difference value between the coordinate value 111 of the immediately preceding still image 101 and the coordinate value 112 of the still image 102 is transmitted from the server as a difference frame 122.

Generally, a still image 10i and a still image 1
0j (however, j = i + 1) if the vertex n in coordinates as follows, still images 10i vertex _{n: (x ni, y ni} , z ni) static image 10j vertex _{n: (x nj, y nj} , difference value of the vertex n in z _nj) difference frames, the still image 10j vertex _{n: (dx nj = x nj} -x ni ( difference frame _{12j) dy nj = y nj -y} ni dz nj = z nj -z ni) Becomes

The reference frame and the difference frame transmitted from the server 1 are transmitted to the gateway 4 in the network 3.
A, 4B received by the data receiving unit 11 (see FIG. 2),
It is stored and held in the input register 51 (see FIG. 5) of the difference frame converter 12. When the frame held in the input register 51 is a reference frame, all the accumulated difference values held in the accumulator 52 are initialized (accumulated difference value = 0), and the reference frame is output from the gate 55 as it is. Is done.

On the other hand, when the frame held in the input register 51 is a difference frame, first, each vertex difference value of the difference frame is added to the accumulated difference value held in the accumulator 52 as shown in FIG. (Step 61), and the arithmetic processing unit 53 calculates the change amount of each vertex from the newly obtained accumulated difference value (Step 62).

For example, the cumulative difference value of the vertex n at the time when the difference frame 12j indicating the still image 10j is received is calculated as follows. Cumulative difference value Vertex n: (Sdx _nj = dx _nj + Sdx _ni (difference frame 12j) Sdy _nj = dy _nj + Sdy _ni Sdz _nj = dz _nj + Sdz _ni )

The arithmetic processing unit 53 calculates the change amount of each vertex based on the new accumulated difference value as follows. For example, the vertex n calculated from the difference frame 12j
Is (Sdx _nj , Sdy _nj , Sdz _nj ), the amount of change d _nj of vertex n between the last transmitted vertex coordinates and still image 10j is d _nj = (Sdx _nj ² + Sdy) _nj ² + Sdz _nj ² ) ^1/2 .

The change amount d _nj of the vertex n calculated by the arithmetic processing unit 53 is compared with a difference precision value p based on a network load or a request from a client in a determination unit 54, and the accumulated difference value of the vertex n is calculated. It is determined whether the transfer is necessary (step 63). If the change amount d _nj of the vertex n is equal to or greater than the difference accuracy value p (step 63: YES), it is determined that the vertex n has moved significantly from the coordinate position of the vertex n transmitted last, and The gate 55 is instructed that the cumulative difference value of n is required to be transferred (step 64).

On the other hand, the change amount d _nj of the vertex n is equal to the difference precision value p.
If it is smaller (step 63: NO), it is determined that the vertex n has hardly moved from the coordinate position of the vertex n transmitted last, and that the accumulated difference value of the vertex n does not need to be transferred. In this way, the amount of change is calculated for each vertex and compared with the difference accuracy value p to determine whether or not each vertex needs to be transferred. Then, only the accumulated difference value of the vertices that have received the transfer necessity instruction is transmitted from the accumulator 52 to the client via the gate 55.

It should be noted that the accumulated difference value of the vertex determined to be unnecessary for transfer is held in the accumulator 52 as it is.
The cumulative difference value of the vertex determined to need transfer is initialized after the transmission is completed (step 65). In FIG. 5, the initialization instruction is issued by delaying the transfer instruction from the determination unit 54 to the gate 55 by the delay unit 56 and inputting it as a reset signal to the accumulator 52, but other methods may be used.

In FIG. 7, two still images 101, 1
If the vertex C moves more than the difference precision value p and the vertices A and B hardly move between 02, the judgment result 13
As shown in FIG. 2, it is determined that only the vertex C needs to be transferred, and a new difference frame 1 including the accumulated difference value of the vertex C is determined.
42 is transmitted to the client side. As a result, a difference frame 142 having a smaller data volume than the difference frame 122 transmitted from the server 1 is newly generated and transmitted from the gateways 4A and 4B.

Therefore, the magnitude of the difference precision value p used to determine whether transfer of the accumulated difference value is necessary is determined by the gateways 4A and 4A.
By controlling according to the network load after B, the data capacity of the difference frame can be increased or decreased. As a result, even when the network load increases and the available bandwidth decreases, the client 2 transmits the difference frame with a small data capacity at a predetermined time interval.
Frame data can be transmitted to A to 2C, and moving images can be transferred without delay even when the load on the network increases.

Further, since such a difference frame conversion process is performed by each gateway provided in the network, it is possible to individually cope with the load of each route of the network, and to optimize each client. Services can be provided. Further, for each vertex, the difference between each frame is accumulated, and the accumulated difference value is transmitted according to the determination of the transfer necessity. Therefore, for each vertex, a new coordinate value is transmitted from the last transmitted coordinate value. An accurate difference value up to the coordinate value can be transmitted, and an image can be accurately reproduced on the client side.

It is to be noted that the differential accuracy value p may be controlled not only in accordance with the network load but also in accordance with a request from the client side. For example, if the processing capacity is reduced on the client side due to other processing, the processing of reproducing frame data from the server is delayed. In such a case, the client transmits a request to increase the difference accuracy value p to the gateway.

Accordingly, the gateway reproduces the moving image without delay even if the load on the client side is reduced, in order to reduce the data volume of the difference frame immediately transmitted to the client. it can. Further, after that, when the load on the client side is reduced, by transmitting a request to reduce the difference precision value p, a difference frame having a large data capacity is transmitted from the gateway, and a moving image with high precision is transmitted to the client. Can be played.

In the above description, the difference frame conversion processing for one client has been described.
Such a difference frame conversion process may be performed individually for each client. As a result, it is possible to provide each client with an optimal service according to the load of the network on each client and the load of each client.

[0038]

As described above, according to the present invention, when a difference frame is received from a server at a relay station provided in the middle of a network from a server to a client, a different time point obtained from the difference frame is used. 2
A difference value between individual graphic data changed between two still images is compared with a difference accuracy value indicating an accuracy of an image generated from the difference frame, and transfer of each difference value is determined according to the comparison result. Is determined, and a new difference frame is generated from the difference value determined to be required to be transferred and transferred to the client side.

Therefore, by controlling the magnitude of the difference precision value according to the network load or the client request, the data capacity of the difference frame transmitted from the relay station to the client side can be adjusted. As a result, even when the network load increases and the available bandwidth decreases, the frame data can be transmitted to the client at predetermined time intervals by transmitting the differential frame having a small data capacity, and the network load increases. But you can transfer videos without delay.

Further, since such a difference frame conversion process is performed by each gateway provided in the network, it is possible to individually cope with the load on each route of the network, and to optimize each client. Services can be provided. Also, since a difference frame with a small data capacity is transmitted in response to a request from the client,
Even if the load on the client increases, the moving image can be reproduced without delay. Further, since it is possible to cope with each route of the network and each client, it is extremely useful in a multicast system in which moving image data is transmitted from a server to a plurality of clients simultaneously in parallel.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a moving image transfer system according to an embodiment of the present invention.

FIG. 2 is a block diagram showing a gateway.

FIG. 3 is a block diagram showing a client.

FIG. 4 is an explanatory diagram showing a configuration of moving image data.

FIG. 5 is a block diagram illustrating a difference frame conversion unit.

FIG. 6 is a flowchart illustrating a difference frame conversion process.

FIG. 7 is an explanatory diagram showing a converted difference frame.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Server, 2A-2C ... Client, 3 ... Network, 4A, 4B ... Gateway, 11 ... Data reception part, 12 ... Difference frame conversion part, 13 ... Data transmission part,
14: Network monitoring unit, 15: Difference accuracy setting unit, 1
6 ... Difference accuracy management unit, 17 ... Command reception unit, 21 ... Data reception unit, 22 ... Image development processing unit, 23 ... Frame buffer, 24 ... Display unit, 25 ... Command input unit, 26 ...
Command transmission unit, 31 to 34: polygon, 41: reference frame, 42 to 44: difference frame, 51: input register, 52: accumulator, 53: arithmetic processing unit, 54:
Judgment unit, 55 ... Gate, 56 ... Delay unit, 101-10
4, 10i, 10j: Still image, 111 to 114 coordinate values, 122 to 124, 12j: Difference frame, 132 to
134 judgment result, 142-144 new difference frame.

Claims (4)

(57) [Claims] 1. A series of moving images made up of a series of still images composed of a large number of graphic data indicating coordinate information, connection information, and the like of a graphic are sequentially converted from all graphic data of the still image at an arbitrary time point. To a difference frame composed of difference values of individual graphic data changed between two still images at different points in time, and these reference frame and difference frame are converted into moving image data from the server to the client via the network. In the moving image transfer method for transferring video data, the relay station provided in the middle of the network from the server to the client, when receiving the difference frame from the server side, transmits the difference frame between the two still images at different time points obtained from the difference frame. The difference value of each changed graphic data and the difference accuracy value indicating the accuracy of the image generated from the difference frame Are compared with each other, the necessity of transfer of individual difference values is determined according to the comparison result, a new difference frame is generated from the difference values determined to be necessary for transfer, and transferred to the client side. Video transfer method. 2. The moving image transfer method according to claim 1, wherein the relay station monitors a load on a network that transfers the reference frame and the difference frame, and transfers a difference value based on the network load obtained by the monitoring. A moving image transfer method characterized by setting a difference accuracy value used for determination of necessity. 3. The moving image transfer method according to claim 1, wherein the relay station sets a difference accuracy value to be used to determine whether transfer of the difference value is necessary in response to a difference accuracy setting request from a client indicating the difference accuracy value. A moving image transfer method. 4. The moving image transfer method according to claim 1, wherein the relay station holds a cumulative difference value determined to be unnecessary for transfer as a cumulative difference value for each graphic data, and receives a new difference frame. In this case, the difference value constituting the received difference frame is added to the corresponding cumulative difference value, and by comparing the added cumulative difference value with the difference precision value, it is determined whether or not the transfer of each cumulative difference value is necessary. A moving image transfer method, wherein a new difference frame is generated from the accumulated difference value determined to need to be transferred and transferred to the client side, and the transferred accumulated difference value is initialized and held.