JP2004072147A - Image distribution system, image reproducing system, and image distribution reproducing system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image distribution system for preventing a data quantity of a not reproduced image stored in a buffer of a client side from being zero resulting in interrupting the reproduction of the image. <P>SOLUTION: A parsing means 12 parses a syntax of code stream data, which results from applying hierarchical compression encoding to moving picture data each frame of which is divided into one or more tiles and pixel values of each tile is subjected to discrete wavelet transform. A code stream generating means 13 performs a plurality of jobs in paralell, generating other code stream data from the code stream data on the basis of the result of analysis. A data read amount detection means 25 detects a read amount per unit time of received code stream data, and a parameter generating means 16 integrates data of the read amounts to discriminate a congestion state of a channel and to produce proper parameters for generating the code stream data. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an image distribution system, an image reproduction system, and an image distribution and reproduction system.
[0002]
[Prior art]
2. Description of the Related Art Conventionally, there is a moving image distribution technology in which moving image data is supplied from a server, a transfer request of moving image data is issued from a client to a server, and a moving image is reproduced by a client based on the received data. . In this technique, download reproduction and stream reproduction are generally known as methods in which a reproduction device on the client side receives and reproduces moving image data from a server.
[0003]
Download reproduction is to reproduce the data downloaded from the server to the buffer of the client. This type is to reproduce the data once it is loaded into the buffer. There is a disadvantage that the playback time is short, but if all the data is received and then played back, there is an advantage that the moving image can be played regardless of the processing load on the server side, the speed of the transmission path, and the congestion situation. .
[0004]
On the other hand, in the streaming reproduction, a reproduction device on the client side continuously requests data from a server and performs a reproduction process of a moving image in parallel with an operation of acquiring the data on a buffer. In this type, since the client continuously receives moving image data, the data in its own buffer is reproduced and discarded, while new data is overwritten. Therefore, there is an advantage that a long-time moving image can be reproduced without being limited by the storage capacity of the buffer.
[0005]
However, there are disadvantages in that the server load increases due to an increase in the number of clients accessing the server at the same time, and the transmission line is easily affected by the speed. Therefore, there is a possibility that a serious problem that reproduction of a moving image is stopped due to an increase in a load on a server or a decrease in speed of a transmission line. In general, as a method for avoiding such a serious influence, a method called scalability for changing the capacity of moving image data is known.
[0006]
Conventionally, MPEG1 / MPEG2 / MPEG4 dedicated to moving images and Motion JPEG which treats still images as continuous frames have been used as image compression / decompression algorithms. Recently, however, the encoding of the latter Motion still images has been used. As for, a new method called Motion JPEG2000 is being standardized as an international standard.
[0007]
[Problems to be solved by the invention]
However, in the conventional moving image distribution technology, distribution contents are generally stored on a server in advance with a plurality of scalabilities, and a user selects the most suitable scalability according to a transmission capability of a communication path and a reproduction capability of a client. It is common to perform streaming playback.
[0008]
In this case, there is not only the inconvenience that the user operating the client software needs to select an appropriate condition in consideration of the restriction of the communication path, but also a problem that once the selection is made, it cannot be changed on the way. . That is, if the communication path suddenly becomes crowded during streaming playback, the image data arrives at the client at a delay with respect to the image playback speed at the client side, so that the buffering at the client performing streaming playback is difficult. However, there is a problem that the unreproduced moving image data becomes 0 and the image reproduction is interrupted.
[0009]
An object of the present invention is to prevent the unreproduced image data amount held in the buffer on the client side from becoming zero and the reproduction of the image to be interrupted even when the streaming reproduction is performed at the distribution destination of the moving image data. It is to prevent.
[0010]
[Means for Solving the Problems]
According to the first aspect of the present invention, there is provided code string data obtained by dividing moving image data into one or a plurality of rectangular regions for each frame, performing discrete wavelet transform on pixel values for each rectangular region, and hierarchically compressing and encoding. Syntax analysis means for analyzing the syntax of the code string data, and code string creation means capable of simultaneously performing a plurality of tasks of creating another code string data from the code string data based on the analysis result. Transmitting means for transmitting each of the code string data created by the code string creating means to a plurality of destinations simultaneously and in parallel via a network; and an amount of reading of the code string data per unit time at each of the destinations Instructing the creation of the code string data by a code string parameter when creating the code string data by the code string creation means created according to the data obtained by integrating A parameter instructing means for adjusting the data amount per unit time of the code string data to be transmitted by the transmission means, an image distribution system comprising a.
[0011]
Therefore, the data amount of the code string data is reduced or increased by a parameter created by judging the congestion state of the communication path from the data obtained by integrating the read amount of the code string data per unit time at the transmission destination of the code string data. Then, even if the amount of code string data to be transmitted per unit time is adjusted and the streaming reproduction is performed at the destination of the moving image data, the data amount of the unreproduced image becomes 0 and the reproduction of the image is interrupted. Can be prevented.
[0012]
According to a second aspect of the present invention, in the image distribution system according to the first aspect, receiving means for receiving the data of the read amount per unit time from each of the transmission destinations, And a parameter creating means for creating the parameter in response.
[0013]
Therefore, the data of the read amount per unit time of the code string data is collected from the code string data distribution destination, and the parameter can be created in the image distribution system.
[0014]
According to a third aspect of the present invention, in the image distribution system according to the first or second aspect, when the number of the destinations is n (n = 1, 2,...) Assuming that the read amount per unit time in An is An and the maximum data distribution capability of the code string data by the image distribution system is B, the communication line usage rate C is
C = ΣAn / B
And the parameters are created in accordance with the value of the usage rate C.
[0015]
Therefore, a parameter can be created by determining the usage rate of the communication line from the data of the read amount of the code string data per unit time at the distribution destination.
[0016]
According to a fourth aspect of the present invention, in the image distribution system according to any one of the first to third aspects, the parameter used by the parameter indicating unit is an image resolution or an image resolution obtained when decoding the code string data. It is a parameter related to the frame rate or a parameter related to the image compression ratio.
[0017]
Therefore, the data amount per unit time of the code string data to be transmitted can be adjusted by the parameters of the image resolution, the frame rate, or the image compression ratio.
[0018]
According to a fifth aspect of the present invention, there is provided code string data obtained by dividing moving image data into one or a plurality of rectangular regions for each frame, performing discrete wavelet transform on pixel values for each of the rectangular regions, and performing hierarchical compression coding. Receiving means for receiving, via a network, decoding means for decoding the received code string data, data reading amount detecting means for detecting the reading amount of the code string data per unit time by the receiving means, A transmission means for transmitting the detected data of the read amount per unit time to a transmission destination of the code string data.
[0019]
Accordingly, the amount of read code string data to be received per unit time is detected and transmitted to the destination of the code string data, so that the transmission destination judges the congestion state of the communication path from the data and transmits the code string to be transmitted. By reducing or increasing the data amount of the data, the data amount per unit time of the code string data to be transmitted is adjusted, and even if the streaming reproduction is performed at the destination of the moving image data, the data amount of the unreproduced image is Can be prevented from becoming zero and the reproduction of the image being interrupted.
[0020]
According to a sixth aspect of the present invention, there is provided the image distribution system according to the second aspect, wherein the code string data is received from the image distribution system, and the data of the read amount per unit time is transmitted to the image distribution system. An image distribution / reproduction system comprising at least one image reproduction system according to claim 5.
[0021]
Therefore, the image reproduction system detects the read amount of the received code string data per unit time, and the image distribution system determines the congestion state of the communication path from the data of the read amount and creates a predetermined parameter for transmission. By reducing or increasing the amount of code string data to be transmitted, the amount of code string data to be transmitted is adjusted per unit time, and even when streaming playback is performed at the destination of moving image data, unplayed images are not Can be prevented from being interrupted due to the data amount of 0 becoming zero.
[0022]
BEST MODE FOR CARRYING OUT THE INVENTION
An embodiment of the present invention will be described.
[0023]
FIG. 1 is a block diagram showing a system configuration of the entire image distribution / reproduction system according to the present embodiment. 1, in this system, a server 1 that implements an image distribution system of the present invention and a client 2 that implements an image reproduction system of the present invention are connected via a communication path 3 via a network.
[0024]
The server 1 includes a magnetic storage device 11 that stores moving image data. The magnetic storage device 11 stores code string data obtained by compressing and encoding original moving image data using the JPEG2000 algorithm. Therefore, the code string data is obtained by dividing the moving image data into one or a plurality of rectangular areas (tiles) for each frame, and performing a discrete wavelet transform (DWT) on each of the tiles to obtain a hierarchical structure. It is created by compression encoding.
[0025]
When distributing the code string data to the client 2, the syntax analysis means 12 analyzes the syntax of the code string data, that is, the information described in the header, and the code string creation means 13 performs the analysis based on the analysis result. Create another code string data from the code string data. Then, the generated code string data is transmitted to the client 2 via the communication path 3 via the communication interface 14 which realizes a transmission unit. The code string creation means 13 can create a plurality of code string data simultaneously and in parallel for a plurality of clients 2. The code string creating means 13 includes code string creating means 131 to 13n (n = 1, 2, 3,... N) for creating one piece of code string data, respectively. Create data.
[0026]
The parameter instructing means 15 instructs a parameter used when the code string creating means 13 creates code string data, and adjusts the data amount per unit time when transmitting the code string data by changing the parameter. .
[0027]
The parameter creating unit 16 creates a parameter designated by the parameter designating unit 15. The details will be described later.
[0028]
The client 2 reproduces an image by streaming reproduction. That is, each client 2 (clients 201, 202, 203,..., 20n) receives the code string data created by any of the code string creating units 131 to 13n through the communication interface 21 that implements the receiving unit. The received code string data is decoded by the decoding unit 23, and the decoded moving image data is buffered in the buffer 22 and displayed on the display device 24.
[0029]
The data read amount detection means 25 is provided in each client 2 for the data read amount per unit time of the code string data received from the server 1, for example, the data read amount An per second (n = 1, 2,...) (Unit: bps) is detected. This detection is continued during the reception of the code string data from the server 1, and the detected value of the data read amount An is determined, for example, by the communication interface 21 every minute during the reception of the code string data from the server 1. The data is transmitted from each client 2 to the server 1 via the client 2.
[0030]
The server 1 receives the information of the detected value of the data read amount An from each client 2, and the parameter creating unit 16 creates the above-described parameters based on each data read amount An.
[0031]
Specifically, assuming that the usage rate of the communication line is C and the total data transfer capability of all the lines (unit: bps), that is, the maximum data distribution capability of the server 1 is B,
C = ΣAn / B
It becomes.
[0032]
As a result, the data read amount An is integrated, and the utilization rate C indicating the congestion state of the communication line is obtained from the integrated data. When the usage rate C exceeds a predetermined threshold value c, it is determined that the communication path 3 is congested, and the amount of code string data per unit time transmitted by each of the code string creation units 131 to 13n is reduced. The above-mentioned parameters are changed so as to be smaller. In this case, the image quality of the image based on the code string data to be transmitted more or less is reduced. Conversely, when the usage rate C falls below the predetermined threshold value c, it is determined that the communication path 3 is free, and the data amount of the code string data per unit time transmitted by each of the code string creation units 131 to 13n is reduced. The above-mentioned parameters are changed so as to increase. In this case, the image quality of the image based on the code string data is improved. It should be noted that a plurality of thresholds c are prepared in a stepwise manner, the above-mentioned parameters are changed in a stepwise manner, and the amount of code string data per unit time transmitted by each of the code string creating means 131 to 13n is adjusted stepwise You may do it. Alternatively, the above-mentioned parameter is changed steplessly according to the magnitude of the usage rate C, and the data amount of the code string data per unit time transmitted by each of the code string creating means 131 to 13n is adjusted steplessly. You may do it.
[0033]
As the parameters, one or more of image resolution (the number of vertical and horizontal dots of the image), S / N (compression rate of the compression algorithm), and frame rate (the number of frames per unit time) can be used. That is, if the image resolution, the S / N, and the frame rate are increased, the amount of code string data to be transmitted also increases.
[0034]
When adjusting the frame rate, all frames are transmitted when the load is small, but the frame rate is reduced as shown in FIG. 2 as the load increases. That is, FIG. 2A shows the original code string data. This code string data includes n frames (bit streams). As shown in FIG. 2B, even-numbered (2, 4,...) Frames are deleted, and By transmitting only one frame, the frame rate can be reduced by half, and the data amount can be reduced by half. When deleting even-numbered (2, 4,...) Frames, only odd-numbered (1, 3,...) Frames from which even-numbered (2, 4,. The header (Header) is rewritten to obtain new code string data.
[0035]
When adjusting the image resolution, all the sub-band data is transmitted in order to transmit high-resolution image data when the load is small. However, as the load increases, transmission from lower sub-bands is stopped. , Low-resolution image data is transmitted.
[0036]
That is, FIG. 3A shows the contents of a code data file including all the original sub-band data. Here, code data is arranged in order of each subband. Delimiters are included in the breaks between the subbands, so that the boundaries can be easily determined. When the load is small, transmission is performed up to the last sub-band (0), but as the load increases, transmission is performed up to sub-band 1 or 2, 3 and lower sub-bands are not transmitted so as to not transmit the subsequent sub-bands. The band data is deleted, and a new code string as shown in FIG. 3B is created.
[0037]
FIG. 4 shows one subband in the code string of FIG. The sub-band contains coded data in order from the MSB of the bit plane. In the example of FIG. 4, the 2-bit plane data is discarded from the LSB of the original data in FIG. 4A, and a new code string shown in FIG. 4B is created to reduce the data amount. .
[0038]
【The invention's effect】
The invention according to Claims 1, 5, and 6 determines the congestion state of a communication channel and reduces or increases the amount of code string data to be transmitted, so that the data per unit time of the code string data to be transmitted is determined. Even if the amount is adjusted and the streaming reproduction is performed at the destination of the moving image data, it is possible to prevent the data amount of the unreproduced image from becoming zero and the reproduction of the image from being interrupted.
[0039]
According to a second aspect of the present invention, in the first aspect of the invention, data of a read amount per unit time of the code string data is collected from a destination of the code string data, and parameters are created in the image distribution system. Can be.
[0040]
According to a third aspect of the present invention, in the first or second aspect of the present invention, a parameter is created by obtaining a communication line utilization rate from data of a read amount of code string data per unit time at a distribution destination. Can be.
[0041]
According to a fourth aspect of the present invention, in the invention according to any one of the first to third aspects, the data per unit time of the code string data to be transmitted is determined by a parameter of an image resolution, a frame rate, or an image compression ratio. The amount can be adjusted.
[Brief description of the drawings]
FIG. 1 is a block diagram of a system configuration according to an embodiment of the present invention.
FIG. 2 is an explanatory diagram in a case where a frame rate of code string data is adjusted in the system of FIG. 1;
FIG. 3 is an explanatory diagram when adjusting the image resolution of code string data in the system of FIG. 1;
FIG. 4 is an explanatory diagram when adjusting the image resolution of code string data in the system of FIG. 1;
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 Image distribution system 2 Image reproduction system 12 Syntax analysis means 13 Code string creation means 14 Transmission means 15 Parameter instruction means 16 Parameter creation means 21 Receiving means 22 Buffer 23 Decoding means 24 Display device 25 Data reading amount detection means

Claims (6)

The moving image data is divided into one or a plurality of rectangular regions for each frame, and pixel values of each of the rectangular regions are subjected to discrete wavelet transform. Parsing means for parsing the syntax,
Code string creating means capable of simultaneously performing a plurality of tasks of creating another code string data from the code string data based on the analysis result,
Transmitting means for simultaneously transmitting each code string data created by the code string creating means to a plurality of destinations via a network;
The code string data is created by the code string creating means created according to the integrated data of the read amount per unit time of the code string data at each of the transmission destinations. Parameter instructing means for instructing creation, adjusting a data amount per unit time of the code string data transmitted by the transmitting means,
An image distribution system comprising: Receiving means for receiving the data of the read amount per unit time from each of the destinations,
Parameter creating means for creating the parameter according to the received data of each read amount,
The image distribution system according to claim 1, further comprising: The parameter creating means, when there are n destinations (n = 1, 2,...), Sets the read amount per unit time at each destination to An, and stores the code string data by the image distribution system. Assuming that the maximum data distribution capacity is B, the communication line usage rate C is
C = ΣAn / B
And the parameter is created according to the value of the usage rate C.
The image distribution system according to claim 1. The parameter used by the parameter instructing unit is a parameter related to an image resolution or a frame rate obtained when decoding the code string data, or a parameter related to a compression ratio of an image.
The image distribution system according to claim 1. Receiving means for dividing moving image data into one or a plurality of rectangular areas for each frame, and performing discrete wavelet transform of pixel values for each of the rectangular areas, and receiving hierarchically compression-coded code string data via a network When,
Decoding means for decoding the received code string data;
A data reading amount detecting unit that detects a reading amount per unit time of the code string data by the receiving unit,
Transmitting means for transmitting the data of the detected read amount per unit time to a destination of the code string data,
An image playback system comprising: An image distribution system according to claim 2,
6. The image reproduction system according to claim 5, wherein the image reproduction system receives the code string data from the image distribution system, and transmits the read amount data per unit time to the image distribution system.
An image distribution / playback system comprising:

Images