JP2002199344A - Multi-media information transmitting server - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a server which does not transmit useless data to a client when special reproduction is performed, such as fast-forwarding or rewinding, in streaming transmission. SOLUTION: A multi-media information transmitting server is provided with a transmission control processing part 14 which retrieves a reproduction start possible point storage part 12 based on reproduction control information which is transmitted from a client terminal 2, calculates a reproduction possible point being the nearest to a reproduction desired point and, then, transmits multi- media information from the calculated reproduction possible point. Reproduction control information is information corresponding to a reproduction point desired by the client terminal. The storage part 12 stores the position of a multiplex packet comprising a frame (I-Frame) which is included in video information and performed in-frame encoding or the number of the multiplex packet together with reproduction time information.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a multimedia information transmitting server apparatus for streaming and transmitting multimedia information in which video and audio are multiplexed.

[0002]

2. Description of the Related Art As one form of multimedia information transmission for transmitting audio, video, data, and the like, multimedia information is stored in a server device in advance, and the information is transmitted in response to a request from a client terminal. is there. At this time, on the client terminal side, a method of starting reproduction after receiving all desired multimedia information,
There is a method of starting reproduction before receiving all desired multimedia information, and performing reception and reproduction of information in parallel. In particular, the latter method uses streaming transmission,
This is what is generally called streaming playback.

Generally, when transmitting multimedia information,
It is not realistic to send original information such as video and audio as it is because the data amount becomes very large. Therefore, it is common to transmit the data by compressing it by some method and to expand it at the time of reception. Also, when comparing video and audio, the amount of information of the video is much larger than that of audio, so compression of video information is essential.

[0004] There are various video information compression methods, and standard compression methods such as MPEG-2 and MP are used.
Something like EG-4 is used. As these features, it is possible to use not only a method of encoding frame information which is original information of a video but also a method of encoding a next frame using information of a certain encoded frame. This utilizes the redundancy of the video information in the time axis direction, and plays a large role in information compression.

For example, in compression using MPEG-4, certain frame information is coded independently (this is called an I frame: intra frame), and the next frame is
Only the result of calculating the difference based on the information of the frame is encoded (this is called a P frame: predictive frame), and the next frame is repeated by repeating the same processing as described above based on the information of the P frame. In addition, information is compressed by suppressing the redundancy in the time axis method.

Here, since the P frame encodes only the difference information from the previous frame, the information amount per frame becomes small. On the other hand, in the case of an I frame, since the encoding is performed based on only the information of the corresponding frame, the amount of information per frame is large. Therefore, when reducing the transmission rate with the same quality, it can be said that the smaller the number of frames to be encoded with I frames, the better the image quality as a whole.

[0007] Considering the case of reproducing from the middle of the multimedia information, the I frame can be reproduced from the I frame because the information is composed of a single frame. On the other hand, since the P frame is encoded based on the difference information from the previous frame,
It is not possible to start playback from a P frame.

Therefore, when reproducing from the middle of the multimedia information, if the point desired by the user to reproduce is coded with a P frame, normal reproduction is performed until the next I frame in the multimedia information appears. No display is made. This can be a source of dissatisfaction for the user in that the reproduction is instructed but not reproduced. In addition, in transmitting unreproducible information via a network, the user is charged for useless information, which is an undesirable situation for the service receiving side.

In streaming playback in a conventional system, a server device that receives a request for desired multimedia information from a client terminal transmits the requested information to the client terminal, while the client terminal side receives the received multimedia information. Only the media information is received and played back, and it is difficult to execute special playback such as pause, fast forward, and rewind.

[0010] These special reproductions can be handled by using a non-standard protocol between the server device and the client terminal. However, in recent years when various platforms exist, a specific server device cannot be specified. Using a protocol that can be used only by the client terminal has been inconvenient for the user.

[0011]

As described above, in the conventional multimedia information transmission, at the time of the special reproduction in the streaming transmission type, the reproducible point and the desired reproduction point do not match, so that the reproduction from the desired reproduction point is not performed. Until the playback point, like the previous image or black screen,
The only way for the client terminal to look at the images was meaningless.

Further, since data between the desired reproduction point and the reproducible point, which is meaningless for the client terminal, has been transmitted, a wasteful transmission data amount occurs, which affects the communication amount to be borne by the client. I was

An object of the present invention has been made in view of the above circumstances, and when performing special reproduction such as fast forward or rewind in a streaming transmission type, reproducible data can be transmitted without transmitting useless data. It is to provide a multimedia information transmission server device.

[0014]

In order to achieve the above object, a multimedia information transmitting server apparatus of the present invention comprises:
It stores multimedia information in which video and audio are multiplexed, and transmits the stored information to a client terminal, and stores a reproduction start possible point in the multimedia information. And reproduction control means for receiving reproduction control information from the client terminal and transmitting information from a reproducible point. Here, the reproduction control information is information corresponding to a reproduction point desired by the client terminal, and the reproduction control means searches the reproduction startable point storage unit and finds a reproduction possible point closest to the reproduction desired point. Is calculated, and multimedia information is transmitted from the calculated reproducible point. The reproducible point in the multimedia information is a position of an intra-coded frame included in video information in the multimedia information, that is, a position of a multiplexed packet including an I-Frame or a multiplexed packet number.

By employing the above configuration, it is possible to start transmission of multimedia information from a reproduction start point close to a desired reproduction point, so that special reproduction such as fast forward and rewind can be performed even in a streaming transmission type. In doing so, there is no need to send useless data to the client.

[0016]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram showing a configuration of a multimedia information transmission system using the multimedia information transmission server device of the present invention. As shown in FIG.
Via a network, a multimedia information transmitting server device 1 and a multimedia information receiving client terminal 2
Exists.

The multimedia information transmitting server device 1 comprises:
A multimedia information storage unit 11 (denoted as “MM information storage” in the figure), a reproducible point management database 12 (denoted as a reproducible point management DB in the figure), a control information reception processing unit 13, and a transmission control processing unit 14 Have. Further, the multimedia information receiving client terminal 2 includes a multimedia information receiving processing unit 21 (in the figure, an “MM information receiving processing unit”).
, A playback control input processing unit 22, a video information output unit 23, an audio information output unit 24, and a playback control method display unit 25.

The multimedia information storage unit 11 has the configuration shown in FIG.
Multimedia information having the structure shown in FIG. FIG. 2 shows an example in which one video stream and one audio stream are multiplexed to generate one multimedia stream (denoted as “multiplexed stream” in the figure). In FIG. 2, V1, V2, and V3 denote packets obtained by encoding a video stream in frame units, and A1, A2, A3,.
Indicates packets obtained by encoding the audio stream in frame units.

The coded video stream includes an intra-frame (I-Frame) coded using only the information in the frame and a frame (I-Frame) coded using the information of the previously coded frame. P-Frame). Since the P-Frame can compress the redundancy using the information of the previously encoded frame, the information after the encoding can be made smaller than that of the I-Frame. In the example of FIG.
Let I-Frame, V2, and V3 be P-Frames. It is also assumed that the audio stream is encoded as a fixed length within a fixed time. When reproducing a video stream encoded in this way, I-Frame information is essential. Then, the head of the I-Frame in the stream is set as the reproduction start point.

On the other hand, an audio stream is not encoded using information between frames, so that it is not necessary to be particularly aware of the reproduction start point.

The middle part of FIG. 2 shows a configuration example of a multimedia stream obtained by multiplexing the video stream and the audio stream. The multiplexed stream is composed of packet units, and a header indicating how the video and audio streams are multiplexed is placed at the beginning of each packet, and an audio packet and a video packet are placed in a payload portion following the header. Are multiplexed by an appropriate method. In the example of FIG. 2, a V1 packet, which is an I-Frame, is divided and multiplexed into a plurality of multiplexed packets M1 and M2, and a V2 packet is multiplexed into M3.

The playable point database 12 has a function of detecting a playable point in the multimedia information, that is, a position of an I frame, and storing the detected position.

The control information reception processing section 13 has a function of receiving reproduction control information from a client. In this embodiment, as a means for transmitting information from the client,
Use POST method specified in HTTP. Of course, any means may be used as long as the means transmits the playback control information from the client to the server.

The transmission control processing unit 14 detects the reproduction start point using the information in the database 12 based on the control request from the control information reception processing unit 13, and reproduces the information stored in the information storage unit 11. Has a function to send from the starting point.

On the other hand, the multimedia information reproduction processing unit 21 in the multimedia information receiving client terminal 2
Has a function of separating multimedia information received from the multimedia information transmitting server device 1 via the network into video and audio information. The separated video information is reproduced in the video information output unit 23, and the separated audio information is reproduced in the audio information output unit 24.

The reproduction control method display section 25 has a function of displaying information for the user to control the reproduction method. For example, button images corresponding to the respective functions are displayed in order to allow the user to perform reproduction control instructions such as reproduction start, reproduction stop, pause, and skip. In addition, a slide bar is displayed as a function for enabling reproduction from an arbitrary position. FIG. 3 shows a display example.

The reproduction control input processing section 22 receives the control method designated by the user based on the reproduction control method display section 25 and sends the control method to the multimedia information transmitting server 1.
It has a function to send necessary instructions.

The operation of the multimedia information transmitting server device 1 and the multimedia information receiving client terminal 2 based on the above configuration will be described below.

Now, streaming reproduction is considered as a method in which the user of the multimedia information receiving client terminal 2 receives and reproduces the multimedia stream stored in the multimedia information transmitting server device 1.

When streaming playback is selected, the multimedia information receiving client terminal 2 can start playback before receiving the entire multimedia stream. Therefore, the multimedia information receiving client terminal 2 does not need to store the entire multimedia stream, so that the storage area can be reduced.

The multimedia information receiving client terminal 2 separates the multimedia stream received from the multimedia information transmitting server device 1 into video and audio information in the multimedia information reproduction processing unit 21.
The separated video information is reproduced in the video information output unit 23, and the separated audio information is output from the audio information output unit 2.
4 is played.

Here, at the time of reproduction, if the total length of the multimedia stream is known in advance, the place where the reproduction is being performed can be indicated on the slide bar shown in the reproduction control method display section 25. FIG. 4 shows an example of the playback location display on the slide bar.

FIG. 4 shows an example of reproducing a stream having a total length of 600 seconds. In this case, the end point A of the slide bar
Indicates the head of the stream, and endpoint B indicates the end of the stream. If the playback time from the end point A to the end point B is 600 seconds, and the playback point is located 100 seconds from the start,
Since one sixth of the whole has been reproduced, the sixth point of the entire slide bar from the end point A is displayed as a reproduction point. Similarly, if you are playing the position of 200 seconds,
From the end point A on the slide bar, two-sixth points of the entire slide bar are displayed as reproduction points.

Here, it is assumed that the user wants to reproduce from the point of 100 seconds when the reproduction point is at the position of 500 seconds. Hereinafter, this is referred to as “skip reproduction processing”. At this time, the user shifts the slide bar of the reproduction control method processing unit 25 to the position of 100 seconds to instruct the reproduction control input unit 22 to reproduce from the position of 100 seconds. That is, the skip reproduction process is instructed to skip to the position of 100 seconds.

When the playback control input processing unit 22 recognizes that the playback is to be performed from the 100 second point, the playback control input processing unit 22 informs the multimedia information transmitting server apparatus 1 of "skip to the 100 second point" as shown in FIG. Transmit playback control information.

FIG. 5 shows an example of the reproduction control information. As shown in FIG. 5, “reproduction time” is used as the skip destination information.
Any of “packet number” and “stream position” can be used. This means that the playback point recognized by the playback control input unit 22 may be managed by any of the playback time, packet number, and stream position. The “judgment method” in FIG. 5 will be described later.

The control information reception processing unit 13 of the multimedia information transmission server device 1 detects that the request from the client is “skip to the 100 second point” by receiving / analyzing the transmitted reproduction control information. Then, the fact is transmitted to the transmission control processing unit 14.

In the transmission control processing section 14, the transmitted "10
When "skip to 0 second point" information is received,
The transmission process of the multimedia stream is interrupted, and the transmission start point is searched based on the information specified by the client.

FIG. 6 shows the configuration of a multimedia stream according to the present embodiment. FIG. 6 shows the configuration of the entire stream that requires 600 seconds of playback time. The stream is composed of a packet unit in which a video stream and an audio stream are appropriately multiplexed. Further, FIG. 6 shows the vicinity of the 100 second point as the skip destination, that is, 90 seconds.
In addition to showing the packet configuration from the second point to the 110 second point, it shows the location where the I-Frame is encoded in the video stream in the packet. In other words, the video stream encoded by I-Frame exists in the packet at the 99-second point and the packet at the 107-second point. It is assumed that all video streams included in other multiplexed packets are encoded in P-Frame.

In the present embodiment, the transmission control processing unit 14, which has received the "skip to 100 second point" information, interrupts the multimedia stream transmission processing and starts transmission based on the information specified by the client. Search for points. Specifically, by comparing the playable point information stored in the multimedia playable point database 12 with the skip destination point, the I-Fram, which is the playable point, is compared.
Detect the position of the packet containing e.

FIG. 7 shows an example of information stored in the multimedia playable point database 12. FIG. 7 shows a packet number including an I-Frame as a reproducible point, a reproduction time of the packet number, and a position of the packet from the head of the stream. What is unique here is the I-Frame
And the reproduction time is stored as data in association with the packet number including the packet number and the position of the packet from the head of the stream.

In FIG. 7, when skipping to the 100 second point, a process for searching the nearest reproducible point from the database is executed. If the skip destination point matches the reproducible point, the point can be set as the reproducible point. If not, the optimum reproducible point is searched from the specified skip destination. There is a need. Here, the search condition is “select a reproducible point immediately before the specified skip destination point”. In this case, the reproduction time field is compared with the skip destination point, and the 99 second point is selected from the registered data. Therefore, it is sufficient to transmit the packet of the packet number 990, that is, the stream from the 792000 byte point from the head if the expression is changed.

By executing this processing, the multimedia receiving client receives the stream after 99 seconds. I-Fram for 99 seconds packet
Since the video stream encoded by e exists, it can be immediately reproduced.

On the other hand, as another search condition, “select the reproducible point closest to the designated skip destination point” is also possible. This is the case where the item of “judgment method” in the reproduction control information of FIG. 5 is “recent”. For example, when the 104 second point is selected as the skip destination point, the playback time field is compared with the skip destination point, and the playable point immediately before the skip destination point and the playable point immediately after the skip destination point are determined. Extract.

In this example, the 99 second point is extracted as the immediately preceding reproducible point, and the 107 second point is extracted as the immediately succeeding reproducible point. Calculating the time difference between the skip destination and these two playable points, the previous playable point is 4 seconds,
It turns out that there is a difference of 3 seconds from the reproducible point immediately after,
As a result, the reproducible point closest to the skip destination point is
It is required to be 107 seconds. Therefore, in this case, it is sufficient to transmit the packet of the packet number 1070, or in other words, the stream starting from 856000 bytes from the beginning.

The above search conditions are merely examples, and other conditions may be provided. The selection of the condition may be left to the server device side, or the client may have a selection function, and may select which condition to use according to a user's instruction. This can be realized by providing an item of “judgment method” in the above-described reproduction control information shown in FIG. 5 and allowing the user to select it. In the embodiment described above,
Although an example has been described in which the optimum reproduction point is obtained using the time information in the reproduction control information, the optimum reproduction point may be obtained based on the packet number or the stream position.

In addition, various modifications can be made without departing from the gist of the present invention.

[0049]

As described above, according to the multimedia information transmitting server apparatus of the present invention, when performing special reproduction such as fast forward and rewind in the streaming transmission type, unnecessary data is not transmitted to the client. , And reproducible data can be transmitted.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of a multimedia information transmission system using a multimedia information transmission server device of the present invention.

FIG. 2 is a diagram showing a structure of multimedia information stored in a multimedia information storage unit 11 in the multimedia information transmission server device.

FIG. 3 is a view showing a display example on a reproduction control method display section 25 in the multimedia information receiving client terminal 2;

FIG. 4 is a diagram showing a display example of a reproduction control method display unit 25.

FIG. 5 is a diagram showing a configuration of reproduction control information supplied from the reproduction control input processing unit 22 to the multimedia information transmitting server device 1.

FIG. 6 is a diagram showing a configuration of a multimedia stream in the embodiment.

FIG. 7 is a diagram showing an example of information stored in a multimedia reproducible point database 12 in the multimedia information transmission server device 1.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 ... Multimedia information transmission server apparatus 11 ... Multimedia information storage part 12 ... Reproducible point management database 13 ... Control information reception processing part 14 ... Transmission control processing part 2 ... Multi Media information receiving client terminal 21 Multimedia information receiving processing unit 22 Playback control input processing unit 23 Video information output unit 24 Audio information output unit 25 Playback control method display unit

 ────────────────────────────────────────────────── ─── Continued on the front page (72) Inventor Jun Sato 70 Yanagimachi, Saiwai-ku, Kawasaki-shi, Kanagawa F-term in Toshiba Yanagicho Plant (reference) 5C053 FA23 GB29 HA21 JA24 LA06 LA07 LA11 LA15 5C064 BA07 BB05 BC01 BC10 BC18 BC23 BD02 BD03 BD05 BD08 BD14

Claims (4)

[Claims] 1. A multimedia information transmitting server device for storing multimedia information in which video and audio are multiplexed and transmitting the stored information to a client terminal. A multimedia information transmission server device, comprising: a reproduction start possible point storage unit for storing the reproduction control information from the client terminal and transmitting information from the reproduction possible point. 2. The reproduction start point storage unit stores a position of a multiplexed packet including a position of an intra-coded frame included in video information multiplexed on the multimedia information, as a reproduction start point. 2. The multimedia information transmitting server device according to claim 1, wherein the reproduction start possible point is stored in association with a reproduction time corresponding to a position of the multiplexed packet. 3. The reproduction control means searches a reproduction start possible point stored in the reproduction start possible point storage unit based on reproduction control information from the client terminal, and obtains a reproduction start possible point obtained as a result of the search. 2. The multimedia information transmitting server device according to claim 1, wherein the multimedia information is transmitted starting from a multiplexed packet including the following. 4. The reproduction start possible point storage unit stores the number of a multiplexed packet including the position of an intra-coded frame included in the video information multiplexed with the multimedia information as a reproduction start possible point. 2. The multimedia information transmitting server device according to claim 1, wherein the reproduction start possible point is stored in association with a reproduction time corresponding to the number of the multiplexed packet.