CN101883270A - A method for inserting related media in independent video streams - Google Patents

Abstract

A method of inserting related media in an independent video stream, collecting the main video stream, and the subordinate video stream to be coordinated with the main video stream, and specifying the time stamp of these multimedia information inserted into the original video stream; for the main video The stream is encoded with an encoder, and the subordinate video stream is encoded separately with an optional encoder; during encoding, the user-defined multimedia information will be encoded into some special frames, which will be compared with ordinary video frames by time stamps , rearrange the position of the frame according to a certain timing; at the receiving end, decode the received video stream, especially consider the decoding of the user's special frame; if the special frame contains data such as text, the text will be displayed in in the appropriate place in the original video stream; if the special frame contains a picture or a dependent video stream, then display that content in a new frame. The invention simulates the teaching effect of a real classroom and provides users with more information and more vivid scenes.

Description

A method for inserting related media in independent video streams

technical field

The invention relates to computer network video, especially the remote real-time transmission technology for processing multi-channel video streams.

Background technique

With the development of network technology, distance teaching has rapidly developed into an important way for people to obtain education. Remote teaching can transmit the teacher's on-site teaching scene to the class in the form of video stream in real time, or compress the video stream into a multimedia file and distribute it to the class, which is a kind of non-real-time video. However, the current video lectures cannot maintain the vividness of live lectures. In the current video lectures, most of the shots are switched between the lecturer and the lecture notes, and the teacher’s own body movements and the content of the lecture notes cannot be displayed to the audience at the same time, so a certain amount of on-site lectures is lost. of vividness. However, the traditional real-time video stream is generally a mixed transmission of one video stream and multiple channels of audio, which is not suitable for the simultaneous transmission of multiple video streams in remote teaching.

In order to show the real-time teaching scene more vividly, it is necessary to display several other subordinate media streams while playing one main media stream. The subordinate media stream is a supplement and annotation to the main media stream, that is, adding multiple shots to the real-time scene and encoding the multiple shots separately; displaying the subordinate media stream with multiple screens at the receiving end to give a vivid scene . The subordinate media stream may be information such as compressed pictures or text.

The patent "A Method for Making Remote Teaching Videos That Simulate the Effects of Real Classroom Teaching" shows a method for making remote teaching videos that simulate the effects of real classroom teaching. In this process, the blackboard video and the lecture video are respectively made, so that the two videos are completely synchronized, and the teaching effect of the real classroom is simulated. This method has certain limitations, it can only handle two video streams, but not more video streams.

The patent "Advertising Method, System and Client During Network Live Buffering" discloses a method, system and client for broadcasting advertisements during network live buffering, but the object it deals with is not for remote video teaching application, but mainly consider the fluency of network video.

Therefore, the current remote video teaching form of a single video stream can no longer meet today's demand for better effects in real-time scenes, and the real-time video transmission technology for multiple video streams needs to be developed.

Contents of the invention

The purpose of the present invention is to simulate the teaching effect of a real classroom and to meet the current demand for better effects in real-time scenes. Insert user-customized multimedia information into the video stream to realize annotations, explanations, supplements, etc. in the video stream. When applied In remote video teaching, the real-time scene of the teaching can be displayed in multiple video screens to provide more vivid videos. The video stream is combined with the user-customized multimedia information, so that the user-customized multimedia information can fully display the scene of video expression as much as possible, so that these multimedia information can express more details, and present the video stream to the user in the form of multi-screen. That is, the present invention can encode and transmit more than one video stream or picture stream, aiming at the real-time video transmission technology of multiple video streams.

Technical scheme of the present invention comprises:

A method for inserting user-customized multimedia information into an original video stream, comprising the following steps:

1. Collect the main video stream and the subordinate video streams to be coordinated with the main video stream, including text, picture or other video streams, and specify the time stamps for inserting these multimedia information into the original video stream.

2. Encode the main video stream with one encoder, and encode the subordinate video streams separately with an optional encoder. When encoding, the multimedia information customized by the user will be encoded into some special frames, and after comparing with the common video frames through time stamps, the positions of the frames will be rearranged according to a certain timing.

3. At the receiving end, decode the received video stream, especially considering the decoding of the user's special frame. If the special frame contains subtitle data such as text, display the text at a proper position of the original video stream as required; if the special frame contains pictures or subordinate video streams, display these contents in a new frame.

Beneficial effect

The method for inserting related multimedia into the video stream provided by the present invention can conveniently insert the user's customized multimedia information into the video stream, so that the display function of the video can be expanded, and the scene can be displayed through multiple pictures at the receiving end, or the details can be supplemented. Give the receiving user more information and more vivid scenes.

Description of drawings

The present invention is further described below in conjunction with accompanying drawings and examples of implementation.

Fig. 1 schematic diagram of encoding process;

Fig. 2 schematic diagram of two-way video stream decoding;

Fig. 3 Encoding structure diagram of the whole system;

Figure 4 overall effect diagram.

Detailed ways

The application implementation steps and advantages of the present invention will be illustrated below with a case of remote video teaching.

During video lectures, the video stream can have two lenses: one for the lecturer and one for the lecture notes. The actions of the lecturer are the main video stream, which we call the main media; and the lecture notes are inserted into the main video stream as multimedia information in the form of pictures for transmission, which we call the secondary stream. In the following description, a subordinate flow is taken as an example to illustrate the implementation steps:

Encode the dependent stream as a special frame in the main stream. In a general video stream, there are three types of frames: I frames, P frames, and B frames. We can customize a new type of frame: U frame, representing user multimedia frame. In this way, the dependent stream can also be encoded as a frame, and the existing frame coding technology can be used to process the dependent stream, but the U frame has nothing to do with the I, P and B frames, and each U frame is intra-frame encoded. And it is not a reference frame for other frames. The process at the encoding end is shown in Figure 1.

At the receiving end, the video stream is decoded according to the encoding method at the sending end. The decoded video stream is displayed in two parts, one screen shows the lecturer's actions, and the other screen shows the lecture notes. In this way, the lecturers can not only see the body language of the lecturer, but also watch the contents of the lecture notes. (See Figure 4)

The schematic diagram of the decoding process is shown in Figure 2:

When encoding multiple video streams, more video frames can be defined, such as U1, U2, etc., and the process of encoding and decoding is similar to that of two video streams. The advantage of this is that all user-attached data are treated as special frames, which are intra-frame coded, and can use different encoding and decoding methods from the main media, which has great flexibility and makes multi-channel The video streams are processed without interfering with each other.

The invention has strong expansibility. There is no limit to the type of data transmitted by special frames. Therefore, we can use special frames to expand the number of video stream channels that can be transmitted; we can also use special frames to carry out other data services, such as some control commands. (see Figure 3)

The present invention is an extension based on the current coding scheme and is compatible with the current coding scheme. The encoding of one video stream is a special case of the encoding scheme of the present invention.

Claims (3)

1. a method of inserting related media in independent video streams is characterized in that, may further comprise the steps:

1), gather main video flowing, and subordinate video flowing that will be collaborative with main video flowing comprises style, picture or other video flowing, and specifies the timestamp in the original video stream of being inserted into of these multimedia messagess;

2), utilize a kind of encoder to encode, the optional encoder of video flowing utilization of subordinate is encoded separately to main video flowing; The multimedia messages of customization will be encoded into some special frames when coding, after the stamp contrast of common video frame elapsed time, rearrange the position of frame according to certain time sequence;

3), at receiving terminal, the video flowing of receiving is decoded, consider the decoding of user's special frames especially; If what this special frames comprised is caption data such as text, then as required with text display on the correct position of original video stream; If what this special frames comprised is picture or subordinate video flowing, then in a new picture, show these contents.

2. method according to claim 1, it is characterized in that: in the described step 2, when multi-path video stream is encoded, can define more frame of video, the data that all users add are all treated as special frames, these frames carry out intraframe coding, can use the encoding and decoding means different with main medium, and the process of Code And Decode and two tunnel video stream class are seemingly.

3. method according to claim 1 is characterized in that: in the described step 3, the data type that described special frames transmits can be extended for transmissible video flowing number of active lanes without limits; Also can carry out other control command data business.

Images