WO2011029369A1

WO2011029369A1 - Video encoding and decoding method, system and video monitoring system

Info

Publication number: WO2011029369A1
Application number: PCT/CN2010/076353
Authority: WO
Inventors: 张莉
Original assignee: 中兴通讯股份有限公司
Priority date: 2009-09-08
Filing date: 2010-08-25
Publication date: 2011-03-17
Also published as: CN101692711A

Abstract

A video encoding and decoding method, system and a video monitoring system are provided in the present invention, all of which can make sure that videos are divided into frame sections by an encoding end, data frames are extracted from the frame sections and a sub-code stream group is formed according to a time sequence by the encoding end; the first frame of the sub-code stream group is encoded by means of an I-frame; a code stream group is composed of the encoded sub-code stream group according to a time sequence; the code stream group is obtained by a decoding end, and the data frames contained in the code stream group are decoded sequently by the decoding end. The video decoding and encoding method, system and video monitoring system in the present invention can be applied for multiple network environments and device performances, and remarkably improve the effect of video play.

Description

Video codec method, system and video monitoring system

The present invention relates to the field of multimedia technologies, and in particular, to a video encoding and decoding method, system, and video monitoring system. Background technique

The encoding method of the video generally includes an I frame, a P frame, and a B frame, wherein the I frame is a self-reference frame (also called an intra-coded frame), that is, the encoding of the I frame does not refer to other data frames, and accordingly, it can be independent. The data of other frames is independently decoded; the P frame is a unidirectional reference frame, that is, the encoding of the P frame needs to refer to the previous one or several frames of the timing, and accordingly, it needs to rely on the decoded data of the previous one or several frames. The decoding is performed; the B frame is a bidirectional reference frame, that is, the encoding of the B frame requires reference to the preceding and following data frames of the timing, and accordingly, it needs to rely on the preceding and following decoded data for decoding. Generally, in view of the continuity and real-time performance of video stream transmission, in order to reduce the delay, the real-time video playback system usually only uses the common encoding method of I and P frames. To further prevent data loss, periodicity is preferred. The I frame coding mode, that is, the I code is set at intervals of the code stream, and a plurality of P frames are set after each I frame. However, in many cases, the video stream usually needs to be transmitted between the encoding end and the decoding end through different communication methods. Therefore, factors such as communication mode, network bandwidth, network status, and terminal device performance will inevitably affect the quality of the stream transmission. And decoding quality, if one or more I frames are lost or damaged during the stream transmission, then multiple P frames cannot be decoded smoothly, thus affecting the display effect of the video.

For example, in the video surveillance system shown in FIG. 1, a front-end monitoring device installed in a monitoring place such as a camera or a cloud platform encodes the collected video data, and forwards the code stream to each client through a streaming server, and different clients. The terminal can use different terminal devices such as mobile phones, desktop computers, and notebook computers. When different clients obtain the code stream from the streaming server, they also receive different networks. The limitations of the network environment. For example, some clients access services through high-speed Ethernet access, some clients access services through medium-speed modulation regulators, and some users can only access services through low-speed unlimited networks due to conditions. These different The access mode results in different network bandwidths. Even if the client uses the same access mode, it is possible to have different network bandwidths at different times due to different network conditions. Therefore, a video code stream with a constant frame rate is in the network bandwidth. In the case of large fluctuations, video data may be lost or delayed. For example, when video decoding, video decoding is usually performed from the I frame. If any I frame is lost or corrupted, it may affect all subsequent P frames. decoding. Some clients also have poor performance of the device, and the code stream processing and display capabilities are poor. Therefore, when the video is played, the client may have serious problems such as delay, flower screen, mosaic, etc., and the clear and smooth playback effect cannot be achieved. Therefore, the existing video codec mode cannot adapt to various network environments and device capabilities of the terminal, and it is difficult to meet the actual needs of the user. Summary of the invention

The technical problem to be solved by the present invention is to provide a video encoding and decoding method, system and video monitoring system that adapt to various network environments or device performances and improve video playback quality.

The technical solution adopted by the present invention to solve the technical problems thereof is:

A video encoding and decoding method includes the following steps:

The encoding end decomposes the continuous video into frame segments, extracts data frames from the frame segments and forms sub-stream groups in chronological order; and encodes the first frame of the sub-code stream group in an I-frame manner And composing the encoded substream group into a code stream group in chronological order;

The decoding end acquires the code stream group, and sequentially decodes the data frames included in the code stream group.

The coding end encodes the other data frames in the substream group as follows:

釆 Use the first frame as a reference and encode in the form of a P frame.

When the data frame is extracted from the frame segment, the extracted data frame is the first frame of the frame segment. The method of decomposing continuous video into frame segments is: decomposing continuous video into frame segments that are connected end to end and containing equal number of frames;

A method of extracting data frames from the frame segments is: extracting data frames at medium intervals from the frame segments.

The method for extracting a data frame from the frame segment is: extracting at least one data frame from the frame segment, and forming a code stream group each time the extracted data frame is formed;

The method for constituting the code stream group in the chronological order is: selecting a sub-stream group corresponding to all the frame segments and forming the code stream group, and selecting at least one code stream group after multiple selections;

The manner in which the decoding end acquires the code stream group is: The decoding end acquires at least one code stream group according to a specific request.

The method further includes:

The decoding end requests to obtain a data group from the decoding end according to the relationship between the real-time delay value and the preset delay threshold.

The decoding end sequentially decodes the data frames included in the code stream group by: the decoding end sequentially decodes the data frames included in some or all of the sub-code stream groups in the code stream group.

Each frame segment includes an odd number of data frames, and the process of extracting data frames from the frame segments and forming a substream group includes:

Extracting a first frame of each frame segment as a first frame of each corresponding substream group;

Extracting odd frames other than the first frame from the frame segments and forming odd sub-stream groups in chronological order, extracting even frames other than the first frame from the frame segments, and forming even sub-streams in chronological order Group

Forming the odd subcode stream group into a singular digital stream group in chronological order, and constituting the even subcode stream group into an even digital stream group in chronological order;

The process of decoding the data frames included in the code stream group by the decoding end includes: The decoding end acquires the odd digital stream group and/or the even digital stream group according to the specific request, and sequentially decodes the data frames in the odd digital stream group and/or the even digital stream group.

Each frame segment contains 25 consecutive data frames, and each frame segment corresponds to a time of 1 second. A video codec system includes an encoding end and a decoding end, the encoding end includes a frame segment decomposing module and a block encoding module, and the decoding end includes a code stream acquiring module and a packet decoding module; Decomposing continuous video into frame segments;

The packet encoding module is configured to: after extracting data frames from the frame segment and forming a sub-code stream group in time sequence, encoding, by using an I frame, the first frame of the sub-code stream group, and encoding The subsequent sub-stream groups form the code stream group in chronological order;

The code stream obtaining module is configured to acquire the code stream group from the encoding end, and send the code stream group to a packet decoding module;

The packet decoding module is configured to sequentially decode data frames included in the received code stream group.

A video surveillance system includes a front-end monitoring device, a streaming server, and a client, the front-end monitoring device is provided with an encoding end, and the encoding end includes a frame segment decomposition module and a packet coding module; The decoding end includes a code stream acquisition module and a packet decoding module, and the frame segmentation module is configured to decompose the video collected by the front-end monitoring device into a frame segment; the packet coding module is configured to use the frame The data frame is extracted in the segment and the sub-stream group is formed in chronological order, and the first frame of the sub-stream group is encoded in an I-frame manner, and the encoded sub-stream group is formed into a code stream in time sequence. Group, the code stream group is further provided to the stream server;

The code stream obtaining module is configured to acquire the code stream group from the stream server by using a communication network;

The packet decoding module is configured to sequentially decode the data frames included in the code stream group acquired by the code stream obtaining module. The invention has the beneficial effects that the video de-encoding method and system of the invention reduces the amount of code stream transmission in the communication network and improves the transmission efficiency, and is therefore applicable to various network environments and device performances; When the data frame in the decoding is decoded, since the P frame in each sub-code stream group only depends on the I frame or the P frame in the group for decoding, the influence of the loss of the individual data frame on the decoding of other data frames is avoided, so the improvement is improved. The video playback effect.

Further, the encoder extracts at least one data frame from the frame segment at a medium interval, and each extracted data frame forms a different sub-code stream group, and the implementation method extracts the data frame from the frame segment to obtain continuous video playback. effect. Moreover, the decoding end can obtain part or all of the code stream according to the network environment or device performance, so that the code stream transmission and decoding are adapted to the network capability and the terminal capability, and the method is more flexible and convenient, thereby further ensuring the video transmission and playback effect.

Furthermore, the video surveillance system of the present invention can adapt to different network environments and device performances. If the network status is not ideal, the processing capacity of the client is large, or the preview requirement is not high, the streaming server only sends the client only to include The code stream group of some data frames in the original video not only improves the transmission efficiency, but also ensures the continuity and real-time performance of the video playback, and improves the video monitoring effect. DRAWINGS

1 is a schematic diagram of an existing video monitoring system;

2 is a flow chart of a specific implementation manner of a video codec method according to the present invention;

3 is a schematic diagram of a video monitoring system of a video codec system according to the present invention;

4 is a schematic diagram of a specific implementation manner of a video monitoring system according to the present invention. detailed description

In the video de-encoding method of the present invention, after the encoding end encodes and decodes the continuous video, the code stream group including only a part of the data frames in the original video is provided to the decoding end according to the network environment and device performance, and the decoding is performed. When the data frame in the code stream group is decoded, the P frame in each sub-stream group depends on only the I frame or the P frame in the group for decoding. The present invention is described below with reference to the accompanying drawings. The detailed description is further described.

2 is a flow chart of a specific implementation manner of a video encoding and decoding method according to the present invention, including the following steps:

Step S100: The encoding end decomposes the continuous video into at least one frame segment, and the decomposition manner and length of the frame segment may be determined according to specific needs. For example, multiple frame segments may be randomly extracted from consecutive videos, and each frame segment includes different frames. The number of frames, or a certain data frame between adjacent frame segments; preferably, in order to achieve the continuity of the code stream in time as much as possible, to ensure the display effect of the video, the continuous video is decomposed into end-to-end, including A frame segment of equal number of frames. Usually, the number of data frames that the human eye can see in 1S is about 25 frames, and the frame rate of mainstream video is also 25 frames per second. Therefore, the time corresponding to each frame segment can be set to one second, that is, each frame segment. Contains 25 data frames.

Step S101: extracting data frames from the frame segments and forming substream groups in chronological order. In this step, there are many ways to extract data frames from a frame segment. For example, one or more data frames can be randomly extracted, multiple data frames can be extracted at equal intervals, and the extracted data frames form a subcode in time sequence. Stream group. In addition, in order to meet various needs of the user, the encoding end may also extract data frames from the frame segment multiple times, and correspondingly form a plurality of different sub-stream groups. Preferably, in order to uniformly extract data frames from the frame segments, a continuous video playback effect is obtained, and the encoder extracts at least one data frame from the frame segments at intervals, and each extracted data frame forms a different substream group. For example, the extracted sequence numbers are 1, 4, 7, 10...3n+l (n is a positive integer), and the data frames are composed of the first sub-stream group, and the extracted numbers are 2, 5, 8, 11... A data frame of 3n+2 (n is a positive integer) constitutes a second substream group.

Step S102: Encoding the first frame of the subcode stream group in an I frame manner. Since the compression ratio of the I frame is much smaller than the P frame, the data amount of the I frame is large. In order to achieve a better transmission effect, this step encodes the first frame of one or more substreams in an I frame manner. For all other data frames in the sub-stream group, the first frame is used as a reference, and the P-frame is used to encode, that is, each sub-stream group contains only one I-frame. Preferably, it may be selected that the first frame of each frame segment must be extracted, Taking the first frame of the corresponding sub-stream group, if a plurality of sub-stream groups are formed after extracting the data frame multiple times from the frame segment, the first frame of each sub-stream group can share the first frame of the corresponding frame segment. frame.

Step S103: constituting the coded sub-stream group into a code stream group according to chronological order. If only one data frame is extracted from each frame segment to form one sub-stream group, only one code stream group is formed; if from each frame The segment extracts the data frame multiple times and extracts one sub-stream group each time. Then, a part of the sub-stream groups corresponding to all the frame segments are selected to form one code stream group, and at least one code stream group is obtained after multiple selections. For example, in step S101, the first sub-stream groups corresponding to all the frame segments are combined to form a first code stream group, and the second sub-code stream groups corresponding to all the frame segments are combined to form a second code stream group, and so on.

Step S104: The decoding end acquires a code stream group. In this step, since the code stream group obtained by the decoding end only includes a part of the data frame in the original video, the amount of data transmitted through a communication network such as a wired network or an Ethernet is obviously higher. Small, so the method is applicable to a variety of network environments and terminal devices, such as some slower network transmission conditions or poor performance of terminal devices, improving data transmission speed and quality.

In order to adapt to the different needs of various users, that is, some clients have better network environment or higher configuration of terminal devices, so a complete code stream is required; some clients have unstable network conditions, and a complete code stream is required when the network is good. When the network is poor, only one or more code stream groups are needed, so the decoder can request partial code stream groups or all code stream groups according to specific requirements. For example, the decoding end can set an acceptable delay threshold, and according to the relationship between the real-time delay value and the delay threshold, it is determined that at least one code stream group is obtained from the decoding end, and when the decoding end needs to acquire the code stream, first According to the network environment and device performance, processing capability, etc., if the real-time delay value is less than the delay threshold, request to obtain a code stream group or a small number of partial code stream groups, otherwise, request a larger number of parts. Code stream group or all code stream groups.

Step S105: sequentially decoding the data frames included in the code stream group, that is, after decoding the I frame in each sub-stream group, the P frames are all dependent on the I frame or other P frames for decoding. In this step, since the decoding of the p frame in each substream group depends only on the decoded data of the I frame or the P frame in the group. Decoding, if an I frame loss occurs, it will only affect the decoding of P frames in the group, and will not affect the decoding of data frames in other sub-stream groups, so the decoder can choose to give up some incomplete sub-segments. The code stream group, and the data restoration in the subsequent sub-stream group is not affected at all, so the partial loss of the data frame is effectively avoided, and the display effect such as the flower screen is poorly improved, and the quality of the video playback is improved to some extent. . The encoding end can also decode some or all of the obtained code stream groups according to specific needs. If the user has low requirements for playing video, or the device performance is poor, or need to run multiple programs at the same time, in order to realize real-time and complete video. Playback (especially for video surveillance systems that require real-time performance and low playback requirements), it is also possible to decode only the data frames in the inner code stream group of the code stream group. After changing the frame rate, it can be realized. Real-time, continuous video playback.

In a preferred implementation manner, the decoding end performs packet decoding on the continuous video to obtain two code stream groups, and the user may request to obtain one or two code stream groups according to specific needs, which specifically includes the following steps:

First, the continuous video is decomposed into a plurality of consecutive frame segments, and each frame segment contains an odd number of data frames. For example, each frame segment contains 25 consecutive data frames, and the corresponding time is 1 second.

Then, the first frame of each frame segment is used as the first frame of each corresponding sub-stream group, and the odd frames other than the first frame are extracted from each frame segment and the odd sub-stream groups are formed in chronological order. Then, even frames other than the first frame are extracted in each frame segment and an even sub-stream group is formed in chronological order. For example, if each frame segment contains 25 consecutive data frames, the sequence numbers other than the first frame are extracted as follows.

3, 5 23 12 data frames, and form odd sub-stream groups in chronological order, the coding method is:

IPPPPPPPPPPPP;

Then, 12 data frames whose sequence numbers are 2, 4, 6, . . . 24 are sequentially extracted, and the even sub-stream groups are formed in chronological order, and the coding mode is the same as the odd sub-code stream group. Then, all the encoded odd sub-stream groups are formed into singular digital stream groups in chronological order, and all encoded even sub-stream groups are chronologically composed into even digital stream groups. Finally, the decoding end specifically requests to acquire the odd digital stream group and/or the even digital stream group according to network conditions, device conditions, and the like, and sequentially decode the data frames in the odd digital stream group and/or the even digital stream group. According to the conventional coding mode, the frame rate of the normal code stream is 25 frames per second, and the frame rate of the code stream group is 13 frames per second, and the amount of data to be transmitted in the communication network is required. Reduced to about half of the original, so it can adapt to a variety of network environments and device performance, and because the data frames are extracted from the original video and form a stream group, the data information contained in each stream group maintains the original video. The continuity, thus significantly improving the display of video playback.

FIG. 3 is a schematic diagram of a video codec system according to the present invention, which includes an encoding end 10 and a decoding end 20. The decoding end 20 can be configured according to specific needs, wherein the encoding end 10 includes a frame segment decomposing module 11 And a block coding module 12, each of which includes a code stream acquisition module 21 and a packet decoding module 22.

The frame segmentation module 11 is configured to decompose the continuous video into at least one frame segment.

The packet encoding module 12 is configured to extract the data frame from the decomposed frame segment and form the sub-code stream group in time sequence, and then encode the first frame of the sub-code stream group in an I frame manner, and also use the coded The substream group constitutes a stream group in chronological order. The block coding module 12 may also form a plurality of sub-stream groups after extracting data frames multiple times from each frame segment, and select sub-stream groups multiple times to form different code stream groups.

The code stream acquisition module 21 is configured to acquire a code stream group from the encoding end 10 through a communication network such as a wired network, an Ethernet network, a 3G network, or a wireless network, and send the code stream group to the packet decoding module 22. Considering the need to adapt to different network conditions or device performance, the code stream acquisition module 21 can obtain some or all of the code stream groups as needed.

The packet decoding module 22 is configured to sequentially decode the data frames included in the code stream group, and may decode some or all of the code stream groups according to device performance and specific needs, or include a code stream group in the middle molecular code stream group. The data frame is decoded. 4 is a schematic diagram of a video surveillance system according to the present invention, including a front-end monitoring device, a stream server 30, and a client. The front-end monitoring device includes a camera, a cloud station, and the like, and is further provided with an encoding end 10, and the encoding end 10 includes a frame segment decomposition. The module 11 and the packet encoding module 12; the client is usually a laptop computer, a desktop computer, a mobile phone, a palmtop computer, etc., wherein a decoding terminal 20 is provided, and the decoding terminal 20 includes a code stream obtaining module 21 and a packet decoding module 22, and the client is wired. A communication network such as a network, an Ethernet, a 3G network, or a wireless network interacts with the streaming server 30.

The frame segmentation module 11 is configured to decompose the continuous video of the front-end monitoring device into at least one frame segment.

The packet encoding module 12 is configured to extract the data frame from the frame segment and form the sub-code stream group in time sequence, and encode the first frame of the sub-code stream group in an I-frame manner, and also use the encoded sub-code. After the stream group composes the code stream group in chronological order, the code stream group is provided to the stream server 30. The packet coding module 12 may form a plurality of substream groups after extracting the data frames multiple times from each frame segment, and select the substream groups multiple times to form different codestream groups.

The code stream obtaining module 21 is configured to obtain a code stream group from the stream server 30 through the communication network. The code stream obtaining module 21 of different clients may request the stream server 30 to acquire a partial code stream group or all code stream groups according to actual conditions. . If a client requests a code stream through an Ethernet request, and the network status is better, the flow server 30 is requested to obtain all the code streams; if another client requests data through the wireless network and the bandwidth is relatively small, the flow server may be used. 30 requests a code stream group, so that when the stream server 30 receives the message, only one of the code stream groups is sent to the client, so that the client's packet loss and delay will be alleviated. In order to properly control the delay effect, the code stream obtaining module 22 may further set a delay threshold, and determine to obtain at least one code stream group from the stream server 30 according to the relationship between the real-time delay value and the delay threshold. .

The packet decoding module 22 is configured to sequentially decode the data frames included in the code stream group, and the packet decoding module 22 can decode the data frames in all the obtained code stream groups according to device performance, processing capability, and actual requirements. Or decoding only the data frames in a part of the code stream group, or The data frames contained in some of the substream groups in the code stream group are decoded.

The video monitoring system of the present invention can adapt to different network environments and device performances. If the network condition is not ideal, the processing capacity of the client is large, or the preview requirement is not high, the streaming server only sends the client only the part of the original video. The code stream group of the data frame not only improves the transmission efficiency, but also ensures the continuity and real-time performance of the video playback, and improves the video monitoring effect. Since the p-frame in each sub-stream group only relies on the I-frame or other P-frames in the group for decoding, effectively avoiding the loss of some data frames in the prior art due to the loss of some data frames, resulting in poor flower screens and the like. display effect. It is to be understood that the specific implementation of the invention is not limited to the description. It is to be understood by those skilled in the art that the present invention can be delineated or substituted without departing from the spirit and scope of the invention.

Claims

Claim

A video encoding and decoding method, comprising the steps of:

2. The method according to claim 1, wherein: the encoding end encodes the other data frames in the substream group:

釆 Use the first frame as a reference and encode in the form of a P frame.

3. The method according to claim 1, wherein: when the data frame is extracted from the frame segment, the extracted data frame is the first frame of the frame segment.

The method according to any one of claims 1 to 3, characterized in that

The method of decomposing continuous video into frame segments is: decomposing continuous video into frame segments that are connected end to end and containing equal number of frames;

5. The method of claim 4, wherein

The method according to claim 5, wherein the method further comprises: the decoding end requesting to obtain the data group from the decoding end according to the relationship between the real-time delay value and the preset delay threshold.

The method according to any one of claims 1 to 3, characterized in that:

The process of decoding the data frames included in the code stream group by the decoding end includes:

The decoding end acquires the odd digital stream group and/or the even digital stream group according to the specific request, and sequentially decodes the data frames in the odd code stream group and/or the even digital stream group.

9. The method according to claim 8, wherein: each frame segment comprises 25 consecutive data frames, and each frame segment corresponds to a time of 1 second.

A video codec system, comprising: an encoding end and a decoding end, wherein: the encoding end comprises a frame segment decomposing module and a block encoding module, and the decoding end comprises a code stream acquiring module and a packet decoding module;

The frame segmentation module is configured to decompose continuous video into frame segments;

The group coding module is configured to extract data frames from the frame segments and form in chronological order After the sub-stream group, the first frame of the sub-stream group is encoded in an I-frame manner, and the encoded sub-stream group is formed into a code stream group in time sequence;

A video surveillance system, including a front-end monitoring device, a streaming server, and a client, wherein: the front-end monitoring device is provided with an encoding end, and the encoding end includes a frame segment decomposition module and a packet encoding module; The decoding end includes a decoding end, and the decoding end includes a code stream acquiring module and a packet decoding module;

The frame segmentation module is configured to decompose the video collected by the front-end monitoring device into a frame segment; the packet coding module is configured to extract a data frame from the frame segment and form a sub-stream group in time sequence And encoding the first frame of the sub-stream group by I frame, and grouping the encoded sub-stream group into a code stream group in time sequence, and then providing the code stream group to the stream Server

And the packet decoding module is configured to sequentially decode the data frames included in the code stream group acquired by the code stream obtaining module.