KR101452368B1 - Video decoding method and apparatus based on parallel processing - Google Patents

Abstract

Even if an open GOP is used in which a part of the image in the current GOP is used as a reference image in the previous GOP by inserting a part of information of the neighboring encoded stream in the separated encoded stream, decoding of the encoded stream is performed in real time parallel processing Based video decoding method and apparatus capable of performing parallel-based video decoding.

Description

TECHNICAL FIELD [0001] The present invention relates to a video decoding method and a video decoding method based on parallel processing,

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video decoding method, and more particularly, to a video decoding method using motion compensation such as MPEG-2, AVC, and HEVC.

In the video coding technology, a coding method such as MPEG-2, AVC, and HEVC (High Efficiency Video Coding) uses a motion prediction and compensation technique which is correlation information between frames as shown in FIG. That is, in the case of the P frame (in the case of the HEVC, since the GPB frame in which the P frame is generalized is used instead, the P frame in the present invention can be replaced with the GPB of the HEVC) and the B frame The B frame can be replaced with the P frame in the present invention), only the difference value is calculated by calculating only the difference with respect to the neighboring I frame or P frame. A typical group access picture (GOP) always includes only one I frame and includes all frames until the next I frame. In the embodiment of FIG. 1, the GOP size is 9 to be.

A GOP is a set of consecutive images starting with an I frame. That is, since the encoding of the B frame is predicted not only from the frame preceding in time but also from the frame after the frame, the coding order and the display order are different as shown in Fig. In order to decode the second frame by the prediction between the frames, the first frame and the fourth frame must be first decoded. Similarly, in order to decode the 8th frame, the 7th frame and the 10th frame have to be decoded first.

FIG. 3 is a diagram showing a frame in which decoding is performed in a plurality of video decoders according to time when a stream encoded with the encoding prediction structure shown in FIG. 1 is divided into GOP units and input to a plurality of video decoders. When the encoded stream of the encoding prediction structure shown in FIG. 1 is separated into GOP units as shown in FIG. 3 and input to a plurality of video decoders, it is impossible to decode the 8th and 9th B frames in the second GOP encoded stream. 4 is a diagram showing a video decoding time for the example of FIG. In order to decode them, a restored image of the seventh P frame is required in the first GOP unit coded stream as shown in Fig. Since the 7th P frame is restored at T5 time, it is possible to restore 8th and 9th B frames at T6 time after T5 time. In addition, in order to decode the 17th and 18th B frames in the third GOP unit encoded stream, a restored image of the 16th P frame is required. Since the 16th P frame is restored at T7 time, it is possible to restore the 17th and 18th B frames at T11 time after T10 time. Therefore, there is a problem that in fact, parallel processing can not be performed.

Korean Patent No. KR 10-0775871 ("Method and Apparatus for Performing Multi-view Video Coding and Decoding Based on Image Synthesis, published by Electronics and Telecommunications Research Institute, 2006.05.18)

An object of the present invention is to provide a decoding method and apparatus for performing parallel processing for real-time video decoding in a video coding technique having motion compensation such as MPEG-2, AVC, and HEVC.

According to another aspect of the present invention, there is provided a method of decoding parallel video based video, comprising: separating an encoded stream into a plurality of groups of pictures (GOP); Decoding the separated sets of pictures in parallel by a plurality of decoders, and providing the decoded images to an output apparatus, wherein the decoding unit decodes the encoded streams into a plurality of groups of pictures (GOPs) Wherein the separated plurality of picture sets are K-th picture, where K is a positive integer, and the separated picture set unit encoded stream is divided into a first I frame belonging to the Kth picture set unit in the encoded stream And a second I frame, wherein the second I frame includes a first I frame, which is a start frame of at least one frame belonging to a (K + 1) And the step of decoding the separated plurality of sets of pictures in parallel by a plurality of decoders, respectively, The K-th separated encoded stream to be decoded by each of the decoders includes a second I frame having the same value as the first I frame which is the start frame of the (K + 1) -th encoded stream And each of the decoders decodes the separated encoded stream independently of frames in the decoded order, prior to the display order of the second I-frame, without information of other separated encoded streams.

Wherein the step of separating the encoded stream into a plurality of group of pictures (GOP) units comprises the steps of: dividing the encoded stream of the K-th separated image aggregation unit into an encoded stream of the image set unit in which the (K + 1) At least one frame which is earlier than the first I frame of the coded stream of the (K + 1) -th picture aggregation unit separated from the at least one frame included in the decoding sequence, but is subsequent to the decoded sequence in the display order.

At least one frame preceding the first I frame of the (K + 1) -th coded image set unit-divided coded stream in the display order and subsequent to the decoding order is the coded stream of the And may be at least one B frame that is read later than the first I frame and is read before the first P frame of the P frame of the (K + 1) th image aggregation encoded stream.

Wherein the step of decoding the separated plurality of sets of pictures in parallel by a plurality of decoders comprises the steps of: decoding the K-th separated encoded streams by at least one B Frames can be decoded using the second I frame of the K-th separated encoded stream.

Wherein the step of decoding the separated plurality of sets of pictures in parallel by a plurality of decoders comprises the steps of: decoding the K-th separated encoded streams by at least one B Frames after the decoding of the second I frame of the K-th separated encoded stream.

Wherein the step of decoding the separated plurality of sets of pictures in parallel by a plurality of decoders comprises a step of decoding the first I frame of the K- The B frame to be read before decoding may not be decoded.

The providing of the decoded image to the output device may provide at least one decoded image to the output device, except for the image generated by the decoder decoding the second I frame.

The step of providing the decoded image to an output device may include storing the decoded image in a memory and providing the decoded image stored in the memory to an output device, And at least one decoded image excluding the decoded image may be stored in the memory.

The step of providing the decoded image stored in the memory to the output device may provide the decoded image to the output device in the order to be displayed on the output device.

The step of storing the decoded images in the memory may store the decoded images in the order of providing the decoded images to the output device.

The decoding method and apparatus according to an embodiment of the present invention may include an encoding method in which an information of a neighboring encoded stream is partially inserted into an encoded stream and an open GOP is used in which a portion of the current GOP is used as a reference image It is possible to perform real-time parallel decoding of the decoding of the encoded stream, thereby improving the decoding performance of the video decoder.

FIG. 1 is a diagram illustrating the relationship between frames decoded using motion prediction and compensation techniques, which are correlation information between frames, arranged in the order in which they are displayed.
FIG. 2 is a diagram illustrating video decoding time of frames for the embodiment of FIG. 1. FIG.
FIG. 3 is a diagram illustrating a case where a stream encoded by the encoding prediction structure shown in FIG. 1 is divided into units of a group of pictures (GOP) and input to a plurality of video decoders, Fig.
FIG. 4 is a diagram illustrating video decoding time of frames for the example of FIG. 3. FIG.
FIG. 5 is a diagram illustrating a structure of a video decoding apparatus based on parallel processing of a group of pictures (GOP) unit according to an embodiment of the present invention.
FIG. 6 is a flowchart illustrating a method of decoding a frame read from a plurality of video decoders when a stream encoded with an encoding prediction parser of FIG. 1 according to an embodiment of the present invention is divided into GOP units and input to a plurality of video decoders. Fig.
7 is a flowchart illustrating a process of decoding a coded stream in parallel by a video decoder according to a decoding method according to an embodiment of the present invention.
FIG. 8 is a conceptual diagram illustrating a frame to be decoded in an encoded stream separated by each video decoder according to a time when an encoded stream is divided according to a decoding method according to an embodiment of the present invention shown in FIG.

While the invention is susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and will herein be described in detail.

It should be understood, however, that the invention is not intended to be limited to the particular embodiments, but includes all modifications, equivalents, and alternatives falling within the spirit and scope of the invention.

The terms first, second, etc. may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component. And / or < / RTI > includes any combination of a plurality of related listed items or any of a plurality of related listed items.

It is to be understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, . On the other hand, when an element is referred to as being "directly connected" or "directly connected" to another element, it should be understood that there are no other elements in between.

The terminology used in this application is used only to describe a specific embodiment and is not intended to limit the invention. The singular expressions include plural expressions unless the context clearly dictates otherwise. In the present application, the terms "comprises" or "having" and the like are used to specify that there is a feature, a number, a step, an operation, an element, a component or a combination thereof described in the specification, But do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or combinations thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the relevant art and are to be interpreted in an ideal or overly formal sense unless explicitly defined in the present application Do not.

FIG. 5 shows a structure of a parallel processing-based video decoding apparatus 100 on a group-of-picture (GOP) basis according to an embodiment of the present invention. A GOP is a set of consecutive images starting with an I frame. A decoding apparatus 100 according to an embodiment of the present invention includes an encoding stream parser 110 for each GOP unit, video decoders 121, 122 and 123 for N GOP units, a restored image storage memory 130, And outputs the video stream as a video image.

The coded stream parser 110 receives the coded video coded streams based on the motion prediction and compensation, separates the coded streams in units of GOP, and inputs them to the N GOP unit video decoders 121, 122 and 123, respectively.

Each of the N GOP-unit video decoders 121, 122, and 123 independently decodes the encoded stream and stores the reconstructed image in the memory 130.

The video output unit 140 outputs the restored video stored in the memory to a monitor or the like in accordance with the display order to reproduce the video in real time. The video output device 140 may arrange and restore the restored video stored in the memory according to the display order.

FIG. 6 is a flowchart illustrating a method of decoding a stream encoded by the encoding stream parser 110 in the encoding prediction structure shown in FIG. 1 according to an exemplary embodiment of the present invention in units of GOPs according to an embodiment of the present invention, 122, and 123 in order of frames read from the plurality of video decoders 121, 122, and 123, respectively. The proposed GOP-based encoded stream parser 110 according to an embodiment of the present invention separates (parses) an encoded stream as shown in FIG. As shown in FIG. 6, the encoded stream parser 110 further includes an I frame of the GOP unit encoded stream to be separated in the next order in the separated encoded stream when dividing the encoded stream into the proposed GOP units. When the encoded stream parser 110 separates the encoded stream into the proposed GOP units, the encoded stream parser 110 decodes the separated encoded stream after the I frame of the GOP unit encoded stream to be separated in the following order, but the display order is earlier than the I frame Lt; / RTI > frames.

 More specifically, if the I frame, which is the start frame of the separated GOP unit encoded stream, is the first I frame, the K-th separated GOP unit encoded stream starts with the first I frame, and the (K + 1) The first I frame of the (K + 1) -th separated GOP encoded stream which is the start frame of the GOP unit encoded stream, and all the frames that are in the order of display time earlier than the I frame and are subsequent to the decoding order. Thus, the K-th separated GOP encoded stream of the present invention includes an I frame as a first I frame and a second I frame, unlike a conventional GOP. Here, the second I frame is the first I frame of the (K + 1) -th separated GOP encoded stream which is the start frame of the (K + 1) -th encoded stream in units of GOP, K is greater than or equal to 1, It is a positive integer with a value less than the total number of streams.

For example, the I frame decoded at the time T8 in the first GOP unit encoded stream shown in FIG. 6 is the same as the I frame decoded at the T1 time in the second GOP encoded stream. As shown in FIG. 6, if the separated GOP encoded stream is decoded, the I frame needs to be decoded twice, but it is possible to perform parallel processing in units of GOPs.

Hereinafter, a stream separated in units of GOP means a stream divided into GOP units proposed by the present invention including two I frames as described above.

Referring to FIG. 7, a description will be given of a step in which an encoded stream separated by a GOP unit according to an embodiment of the present invention is decoded by each of the GOP-based video decoders 121, 122, and 123. FIG. First, the encoded-stream parser 110 divides an encoded stream into I-frames of a GOP-encoded stream of the next sequence, and then divides the encoded stream into GOP units including all frames subsequent to the decoded sequence on the display (S210). Then, the encoded stream parser 110 inputs the divided GOP-encoded stream into each of the GOP-based video decoders 121, 122, and 123 (S211). According to an embodiment of the present invention, the encoded stream of the first GOP unit in FIG. 6 is input to the first video decoder 121 of the GOP unit, and the encoded stream of the second GOP unit is input to the second video decoder 122, and the N-th GOP encoded stream may be input to the N-th video decoder 123 of the GOP unit.

The first frame of the divided encoded stream is an I frame. The video decoders 121, 122, and 123 decode the I frame, which is the first frame of the encoded stream divided at the time T1, and store the decoded I frame in the memory (S220).

The video decoders 121, 122, and 123 read the next frame of the divided encoded stream (S230). The video decoders 121, 122, and 123 determine whether the read frame is a B frame or a previous frame on the display rather than the I frame (S231). As shown in FIG. 6, if the frame read as the second and third GOP encoded streams is a B frame, the video decoders 121, 122 and 123 can not decode the B frame as described above, That is, the next frame is read from the encoded stream without decoding until the encoded stream of the next frame after the first I frame is displayed in the display order. For example, in the embodiment of the present invention shown in FIG. 6, the second video decoder 122 receiving the encoded stream of the second GOP unit reads the next frame without decoding the 8th and 9th B frames, Similarly, the third video decoder (not shown) receiving the encoded stream of the third GOP unit does not decode the 17th and 18th B frames. If it is determined that the frame read in step S231 is a P frame (S232), the video decoders 121, 122, and 123 decode the P frame at time T2 and store the decoded P frame in the memory (S233).

When the T2 time decoding ends, the video decoders 121, 122, and 123 read the next frame of the divided encoded stream (S241). The video decoders 121, 122, and 123 determine whether the read frame is an I frame (S242). If the read frame is not an I frame but a B frame or a P frame, the video decoders 121, 122, and 123 decode the read frame at the time T and store the video in the memory at step S243. On the other hand, if the read frame is an I frame, the I frame that is not located in the first frame in the separated encoded stream is an I frame added from another encoded stream. Therefore, when the read frame is an I frame, the I frame is used to decode the next frame, (S244). For example, in one embodiment of the present invention shown in FIG. 6, each of the video decoders 121, 122, and 123 in units of GOPs decodes I (I) Frame, the I-th frame 19 of the second GOP-based encoded stream and the I-th frame 28 of the third-GOP-encoded stream are used for decoding after decoding and are not stored in the reconstructed image storage memory 130, That is, not displayed.

Then, the video decoders 121, 122, and 123 determine whether there is a next frame in the separated encoded stream (S245). If there is a next frame in the separated encoded stream, the video decoders 121, 122, and 123 continue from the step of reading the next frame (S241). If there is no next frame, the video decoders 121, 122, and 123 finish decoding the stream.

FIG. 8 illustrates a time for decoding an encoded stream separated by each video decoder when an encoded stream is divided according to the decoding method according to an embodiment of the present invention shown in FIG. As shown in the figure, each of the GOP-based video decoders 121, 122, and 123 can process the encoded streams in units of GOP in parallel within T10 hours.

Claims (10)

A parallel processing based video decoding method,
Separating an encoded stream into a plurality of group of pictures (GOP) units;
Decoding the separated plurality of sets of pictures in parallel by a plurality of decoders,
And providing the decoded image to an output device,
Wherein in the step of dividing the encoded stream into units of a plurality of groups of pictures (GOPs), the separated plurality of sets of pictures are arranged in a Kth order, where K is a positive integer, At least one frame starting from a first I frame which is a start frame of an encoding stream separated by a K-th separation in a coded stream, and a second I frame, wherein the second I frame includes a K + A first I frame, which is a start frame of at least one frame of an encoded stream of a first separated image set unit,
Wherein the step of decoding the separated plurality of sets of pictures in parallel by a plurality of decoders comprises decoding each of the separated encoded streams by the respective decoders, wherein each of the decoders decodes the K- And a second I frame having the same value as the first I frame which is the start frame of the (K + 1) -th encoded video stream, And decodes the frames that are prior to the display order and later in the decoding order than the second I frame without information of other separated encoded streams. The method according to claim 1,
The step of separating the encoded stream into a plurality of group of pictures (GOP)
Wherein the K-th coded image-set encoded stream is a K + 1-th coded image-set encoded stream, Further comprising at least one frame that is previous in display order and later in decoding order than a first I frame that is a start frame. 3. The method of claim 2, wherein at least one of the frames preceding the first I frame, which is the start frame of the encoded stream in the (K + 1) At least one of which is read earlier than the first I frame which is the start frame of the encoded image set unit encoded stream and which is read before the first P frame of the P frame of the (K + 1) And a B frame of the video frame. 4. The method of claim 3, wherein the step of decoding the separated plurality of sets of pictures in parallel by a plurality of decoders comprises the steps of: And decodes the at least one B frames added from the encoded stream of the image set unit by using the second I frame of the K-th separated encoded image stream unit. The method of claim 4, wherein the step of decoding the separated plurality of sets of pictures in parallel by a plurality of decoders comprises the steps of: (a) dividing the (K + 1) And decodes at least one B frames added from the encoded stream of the image aggregation unit after decoding the second I frame of the K-th separated image aggregation encoded stream. The decoding method according to claim 5, wherein the step of decoding the separated plurality of sets of pictures in parallel by a plurality of decoders comprises the steps of: decoding the first I frame, which is the start frame of the K- And decodes the B frame to be read before decoding the P frame among at least one B frame to be decoded and read. The method as claimed in claim 6, wherein the step of providing the decoded image to an output device comprises providing at least one decoded image to an output device except for an image generated by the decoder decoding the second I frame Video decoding method. The method of claim 7, wherein providing the decoded image to an output device comprises: storing the decoded image in a memory; and providing the decoded image stored in the memory to an output device, Wherein at least one decoded image excluding an image generated by decoding the second I frame is stored in the memory. The method of claim 8, wherein providing the decoded image stored in the memory to an output device provides the decoded image to the output device in the order to be displayed on the output device. 10. The video decoding method of claim 9, wherein the step of storing the decoded images in a memory comprises arranging and storing the decoded images in order to provide the decoded images to the output device.

Images