KR101531036B1 - 3 demension decoding apparatus and method thereof - Google Patents

Abstract

The present invention is characterized in that the number of threads of a first decoder for decoding a HEVC data stream of a color image among decoding parameters is set to a default value and a second number of threads for decoding a HEVC data stream of a depth image among the decoding parameters The number of threads of the decoder is set to K less than N, the image information is grasped in header information of the HEVC data stream for each decoding parameter, the image information difference of the HEVC data stream for each decoding parameter is grasped, And increases the number of threads of the decoder corresponding to the HEVC data stream having the long image information in addition to the number of the default threads, thereby improving the performance of the 3D decoder.

Description

[0002] 3D image decoding apparatus and method [0003]

The present invention relates to a 3D image decoding technique.

Currently, 3D image technology is adopted in the field of image such as TV and movie. These three-dimensional imaging technologies are largely divided into stereo 3D technology and multi-point 3D technology that does not wear glasses.

Among them, 3D 3D technology, which requires wearing glasses, has attracted attention. For multi-view 3D video systems, the 3D format of multiple-view color + depth (MVD) is widely used.

MVD images for multi-view 3D have much more information than conventional 2D and stereo 3D. Therefore, high efficiency video coding (HEVC) is being developed as a video compression technology for processing a large amount of information.

HEVC is a new video coding standard developed after H.264 / AVC (Advanced Video Coding) and has better coding efficiency than H.264 / AVC. The coding structure of HEVC is almost similar to H.264 / AVC, but there are two differences.

First, video data is compressed using a highly flexible quadtree partition scheme. That is, the HEVC can compress images of various resolutions into optimal partitions by the quad tree partition structure. The compression structure of the HEVC consists of three units: a coding unit (CU), a prediction unit (PU), and a transform unit (TU). CU is similar to H.264 / AVC macroblock, and represents the basic unit for compression. In HEVC, the size of CU can be variously different from the size of H.264 / AVC macroblock.

In the current High Efficiency Video Coding (HEVC), the CU size can range from 8x8 to 64x64, and the largest CU (LCU) and the smallest CU (SCU) are described in the sequence parameter set . Also, one CU is compressed into one or more PUs, and TUs can have various sizes within one CU.

The second is that the HEVC standard includes new coding tools for coding efficiency. New intraprediction directions have been added, and large transforms (16x16, 32x32), asymmetric motion partitions (AMP), and sample adaptive offset (SAO) have been added.

Therefore, the decoding process of HEVC is complicated compared with that of H.264 / AVC. Therefore, implementation technology for fast decoding is required to use image compression technology using HEVC.

SUMMARY OF THE INVENTION The present invention provides a 3D image decoding method and apparatus capable of efficiently decoding a 3D image encoded using the HEVC standard.

According to another aspect of the present invention, there is provided a 3D image decoding apparatus. The 3D image decoding apparatus includes a plurality of extracting units each receiving an HEVC data stream for each decoding parameter and extracting and providing a NAL unit from the HEVC data stream and a plurality of extracting units corresponding to the plurality of extracting units, A synchronization unit for synthesizing a decoded image of the same frame output from each of the plurality of decoders and outputting the decoded image as a single image frame; The number of threads of the first decoder for decrypting the HEVC data stream of the color image among the decryption parameters is set to N and the number of threads of the second decoder for decrypting the HEVC data stream of the depth image among the decryption parameters is set to Lt; RTI ID = 0.0 > N < / RTI > .

Wherein the thread control unit grasps the image information of the HEVC data stream for each decoding parameter from the header information of the HEVC data stream for each decoding parameter and has image information with a long decode processing time according to the image information difference of the HEVC data stream for each decoding parameter The number of threads of the decoder corresponding to the HEVC data stream is increased in addition to the number of the default threads.

Alternatively, the thread control unit increases the number of threads in addition to the default number of threads for at least one of the plurality of decoders in accordance with a user command signal.

The image information is at least one of an information amount, a resolution, and an image quality.

Wherein the thread control unit compares the image information of the HEVC data streams of the color image among the decoding parameters with the image information of the HEVC data streams of the depth image among the decoding parameters, The number of threads of the decoder corresponding to the HEVC data stream having image information is increased in addition to the number of the default threads.

According to another aspect of the present invention, there is provided a 3D image decoding method. The 3D image decoding method sets N number of threads of the first decoder for decoding the HEVC data stream of the color image among the decoding parameters with a default value and decodes the HEVC data stream of the depth image among the decoding parameters A second step of setting the number of threads of the second decoder to K number of less than N, a second step of grasping image information in header information of the HEVC data stream for each decoding parameter, the image information of the HEVC data stream for each decoding parameter And a fourth step of increasing the number of threads of the decoder corresponding to the HEVC data stream having image information with a longer decode processing time, in addition to the number of the default threads.

The fourth step increases the number of threads in proportion to the image information difference of the HEVC data stream for each decoding parameter.

According to the embodiment of the present invention, the decoding performance of a plurality of decoders is adjusted to be constant by adjusting the number of threads of the decoder, thereby improving the 3D image decoding speed for the HEVC data stream.

1 is a block diagram of a 3D image decoding apparatus according to a first embodiment of the present invention.
FIG. 2 is a diagram illustrating frame-by-frame parallel processing performed by each decoder unit according to an embodiment of the present invention.
3 is a block diagram of a 3D image decoding apparatus according to a second embodiment of the present invention.
4 is an operation flowchart of a 3D image decoding apparatus according to a second embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily carry out the present invention. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. In order to clearly illustrate the present invention, parts not related to the description are omitted, and similar parts are denoted by like reference characters throughout the specification.

Hereinafter, a 3D image decoding apparatus and method according to an embodiment of the present invention will be described in detail with reference to the drawings.

Prior to the description, the 3D image decoding apparatus and method according to the embodiment of the present invention are based on HEVC technology, and can be applied to stereo 3D decoding or multi-view 3D decoding. Hereinafter, only the case where the present invention is applied to the multi-view 3D decoding will be described for convenience of explanation, and it will be understood by those of ordinary skill in the art that the present invention is applied to the stereo 3D decoding through the following description.

FIG. 1 is a block diagram of a 3D image decoding apparatus according to a first embodiment of the present invention, which is applied to multi-view 3D decoding. As shown in FIG. 1, the 3D image decoding apparatus according to the first embodiment of the present invention includes first through fourth extracting units 110 through 140, first through fourth decoders 210 through 240, a thread control unit 300 and a synchronization unit 400.

The first to fourth extracting units 110 to 140 input the HEVC data streams for two color images and two depth images according to the format of the MVD image for the 3D point again And extracts one NAL (Network Abstract Layer) unit from the data stream.

That is, the first extracting unit 110 receives the HEVC data stream for the color 0 image for each frame, sequentially extracts one NAL unit from the input data stream, and provides it to the first decoder 210. The second extracting unit 120 receives the HEVC data stream for the color 1 image for each frame and sequentially extracts one NAL unit from the input data stream and provides it to the second decoder 220.

The third extracting unit 130 receives the HEVC data stream for the depth 0 image for each frame, sequentially extracts one NAL unit from the input data stream, and provides it to the third decoder 230. The fourth extracting unit 140 receives the HEVC data stream for the depth 0 image for each frame, sequentially extracts one NAL unit from the input data stream, and provides it to the fourth decoder 240.

In this case, the number of extracting units is four according to the number of images classified according to the format of the MVD image. However, when the number of images forming one frame increases, the number of extracting units will also increase correspondingly. In this case, a variable value for determining the number of extraction units is called a decoding parameter, and the decoding parameters applied to FIG. 1 are color 0, color 1, depth 0, and depth 1.

Next, the first to fourth decoders 210 to 240 are configured as many as the number of extraction units, and one decoder corresponds to one extraction unit. That is, the first decoder 210 corresponds to the first extracting unit 110, the second decoder 220 corresponds to the second extracting unit 120, and the third decoder 230 corresponds to the third extracting unit 130, and the fourth decoder 240 corresponds to the fourth extraction unit 140. [

Each of the decoders 210 to 240 decodes NAL units received from the extracting unit using a plurality of threads according to a frame-by-frame parallel processing method.

Hereinafter, a frame-by-frame parallel processing method will be briefly described with reference to FIG. FIG. 2 is a diagram illustrating a frame-by-frame parallel processing performed by each decoder unit according to an embodiment of the present invention, in which one decoder uses four threads.

The frame-by-frame parallel processing decodes one frame in one thread, and concretely, it is a processing technique for simultaneously decoding a plurality of frames using a plurality of threads.

That is, the frame-by-frame parallel processing decodes the NAL unit of the first frame in the first thread (thread 1) when the NAL unit of the first frame (Frame 1) is input to one decoder. In this case, the coded NAL unit is composed of CTB (Coding Tree Block) (CTU) which is a basic unit of compression. Therefore, the first thread (thread 1) decodes the NAL unit of the first frame in CTB units.

In decoding the NAL unit in the first thread (Thread 1), the NAL unit of the second frame (Frame 2) is input to the corresponding decoder, and the NAL unit of the second frame (Frame 2) ) In the unit of CTB. In decoding the NAL unit in the first and second threads (thread 1, thread 2), the NAL unit of the third frame (Frame 3) is input to the corresponding decoder, and the NAL unit of the third frame (Frame 3) The third thread (thread 3) decodes in units of CTB. In decoding the NAL unit in each of the first to third threads (thread 1 to thread 3), the NAL unit of the fourth frame (Frame 4) is input to the corresponding decoder, and the NAL unit of the fourth frame Is decoded in the fourth thread (thread 4) in units of CTB.

If the decoding of one frame input from the first thread (thread 1) is completed during decoding in the first to fourth threads (thread 1 to thread 4), the NAL unit of the fifth frame is input to the corresponding decoder , And the first thread (thread 1) decodes the fifth frame independently of the operation of the second to fourth threads (thread 2 to thread 4).

Therefore, if the frame-by-frame parallel processing is performed in one decoder, the image is decoded in a shorter time.

On the other hand, the decoding speed of the color image and the decoding speed of the depth image in one thread are not the same. Generally, the decoding speed for a color image of one frame in a thread is slower than the decoding speed for a depth image of one frame.

The decoding speed of the color image is slower than the decoding speed of the depth image because the data amount (or the amount of information) of the color image is larger than the data amount (or the amount of information) of the depth image.

For this reason, when the number of threads of the first to fourth decoders 210 to 240 is the same, the output speeds of the first to fourth decoders 210 to 240 are different.

When the output speeds of the respective decoders 210 to 240 are changed, the synchronizer 400 collects the outputs of the decoders 210 to 240 into one frame image, The greater the processing speed difference of each decoder, the longer the waiting time in the synchronizer 300 becomes, which causes the overall decryption processing speed to be slowed down.

Therefore, the apparatus and method according to the embodiment of the present invention allow the processing speeds of the plurality of decoders to be substantially equal, thereby speeding up the overall decryption processing speed.

The thread control unit 300 is configured to speed up the overall decryption processing speed. The thread control unit 300 receives the user setting input signal generated in response to the setting instruction of the user and sets the number of threads for at least one of the first to fourth decoders 210 to 240 according to the user setting input signal Adjust.

The thread control unit 300 automatically sets the default number of threads for each of the decoders 210 to 240 when there is no separate user setting command.

At this time, the number of default threads is set to be larger than the number of the default threads in the third and fourth decoders 230 and 240 which perform decoding of depth spirituality. The thread control unit 300 sets the number of threads of the first and second decoders 210 and 220 equal to or less than the number of threads of the third and fourth decoders 230 and 240 If an invalid user command is received, the user is asked to reset the number of threads through an alarm.

Here, the number of threads of the first and second decoders 210 and 220 corresponding to the color image may be the same or different, and the number of threads of the third and fourth decoders 230 and 240 corresponding to the depth image may be the same Or may be different.

Finally, the synchronizing unit 400 receives the decoded image from the first to fourth decoders 210 to 240 and decodes the decoded image of color 0, color 2, depth 0 and depth 1 of the same frame from each of the decoders 210 to 240 It makes a single image frame and outputs it to the outside.

Hereinafter, a 3D image decoding apparatus and method according to a second embodiment of the present invention will be described with reference to FIGS. 3 and 4. FIG.

3 is a block diagram of a 3D image decoding apparatus according to a second embodiment of the present invention. As shown in FIG. 3, the 3D image decoding apparatus according to the second embodiment of the present invention is substantially the same as the 3D image decoding apparatus according to the first embodiment of the present invention.

However, in the first embodiment of the present invention, the thread control unit 300 adjusts the number of threads of each of the decoders 210 to 240 according to a user's input. However, in the second embodiment of the present invention, The number of threads of each of the decoders 210 to 240 is adjusted.

Therefore, in the second embodiment of the present invention, the operation of the thread control unit 300 is different from that of the first embodiment of the present invention, and the remaining configuration is the same as that of the first embodiment of the present invention.

The thread control unit 300 according to the second embodiment of the present invention receives header information of HEVC data streams of color 0, color 2, depth 0, and depth 1, decodes the information amount, image quality, resolution and the like of the data stream included in the header information (Hereinafter, referred to as "image information") of factors affecting the processing time.

The thread control unit 300 basically defaults to the number of threads for the color image and the number of threads for the depth image, and the number of threads for the color image is larger than the number of threads for the depth image.

In this state, the thread control unit 300 adjusts the number of threads of each of the decoders 210 to 240 in such a manner as to increase the number of threads in accordance with image information obtained from header information of HEVC data streams of color0, color2, depth0, and depth1 do. Of course, the thread control unit 300 increases the number of threads as the amount of information is large or the image quality or resolution is high.

Hereinafter, the operation of the 3D image decoding apparatus according to the second embodiment of the present invention will be described with reference to FIG. 4 is an operation flowchart of a 3D image decoding apparatus according to a second embodiment of the present invention.

The thread control unit 300 controls the number of threads of the first to fourth decoders 210 to 240 according to a default value. The first and second decoders 210 and 220, which process decoded color images, And the third to fourth decoders 230 and 240 that process the decoded data are set to three.

In this state, the first extracting unit 110 inputs the HEVC data stream for the color 0 image for each frame, the second extracting unit 120 inputs the HEVC data stream for the color 1 image for each frame, The extracting unit 130 inputs the HEVC data stream for the depth0 image for each frame, and the fourth extracting unit 140 inputs the HEVC data stream for the depth1 image for each frame at step S401.

Header information of HEVC data streams of color0, color1, depth0 and depth1 or HEVC data streams of color0, color1, depth0 and depth1 are provided to the thread control unit 300 (S402)

The thread control unit 300 reads the header information of the data stream and distinguishes whether the image input to each of the decoders 210 to 240 is a color image or a depth image (S403).

In addition, the thread control unit 300 determines image information (e.g., information amount, image quality, resolution) of each data stream by reading header information of each data stream (S404). In FIG. 4, the processing speed variable value represents image information differently.

If the image information of each data stream is different, the thread control unit 300 increases the number of threads of a decoder for processing a data stream having a large decoding throughput (S405).

For example, to increase the number of threads,

First, the image information difference of each data stream is identified based on at least one of image quality and resolution, and the number of threads of the decoder corresponding to the data stream is increased in proportion to the image information difference. For example, if the topology or image quality of the data stream depth1 is higher than that of the other three data streams and the resolution or image quality of the other three data streams is the same, the number of threads of the decoder 240 decoding the depth1 image is increased.

The second is to divide the color image and the depth image to increase the number of threads. The difference is determined based on at least one of information amount, image quality or resolution of the data stream of the color0 image and the color1 image, And increases the number of threads of the decoder corresponding to a data stream having a large amount of information or a high image quality or resolution. Also depth0 and depth1 compare image information with each other like color0 image and color1 image, and increase the number of threads when there is a difference.

In addition to the first and second cases, the larger the difference in image information, the greater the number of threads can be increased. That is, if the difference in image information is one level, one thread is increased, and if the difference in image information is two levels higher than one level, two threads are increased.

When the number of threads of each of the decoders 210 to 240 is determined by the thread control unit 300 and the first to fourth extraction units 110 to 140 extract NAL units from the respective data streams and provide them to the corresponding decoders (S407). The decoder decodes the NAL unit for each frame by performing the frame-by-frame parallel processing using the determined number of threads (S408).

In accordance with this decoding operation, each of the decoders 210 to 240 outputs the decoded color0 video frame, color1 video frame, depth0 video frame, and depth1 video frame at substantially the same time, and provides them to the synchronization unit 400. [ At this time, four output image frames are separated from the same image frame.

The synchronization unit 400 receives four image frames with almost no waiting time, combines the received four image frames into one image frame, and outputs the synthesized image frame (S409).

The embodiments of the present invention described above are not only implemented by the apparatus and method but may be implemented through a program for realizing the function corresponding to the configuration of the embodiment of the present invention or a recording medium on which the program is recorded, The embodiments can be easily implemented by those skilled in the art from the description of the embodiments described above.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, It belongs to the scope of right.

110 - 140: Extracting unit 210 - 240: Decoder
300: thread control unit 400:

Claims (8)

A plurality of extracting units for respectively receiving HEVC data streams for respective decoding parameters, extracting NAL units from the HEVC data stream,
A first decoder corresponding to each of the plurality of extracting units and performing decoding on the NAL unit provided by the extracting unit through frame parallel processing and decoding the HEVC data stream of the color image among the decoding parameters, And a second decoder for performing a decoding process on the HEVC data stream of the depth image among the decoding parameters,
A synchronization unit for synthesizing the decoded images of the same frame output from each of the plurality of decoders,
Setting a default value of the number of threads or adjusting a default number of threads for at least one of the plurality of decoders, setting a default value of the number of threads of the first decoder to N, And a thread control unit for setting the number of default values to K, which is smaller than N. The method of claim 1,
Wherein the thread control unit grasps the image information of the HEVC data stream for each decoding parameter from the header information of the HEVC data stream for each decoding parameter and has image information with a long decode processing time according to the image information difference of the HEVC data stream for each decoding parameter And increases the number of threads of at least one decoder among the plurality of decoders corresponding to the HEVC data stream to the number of the default threads. The method of claim 1,
Wherein the thread control unit increases the number of threads in addition to the number of the default threads for at least one decoder among the plurality of decoders according to a user command signal. 3. The method of claim 2,
Wherein the image information is at least one of an information amount, a resolution, and an image quality. 5. The method of claim 4,
Wherein the thread control unit compares the image information of the HEVC data streams of the color image among the decoding parameters with the image information of the HEVC data streams of the depth image among the decoding parameters, And increases the number of threads of at least one decoder among the plurality of decoders corresponding to the HEVC data stream having image information, in addition to the number of the default threads. Wherein a default value of the number of threads of the first decoder for decoding the HEVC data stream of the color image among the decoded parameters among the plurality of decoders is set to N and the HEVC data stream of the decoded image among the plurality of decoders A first step of setting a default value of the number of threads of the second decoder to be decoded to K smaller than N,
A second step of grasping image information from header information of the HEVC data stream for each decoding parameter,
A third step of recognizing image information difference of the HEVC data stream for each decoding parameter, and
And a fourth step of increasing the number of threads of the first decoder or the second decoder corresponding to the HEVC data stream having the decoded image information with the image information longer than the number of the default threads. The method of claim 6,
Wherein the image information is at least one of an information amount, a resolution, and an image quality. 8. The method of claim 7,
Wherein the fourth step increases the number of threads of the first decoder or the second decoder in proportion to the image information difference of the HEVC data stream for each decoding parameter.

Images