KR102468223B1 - Apparatus for transmitting 3dtv broadcasting and method for selecting reference video thereof - Google Patents

Abstract

The 3DTV broadcast transmission device compares the picture quality of the left video and the right video, selects a video having a good quality among the left video and the right video as a reference video, selects the remaining video as an additional video, and selects the reference video and the additional video. After encoding the video, information on the encoded base video, the encoded additional video, and the video selected as the base video is multiplexed and transmitted.

Description

Apparatus for transmitting 3DTV broadcasting and its reference video selection method

The present invention relates to a 3DTV broadcast transmission device and a method for selecting a reference video therefor, and more particularly, to a method for selecting a video suitable for use as a reference video from a left video and a right video for a fusion-type 3DTV broadcast service in a 3DTV broadcast transmission apparatus It is about.

Convergence 3DTV [3-Dimensional Television] broadcasting service is a method of providing a 3DTV broadcasting service by using left and right images with asymmetric resolution, and uses a reference image with high resolution and an additional image with low resolution.

The left and right images for expressing 3D information can be acquired through binocular cameras, and since they are images taken from different sides, even the original images may be different from the left and right images due to inconsistency in focus and the influence of lighting depending on the situation. The quality of the right image may be different.

In the existing convergence type 3DTV broadcasting service environment, one of the left and right images is set as a reference image regardless of the quality of the left and right images. However, since the subjective picture quality of the convergence type 3DTV (i.e., the picture quality of the 3D synthesized video) is affected by the picture quality of the reference video, if the image with the lower quality among the left and right images is set as the reference picture, the picture quality of the convergence type 3DTV will also deteriorate. can Therefore, it is necessary to select a high-quality image among the left and right images as a reference image, and a method for determining a clearer image among the left and right images in a convergence type 3DTV service environment is required.

An object to be solved by the present invention is to provide a 3DTV broadcast transmission device and method for selecting a base image for selecting a video suitable for use as a reference video among a left video and a right video in a convergence type 3DTV broadcasting service environment.

According to an embodiment of the present invention, a method for selecting a base video in a 3DTV broadcast transmission device is provided. The method for selecting a base image includes comparing the image quality of a left image and a right image, selecting an image having a good quality among the left image and the right image as a base image, and selecting the remaining images as additional images; The method includes encoding the additional video, and multiplexing and transmitting information on the encoded base video, the encoded additional video, and the video selected as the base video.

According to an embodiment of the present invention, it is efficient in that a separate algorithm for selecting a reference image is not additionally required by determining the quality of the left and right images by using the result of general video encoding.

In addition, by selecting an image having excellent quality among the left and right images as a reference image, a convergence type 3DTV service of high quality can be provided.

1 is a diagram illustrating a 3DTV broadcast transmission device according to an embodiment of the present invention.
2 is a diagram illustrating a 3DTV broadcast transmission method according to an embodiment of the present invention.
3 is a diagram illustrating a reference image selection method of an image selector according to an embodiment of the present invention.
4 is a diagram illustrating a 3DTV broadcast transmitting device according to a second embodiment of the present invention.
5 is a diagram illustrating a 3DTV broadcast transmission device according to a third embodiment of the present invention.
6 is a diagram illustrating a 3DTV broadcast receiving device according to a first embodiment of the present invention.
7 is a flowchart illustrating a method for receiving 3DTV broadcasting according to an embodiment of the present invention.
8 is a diagram showing a 3DTV broadcast receiving device according to a second embodiment of the present invention.

Hereinafter, with reference to the accompanying drawings, embodiments of the present invention will be described in detail so that those skilled in the art can easily carry out the present invention. However, the present invention may be implemented in many different forms and is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the specification and claims, when a certain component is said to "include", it means that it may further include other components without excluding other components unless otherwise stated.

Now, a 3DTV broadcasting transmission device and a method for selecting a reference video thereof according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

FIG. 1 is a diagram illustrating a 3DTV broadcast transmission device according to a first embodiment of the present invention, and FIG. 2 is a flowchart illustrating a 3DTV broadcast transmission method according to an embodiment of the present invention.

The content transmitted by the 3DTV broadcast transmission device 100 shown in FIG. 1 may be video, video, text, etc. Among the video, in particular, the three-dimensional (3D) video is a left video and a right video with different viewpoints. Can include video. The left image and the right image may have the same resolution. One of the left and right images may be used as a reference image and the other may be used as an additional image. The additional image may refer to an image used to construct a 3D image paired with a reference image, and may have a resolution lower than that of the reference image.

Referring to FIG. 1, the 3DTV broadcast transmission device 100 includes an image selection unit 110, a down-sampling conversion unit 120, first and second video encoders 130 and 140, and a multiplexer ( 150) and a transmission unit 160.

Referring to FIG. 2 , the image selector 110 selects one image as a reference image from among the left and right images acquired through the binocular camera, and selects the other image as an additional image (S210). The image selection unit 110 transfers the additional video to the downsampling conversion unit 120 and transfers the base video to the first video encoder 130 . In addition, the image selector 110 transmits signaling information including information on the image selected as the reference image to the multiplexer 150 . Even if the left and right images have the same resolution, the image quality between the left and right images may not be the same depending on a shooting situation and a difference in subsequent processing. Accordingly, the image selector 110 may determine the quality of the left and right images, and select an image having a good quality among the left and right images as a reference image. The image selection unit 110 may utilize the result of image encoding as a method for determining the image quality of the left and right images. That is, the image selection unit 110 determines the image quality of the left image and the right image by using the fact that the result of image encoding is different depending on the sharpness of the image.

The downsampling converter 120 downsamples the additional video and transmits it to the second video encoder 140 (S220). For example, the left and right images may be UHD (Ultra High-Definition) resolution images, and the additional image may be downsampled to HD resolution.

The first video encoder 130 encodes the base video using a predetermined video encoding method to generate base video encoding data (S230).

The second video encoder 140 encodes the additional video using a predetermined video encoding method to generate additional video encoding data (S240).

The multiplexer 150 multiplexes the base video encoding data, the additional video encoding data, and signaling information (S250).

The transmission unit 160 transmits the multiplexed base video encoding data, additional video encoding data, and signaling information (S260). The transmitter 160 may convert the base layer data and the enhancement layer data into at least one transport stream and transmit the converted at least one transport stream. Here, the transport stream may be referred to as a physical layer pipe (PLP) stream, or other terms may be used depending on the type of network to be transmitted.

3 is a diagram illustrating a reference image selection method of an image selector according to an embodiment of the present invention.

Referring to FIG. 3 , the image selector 110 compares the image quality between the left and right images in order to select a good image quality. Left and right images are acquired through binocular cameras. Therefore, the similarity between the left image and the right image is very high. The image selector 110 compares the image quality between the two images through the sharpness of the left and right images having a high degree of similarity, and uses the result of image encoding as a method for determining the sharpness of the left and right images. Since commonly used video encoding includes a process of converting the frequency of video signals, if the video selector 110 appropriately uses the result of video encoding, it is possible to compare the picture quality between the left video and the right video having a difference in sharpness.

The image selector 110 calculates a peak signal to noise ratio (PSNR) or bit rate of the encoded images for each of the left and right images (S310). Encoded images for each of the left and right images may be obtained using a separate image encoder.

The image selector 110 compares the PSNR or bit rate between the encoded images of the left and right images respectively (S320).

Based on the comparison result, the image selector 110 selects an image having high sharpness among the left and right images as a reference image (S330), and selects the remaining images as additional images (S340).

Specifically, the image selector 110 may select a reference image by measuring the PSNR of the encoded image corresponding to the left image and the PSNR of the encoded image corresponding to the right image and comparing the PSNRs of the two images. The sharpness of the left and right images can be determined using the PSNR of the encoded image. PSNR can be calculated using Equations 1 and 2.

Here, MAX is the maximum value that one pixel of the image can represent. For example, if one pixel represents 8 bits, MAX may be 255. Mean Square Error (MSE) is a noise value, and can be calculated by averaging the sum of squares of pixel-to-pixel differences between the original image and the image with data loss, as shown in Equation 2. The unit of PSNR uses dB, and if MSE is 0, PSNR has an infinite value.

In Equation 2, C and R represent the original image and the encoded image, respectively, i and j represent the position of a pixel, and X and Y represent the horizontal and vertical sizes of the image, respectively.

If the left and right images with different sharpness are coded to have similar bit rates, the high-definition image has greater distortion due to encoding than the low-definition image, resulting in a higher MSE and therefore a lower PSNR. Accordingly, the image selection unit 110 may select an image having a lower PSNR between the PSNR of the encoded image corresponding to the left image and the PSNR of the encoded image corresponding to the right image as the base image.

Alternatively, the image selector 110 may select a base image by comparing the bit rate of the encoded image corresponding to the left image with the bit rate of the encoded image corresponding to the right image. In image encoding, since an image with high definition contains relatively more high-frequency components than an image with slightly lower definition, a lower quantization parameter (QP) is required to encode the image closer to the original image. Therefore, if the left image and the right image having different sharpness are coded with the same QP, the bit rate of the image having high sharpness is relatively large. Accordingly, the image selection unit 110 may select an image having a higher bit rate between the bit rate of the encoded image corresponding to the left image and the encoded image corresponding to the right image as the base image.

4 is a diagram illustrating a 3DTV broadcast transmitting device according to a second embodiment of the present invention.

Referring to FIG. 4 , the 3DTV broadcast transmitting device 100a does not use independent video encoders for each of a base video and an additional video. That is, the 3DTV broadcast transmitting device 100a may include one hierarchical video encoder 170 instead of the first and second video encoders 130 and 140 .

The base image and the down-sampled auxiliary image are transferred to the hierarchical image encoder 170 . The hierarchical video encoder 170 generates a base layer stream by encoding the downsampled auxiliary picture, and generates an enhancement layer stream by encoding a correlation picture between an image decodable from the base layer data (ie, the auxiliary picture) and the reference picture. do. Here, the correlation image means an image generated by using a decodeable image (ie, an additional image) from a base layer stream and a reference image. At this time, since the additional video is down-sampled by the down-sampling conversion unit 120 and input to the hierarchical video encoder 170, a resolution difference between the base video and the down-sampled additional video occurs. Therefore, the hierarchical image encoder 170 upsamples the decodable image from the base layer stream again to compensate for the resolution difference between the base image and the downsampled additional image, and then compresses the correlation image between the upsampled image and the reference image. Thus, an enhancement layer stream can be created.

The base layer stream, the enhancement layer stream, and signaling information are multiplexed by the multiplexer 150 and transmitted through the transmitter 160.

In this way, when the base layer stream, the enhancement layer stream, and signaling information are multiplexed and transmitted, the 3DTV broadcast receiving device can obtain an additional video from the base layer stream and generate a base video using the base layer stream and the enhancement layer stream. can be obtained

5 is a diagram illustrating a 3DTV broadcast transmission device according to a third embodiment of the present invention.

Referring to FIG. 5 , the 3DTV broadcast transmission device 100b is different from the 3DTV broadcast transmission device 100 according to the first embodiment in that it further includes an additional information encoder 180 .

The additional information encoder 180 generates additional information for improving the quality of an additional video. The additional information encoder 180 may generate additional information using the additional video, base video encoding data, and additional video encoding data. The additional information may include information indicating which of the base video and the additional video data is used to improve the quality of the additional video. The receiving device can improve the quality of the additional video by substituting it with better image information between the base video and the additional video in order to expand the lower resolution additional video to the same size as the base video. Using this principle, the additional information can be created. Using the additional information generated in this way, the 3DTV broadcast receiving device can improve the quality of the additional video to be close to that of the reference video.

The additional information encoder 180 may transmit the additional information to the multiplexer 150 after encoding the additional information according to a predetermined encoding method. The base video encoding data, the additional video encoding data, the signaling information, and the encoded additional information are multiplexed by the multiplexer 150.

6 is a diagram illustrating a 3DTV broadcast reception device according to the first embodiment of the present invention, and FIG. 7 is a flowchart illustrating a 3DTV broadcast reception method according to an embodiment of the present invention.

Referring to FIG. 6, the 3DTV broadcast receiving device 600 includes a receiver 610, a demultiplexer 620, first and second video decoders 630 and 640, an upsampling converter 650, and a combiner ( 660).

Referring to FIG. 7 , the receiving unit 610 receives at least one transport stream transmitted from the 3DTV broadcast transmission device 100 (S710) and transfers the received at least one transport stream to the demultiplexer 620.

The demultiplexer 620 demultiplexes the base video coded data, additional video coded data, and signaling information multiplexed by the 3DTV broadcast transmission device 100 and separates them into base video coded data, additional video coded data, and signaling information (S720). ). The demultiplexer 620 transfers the base video encoded data to the first video decoder 630, the additional video encoded data to the second video decoder 640, and transfers signaling information to the synthesis unit 660. do.

The first image decoder 630 decodes the base image encoding data using a predetermined image decoding method to obtain a base image (S730). In this case, the first image decoder 630 may be a hierarchical image decoder. When the first video decoder 630 is a hierarchical video decoder, the base layer stream and the enhancement layer stream generated by the hierarchical encoding of the hierarchical video encoder 170 may be hierarchically decoded to obtain a base video. .

The second video decoder 640 decodes the additional video encoding data using a predetermined video decoding method to obtain an additional video (S740). When the first video decoder 630 is a hierarchical video decoder, the second video decoder 640 decodes the base layer stream generated by the hierarchical encoding of the hierarchical video encoder 170 using a video decoding method. Additional images can be acquired. The up-sampling conversion unit 250 up-samples the additional video and transmits it to the synthesis unit 260 (S750). Since the additional video is downsampled by the downsampling conversion unit 120 of the 3DTV broadcast transmission device 100 and transmitted, the upsampling conversion unit 250 converts the additional video for synthesis with the reference video in the synthesis unit 260. The original resolution is restored by upsampling.

The compositing unit 260 synthesizes the base video and the up-sampled additional video based on the signaling information and provides a 3D service (S760). The synthesis unit 260 may determine whether the image selected as the reference image is a left image or a right image based on the signaling information, and display images corresponding to the left and right images, respectively.

FIG. 8 is a diagram illustrating a 3DTV broadcast receiving device according to a second embodiment of the present invention, and may be a 3DTV broadcast receiving device corresponding to the 3DTV broadcast transmitting device 100b shown in FIG. 5 .

Referring to FIG. 8 , the 3DTV broadcast receiving device 600a is different from the 3DTV broadcast receiving device 600 according to the first embodiment in that it further includes an additional information decoder 670 and a picture quality improving unit 680. there is

When additional information is additionally multiplexed and transmitted by the 3DTV broadcast transmission apparatus 100b, the demultiplexing of the demultiplexer 620 separates base video encoding data, additional video encoding data, signaling information, and encoded additional information, The encoded side information is delivered to the side information decoder 670.

The additional information decoder 670 decodes the encoded additional information using a predetermined decoding method to obtain additional information, and transmits the additional information to the picture quality improving unit 680.

The picture quality improving unit 680 uses the additional information to improve the picture quality of the additional video to be close to that of the reference video.

In addition, a 3D service may be provided by synthesizing a base image and an additional image having improved image quality by the synthesis unit 660 .

At this time, when synthesizing the additional video and the base video, signaling information including playback time of the base video, playback time of the additional video, and error information between the playback time of the base video and the playback time of the additional video is separately transmitted to synchronize them. can

Embodiments of the present invention are not implemented only through the devices and/or methods described above, and may be implemented through a program that realizes functions corresponding to the configuration of the embodiments of the present invention or a recording medium on which the program is recorded. Implementation can be easily implemented by an expert in the technical field to which the present invention belongs based on the description of the above-described embodiments.

Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements of those skilled in the art using the basic concept of the present invention defined in the following claims are also included in the scope of the present invention. that fall within the scope of the right.

Claims (10)

delete As a method for selecting a reference video in a broadcast transmission device,
Calculating a peak signal to noise ratio (PSNR) of an encoded image for each of a left image and a right image constituting a three-dimensional image;
Selecting a video having a lower PSNR among the PSNRs of the encoded video for each of the left video and the right video as a base video and selecting the remaining video as an additional video;
Downsampling the additional video;
encoding the base picture and the downsampled additional picture; and
Multiplexing and transmitting information on the encoded base image, the encoded downsampled additional image, and an image selected as the base image
Reference image selection method comprising a. As a method for selecting a reference video in a broadcast transmission device,
Calculating the bit rate of the encoded image for each of the left and right images constituting the three-dimensional image;
Selecting an image having a higher bit rate among bit rates of encoded images for each of the left image and the right image as a reference video and selecting the remaining video as an additional video;
Downsampling the additional video;
encoding the base picture and the downsampled additional picture; and
Multiplexing and transmitting information on the encoded base image, the encoded downsampled additional image, and an image selected as the base image
Reference image selection method comprising a. In paragraph 2 or 3,
The coding step is
Encoding the downsampled additional video to generate a base layer stream; and
and generating enhancement layer data by encoding a correlation image between the reference image and an image decoded from the base layer stream. In paragraph 2 or 3,
Encoding additional information for improving the picture quality of the additional video
Including more,
The transmitting step includes multiplexing the encoded base image, the encoded downsampled additional image, information of the image selected as the base image, and the encoded additional information. delete As a broadcast transmission device,
A peak signal to noise ratio (PSNR) of an encoded image for each of the left and right images constituting a three-dimensional image is calculated, and the PSNR of the encoded image for each of the left and right images is calculated. An image selection unit that selects an image having a lower PSNR among the images as a reference image and selects the remaining images as additional images;
a downsampling converter for downsampling the additional video;
An image encoder for encoding the base image and the downsampled additional image; and
A multiplexer for multiplexing information on the encoded base image, the encoded downsampled additional image, and the image selected as the base image
A broadcast transmission device comprising a. As a broadcast transmission device,
The bit rate of the encoded image for each of the left and right images constituting the three-dimensional image is calculated, and the image having the higher bit rate among the bit rates of the encoded image for each of the left and right images An image selection unit that selects as a reference image and selects the remaining images as additional images;
a downsampling converter for downsampling the additional video;
An image encoder for encoding the base image and the downsampled additional image; and
A multiplexer for multiplexing information on the encoded base image, the encoded downsampled additional image, and the image selected as the base image
A broadcast transmission device comprising a. In paragraph 7 or 8,
The video encoder performs hierarchical video encoding to generate a base layer stream by encoding the downsampled additional video, and to generate enhancement layer data by encoding a correlation image between the reference video and an image decoded from the base layer stream. A broadcast transmission device that performs. In paragraph 7 or 8,
Additional information encoder for generating additional information for improving the image quality of the additional video and encoding the additional information
Including more,
The multiplexer multiplexes the encoded reference video, the encoded down-sampled additional video, information of the video selected as the reference video, and the encoded additional information.

Images