JP2020145584A - Transmitting apparatus, receiving apparatus and program - Google Patents

Abstract

To realize high quality video transmission by reducing deterioration of image quality at light compression time, generated by dividing a video frame.SOLUTION: A transmitting apparatus 1 comprises: a video dividing section 11 that generates L (L≥2) division frames from a video frame by dividing the video frame in a horizontal direction using K (K≥1) horizontal direction pixel columns within the video frame as a division unit, without dividing an input video frame in a vertical direction; a video compressing section 12 that generates L compressed division frames from L division frames by performing compression processing using a light compression technology which uses M (M≥1) horizontal direction pixel columns within the division frame as a compression unit, for each of L division frames; and a transmitting section 13 that transmits L compressed division frames through a transmission channel 3 to a receiving apparatus 2.SELECTED DRAWING: Figure 1

Description

The present invention relates to a transmitter, a receiver, and a program.

One of the video compression technologies is a compression technology called light compression. This compression technology has a compression rate of about a fraction and is not high, but can perform compression processing with low delay, and there are LLVC (Low Latency Video Codec), VC2, TICO (Tiny Codec) / JPEG-XS. (See, for example, Non-Patent Documents 1 to 4).

In each of the light compression technologies, the Wavelet transform is applied every few lines in units of horizontal pixel strings (hereinafter, appropriately referred to as "lines") in one video frame, and the video signal is divided into low frequency components and high frequency components. It is divided into two parts, and compression processing is performed by performing entropy coding or the like centering on high frequency components. Since the processing unit is completed every few lines, compression processing is possible with a low delay of several lines.

When lightly compressing and transmitting video in order to perform video transmission that requires low delay at the level of several lines using a line with a limited transmission capacity, depending on the conditions of the transmission capacity, the transmission capacity can be simply compressed. May be missing. At this time, there is a method of dividing the video, lightly compressing the divided video, and transmitting the divided video using a plurality of lines.

For light compression, a system specialized for a specific format such as 2K / 4K / 8K is required. However, when transmitting video in different formats, if the video is divided into a common format, it becomes possible to flexibly deal with it by constructing a system in that common format.

As a method for dividing the video, there are methods such as 2SI (2-Simple Interleave Division) and SQD (Quare Division). 2SI is a method in which even-numbered and odd-numbered lines of a video frame are sampled in the horizontal direction at intervals of 2 pixels and divided into four to generate a divided frame. SQD is a method of generating a divided frame by dividing the video frame into four in a rice field shape.

SMPTE RDD34, "LLVC-Low Latency Video Codec for Network Transfer" SMPTE ST 2042, "VC-2 Video Compression" SMPTE RDD35, "TICO Lightweight Codec Used in IP Networked or in SDI Infrastructure" ISO / IEC 21122 “Information technology --Low-latency light weight image coding system”

As a result of diligent studies, the present inventors have divided the video frame by the conventional division method 2SI or SQD and then lightly compressed each divided frame, as compared with the case where the undivided video frame is lightly compressed as it is. We obtained the finding that the image quality deteriorates.

Since 2SI integrates discrete pixels in a video frame to generate a divided frame, it is considered that a significant increase in high frequency components leads to deterioration of image quality.

Since the SQD divides the video frame into a T-shape, the parts with high frequency components in a part of the video frame are concentrated in some of the divided frames, and the image quality deteriorates in some of the divided frames. Conceivable.

Therefore, an object of the present invention is to provide a transmitting device, a receiving device, and a program that reduce the deterioration of image quality during light compression caused by dividing a video frame and realize high-quality video transmission.

The transmission device according to the first aspect does not divide the input video frame in the vertical direction, but horizontally divides the video frame using K (K ≧ 1) horizontal pixel rows in the video frame as a division unit. For each of the video division unit that generates L (L ≧ 2) divided frames from the video frame by dividing in the direction and the L divided frames, M (M) in the divided frame. A video compression unit that generates L compression division frames from the L division frames and a transmission line by performing compression processing using a light compression technique using the horizontal pixel sequence of ≧ 1) as a compression unit. The gist is to include a transmission unit that transmits the L compression-divided frames to the receiving device via the transmission unit.

The receiving device according to the second aspect is the receiving unit that receives L (L ≧ 2) compression division frames from the transmission device via the transmission line, and the L compression division frames for each of the L compression division frames. L division frames are generated from the L compression division frames by performing decoding processing using a light compression technique in which M (M ≧ 1) horizontal pixel strings in the compression division frame are used as the decoding unit. The L divided frames are vertically combined with the K (K ≧ 1) horizontal pixel strings in each divided frame as a coupling unit without horizontally combining the video decoding unit and the L divided frames. It is a gist to include a video coupling portion that restores the original video frame corresponding to the L divided frames by combining in the direction.

The program according to the third aspect does not divide the video frame input to the transmission device in the vertical direction, and uses K (K ≧ 1) horizontal pixel sequences in the video frame as a division unit for the video. A video division process that generates L (L ≧ 2) divided frames from the video frame by dividing the frame in the horizontal direction, and M in the divided frame for each of the L divided frames. Video compression processing that generates L compression division frames from the L division frames by performing compression processing using a light compression technology that uses a number of (M ≧ 1) horizontal pixel strings as a compression unit. The gist is to execute a transmission process of transmitting the L compression-divided frames to a receiving device via a transmission line.

The program according to the fourth aspect is for the reception process of receiving L (L ≧ 2) compression division frames from the transmission device via the transmission line to the receiving device, and for each of the L compression division frames. Then, by performing a decoding process using a light compression technique in which M (M ≧ 1) horizontal pixel strings in the compression division frame are used as the decoding unit, L divisions are performed from the L compression division frames. The video decoding process for generating a frame and the L division using the K (K ≧ 1) horizontal pixel strings in each division frame as a combination unit without horizontally combining the L division frames. The gist is to execute a video combining process that restores the original video frames corresponding to the L divided frames by combining the frames in the vertical direction.

According to the present invention, it is possible to provide a transmission device, a reception device, and a program that reduce deterioration of image quality during light compression caused by dividing a video frame and realize high-quality video transmission.

It is a figure which shows the system configuration which concerns on embodiment. It is a figure which shows the multi-format transmission which concerns on embodiment. It is a figure which shows the operation example of the image division part which concerns on embodiment. It is a figure which shows the compression process in a light compression technique TICO. It is a figure which shows a mode that the image compression unit compresses the division frame output by the image division unit which concerns on embodiment using TICO as a light compression technique. It is a figure which shows the image division by 2SI. It is a figure which shows the image division by SQD. It is a figure which shows the PSNR difference [dB] with respect to the case without division for LLD, 2SI, and SQD. It is a figure which shows the simulation evaluation of the image quality when the LLD which concerns on embodiment is compared with 2SI and SQD.

An embodiment will be described with reference to the drawings. In the description of the drawings, the same or similar parts are designated by the same or similar reference numerals.

(System configuration)
First, the system configuration according to this embodiment will be described. FIG. 1 is a diagram showing a system configuration according to the present embodiment.

As shown in FIG. 1, the system according to the present embodiment includes a transmitting device 1 and a receiving device 2. The transmission device 1 transmits video to the reception device 2 via the transmission line 3.

The standardization and practical application of an IP production system that produces programs using IP (Internet Protocol) technology is being discussed. If this program production system is used, the video and audio materials required for program production are transmitted from the relay site to the broadcasting station using the IP line, which is the existing infrastructure, so that the broadcasting station can produce the program. Remote production becomes possible. In order to realize IP remote production, it is necessary to share video and audio materials in near real time between the relay site and the broadcasting station.

For example, the transmitting device 1 is provided at the relay site, and the receiving device 2 is provided at the broadcasting station. The transmission line 3 is a wired transmission line composed of an IP line between the relay site and the broadcasting station.

When the transmitting device 1 transmits the video material necessary for program production to the receiving device 2 (broadcasting station), the broadcasting station can produce the program instead of the relay site.

The transmission capacity of the IP line (transmission line 3) is limited. In order to share the video in real time between the relay site and the broadcasting station using this IP line, the transmission device 1 performs a compression process using a light compression technique after the video is divided.

Video materials in a plurality of formats such as 2K / 4K / 8K are input to the transmission device 1 from a camera or the like. In order to efficiently transmit video materials in such a plurality of formats by the same transmission device 1, the transmission device 1 divides the video so as to unify the video formats into the 2K format as shown in FIG. Do.

In the example shown in FIG. 2, the transmission device 1 divides the 4K format video frame into four 2K format divided frames, and divides the 8K format video frame into 16 2K format divided frames. The details of the video division will be described later. It should be noted that FIG. 2 shows the concept of video division, and the shape of the division frame and the like are different from the actual ones.

By unifying to the 2K format in this way, it is possible to standardize the processing from light compression to IP packetization, and the same transmission device 1 can support a plurality of formats. Therefore, it is possible to flexibly deal with production sites where 2K / 4K / 8K video materials are mixed, and it is possible to efficiently transmit video from the viewpoint of line use because it is unified in a small 2K format.

(Transmitting device)
Next, the transmission device 1 according to the present embodiment will be described.

As shown in FIG. 1, the transmission device 1 includes a video division unit 11, a video compression unit 12, and a transmission unit 13. A video signal is input to the video dividing unit 11 from a camera or the like, and the video is divided for each video frame included in the video signal.

The video division unit 11 divides a video frame using K (K ≧ 1) lines in the input video frame as a division unit, and generates L (L ≧ 2) division frames from the video frame. Then, L divided frames are output to the video compression unit 12. The line means a horizontal component for one pixel in a video frame, and corresponds to a horizontal pixel sequence.

The video dividing unit 11 divides the video frame in the horizontal direction to generate the divided frame without dividing the input video frame in the vertical direction. Such a new method of video division is called LLD (Linear Line Division).

Since the video compression unit 12 performs compression processing every several lines, the video division unit 11 also divides the video in line units. In the present embodiment, the number of lines "K" corresponding to the division unit in which the video division unit 11 performs the division processing is equal to the number of lines "M" corresponding to the compression unit in which the video compression unit 12 performs the compression processing. In the following, an example in which K = M = 2 will be described, but K and M may be “1” or K and M may be “3” or more depending on the method of the light compression technique. There may be.

The video division unit 11 determines the value of L according to the format of the video frame so that the format of the division frame is a predetermined format (that is, 2K format). Specifically, the video dividing unit 11 determines that L = "4" when the video frame is in the 4K format. The video dividing unit 11 determines that L = "16" when the video frame is in the 8K format.

FIG. 3 is a diagram showing an operation example of the video dividing unit 11. FIG. 3 shows an example of dividing an 8K format video frame by LLD.

First, as shown in FIG. 3A, the video dividing unit 11 divides a video frame using two lines consecutive in the vertical direction as a dividing unit. When generating one divided frame, the video dividing unit 11 extracts N (N ≧ 2) lines from the video frame at intervals in the vertical direction, thereby (2 × N). Generate a split frame consisting of the lines of.

In the example shown in FIG. 3A, the video dividing unit 11 extracts the first line, the second line, the ninth line, the tenth line, and the like of the video frame, and one piece composed of these extracted lines. Generate a split frame for. This divided frame is a 4K format divided frame.

The video dividing unit 11 extracts the third line, the fourth line, the eleventh line, the twelfth line, and the like of the video frame, and generates one divided frame composed of these extracted lines. This divided frame is a 4K format divided frame.

The video dividing unit 11 extracts the 5th line, the 6th line, the 13th line, the 14th line, and the like of the video frame, and generates one divided frame composed of these extracted lines. This divided frame is a 4K format divided frame.

The video dividing unit 11 extracts the 7th line, the 8th line, the 15th line, the 16th line, and the like of the video frame, and generates one divided frame composed of these extracted lines. This divided frame is a 4K format divided frame.

As a result, the video dividing unit 11 generates four divided frames in the 4K format as shown in FIG. 3 (b).

Secondly, as shown in FIG. 3B, the video dividing unit 11 further divides each of the four divided frames in the 4K format into four, with two lines consecutive in the vertical direction as the dividing unit. Generates 16 split frames in 2K format. The method of generating the 2K format divided frame from the 4K format divided frame is the same as the method of generating the 4K format divided frame from the 8K format video frame.

As a result, as shown in FIG. 3C, the video division unit 11 generates 16 division frames in the 2K format and outputs the 16 division frames in the 2K format to the video compression unit 12.

As shown in FIGS. 3 (b) and 3 (c), the divided frame has an elongated shape in the horizontal direction.

An example of dividing an 8K format video frame into four 4K format divided frames and then dividing four 4K format divided frames into 16 2K format divided frames has been described. However, the video frame in the 8K format may be directly divided into 16 divided frames in the 2K format without going through the 4K format.

Further, although an example in which the input video frame is in the 8K format has been described, the input video frame may be in the 4K format. When the input video frame is in the 4K format, the video division unit 11 performs the division processing in the same manner as described above, and divides one video frame in the 4K format into four video frames in the 2K format. ..

The video compression unit 12 performs compression processing on each of the L division frames output by the video division unit 11 using a light compression technique in which M lines in the division frame are used as compression units. L compression division frames are generated from the L division frames, and L compression division frames are output to the transmission unit 13.

In the light compression technology, since the compression processing unit is completed for each M line (M = 2 in this embodiment), the compression rate is about a fraction, but the compression processing can be performed with low delay. it can.

In this embodiment, an example in which TICO is used as the light compression technique will be described, but other methods such as LLVC and VC2 may be used.

FIG. 4 is a diagram showing a compression process in TICO.

As shown in FIG. 4, TICO performs wavelet transform in the horizontal direction and the vertical direction for two consecutive lines, and obtains four pixel information (LL, HL, LH, HH) with a high frequency component and a low frequency component. Divide into. The LL region is the lowest frequency signal obtained by down-sampling the original video frame in the horizontal and vertical directions by 1/2. By using LL and HL, LH, and HH, it is possible to restore a video frame having the original number of pixels. After that, lossy compression is performed by reducing the amount of information by entropy coding or the like. Here, since it is difficult for human vision to recognize the high frequency component of the image, the high frequency component is mainly lossy compressed.

FIG. 5 is a diagram showing a state in which the video compression unit 12 compresses the division frame output by the video division unit 11 using TICO.

As shown in FIG. 5 (b), the first and second lines of the divided frame correspond to the first and second lines of the original video frame shown in FIG. 5 (a), and the third and fourth lines of the divided frame are shown in FIG. The configuration corresponds to the 9th and 10th lines of the original video frame shown in 5 (a).

The divided frame has a different configuration from the original video frame, but by combining the divided unit and the compression unit, the wavelet transform processing performed in units of two lines becomes the same, so during light compression caused by the division. It is possible to reduce the deterioration of image quality.

The transmission unit 13 transmits the L compression division frames output by the video compression unit 12 to the reception device 2 via the transmission line 3. The transmission unit 13 has an IP packetization unit 13a and a transmission buffer 13b.

The IP packetization unit 13a converts the L compression division frames output by the video compression unit 12 into IP packets, and outputs the IP packets to the transmission buffer 13b.

The transmission buffer 13b temporarily stores the IP packet output by the IP packetization unit 13a, and outputs the IP packet to the transmission line 3.

The transmission unit 13 may transmit L compressed and divided frames in parallel using a plurality of communication lines. For example, the transmission unit 13 transmits L compression division frames in parallel via L communication lines.

(Receiver)
Next, the receiving device 2 according to the present embodiment will be described. The receiving device 2 according to the present embodiment reverses the processing performed by the transmitting device 1 described above.

As shown in FIG. 1, the receiving device 2 includes a receiving unit 21, a video decoding unit 22, and a video coupling unit 23.

The receiving unit 21 receives L compression-divided frames from the transmitting device 1 via the transmission line. The receiving unit 21 has a receiving buffer 21a and an IP depackaging unit 21b.

The reception buffer 21a temporarily stores IP packets received from the transmission device 1 via the transmission line 3, and outputs the IP packets to the IP depacketization unit 21b.

The IP depacketization unit 21b converts the IP packet output by the reception buffer 21a into an IP depacket, and outputs L compressed and divided frames to the video decoding unit 22.

The video decoding unit 22 performs a decoding process for each of the L compression-divided frames output by the IP depackaging unit 21b using a light compression technique in which M lines in the compression-division frame are used as decoding units. By doing so, L division frames are generated from the L compression division frames, and the L division frames are output to the video coupling unit 23. In this embodiment, M = 2.

The video combining unit 23 does not horizontally combine the L divided frames output by the video decoding unit 22, but vertically combines the L divided frames with the K lines in each divided frame as the combining unit. By doing so, the original video frame corresponding to the L divided frames is restored, and the restored original video frame is output. In this embodiment, K = 2.

For example, the video coupling unit 23 vertically combines 16 divided frames in the 2K format shown in FIG. 3 (c) with two lines as the coupling unit in the vertical direction to form 4 in the 4K format shown in FIG. 3 (b). Generate individual split frames.

Then, the video coupling unit 23 joins the four divided frames of the 4K format shown in FIG. 3B in the vertical direction with the two lines as the coupling unit to form the source of the 8K format shown in FIG. 3A. Restore the video frame of.

However, the video coupling unit 23 may directly restore the original video frame in the 8K format from the 16 divided frames in the 2K format without going through the 4K format.

(Comparison example of video division)
Next, 2SI and SQD will be described as a comparative example of the LLD according to the present embodiment.

FIG. 6 is a diagram showing video division by 2SI.

As shown in FIG. 6, the 2SI samples each of the even and odd lines of the video frame in the horizontal direction at intervals of 2 pixels and divides them into four. Since the divided frame obtained by dividing into four is composed of sampling of discrete pixels of the original video frame, the high frequency component increases. Therefore, the deterioration of image quality when compressed / decrypted increases.

According to the LLD according to the present embodiment, the divided frame is composed of pixels that are continuous in the horizontal direction, and the increase of high frequency components can be suppressed, so that the deterioration of image quality can be suppressed as compared with 2SI.

FIG. 7 is a diagram showing video division by SQD.

As shown in FIG. 7, the SQD divides the video frame into four in a rice field shape. Further, each divided video frame is divided into four by a paddy character. Even if the video frame contains low frequency components and high frequency components as a whole, the low frequency components or high frequency components are concentrated in a specific divided frame among the four divided frames obtained by the four divisions. In some cases.

That is, when the video frame is divided, a divided frame in which low frequency components are concentrated and a divided frame in which high frequency components are concentrated can be obtained. Here, if the same compression ratio is used for all the divided frames, the image quality deterioration greatly increases when the divided frames in which the high frequency components are concentrated are compressed and decoded.

According to the LLD according to the present embodiment, since the divided frame has a frequency component similar to that of the original video frame and the high frequency component is less likely to be biased like the SQD, deterioration of image quality can be suppressed as compared with the SQD.

(Example of effect of embodiment)
Next, as an example of the effects of the transmitting device 1 and the receiving device 2 shown in FIG. 1, a simulation evaluation of image quality when the LLD is compared with the 2SI and the SQD will be described.

A plurality of types of 8K images were divided into 16 using 2SI, SQD, and LLD, and light compression was performed by changing the compression ratio. After decoding, the image was integrated into an 8K image, and a PSNR (Peak Signal-to-Noise Radio) with the original image before division was obtained. Since TICO is applied to light compression and the unit of compression processing is 2 lines, the number of LLD division lines is also 2 lines. The PSNR when the 8K image was lightly compressed and decoded without being divided was also obtained (referred to as "no division"). The compressibility was varied in the range of 1/2 to 1/8. The evaluation image used was an ultra-high definition, wide color gamut standard image provided by the Institute of Image Information and Television Engineers.

FIG. 8 shows the PSNR difference [dB] from the case without division. However, the compression rate is 1/6. FIG. 9 shows the PSNR [dB] for each compression rate of the image "Stained Glass". As a result of the evaluation, 2SI deteriorated by 4 to 9 [dB] in all images. The SQD deteriorated by a maximum of 4 [dB] depending on the image, but there are some images of less than 1 [dB]. This is because the deterioration of the image quality of the SQD is affected by the bias of the high frequency component that differs for each image. In LLD, the deterioration of image quality was within 1 [dB] in all images, and better characteristics than 2SI and SQD were obtained, and better characteristics than 2SI and SQD were obtained in all images.

Therefore, in the evaluation of image quality, it was found that the LLD according to the present embodiment had better PSNR characteristics than the existing 2SI and SQD, and the image quality was less deteriorated, and the effectiveness of the LLD was confirmed.

(Other embodiments)
A program for causing the computer to execute each process performed by the transmitting device 1 and a program for causing the computer to execute each process performed by the receiving device 2 may be provided. The program may be recorded on a computer-readable medium. Computer-readable media can be used to install programs on a computer. Here, the computer-readable medium on which the program is recorded may be a non-transient recording medium. The non-transient recording medium is not particularly limited, but may be, for example, a recording medium such as a CD-ROM or a DVD-ROM.

Further, the functional unit (circuit) for executing each process performed by the transmission device 1 may be integrated, and the transmission device 1 may be configured as a semiconductor integrated circuit (chipset, SoC). Similarly, a functional unit (circuit) that executes each process performed by the receiving device 2 may be integrated, and the receiving device 2 may be configured as a semiconductor integrated circuit (chipset, SoC).

Although one embodiment has been described in detail with reference to the drawings above, the specific configuration is not limited to the above, and various design changes and the like can be made without departing from the gist. ..

1: Transmission device 2: Reception device 3: Transmission path 11: Video division unit 12: Video compression unit 13: Transmission unit 13a: IP packetization unit 13b: Transmission buffer 21: Reception unit 21a: Reception buffer 21b: IP depacketization unit 22: Video decoding unit 23: Video coupling unit

Claims (8)

By dividing the video frame in the horizontal direction using K (K ≧ 1) lines in the video frame as a division unit without dividing the input video frame in the vertical direction, L pieces from the video frame. A video division unit that generates a division frame of (L ≧ 2) and
Each of the L divided frames is subjected to a compression process using a light compression technique in which M (M ≧ 1) horizontal pixel sequences in the divided frame are used as a compression unit, thereby performing the L pieces. A video compression unit that generates L compression division frames from the division frames of
A transmission device including a transmission unit that transmits the L compression-divided frames to a reception device via a transmission line. The transmitter according to claim 1, wherein the value of K is equal to the value of M. When generating one of the L division frames, the video division unit creates N horizontal pixel sequences (N ≧ 2) from the video frame at intervals in the vertical direction. The transmitting apparatus according to claim 1 or 2, wherein the one divided frame composed of (K × N) horizontal pixel sequences is generated by extraction. Any one of claims 1 to 3, wherein the video dividing unit determines the value of L according to the format of the video frame so that the format of the divided frame is a predetermined format. The transmitter according to item 1. The video dividing unit divides the video frame in the horizontal direction using the K horizontal pixel sequences (K ≧ 2) that are continuous in the vertical direction as the dividing unit.
The image compression unit according to any one of claims 1 to 4, wherein the image compression unit performs the compression process using the M (M ≧ 2) horizontal pixel sequences that are continuous in the vertical direction as the compression unit. The transmitter described. A receiver that receives L (L ≧ 2) compression division frames from the transmission device via the transmission line, and
Each of the L compression-divided frames is subjected to a decoding process using a light compression technique in which M (M ≧ 1) horizontal pixel strings in the compression-divided frame are used as decoding units. A video decoding unit that generates L division frames from L compression division frames,
By combining the L divided frames in the vertical direction with the K (K ≧ 1) horizontal pixel strings in each divided frame as the coupling unit without combining the L divided frames in the horizontal direction. A receiving device including a video coupling unit that restores the original video frame corresponding to the L divided frames. To the transmitter
The video frame is divided horizontally by using K (K ≧ 1) horizontal pixel sequences in the video frame as a division unit without dividing the input video frame in the vertical direction. Video division processing that generates L (L ≧ 2) division frames from
Each of the L divided frames is subjected to a compression process using a light compression technique in which M (M ≧ 1) horizontal pixel sequences in the divided frame are used as a compression unit, thereby performing the L pieces. Video compression processing that generates L compression division frames from the division frame of
A program characterized by executing a transmission process of transmitting the L compression-divided frames to a receiving device via a transmission line. To the receiver
Reception processing for receiving L (L ≧ 2) compression division frames from the transmission device via the transmission line, and
Each of the L compression-divided frames is subjected to a decoding process using a light compression technique in which M (M ≧ 1) horizontal pixel strings in the compression-divided frame are used as decoding units. Video decoding processing that generates L division frames from L compression division frames, and
By combining the L divided frames in the vertical direction with the K (K ≧ 1) horizontal pixel strings in each divided frame as the coupling unit without combining the L divided frames in the horizontal direction. , A program characterized by executing a video combination process for restoring the original video frames corresponding to the L divided frames.

Images