KR20200082332A - Transmitting and receiving apparatus for providing a real-time image and a high-quality image - Google Patents

Abstract

According to the present invention, disclosed is a transmission/reception device for providing a real-time image which may have damage to image quality but have no time delay and a high-quality image which may have time delay but have high-quality equal to or better than increased quality of a real-time image and generating a real-time image and a high-quality image using the same source packet. A reception device of the transmission/reception device includes a reception unit, a first decoding unit, and a second decoding unit.

Description

Transceiver for providing real-time video and high-quality video together{TRANSMITTING AND RECEIVING APPARATUS FOR PROVIDING A REAL-TIME IMAGE AND A HIGH-QUALITY IMAGE}

The present invention relates to an apparatus for transmitting and receiving video, and more specifically, it may provide a high-quality video having the same or better image quality than a real-time video, although image quality damage may occur but real-time video with no delay time and time delay may occur. It relates to a transceiver for generating real-time video and high-quality video using the same source packet.

Conventionally, when a part of data is damaged or lost when data is transmitted from a transmitting device to a receiving device as in Non-Patent Document 1, an automatic repeat reQuest (ARQ) that requests retransmission of the damaged or lost data from the receiving device side to the transmitting device side ) Method, but it is difficult to apply the ARQ method to real-time broadcasting or real-time video transmission.

Recently, even if damage or loss of the original data occurs during transmission, the data containing the recovery data that can recover it is generated and transmitted, and even if some of the data received by the receiving device is damaged or lost, it is recovered. An application layer forward error correction (AL-FEC) encoding method capable of restoring original data from received data including data is used.

The AL-FEC encoding technology has a non-systematic code characteristic as in Non-Patent Document 2 or a systemic code characteristic as in Non-Patent Document 3.

If the AL-FEC encoding technology having unstructured code characteristics is, for example, LDPC (Low Density Parity Check) technology, it is assumed that 10% of recovery data per 100 original data is needed by referring to the average loss rate of the network. On the side, A1, A2,… , Based on the raw data of A100, a total of 110 new data B1, B2,… , B110 is generated and transmitted to the receiving device. The receiving devices are B1, B2,… , If any of 100 or more of B110 data is received, A1, A2,… , The entire A100 can be restored. However, if the receiving device receives less than 100 out of a total of 110 data, A1, A2,… However, none of the A100 data can be restored.

The AL-FEC encoding technique having a structural code characteristic is, for example, A1, A2,..., As a RaptorQ application layer forward error correction (RaptorQ AL-FEC) technique. , Based on the raw data of A100, a total of 110 pieces of data are generated, of which 100 are A1, A2,… , Use the original data of A100 as it is, B101, B102,… , Creating only B110, A1, A2,… , A100, B101, B102,… , B110 transmits a total of 110 data to the receiver. Where A1, A2,… , A100 is a source packet, B101, B102,… , B110 is a recovery packet. The receiving device receives less than 100 out of a total of 110 data, A1, A2,… , Even if the entire original data of A100 cannot be restored, A1, A2,… , The data received from A100 can be effectively used as it is. In addition, the receiving device receives more than 100 out of 110 data, A1, A2,… , Even if data not received during A100 occurs, the entire original data can be restored. However, the receiving device may fail to restore data even if it receives more than 100 data. For example, if 100 are received, the probability of failure is 1/100, and if 101 is received, the probability of failure is significantly reduced to 1/10000.

The AL-FEC encoding technique having a structural code characteristic can secure a target restoration reliability by setting the amount of recovered data in consideration of the average network loss rate and the allowable restore failure rate.

Conventional AL-FEC encoding technology having unstructured code characteristics or structured code characteristics, the source packets A1, A2,... , Since the A100 is collected and the recovery packet is generated and the source packet and the recovery packet are simultaneously transmitted, time delay may occur due to transmission, and it is difficult to apply to real-time video transmission.

1. RFC 3366 "Advice to link designers on link Automatic Repeat reQuest (ARQ)" 2. RFC 5170 "Low Density Parity Check (LDPC) Staircase and Triangle Forward Error Correction (FEC) Schemes" 3. RFC 6330 "RaptorQ Forward Error Correction Scheme for Object Delivery"

In order to solve the above problem, the present invention uses the characteristics of AL-FEC encoded data having structural code characteristics, transmits the source packet to the receiving device as soon as the source packet is generated, and transmits the recovery packet at a set time. By doing so, there is provided a transmission/reception device that minimizes time delay due to transmission.

The present invention provides a high-quality video having the same or improved image quality than a real-time video, although real-time video and time delay without delay may occur, but it can generate real-time video and high-quality video using the same source packet. It provides a transmitting and receiving device.

The transmitting and receiving device for providing both a real-time video and a high-quality video according to an embodiment of the present invention for solving the above problems includes a transmitting device and a receiving device, and the receiving device receives a source packet from a transmitting device in real time. A receiving unit that receives and receives a recovery packet at a set time; It includes a first decoding unit for decoding the source packet to generate a real-time image and a second decoding unit for generating a high-quality image with the same or improved image quality from the real-time image using the recovery packet. The decoding unit may be characterized by using the same source packet.

The receiving device receives the source packet and the recovery packet from the receiving unit and temporarily stores them, provides the source packet to the first decoding unit upon receiving the received packet, and temporarily stores the Nth group source packet and the recovery packet composed of the set data size. If so, it may be characterized in that it further comprises a buffer for providing the source packet and the recovery packet of the N-th group to the second decoding unit.

The transmitting device includes: a source packet generating unit receiving a source image from an imaging device and generating a source packet; After collecting the source packet with a set data size, the recovery packet generation unit for generating a recovery packet based on the collected source packet and the source packet as it is generated are transmitted to a receiving device, and the recovery packet is transmitted at a set time. It may be characterized in that it comprises a transmitter.

The set timing may be characterized in that the recovery packet of the N-th group is transmitted between the source packets of the N-th group.

The present invention can provide a high-quality image having the same or improved image quality than a real-time image, although a real-time image and a time delay without a delay may occur, and a real-time image can be utilized to control a moving object, and a high-quality image is a reconnaissance image. Or it can be used as a relay video.

1 is a block diagram showing a transmitting and receiving device according to an embodiment of the present invention.
FIG. 2 is a block diagram showing a mobile body in which the transmission device of FIG. 1 is installed.
FIG. 3 is a block diagram showing the transmission device of FIG. 1 in detail.
4 is a detailed block diagram of the receiving device of FIG. 1 connected to one display device.
5 is a detailed block diagram of the receiving device of FIG. 1 connected to two display devices.
6 is an example of a screen of a display device.
7 to 9 are examples showing in detail the transceiver of FIG. 1.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings and the contents described in the accompanying drawings, but the present invention is not limited or limited by the embodiments.

1 is a block diagram showing a transmitting and receiving device according to an embodiment of the present invention, the transmitting and receiving device 10 for real-time packet transmission includes a transmitting device 100 and a receiving device 200. The transmitting device 100 encodes the source image to generate a source packet and a recovery packet, and transmits the source packet and the recovery packet to the receiving device 200. The receiving device 200 receives and decodes the source packet and the recovery packet to generate an image.

FIG. 2 is a block diagram showing a moving body in which the transmitting apparatus of FIG. 1 is installed, and the transmitting apparatus 100 is mounted on the moving body 400. The moving object may be an autonomous vehicle or a drone, but is not limited thereto.

The moving object 400 further includes an imaging device 300 that provides a source image to the transmitting device 100, includes control means 410 for real-time driving control, and transmits information about real-time driving control ( 100) or through a separate communication means (not shown). The real-time image may be used as an image for real-time driving control of the moving object.

FIG. 3 is a block diagram showing the transmission device of FIG. 1 in detail. The transmission device 100 includes a source packet generation unit 110, a recovery packet generation unit 120, and a transmission unit 130. The source packet generator 110 receives a source image from the imaging device 300 and encodes the source image to generate a source packet. The recovery packet generation unit 120 generates a recovery packet based on the source packet. More specifically, the recovery packet generation unit 120 collects the source packet to a set data size, and then generates a recovery packet based on the collected source packet.

When the recovery packet transmission is completed, the recovery packet generation unit 120 deletes the collected source packet and recovery packet. For example, the recovery packet generation unit 120 includes a low-capacity memory because the transmission device 100 is mounted on a moving object 400 such as a lightweight drone, and when the transmission of the recovery packet is completed in consideration of memory availability Delete the collected source and recovery packets.

The recovery packet generator 120 performs application layer forward error correction (AL-FEC) encoding with systemic code and fountain code characteristics to recover the source packet. Create a recovery packet. For example, the recovery packet generator 120 may generate a recovery packet for restoring a source packet by performing RaptorQ AL-FEC (Application Layer Forward Error Correction) encoding.

The transmitting unit 130 transmits the source packet to the receiving device as soon as the source packet is generated, and transmits the recovery packet at a set time. When the set time is set, the source device sequentially transmits the source packet to the receiving device immediately after generation, but remembers this and gathers a predetermined number of source packets. It may be a time to transmit to the receiving device. In addition, it may be different depending on the size of one source packet, a predetermined number, or the computing power of the transmitting device 100.

The transmitting unit 130 may transmit the N-th group recovery packet of the N-th group after transmitting the N-th group source packet having the set data size. In more detail, the transmitting unit 130 includes a buffer 220 for guaranteeing the reception time of a packet in the receiving device 200 to transmit the Nth group recovery packet between the N+1 group source packets. have.

The present invention can generate a real-time image through a source packet transmitted in real time and use it as an image for real-time driving control of a moving object, and generate a high-quality image through a recovery packet and use it as a reconnaissance image or a relay image. In addition, the present invention prevents transmission of the source packet of the N+1 group, which needs to be reproduced in real time at the receiving device 200 side, by inserting and transmitting the recovery packet of the N group between the source packets of the N+1 group. In addition, it can reduce the risk of multiple recovery packets being lost at once due to network errors.

The transmitting device 100 may transmit the recovery packet to the receiving device 200 after transmission of all source packets is completed. For example, the transmission device 100 mounted on the drone is equipped with a separate memory (not shown) in order to improve stability for real-time transmission of the source packet, and stores the recovery packet in the memory, and receives when the drone has completed driving. The wired or wireless connection to the device 200 may transmit the recovery packet stored in the memory to the receiving device 200.

Each of the source packet generation unit 110 and the recovery packet generation unit 120 may include an encoding module and a packetization module. The reason why the transmitting device 100 is divided into the source packet generation unit 110 and the recovery packet generation unit 120 is to provide a real-time video and a high-quality video with a time delay that can be deteriorated in the receiving device 200. It is for. Also, to remove the time delay of the real-time video decoded only with the source packet.

FIG. 4 is a detailed block diagram of the receiving device of FIG. 1 connected to one display device, and FIG. 5 is a detailed block diagram of the receiving device of FIG. 1 connected to two display devices. ) Includes a receiver 210, a buffer 220, a first decoder 230 and a second decoder 230. The receiving device 200 may be installed in a remote controller, a mobile terminal or a remote server configured with the mobile body 400.

The receiving unit 210 receives the source packet from the transmitting device in real time, and the first decoding unit 230 decodes the source packet to generate a real time image. The receiving unit 210 receives the recovery packet at a set time, and the second decoding unit 240 generates a high quality image through the source packet and the recovery packet. The real-time image is an image in which image quality may be damaged, but time delay due to transmission is reduced compared to the prior art. A high-quality video is a video that may have a time delay than a real-time video, but has the same or reduced image quality than a real-time video. Also, a high-quality video is a video that conforms to the original quality of the source video. The first decoding unit 230 and the second decoding unit 240 use the same source packet.

The second decoder 240 may provide a high quality image to the display device 500 as illustrated in FIG. 4, and may provide a high quality image to the second display device 520 as illustrated in FIG. 5. have. The second display device 520 may include a function of storing an image.

The buffer 220 temporarily receives and stores the source packet and the recovery packet from the receiving unit 210, and provides the source packet to the first decoding unit 230 upon receiving the source packet. And the recovery packet are temporarily stored, and the Nth group source packet and the recovery packet are provided to the second decoder 240. The buffer 220 guarantees the reception time of the packet to the reception device 200.

When the buffer 220 provides the Nth group source packet and the recovery packet to the second decoder, the Nth group source packet and the recovery packet may be deleted. For example, if the buffer 220 is a small remote controller that is one set with the moving object 400, it includes a low-capacity memory, and if the source packet and the recovery packet of the N group are provided to the second decoding unit in consideration of the availability of the memory, The source packet and the recovery packet of the Nth group are deleted.

FIG. 6 is an example of a screen of the display device, and FIGS. 6A and 6B can be applied to the display device 500 of FIG. 4, and FIG. 6C is the first display of FIG. 5 It may be applied to the device 510 and the second display device 520.

The present invention can provide a half-half method of dividing the first image and the second image into one display device 500, and the first image is displayed in a partial region of the second image by one display device 500. A picture-in-picture (pip) method may be provided, and a first image and a second image may be displayed on each of the two display devices 510 and 520.

In order to transmit an image using a conventional AL-FEC having unstructured code characteristics, the original data A1, A2,... , A100 waited to be generated, and from then on, B1, B2,… , B110 is generated to start transmission to the receiving device, and B1, B2,… , Waiting for at least 100 of B110 to reach, then A1, A2,… , Since A100 is decoded, time delay occurs due to transmission and reception, and it is difficult to apply to real-time video transmission.

The present invention uses AL-FEC having structural code characteristics, and A1, A2, ... on the transmitter 100 side. , As soon as each A100 is generated, it is immediately transmitted to the receiving device 200 side, and it is memorized and later B101, B102,… , B110 only generates and sends additionally, A1, A2,… on the receiving device 200 side. , Regardless of whether the A100 arrives completely, it is possible to minimize the time delay by reproducing the real-time video by using it as soon as each arrives. A1, A2,… , If there is data that failed to reach among A100, B101, B102,… that arrived later... , B110 can be restored to provide high-quality images. According to the present invention, there is a possibility that image quality may be damaged in this way, but a real-time image with a minimum time delay and a time delay may occur, but a high-quality image with a completely recovered transmission image may be provided together, and real-time using the same source packet. You can create images and high-quality images.

Real-time video with no delay even if there is some damage to the video is advantageous in controlling the moving object through the video. In addition, the reconnaissance video or relay video should be a high-quality video recovered from image quality loss even if there is some time delay to provide reliable information.

The present invention can utilize a real-time image to control the moving object, and can use a high-quality image as a reconnaissance image or a relay image. For example, high-quality images may be used as various information such as military reconnaissance images, documentary images, or sports relay images.

7 to 9 are examples of the transmission/reception device of FIG. 1 in detail. In the transmission device 100, the video encoder 110a corresponds to the source packet generator 110, and the AL-FEC encoder 120 Corresponds to the recovery packet generator 120. In the receiving apparatus 200, the receiving unit 210 can be performed in place of the function of the buffer 220, the first video decoder 230a corresponds to the first decoding unit 230, and the AL-FEC decoder 240a Corresponds to the second decoding unit 240. The second decoder 240 may further include a second video decoder 240b.

The AL-FEC decoder 240a may generate a source packet that has undergone a recovery process as shown in FIG. 7 and provide it to the first video decoder 230a. In this case, the first video decoder 230a may provide a real-time image according to a user's selection or a high-quality image reconstructed from the real-time image. That is, the first video decoder 230a may selectively generate real-time images and high-quality images.

As illustrated in FIG. 8, the AL-FEC decoder 240a may provide the source packet that has undergone the recovery process to the second video decoder 240b, and the second video decoder 240b may receive the source packet that has undergone the recovery process. Using it, it is possible to generate a high quality image with reduced image quality damage and provide a high quality image to the display device 500. Also, the second video decoder 240b may provide a high quality image to the second display device 520 as illustrated in FIG. 9.

10: transmitting and receiving device 100: transmitting device
200: receiving device 300: imaging device
400: moving object 500: display device

Claims (4)

In the transmitting and receiving device for providing a real-time image and a high-quality image together including a transmitting device and a receiving device,
The receiving device,
A receiving unit for receiving the source packet from the transmitting device in real time and receiving the recovery packet at a set time;
A first decoding unit for decoding the source packet to generate a real-time image and
A second decoding unit for generating a high-quality image having the same or improved image quality and real-time image using the recovery packet,
The first decoding unit and the second decoding unit transmitting and receiving device for providing a real-time image and high-quality image, characterized in that using the same source packet. According to claim 1,
The receiving device,
When the source packet and the recovery packet are received from the receiving unit and temporarily stored, the source packet is provided to the first decoding unit as received, and when the source packet and the recovery packet of the N group consisting of the set data size are temporarily stored, the N packet of the N group is received. Transmitting and receiving device for providing a real-time video and high-quality video together, characterized in that it further comprises a buffer for providing the source packet and the recovery packet to the second decoder. According to claim 1,
The transmitting device,
A source packet generating unit receiving the source image from the imaging device and generating a source packet;
A recovery packet generation unit that collects the source packets at a set data size and then generates a recovery packet based on the collected source packets;
And a transmitting unit that transmits the source packet to a receiving device as it is generated and transmits a recovery packet at a set time. According to claim 1,
The set timing is a time period for transmitting a high-quality image together with a real-time image, characterized in that the transmission time is inserted between the source packets of the N+1 group.

Images