KR101706573B1

KR101706573B1 - Device and method for multicast screen mirroring

Info

Publication number: KR101706573B1
Application number: KR1020150094739A
Authority: KR
Inventors: 신연철; 이규진; 최성현; 장희석; 정창훈; 한진환
Original assignee: 서울대학교산학협력단; 주식회사 인프라웨어
Priority date: 2015-07-02
Filing date: 2015-07-02
Publication date: 2017-02-15
Also published as: KR20170004418A

Abstract

A multicast screen mirroring apparatus and method are provided. A multicast screen mirroring method includes compressing a video frame to generate a compressed video frame, encoding the compressed video frame using Network Coding to generate an encoded packet, and transmitting the encoded packet via multicast And a control unit.

Description

&Lt; Desc / Clms Page number 1 > DEVICE AND METHOD FOR MULTICAST SCREEN MIRRORING &

The present invention relates to a multicast screen mirroring apparatus and method, and more particularly, to a multicast screen mirroring apparatus and method capable of providing screen mirroring to multiple users.

2. Description of the Related Art Various electronic devices such as a smart phone, a tablet PC, and the like have been developed and various applications implemented in such electronic devices are being provided. Of these various applications, a screen mirroring application is an application that allows a screen of one electronic device to be displayed simultaneously on another electronic device. For example, in an application where an electronic board is used, the teacher can conduct the lesson through the electronic board, and the student can be trained through the electronic board and screen-mirrored electronic device.

Assuming that the electronic blackboard is used as described above, when a screen of one user is mirrored to a plurality of users, such as a situation in which an electronic blackboard screen is mirrored to electronic devices of a plurality of students in a classroom, Point access point (AP) or mirroring through a WiFi-Direct connection. At this time, data can be transmitted unicast or multicast to the multi-user electronic device for screen mirroring. In the case of unicast data transmission, the bandwidth used increases proportionally as the number of receiving users increases, and transmission is impossible when a certain number of receiving users attempt to receive data at the same time. In addition, when transmitting data by multicast, existing wireless LANs do not transmit ACK packets due to collision of ACK, and thus can not guarantee reliability. Thus, the prior art related to screen mirroring has a problem that is unsuitable for screen mirroring to a large number of users.

[Related Technical Literature]

1. Wide area mirroring router (Korean Patent Application No. 10-2014-7009591)

Accordingly, an object of the present invention is to provide a multicast screen mirroring apparatus and method that can provide high scalability and high reliability in mirroring a screen to multiple users.

Another object of the present invention is to provide a multicast screen mirroring apparatus and method capable of safely restoring packets even if some packets are lost by adopting a network coding (NC) scheme.

The problems of the present invention are not limited to the above-mentioned problems, and other problems not mentioned can be clearly understood by those skilled in the art from the following description.

According to an aspect of the present invention, there is provided a method for mirroring a multicast screen, the method comprising: generating a compressed video frame by compressing a video frame; encoding the compressed video frame by network coding; And generating an encoded packet, and transmitting the encoded packet in a multicast manner.

According to another aspect of the present invention, the video frame is characterized in that the screen is captured data for each capture period.

According to another aspect of the present invention, the step of generating a compressed video frame comprises compressing a video frame based on at least one of a bit rate, a frame rate, a resolution, and an I frame interval input from a user The method comprising the steps of:

According to still another aspect of the present invention, the step of generating an encoded packet includes generating one or more original packets by separating a compressed video frame into packets of a predetermined length, grouping the original packets into a predetermined number And encoding the data by network coding on a block-by-block basis.

According to another aspect of the present invention, the step of encoding by network coding on a block-by-block basis includes generating an encoding packet by linearly combining a coupling coefficient and a data value of an original packet.

According to another aspect of the present invention, the step of multicasting the encoded packet includes transmitting the original packet together with the encoded packet in a multicast manner.

According to another aspect of the present invention, an encoded packet includes a header and payload, the header including a number of original packets in the block, an original packet order in the block, a block ID, a video frame ID, byte size, a combining coefficient, and a sequence of original packets in a video frame.

According to an aspect of the present invention, there is provided a multicast screen mirroring apparatus including a video encoder module configured to compress a video frame to generate a compressed video frame, a network coded ) To generate an encoded packet, and a traffic shaper module configured to multicast the encoded packet.

According to an aspect of the present invention, there is provided a computer-readable recording medium that compresses a video frame to generate a compressed video frame, encodes the compressed video frame into network coding, An encoded packet, and a set of instructions to cause the encoded packet to be transmitted in a multicast manner.

According to an aspect of the present invention, there is provided a method for mirroring a multicast screen, comprising: checking a video frame ID of a header of a received packet; determining whether a video frame ID matches an ID of a video frame currently being received The method comprising the steps of: collecting a received packet by a block ID of a header; storing a data value at a predetermined position on a video frame with reference to a received packet when the received packet is an original packet; And performing network coding decoding on the block when a predetermined number or more is received.

According to another aspect of the present invention, the multicast screen mirroring method further comprises, prior to the step of collecting received packets by block ID of the header, if the video frame ID does not match the ID of the video frame currently being received, Decoding the video frame and allocating a buffer memory for storing the new video frame.

According to another aspect of the present invention, a multicast screen mirroring method includes storing a data value of a decoding packet in a predetermined position on a video frame when decoding is successful, and a receiving packet having a block ID indicating a block succeeding decoding The method comprising the steps of:

According to another aspect of the present invention, a multicast screen mirroring method includes the steps of removing a linearly dependent receive packet when decoding fails, and receiving a receive packet having a block ID indicating a block that failed decoding Further comprising the step of waiting.

According to an aspect of the present invention, there is provided a multicast screen mirroring apparatus, which is configured to check a video frame ID of a header of a received packet, The received packet is collected according to the block ID of the header. If the received packet is the original packet, the data value is stored at a predetermined position on the video frame with reference to the received packet. If the packet included in the block indicated by the block ID is a predetermined And a network coding decoder configured to perform network coding decoding on the block if the number of received blocks is equal to or greater than a predetermined number.

According to an aspect of the present invention, there is provided a computer readable recording medium for verifying a video frame ID of a header of a received packet, and when a video frame ID matches an ID of a video frame currently being received , The received packet is collected according to the block ID of the header, and when the received packet is the original packet, the data value is stored at a predetermined position on the video frame with reference to the received packet, and if the packet included in the block indicated by the block ID is a predetermined number And if so received, performing a network coding decoding on the block.

The details of other embodiments are included in the detailed description and drawings.

The present invention can provide high scalability and high reliability in mirroring a screen to multiple users.

In addition, the present invention adopts a network coding scheme to securely recover a packet even if some packets are lost.

The effects according to the present invention are not limited by the contents exemplified above, and more various effects are included in the specification.

1 is a schematic diagram of a multicast screen mirroring system in accordance with an embodiment of the present invention.
2 is a schematic view of a transmitting apparatus in a multicast screen mirroring apparatus according to an embodiment of the present invention.
3 is a flowchart of a transmission method in a multicast screen mirroring method according to an embodiment of the present invention.
4 is a schematic diagram of an encoding packet according to an embodiment of the present invention.
FIG. 5 is a schematic view of a receiving apparatus in a multicast screen mirroring apparatus according to an embodiment of the present invention. FIG.
6 is a flowchart of a transmission method in a multicast screen mirroring method according to an embodiment of the present invention.
7 is a schematic diagram of a multicast screen mirroring apparatus according to another embodiment of the present invention.
Figure 8 is a graph of encoding yield in a multicast screen mirroring apparatus and method according to an embodiment of the present invention.
9A and 9B are graphs of decoding yields in a multicast screen mirroring apparatus and method according to an embodiment of the present invention.
10A and 10B are a measurement topology and a graph for explaining a packet loss rate in a multicast screen mirroring apparatus and method according to an embodiment of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS The advantages and features of the present invention, and the manner of achieving them, will be apparent from and elucidated with reference to the embodiments described hereinafter in conjunction with the accompanying drawings. The present invention may, however, be embodied in many different forms and should not be construed as being limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. Is provided to fully convey the scope of the invention to those skilled in the art, and the invention is only defined by the scope of the claims.

Like reference numerals refer to like elements throughout the specification.

Each block of the accompanying block diagrams and combinations of the steps of the flowcharts may be performed by algorithms or computer program instructions comprised of firmware, software, or hardware. These algorithms or computer program instructions may be embedded in a processor of a general purpose computer, special purpose computer, or other programmable digital signal processing device, so that the instructions that are executed by a processor of a computer or other programmable data processing apparatus Generate means for performing the functions described in each block or flowchart of the block diagram. These algorithms or computer program instructions may also be stored in a computer usable or computer readable memory capable of directing a computer or other programmable data processing apparatus to implement a function in a particular manner, It is also possible for instructions stored in a possible memory to produce a manufacturing item containing instruction means for performing the function described in each block or flowchart of each block diagram. Computer program instructions may also be stored on a computer or other programmable data processing equipment so that a series of operating steps may be performed on a computer or other programmable data processing equipment to create a computer- It is also possible that the instructions that perform the processing equipment provide the steps for executing the functions described in each block of the block diagram and at each step of the flowchart.

Also, each block or each step may represent a module, segment, or portion of code that includes one or more executable instructions for executing the specified logical function (s). It should also be noted that in some alternative embodiments, the functions mentioned in the blocks or steps may occur out of order. For example, two blocks or steps shown in succession may in fact be performed substantially concurrently, or the blocks or steps may sometimes be performed in reverse order according to the corresponding function.

It is to be understood that each of the features of the various embodiments of the present invention may be combined or combined with each other partially or entirely and technically various interlocking and driving is possible as will be appreciated by those skilled in the art, It may be possible to cooperate with each other in association.

Various embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

1 is a schematic diagram of a multicast screen mirroring system in accordance with an embodiment of the present invention. Referring to FIG. 1, a multicast screen mirroring system 1000 is comprised of a transmitting apparatus 100, a receiving apparatus 200, and an access point 300 as multicast screen mirroring apparatuses.

The transmitting apparatus 100 is a multicast screen mirroring apparatus, which means an electronic device for transmitting data to be screen-mirrored to a plurality of users. Various types of electronic devices can be used as the transmission device 100 and various devices such as a smart phone, a notebook computer, a digital broadcasting terminal, a PDA (personal digital assistant), a portable multimedia device (PMP) Player) and the like can be used. For example, the transmitting apparatus 100 may be an electronic device such as an electronic board. A more detailed description of the transmitting apparatus 100 will be described in detail with reference to FIG. 2 and FIG.

The receiving apparatus 200 is a multicast screen mirroring apparatus, which means an electronic device that a plurality of users can use to receive data to be screen mirrored. A variety of electronic devices can be used as the receiving device 200, and the type of electronic device that can be used as the receiving device 200 can be the same as that of the transmitting device 100. For example, the receiving device 200 may be an electronic device such as a smart phone. The receiving apparatus 200 communicates with the transmitting apparatus 100 through the access point 300 and the access point 300 transmits data from the transmitting apparatus 100 to the plurality of receiving apparatuses 200 in a multicast manner . A more detailed description of the receiving apparatus 200 will be described in detail with reference to Figs. 5 and 6. Fig.

In addition, the multicast screen mirroring device may function as a transmitting device 100 or as a receiving device 200 in one device. That is, through the mode setting of the multicast screen mirroring device, one device may function as the transmitting device 100 in a particular case, and may function as a receiving device 200 in another specific case. This will be described in detail with reference to FIG.

2 is a schematic view of a transmitting apparatus in a multicast screen mirroring apparatus according to an embodiment of the present invention. 3 is a flowchart of a transmission method in a multicast screen mirroring method according to an embodiment of the present invention. Referring to FIG. 2, the transmission apparatus 100 includes a UI module 110, a screen capture module 120, a video encoding module 130, a network coding encoder module 140, and a traffic shaper module 150. Hereinafter, referring to FIG. 2 and FIG. 3 together for a more detailed description of the transmission apparatus 100, FIG.

First, the transmitting apparatus 100 sets environment variables of a video codec, a network coding codec, and a traffic shaper through the UI module 110, and receives an input for starting and stopping transmission. The UI module 110 may comprise simple buttons. For example, the UI module 110 may include buttons that can start or stop the transmission. In addition, the UI module 110 may receive a multicast address and a port number through an input window for specifying a multicast address and a port number for multicast transmission. The UI module 110 may be configured to receive a video codec setting value associated with video encoding. For example, the UI module 110 may receive a video codec setting value such as a bit rate, a resolution, a frame rate, an I frame interval, and the like. The UI module 110 may be configured to receive a set value associated with network coding encoding. For example, the UI module 110 may receive various variables such as the length L of the original packet, the number K of original packets that are grouped and constitute the block, and the like. In addition, the UI module 110 can receive a variable such as a token rate of a traffic shaper and a maximum number of tokens. The respective setting values and variables will be described later.

Upon receiving the transmission start input from the UI module 110, the set variables are applied and the operation of the screen capture module 120 is started.

When the transmission start input is received, the screen capture module 120 of the transmission apparatus 100 captures a screen for each capture period and inputs it to the video encoder module. The capture period is determined by the reciprocal of the frame rate value which is the number of frames to be reproduced per second. For example, if a frame rate of 20 frames / sec is set through the UI module 110, the capture period is 0.05 second.

Next, the video encoder module of the transmission apparatus 100 compresses the video frame to generate a compressed video frame (S300).

The video encoder module is a module for compressing data of a screen captured by a video frame, i.e., the screen capture module 120, to reduce the amount of data. Various widely used codecs such as H.264 and MPEG 4 can be used in the video encoder module. The set values used for forming the compressed video frame in the video encoder module may be input from the user through the UI module 110 and determined. For example, the UI module 110 can adjust the bit rate for determining the total amount of transmission, the frame rate for determining the number of video frames to be reproduced per second, the resolution, and the I frame interval for determining the range in which independent decoding is possible. The video encoder module may compress a video frame based on at least one of a bit rate, a frame rate, a resolution, and an I frame interval input from a user. The video encoder module transmits the compressed video frame generated through the compression process to the network coding encoder module 140.

Then, the network coding encoder module 140 of the transmission apparatus 100 encodes the compressed video frame into network coding to generate an encoding packet (S310).

More specifically, the network coding encoder module 140 separates the compressed video frame received from the video encoder module into packets having a predetermined length to generate one or more original packets (S311).

In particular, the network coding encoder module 140 separates the compressed video frames into equal lengths (L). These separated packets are defined as original packets. The length L can be set through the UI module 110, for example. If the compressed video frame does not divide by L, the last original packet has L ', which is a packet length less than L. In the original packets, the order of the original packets in the video frame is recorded.

Then, the network coding encoder module 140 groups the original packets by a predetermined number to generate one or more blocks (S312).

The network coding encoder module 140 combines the original packets with a predetermined number (K) to generate one or more blocks. If the total number of original packets is not divided by K, the last block is made up of K 'original packets smaller than K. Each block has a block ID (BID) for identifying the corresponding block.

Then, the network coding encoder module 140 encodes the data into network coding on a block-by-block basis (S313), and attaches a header to the encoded packet (S314).

The network coding encoder module 140 encodes the data using a random linear network coding (RLNC) on a block-by-block basis. Specifically, encoding is performed on a block basis to generate (N-K) number of encoding packets per block. Therefore, the transmitting apparatus 100 transmits K original packets and (N-K) number of encoding packets per block, and therefore transmits a total of N packets per block. If the last block is composed of K 'original packets smaller than K, the last block additionally generates (N'-K') encoded packets and transmits N 'packets.

A procedure for additionally generating an encoding packet by random linear network coding is as follows. An encoding packet is generated by linear combination of K arbitrary combining coefficients and data values of the original packet. If there is an original packet of length L ', then the rear part (L-L') of the short original packet is regarded as 0 and encoded. In this case, the linear combination operation is performed by a finite field operation. In order to increase the speed of the finite field operation and maintain proper memory usage, the field size is 8 bytes, the symbol size is 1 byte, and the multiplication and division The calculations use a 2D lookup table. The process can be expressed as follows.

[Equation 1]

yi = a_1 x_1i + a_2 x_2i ++ a_K x_Ki,

Where yi denotes the i-th byte of the generated encoded packet and _ji denotes the i-th byte of the j-th original packet. a_j is an arbitrary coefficient value multiplied by the jth original packet. If we change it into matrix form, it is as follows.

Y = AX,

Y is an (N-K) × L size result matrix, A is a (N-K) × K size coupling coefficient matrix, and X is a K × L original packet matrix.

For a more detailed description of the encoding packet generated through the encoding process as described above, reference is also made to FIG.

4 is a schematic diagram of an encoding packet according to an embodiment of the present invention.

Referring to FIG. 4, the encoded packet includes a header and a payload. That is, a header is added to the encoding packet for the decoding process. The header includes the number of original packets K in the block, the original packet order sys in the block, the block ID BID, the video frame ID FID, the total byte size of the video frame, Coefficient and the order of the original packets in the video frame (fseq). That is, K represents the number of original packets in a block, sys represents a number of original packets for the original packet, BID is a block ID for identifying a block, and FID is a video frame ID for identifying a video frame. Flength is the total byte size of the video frame, and K 'and L' are obtained using the corresponding value and K, value. And Coefficient transfers the value of the coupling coefficient used in encoding. fseq is recorded in the order of the packets in the video frame when the video frame is initially cut in packets. Referring to these header values, the receiving apparatus 200 can construct a video frame in order. The payload records the value of the data. In the case of the original packet, payload and fseq are transmitted without being encoded, and the additional encoded packet is transmitted by encoding payload and fseq.

When the encoding is completed as described above, the network coding encoder module 140 delivers the original packet and the encoded packet to the traffic shaper module 150. In this case, one packet is transmitted in one block, and one packet is transmitted in the next block, and all the packets are transmitted by transmitting one packet every block by block. By using such a video-frame-based interleaving scheme, it is robust to a wireless channel in which a burst packet error occurs.

Then, the traffic shaper module 150 multicasts the original packet and the encoded packet (S320).

The traffic shaper module 150 is a module for transmitting packets received from the network coding encoder module 140 by controlling the amount of transmission and multicasting. The traffic shaper module 150 is a module for preventing a packet loss due to a buffer overflow that may occur when a large amount of packets are transmitted at one time. For example, in the case of screen mirroring through WLAN multicast, the transmitting apparatus 100 is transmitted on the unicast uplink to the access point 300, and the access point 300 transmits the packet on the multicast downlink To the receiving apparatus 200. At this time, since the rate of the unicast uplink is relatively fast compared with the rate of the multicast downlink, if a large amount of packets are transmitted at once, packets that are stored in the buffer can not be sent from the access point 300. At this time, if the number of packets stored exceeds a predetermined amount, a buffer overflow may occur which can not be stored any more and is discarded. Accordingly, the traffic shaper can use a variety of traffic shaper algorithms such as a token bucket, etc. to prevent the buffer overflow as described above.

If the traffic shaper uses a token bucket algorithm, the traffic shaper periodically generates tokens. The token generation period is the inverse of the token rate. The traffic shaper can hold tokens up to a preset maximum number, and can not have more than the maximum number. At this time, the network coding encoder module 140 transmits a packet corresponding to the number of tokens that it has when receiving the packet, and stores the packet more than the number of tokens. The stored packets are then transmitted one by one when the token is generated. The token rate and the maximum number of tokens may be determined by the size of the buffer and the transmission rate of the links on the network.

Hereinafter, a receiving apparatus 200 for receiving a packet transmitted through the encoding process as described above among the multicast screen mirroring apparatus will be described.

FIG. 5 is a schematic view of a receiving apparatus in a multicast screen mirroring apparatus according to an embodiment of the present invention. FIG. 6 is a flowchart of a transmission method in a multicast screen mirroring method according to an embodiment of the present invention. Referring to FIG. 5, the receiving apparatus 200 includes a UI module 210, a screen reproducing module 220, a video decoder module 230, and a network coding decoder module 240. 5 and 6 together for a more detailed description of the receiving apparatus 200. FIG.

First, the receiving apparatus 200 can receive an input for starting and stopping reception through the UI module 210. [ The UI module 210 may comprise simple buttons. For example, the UI module 210 may include buttons that can start or stop receiving. Also, the UI module 210 can receive a multicast address and a port number through an input window for specifying a multicast address and a port number for multicast transmission. When receiving an input for starting reception via the UI module 210, the socket is opened to receive a packet for the set address and port number, and the received packet is input to the network coding decoder module 240.

The network coding decoder module 240 of the receiving apparatus 200 collects and decodes the received packets received from the sockets on a block basis, restores the compressed video frames with the obtained packets, and inputs the recovered video frames to the video decoder module 230.

Specifically, when a received packet arriving at the receiving apparatus 200 arrives (S600), the network coding decoder module 240 confirms the video frame ID (FID) of the header of the received packet (S605).

If the video frame ID (FID) does not match the ID of the video frame currently being received, the network coding decoder module 240 performs decoding for the previous video frame (S610).

That is, when the video frame ID (FID) does not match the ID of the video frame currently being received, the network coding decoder module 240 determines that the reception of one video frame is completed, And transmits the data for the corresponding video frame to the video decoder module 230. [ To reduce the delay in the screen mirroring application, the network coding decoder module 240 inputs the previous video frame to the video decoder module 230 at the time the next frame is received without buffering the previous video frame.

Then, the network coding decoder module 240 allocates a buffer memory for storing a new video frame (S615).

The network coding decoder module 240 finds the L or L 'value and the K or K' value and the total number of blocks of the corresponding video frame through the data value contained in the header of the received packet. In addition, the network coding decoder module 240 allocates a buffer memory for storing a video frame to be newly completed according to a Flength value indicating a total byte size of a video frame, and then stores the received packet in accordance with the block ID (BID).

Then, if the video frame ID (FID) matches the ID of the video frame currently being received or if the video frame ID (FID) does not match the ID of the video frame currently being received, but a buffer memory for that video frame is assigned , The network coding decoder module 240 checks whether the corresponding block ID (BID) is necessary for decoding (S620).

The network coding decoder module 240 checks the header of the received packet to check the block ID (BID) of the block including the received packet, and determines whether or not decoding of the block indicated by the corresponding block ID is completed. If decoding of the corresponding block is completed, it is determined that decoding of the corresponding block ID (BID) is not necessary, and the received packet is deleted (S625).

If decoding of the corresponding block ID (BID) is required, the network coding decoder module 240 collects received packets for each block ID (BID) of the header (S630).

Then, the network coding decoder module 240 determines whether the received packet is an original packet (S635).

If the received packet is an original packet, the network coding decoder module 240 refers to the received packet and stores the data value at a predetermined position on the video frame (S640).

The network coding decoder module 240 checks the original packet sequence (sys) in the block in the header of the received packet and determines the sequence (fseq) of the original packets in the video frame in the header of the received packet, And inputs the data value at a predetermined position on the video frame, and then stores the data value.

If the received packet is not the original packet and the received packet is the original packet, the network coding decoder module 240 checks whether the corresponding block ID (BID) of the received packet is a decoding condition (S645).

In the random linear network coding scheme used in the network coding decoder module 240, when K pieces of linearly independent packets among the total N packets transmitted from the transmitting apparatus 100 are received corresponding to one block, K original packets are restored It is possible. Accordingly, when the network coding decoder module 240 receives K received packets for the corresponding block ID (BID), it meets a decoding condition that can attempt decoding. If the block ID (BID) identifies the last block, if the network coding decoder module 240 receives K 'received packets for the corresponding block ID (BID), it meets a decoding condition that can attempt decoding .

If the block ID (BID) is not a decoding condition, that is, if a packet included in a block indicated by a block ID (BID) is received less than a predetermined number K, the received packet is stored (S650) .

When the corresponding block ID (BID) is a decoding condition, that is, when a predetermined number or more of packets included in a block indicated by the block ID (BID) are received, decoding is performed on the block.

Specifically, it is checked whether all rows of the coefficient matrix are linearly independent (S655).

As described above, in the random linear network coding scheme, when K pieces of linearly independent packets among the total N packets transmitted from the transmitting apparatus 100 in response to one block are received, K original packets can be recovered, Is satisfied, that is, when K received packets are received, all of the K received packets must be linearly independent so that decoding can succeed. Accordingly, it is determined that all of the K received packets are linearly independent and that decoding fails if there is a linearly dependent received packet in any one of the K received packets.

If decoding fails, that is, if all the rows of the Coefficient matrix are not linearly independent, the network coding decoder module 240 removes linearly dependent received packets (S660) And waits for a received packet having a block ID indicating a failed block.

If decoding is successful, that is, if it is determined that the received K received packets are linearly independent, the original packet is recovered through LU decomposition (S665).

The decoding process in the network coding decoder module 240 is a process of obtaining an inverse matrix of a coefficient matrix and multiplying the inverse matrix by a received packet. In the process of obtaining the inverse matrix of the coupling coefficient matrix, it is calculated by using LU decomposition. However, the decoding speed can be increased because the operation speed is faster than the general Gaussian elimination method. In the last packet of the last block, the packet with length L 'is removed. The recovered packet through such a decoding process is referred to as a decoding packet.

Then, the network coding decoder module 240 stores the data value of the decoded packet at a predetermined position on the video frame (S670), and deletes the received packet having the block ID indicating the decoded block (S675).

The network coding decoder module 240 checks the header of the decoded packet to determine the order (fseq) of the original packets in the video frame of each packet, stores the data value at a predetermined position on the video frame, And deletes the received packet having the block ID indicating it. In addition, the network coding decoder module 240 also deletes received packets for the same block ID (BID) received thereafter (S625).

When the decoding process for the video frame in the network coding decoder module 240 is completed, the data for the corresponding video frame is transmitted to the video decoder module 230. The video decoder module 230 is a module for decoding the video frame before decoding, which is generated by the network coding decoder module 240, into a reproducible picture. The video decoder module 230 performs decoding using the same video codec as that used in the video encoder module of the transmitting apparatus 100. The video decoder module 230 inputs the screen data generated through decoding to the screen reproduction module 220.

The screen playback module 220 operates to reproduce the decoded video frames decoded by the video decoder module 230 on the screen. If the screen size of the receiving apparatus 200 is larger than the resolution of the video, the screen reproducing module 220 enlarges and reproduces the video frame in proportion to the screen size.

In the multi-screen mirroring apparatus and method according to an embodiment of the present invention, multicast transmission using network coding is performed in a screen mirroring to multiple users, so that even if the number of receiving apparatuses 200 increases, High transmission may be possible.

In the multi-screen mirroring apparatus and method according to an embodiment of the present invention, unlike the related art, a network coding encoder module 140 and a network coding decoder module 240 are used, and a network coding encoder module 140 and a network The coding decoder module 240 can be efficiently designed so that highly reliable multicast transmission can be achieved without increasing the delay in multicast transmission.

In addition, in the multi-screen mirroring apparatus and method according to an embodiment of the present invention, efficiency is improved by encoding in units of video frames, and additional delay due to coding can be reduced. Also, in the multi-screen mirroring apparatus and method according to an embodiment of the present invention, interleaving is performed for each video frame and a packet is transmitted to provide a bursty error robust service.

Also, in the multi-screen mirroring apparatus and method according to an embodiment of the present invention, the transmission amount can be adjusted through the traffic shaper to minimize the packet loss due to the buffer overflow.

Also, in the multi-screen mirroring apparatus and method according to an embodiment of the present invention, even if only K packets out of N packets transmitted from the transmitting apparatus 100 are received by the receiving apparatus 200, decoding can be normally completed. When the receiving apparatus 200 receives K packets, the probability that they are linearly independent from each other is approximately 99.7%. In addition, when the receiving apparatus 200 receives (K + 1) packets, the probability that there are K linearly independent packets is approximately 99.998%. In addition, if the original packet can not be restored, it is possible to restore only the packets received in the original packet form. Therefore, if N is increased, the probability of receiving K linearly independent packets increases, so that the reliability can be adjusted according to the ratio. In addition, even if some of the N packets received by the transmission apparatus 100 are lost, decoding can be normally completed.

7 is a schematic diagram of a multicast screen mirroring apparatus according to another embodiment of the present invention. The multicast screen mirroring apparatus shown in FIG. 7 is a transmitting and receiving apparatus 400 having a transmitting apparatus 100 shown in FIG. 2 and a receiving apparatus 200 shown in FIG. 5 combined. The transmitting apparatus 400 includes a UI module 410, A video encoding module 130, a network coding encoder module 140, a traffic shaper module 150, a screen reproduction module 220, a video decoder module 230 and a network coding decoder module 240 do.

Referring to FIG. 7, the transceiver 400 as a multicast screen mirroring apparatus can function as the transmitting apparatus 100 shown in FIG. 2 or function as the receiving apparatus 200 shown in FIG. 5 as an option. Specifically, when the transmitting / receiving device 400 receives an input to use the transmitting / receiving device 400 as the transmitting device 100 via the UI module 110, the transmitting / receiving device 400 includes a UI module 110, The transmitter 120 may function as the transmitter 100 as shown in FIG. 2 using the video encoder module, the network coding encoder module 140, and the traffic shaper module 150. In addition, when the transmission / reception device 400 receives an input to use the transmission / reception device 400 as the reception device 200 via the UI module 110, the transmission / reception device 400 includes a UI module 110, The video decoder module 230 and the network coding decoder module 240 may be used to function as the receiving device 200 as shown in FIG. Here, an application executed and set for transmission is called a transmission application, and an application executed and set for reception is called a reception application. Accordingly, as the multicast screen mirroring apparatus according to another embodiment of the present invention, the transceiving apparatus 400 can transmit / receive multicast screen mirroring only by application layer, and can be used universally without changing the apparatus.

Figure 8 is a graph of encoding yield in a multicast screen mirroring apparatus and method according to an embodiment of the present invention. FIG. 8 is a graph of the encoding yield when an electronic writing board terminal (Odroid XU) is used as a transmitting apparatus.

The encoding yield refers to the total number of bits of a packet used for encoding per second. The encoding yield is an indicator of the performance of the network coding codec. When the value is larger than the video bit rate, the module can transmit without additional delay.

Referring to FIG. 8 showing the encoding yield result graph using Odroid XU, the delay time for processing one block increases according to the number K of original packets constituting one block, and accordingly, It can be seen that the encoding yield decreases as the number of original packets K constituting one block increases. The reason is that as the number of original packets (K) constituting one block increases, the number of required operations increases proportionally. The results show that the encoding yield is more than 20 Mb / s when K is 20 or less, and it is generally confirmed that the encoding yield is sufficiently high when the bit rate of HD video is 3 Mb / s have.

9A and 9B are graphs of decoding yields in a multicast screen mirroring apparatus and method according to an embodiment of the present invention. FIG. 9A is a decoding yield graph for Embodiment 1 in which Galaxy Note 10.1 2014 edition is used as a receiving apparatus, and FIG. 9B is a decoding yield graph for Embodiment 2 in which Galaxy Note 10.1 2012 edition is used as a receiving apparatus. FIGS. 9A and 9B are graphs of decoding yields according to various K values, and graphs of decoding yields according to the ratio of original packets at respective K values are shown.

The decoding yield is the total number of bits of a packet that can be decoded per second. The decoding yield is an indicator of the performance of the network coding codec. When the corresponding value is greater than the video bit rate, the module is able to receive without additional delay.

The decoding yield depends on the reception rate of the original packet when K packets are received. When all the received packets are original packets, a separate decoding process is unnecessary, and thus a large yield of 200 Mb / s or more is obtained. The lower the reception rate of the original packet is, the more the number of operations required in the decoding process increases and the decoding yield decreases. Also, the larger the value of K, the greater the amount of computation as well as the encoding yield, and the decoding yield decreases. However, it can be confirmed that decoding yields of 9 Mb / s or more are obtained in both the first and second embodiments. In the case of Embodiment 1, the result is about 1.7 times faster than that in Embodiment 2. This is because the CPU clock difference of each terminal used in Embodiment 1 and Embodiment 2 (Embodiment 1: 2300 MHz, Embodiment 2 : 1400 MHz).

10A and 10B are a measurement topology and a graph for explaining a packet loss rate in a multicast screen mirroring apparatus and method according to an embodiment of the present invention. 10A is a schematic diagram showing the positions of the experimental environment and the access point, the transmitting apparatus, the first receiving apparatus, the second receiving apparatus, and the third receiving apparatus for measuring the packet loss rate, and the coordinates for each position are based on the origin The unit is in meters (m). FIG. 10B is a graph showing a packet loss rate (PLR) of the first receiver, the second receiver, and the third receiver when N is 5, 10, and 15, respectively.

Packet loss rate refers to the number of video packets lost relative to the number of packets transmitted and is widely used as a performance indicator. In order to measure the packet loss rate, a wireless LAN access point is connected to an electronic board terminal (Odroid XU) as a transmitting device, and a first receiving device, a second receiving device, and a third receiving device are disposed at three locations in a classroom environment Performance was measured. The access point used in the experiment was MMC-Tech's MW-5150 AP and measured in the 5 GHz band. In the experiment, K was set to 5, and N was changed to 5, 10, and 15. In the case of N = 5, as a comparative example, only the original packet is transmitted without transmitting the encoding packet, and therefore, the result is the same as that in the prior art that does not use network coding. As the video coding parameters, the bit rate is set to 1 Mb / s, the GOP is set to 20, the FPS is set to 20, and the resolution is set to 1280x720. The token period is set to 0.277 ms, and the maximum number of tokens is set to 180. The multicast transmission rate was set at 54 Mb / s. The first receiving device, the second receiving device, and the third receiving device used the Galaxy Note 10.1 2012 edition.

As a result of the measurement, it has been confirmed that a packet loss rate of 10% or more is measured in the case of a packet which is not transmitted using network coding with N = 5, that is, a comparative example in which only an original packet is transmitted. On the other hand, in the case of N = 10 and 15, a packet loss rate of less than 1%, which is a packet loss rate requirement in general video streaming, can be obtained.

The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. The software module may reside in a RAM memory, a flash memory, a ROM memory, an EPROM memory, an EEPROM memory, a register, a hard disk, a removable disk, a CD-ROM or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, which is capable of reading information from, and writing information to, the storage medium. Alternatively, the storage medium may be integral with the processor. The processor and the storage medium may reside within an application specific integrated circuit (ASIC). The ASIC may reside within the user terminal. Alternatively, the processor and the storage medium may reside as discrete components in a user terminal.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the present invention is not limited to the disclosed exemplary embodiments, but various changes and modifications may be made without departing from the spirit and scope of the invention. Therefore, the embodiments disclosed in the present invention are not intended to limit the scope of the present invention but to limit the scope of the technical idea of the present invention. The scope of protection of the present invention should be construed according to the following claims, and all technical ideas within the scope of equivalents should be construed as falling within the scope of the present invention.

100: transmitting apparatus
110: UI module
120: Screen Capture Module
130: Video encoding module
140: Network coding encoder module
150: Traffic Shaper Module
200: Receiver
210: UI module
220: Screen playback module
230: Video decoder module
240: Network coding decoder module
300: access point
400: Transceiver
410: UI module
1000: Multicast Screen Mirroring System

Claims

Compressing a video frame to generate a compressed video frame;
Separating the compressed video frame into packets of a predetermined length to generate one or more original packets;
Grouping the original packets by a predetermined number to generate one or more blocks;
Generating encoded packets by encoding the encoded data in a network coding manner on a block-by-block basis; And
And multicasting the encoded packet with the original packet,
When a packet of a predetermined number or more of the received packets received in correspondence with the block is received in an apparatus that has received the received packet, the receiving packet including the original packet and the encoded packet includes Wherein the network coding decoding is configured to perform the multicast screen mirroring.

The method according to claim 1,
Wherein the video frame is data captured by the screen for each capture period.

The method according to claim 1,
Wherein the step of generating the compressed video frame comprises compressing the video frame based on at least one of a bit rate, a frame rate, a resolution, and an I frame interval input from a user Multicast screen mirroring method.

delete

The method according to claim 1,
Wherein encoding on a block-by-block basis comprises generating the encoded packet by linear combination of a coupling coefficient and a data value of an original packet.

delete

6. The method of claim 5,
Wherein the encoded packet includes a header and payload,
Wherein the header comprises a number of original packets in the block, an original packet order in the block, a block ID, a video frame ID, a total byte size of the video frame, Wherein the multicast screen includes a sequence of original packets.

A video encoder module configured to compress a video frame to generate a compressed video frame;
Generating one or more original packets by dividing the compressed video frame into packets of a predetermined length, generating one or more blocks by grouping the original packets by a predetermined number, and performing network coding on the block- A network coding encoder module configured to encode the encoded packet to generate an encoded packet; And
And a traffic shaper module configured to multicast the encoded packet with the original packet,
When a packet of a predetermined number or more of the received packets received in correspondence with the block is received in an apparatus that has received the received packet, the receiving packet including the original packet and the encoded packet includes Characterized in that the network coding decoding is configured to perform the multicast screen mirroring.

Compressing the video frame to generate a compressed video frame,
Separating the compressed video frame into packets of a predetermined length to generate one or more original packets,
Grouping the original packets by a predetermined number to generate one or more blocks,
And encodes the encoded data by network coding to generate an encoded packet,
A set of instructions to cause the encoded packet to be multicastly transmitted with the original packet,
When a packet of a predetermined number or more of the received packets received in correspondence with the block is received in an apparatus that has received the received packet, the receiving packet including the original packet and the encoded packet includes Wherein the network coding decoding is configured to be performed.

Confirming the video frame ID of the header of the received packet;
If the video frame ID matches the ID of the video frame currently being received,
Collecting received packets for each block ID of the header;
If the received packet is an original packet, storing the data value at a predetermined position on the video frame with reference to the received packet; And
And performing network coding decoding on the block when the packet included in the block indicated by the block ID is a predetermined number or more of linearly independent packets.

11. The method of claim 10,
If the video frame ID does not match the ID of the video frame currently being received, prior to the step of collecting received packets by block ID of the header,
Performing decoding on a previous video frame; And
And allocating a buffer memory to store a new video frame. &Lt; Desc / Clms Page number 19 >

11. The method of claim 10,
If the decoding succeeds,
Storing a data value of a decoding packet at a predetermined location on the video frame; And
Further comprising the step of deleting a received packet having a block ID indicating a block succeeding the decoding.

11. The method of claim 10,
If the decoding fails,
Removing linearly dependent received packets; And
Further comprising the step of waiting for a received packet having a block ID indicating a block that failed to decode said multicast screen.

And to identify the video frame ID of the header of the received packet,
If the video frame ID matches the ID of the video frame currently being received,
Collecting received packets for each block ID of the header,
If the received packet is an original packet, storing the data value at a predetermined position on the video frame with reference to the received packet,
And a network coding decoder configured to perform network coding decoding on the block if the packet included in the block indicated by the block ID is a packet having a linear number or more than a predetermined number.

Confirms the video frame ID of the header of the received packet,
If the video frame ID matches the ID of the video frame currently being received,
Collecting received packets for each block ID of the header,
If the received packet is an original packet, storing the data value at a predetermined position on the video frame with reference to the received packet,
And a set of instructions for causing network coding decoding of the block to be performed when the packet included in the block indicated by the block ID is a packet having a predetermined number or more of linear independence.