KR20200075709A - Method for transporting multimedia using FEC - Google Patents

Abstract

An ultra-narrowband multimedia transmission method using forward error correction (FEC) is provided. The ultra-narrowband multimedia transmission method of a surveillance and reconnaissance aircraft using FEC comprises the steps of allowing an electronic device to: collect or calculate network state information generated while transmitting and receiving video data; encode the video data based on a video encoding rate; input network status information into an optimal DNN model and dynamically and variably output FEC parameters to be applied during FEC encoding of the video data according to network status information; generate a packet by performing FEC encoding of the encoded video data using the variably output FEC parameters; and transmit the generated packet.

Description

Method for transporting ultra-narrowband multimedia using FEC

The present invention relates to a method for transmitting ultra-narrowband multimedia using FEC, and more particularly, to a method for transmitting ultra-narrowband multimedia using FEC capable of dynamically varying FEC variables and transmitting data in consideration of network state and video encoding. will be.

2. Description of the Related Art With the recent development of computer and communication technologies, there is an increasing demand for services that can transmit multimedia data such as voice and video in real time over a network, and for multimedia processing technologies and transmission technologies that can satisfy these demands. Research is actively underway.

If the existing communication service was a text-oriented service such as FTP (File Transfer Protocol), Tenet, or e-mail through the Internet, recently, various services requiring real-time transmission of multimedia data such as YouTube and IPTV have appeared. In particular, as devices that support UHD (Ultra High Definition) or higher images become common, the demand for high quality image transmission and reception is increasing. However, as the total data to be transmitted is greatly increased, the probability of packet loss and bit error between streaming services is also greatly increased.

As a representative method used to compensate for this problem, there is a forward error correction (FEC) technology in the application layer, and various video streaming technologies based on the forward error correction technology have been studied until recently.

However, the existing FEC parameter determination algorithm is a fixed method for variable data, and there is a limit in determining FEC parameters for adaptive transmission and reception data. That is, the biggest disadvantage of the existing multimedia transmission protocol is that, when using the FEC parameter determination algorithm for determining the raptor encoding parameter, it is impossible to change the packet size.

Assuming that the FEC encoder uses the same encoding block size for multimedia transmission, if the packet size cannot be changed, the number of packets is also not changed and fixed. Since the number of packets is a parameter that greatly affects the throughput as well as the code rate of the encoding block, a technique capable of adaptively adjusting FEC parameters according to a communication environment is required.

Domestic published patent No. 10-2018-0052651 (published on May 18, 2018)

In order to solve the above-mentioned problems, a technical problem to be achieved by the present invention is to propose a super narrow-band multimedia transmission method using FEC capable of dynamically adjusting a packet size in an encoding block by applying a deep neural network structure.

The problems of the present invention are not limited to those mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

As a means for solving the above-described technical problem, according to an embodiment of the present invention, the ultra-narrowband multimedia transmission apparatus using Forward Error Correction (FEC) receives network state information generated during transmission and reception of image data. Network information providing unit to provide; A video encoder for encoding the video data based on a video encoding rate; An FEC variable controller for dynamically outputting FEC parameters to be applied when encoding FEC of the image data by dynamically inputting network state information provided from the network information providing unit into an optimal DNN model according to the network state information; An FEC encoding and packet generation unit that generates a packet by FEC encoding the encoded video data using FEC parameters that are variablely output from the FEC variable control unit; And a communication interface unit that wirelessly transmits the generated packet.

The network status information includes a packet loss rate (PLR) and a data rate, and the FEC parameters include symbol size (s, symbol size), symbol number (k, symbol number), and It includes a packet size (p, packet size), and the FEC variable control unit dynamically calculates a packet size according to the packet loss rate and data rate, and the FEC encoding and packet generation unit dynamically calculates the FEC variable control unit. A packet is generated after FEC encoding the encoded video data based on the packet size.

The video encoder provides video encoding rate information applied during encoding to the FEC variable controller, and the FEC variable controller includes the video encoding rate information input from the video encoder together with the network state information and the optimal DNN model. Enter to dynamically output the FEC parameters.

The optimal DNN model is selected by the DNN model selection device, and the DNN model selection device is a symbol capable of maximum throughput while varying the network state information, which is calculated using a multimedia transmission protocol algorithm, within a set test range, respectively. An actual value storage unit for storing a combination of the size (s), the number of symbols (k), and the packet size (p) as an actual value (symbol size, number of symbols, packet size); A DNN model generation unit generating m different DNN models in which at least one of the number of layers used for DNN learning and the number of neurons for each layer is different; For each of the m DNN models, the network state information is input while varying within a set test range to calculate a predicted value consisting of n (symbol size, symbol count, and packet size) for each of the m DNN models. A predicted value calculating unit; And a DNN model selector for selecting the optimal DNN model by comparing the predicted value and the actual value for each of the m DNN models.

The DNN model selector calculates an MSE value for the symbol size, an MSE value for the number of symbols, and an MSE value for the packet size using a Mean Square Error (MSE) method, and adds the calculated MSE values. The DNN model with the smallest result is selected as the optimal DNN model.

On the other hand, according to another embodiment of the present invention, the ultra-narrowband multimedia transmission method using Forward Error Correction (FEC), (A) the electronic device collects network state information that occurs while transmitting and receiving video data Or calculating; (B) the electronic device encoding the video data based on a video encoding rate; (C) the electronic device dynamically inputs the network state information provided in step (A) into an optimal DNN model and dynamically outputs FEC parameters to be applied when FEC encoding the video data according to the network state information. step; (D) generating, by the electronic device, a packet by FEC encoding the encoded video data using FEC parameters that are variablely output in the step (C); And (E) the electronic device transmitting the packet generated in step (D).

The network status information includes a packet loss rate (PLR) and a data rate, and the FEC parameters include symbol size (s, symbol size), symbol number (k, symbol number), and It includes a packet size (p, packet size), in step (C), the electronic device calculates a packet size dynamically according to the packet loss rate and data rate, and in step (D), the electronic device A. After FEC encoding the encoded video data based on the packet size dynamically calculated in step (C), a packet is generated.

In step (B), the electronic device provides video encoding rate information applied during the encoding, and in step (C), the electronic device receives the video encoding rate information input from step (B). The FEC parameters are inputted to the optimal DNN model together with the network status information, and the FEC parameters are variably output according to the network status information and the video encoding rate information.

Prior to the step (A), the (F) DNN model selection device is calculated while varying the network state information and the video encoding rate within a set test range using a multimedia transmission protocol algorithm, a symbol capable of maximum throughput Storing a combination of the size (s), the number of symbols (k) and the packet size (p) as an actual value consisting of (symbol size, number of symbols, packet size); (G) generating, by the DNN model selection apparatus, m different DNN models in which at least one of the number of layers used for DNN learning and the number of neurons for each layer is different; (H) The DNN model selection device inputs the network state information and the video encoding rate for each of the m DNN models while varying within a set test range, and then generates n (symbol sizes) for each of the m DNN models. Calculating a prediction value consisting of', the number of symbols' and the packet size'); And (I) the DNN model selection device selecting the optimal DNN model by comparing the predicted value and the actual value for each of the m DNN models.

According to the present invention, when a FEC parameter is determined, a deep neural network structure is applied to dynamically change the packet size in an encoding block, thereby optimizing peak signal to noise ratio (PSNR) and maximizing transmission efficiency.

In addition, according to the present invention, DNN learning is performed by receiving network state information corresponding to a network layer and video encoding rate information corresponding to an application layer among OSI layers, and thus dynamically according to network state information and video encoding rates. By outputting the variable symbol size, number of symbols, and packet size, the packet size used for data transmission can be adaptively adjusted according to the state of the network and application.

In addition, according to the present invention, for example, the (32-32-16-16) DNN structure may be used, for example, to optimize image quality and image transmission efficiency for the first time, and thus, packet loss rate (PRL) and DR ( Data Rate) and Video Encoding Rate to improve the throughput by selecting the appropriate packet size. That is, according to the present invention, PSNR can send more video data at a similar level based on the same network environment, which can improve video transmission efficiency in limited network resources or congested traffic environments.

The effects of the present invention are not limited to those mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

1 is a block diagram showing an ultra-narrowband multimedia transmission apparatus 100 according to an embodiment of the present invention,
2 is a diagram illustrating an apparatus for selecting a DNN model according to an embodiment of the present invention;
3 is a diagram showing an optimal DNN model selected by the DNN model selector, that is, five fully connected neural network structures,
4 is a flowchart illustrating a method for transmitting ultra-narrowband image data using forward error correction according to an embodiment of the present invention, and
5 is a flowchart illustrating a DNN model selection method according to an embodiment of the present invention.

Prior to explaining the details for the practice of the present invention, terms or words used in the specification and claims can be properly defined by the inventor in order to describe his or her invention in the best way. Based on the principle that it should be interpreted as a meaning and concept consistent with the technical details of the present invention.

In addition, it should be noted that when it is determined that the detailed description of the known functions and configurations related to the present invention may unnecessarily obscure the subject matter of the present invention, the detailed description is omitted.

Hereinafter, with reference to the accompanying drawings, a specific technical content to be carried out in the present invention will be described in detail.

Each configuration of the devices 100 and 200 shown in FIGS. 1 and 2 indicates that they can be functionally and logically separated, and that each configuration is divided into separate physical devices or generated by a separate code. It is not meant that the average expert in the technical field of the present invention can easily infer.

The devices 100 and 200 may be installed in a predetermined data processing device to implement the technical idea of the present invention.

In addition, the devices 100 and 200 according to an embodiment of the present invention use an electronic device capable of installing and executing programs such as a microprocessor, memory, field programmable gate array (FPGA), and application specific integrated circuit (ASIC). Can be implemented.

1 is a block diagram illustrating an ultra-narrowband multimedia transmission apparatus 100 according to an embodiment of the present invention.

The apparatus 100 shown in FIG. 1 may be installed in a military equipment facility capable of transmitting and receiving multimedia data including a video, such as a surveillance reconnaissance aircraft and a surveillance vehicle in wireless communication with a control center or a control center.

Referring to Figure 1, the ultra-narrowband multimedia transmission apparatus 100 according to an embodiment of the present invention is a network information providing unit 110, video encoder 120, FEC variable control unit 130, FEC encoding and packet generation unit ( 140) and a communication interface unit 150.

The network information providing unit 110 collects or calculates network state information generated during transmission and reception of image data and provides it to the FEC variable control unit 130. The network status information may include a plurality of information related to network communication, such as a packet loss rate (PLR), a data rate (DR), a delay, and a bandwidth. The method of obtaining the network status information may use one of various well-known technologies.

The video encoder 120 encodes video data to be transmitted based on a video encoding rate. In addition, the video encoder 120 provides a video encoding rate (VER) used for encoding of image data to the FEC variable control unit 130.

The FEC variable control unit 130 inputs the network status information and the video encoding rate (VER) provided from the network information providing unit 110 into the optimal DNN model, and applies FEC parameters to be applied when FEC encoding of image data to the network status information. Therefore, it can be output dynamically variable.

The FEC variable control unit 130 may use a packet loss rate (PLR) and a data rate (DR) among a plurality of network state information as inputs of an optimal DNN model, which is, of course, changeable.

In addition, the FEC parameters dynamically changed by the FEC variable control unit 130 include a symbol size (s, symbol size), a symbol number (k, symbol number), and a packet size (p, packet size) of the raptor encoding block. do. The symbol size, the number of symbols, and the packet size are one of the variables affecting optimal image quality and transmission efficiency.

The optimal DNN model used by the FEC variable control unit 130 is a model learned and selected by the DNN model selection device 200 to be described later with reference to FIG. 2.

The FEC encoding and packet generation unit 140 uses the FEC parameters that are dynamically variable and output according to the packet loss rate (PLR), data rate (DR), and video encoding rate (VER) in the FEC variable control unit 130 to perform video. The encoder 120 may FEC-encode the encoded video data and generate a packet according to the packet size included in the FEC parameters. That is, the FEC encoding and packet generator 140 generates a packet after FEC encoding the encoded video data based on the packet size dynamically calculated by the FEC variable controller 130.

The communication interface unit 150 wirelessly transmits the packet generated by the FEC encoding and packet generation unit 140 to the target, and provides the communication result related to the transmission to the network information providing unit 110.

2 is a diagram illustrating a DNN model selection device 200 according to an embodiment of the present invention.

The DNN model selection device 200 shown in FIG. 2 may be provided in a server of the control center or implemented as a separate computer device to learn and select the DNN model, and the selected optimal DNN model to the image data transmission device 100 It can be designed to be installable and executable. The DNN model selection device 200 may select an optimal DNN model for AL-FEC (Application Layer FER) parameter determination.

Referring to FIG. 2, the DNN model selection device 200 according to an embodiment of the present invention includes an actual value storage unit 210, a DNN model generation unit 220, a predicted value calculation unit 230, and a DNN model selection unit 240 ).

The actual value storage unit 210 varies the network state information and the video encoding rate (VER) calculated using a multimedia transmission protocol algorithm within a set test range, and the maximum throughput enables symbol size (s) and number of symbols ( The combination of k) and packet size p can be stored as an actual value consisting of (symbol size, number of symbols, packet size) (hereinafter referred to as (s, k, p)).

A combination of (s, k, p) stored according to the network state information and the video encoding rate in the actual value storage unit 210, that is, a plurality of actual values may be used as a DNN training sample for finding an optimal model parameter of the DNN structure . Actual values for use as a DNN training sample may be extracted by a separate electronic device (not shown) such as a PC or the DNN model selection device 200 by executing a multimedia transmission protocol algorithm. Hereinafter, a case where the electronic device (not shown) extracts will be described as an example.

First, in order to extract a sample, an electronic device (not shown) can change the packet loss rate (PLR), data rate (DR), and video encoding rate (VER) in units of a predetermined unit (Unit Step). k, p) extract the combination. Table 1 shows the ranges of the packet loss rate (PLR), data rate (DR), and video encoding rate (VER) applied during sample extraction.

Database Test range Unit step Packet Loss Rate (PLR) 0.001%~0.04% 0.001% or 0.002% Data Rate (DR) 0.1Mbps ~ 1Mbps 0.05Mbps Video Encoding Rate (VER) 0.1Mbps ~ 1Mbps 0.1Mbps

The test range and the unit step shown in [Table 1] are examples used to train the DNN model and select the optimal DNN, but are not limited thereto, and can be changed.

The multimedia transmission protocol algorithm used for sample extraction is an existing algorithm for determining FEC parameters in the multimedia transmission protocol.

First, an electronic device (not shown) defines problems such as symbol size (s), number of symbols (k), and code rate (c) of a raptor encoding block for optimal image quality and transmission efficiency.

And, the electronic device (not shown) uses a global search-based algorithm to find the solution of the problem defined above. That is, the algorithm generates usable combinations of (s,k) and searches for s, k and c to maximize throughput within the combination. It provides a finite number of available (s, k) combinations, and c is a value that can be determined according to a given (s, k) combination and network condition, i.e., packet loss rate (PLR).

The parameter determination algorithm is as follows.

First, the electronic device (not shown) initializes a set in which parameter candidates are stored, and generates all available combinations of (s, k).

Second, the electronic device (not shown) selects one of the generated (s, k) combinations, and based on the selected (s, k), the number of packets corresponding to the raptor encoding block and the minimum number of packets required for raptor decoding, The code rate (c) is determined in consideration of the packet loss rate (PLR).

Third, the electronic device (not shown) calculates whether or not a buffer underflow of the receiving end can be prevented for the selected (s, k) combination.

Fourth, the electronic device (not shown) calculates the throughput of the application layer with the combination of (s, k) and c determined above, and a fixed packet size (p). That is, the electronic device (not shown) finds the maximum raptor encoding block size by combining the symbol size (s) and the number of symbols (k).

Fifth, the electronic device (not shown) returns the parameter candidate in M _cnd as the solution to the above problem if the above process is performed on the (s, k) combination. If there is a combination (s, k) that has not been searched yet, the above operation is repeated.

An electronic device (not shown) changes the packet loss rate (PLR), data rate (DR), and data encoding rate (VER) in the above-described operation to a predetermined unit (Unit Step), while changing the packet loss rate (PLR) and data rate A combination of (s, k, p) capable of maximum throughput for (DR) and video encoding rate (VER) is extracted.

For example, first, the electronic device (not shown) extracts a combination of (s, k, p) capable of maximum throughput when (PLR, DR, VER)=(0.1, 0.1, 0.0001), and then (PLR , DR, VER) = (0.1, 0.1, 0.0002), extracts a combination of (s, k, p) capable of maximum throughput, and finally (PLR, DR, VER) = (1, 1, by this method) 0.04), extracts (s, k, p) combinations capable of maximum throughput.

The actual values extracted by the above-described operation are stored in the actual value storage unit 210.

The DNN model generation unit 220 generates m different DNN models in which at least one of the number of layers used for DNN learning (that is, of the DNN structure) and the number of neurons for each layer is different. In order to find the optimal model parameters, that is, to select the optimal DNN model, the DNN model generation unit 220 has 3, 5, and 7 hierarchies, 64, 32, 16, and 8 And it is possible to create a DNN model using the number of neurons for each of the four layers, and the number of layers and the number of neurons used at this time is not limited to this.

In the general DNN structure, the dimensionality is reduced from the lower layer to the upper layer. Accordingly, the DNN model generation unit 220 may configure the model in a form in which the number of neurons decreases as the hierarchy goes higher.

The predicted value calculating unit 230 inputs the network state information (PLR, DR) and video encoding rate (VER) for each of the m DNN models generated by the DNN model generator 220 while varying within a set test range, For each of the m DNN models, n prediction values consisting of (symbol size, symbol count, and packet size) (hereinafter referred to as (s', k', p')) may be calculated.

The test range and change unit of the network state information (PLR, DR) and video encoding rate (VER) used by the predicted value calculator 230 to calculate the predicted values are values used by the electronic device (not shown) when extracting actual values. Is the same as

In order to find the optimal model parameter of the DNN structure, a plurality of samples (ie, actual values) extracted from an electronic device (not shown) are stored in the actual value storage unit 210 and each DNN is calculated by the predicted value calculation unit 230. When prediction values are calculated for each model, the DNN model selector 240 may compare the predicted values calculated for each m DNN models and the stored actual values to select the optimal DNN model with the least error.

The DNN model selector 240 uses the Mean Square Error (MSE) method to measure the MSE value for the symbol size (s), the MSE value for the number of symbols (k), and the MSE for the packet size (p). The DNN model having the smallest result after adding the calculated MSE values can be selected as an optimal DNN model.

For example, the predicted value calculator 230 implements the proposed DNN structure in MatConvNet, a Matlab-based deep learning library, for experimentation. The DNN model selector 240 uses an ADAM optimization method for learning the proposed method, and performs learning for a total of 500 generations (epoch). The learning rate is 0.0001, the weight decay value is 0.0001, and the size of the mini-batch is set to 32 when learning. The set value is not limited to this as an example for the experiment.

The DNN model selector 240 uses the MSE between the predicted (s', k', p') and the actual value (s, k, p) as a cost function for learning, and the cost function (J) is [ Equation 1].

In Equation 1, the values s, k, and p can have s={16, 32, 64, 128}, k={128, 256, 512}, and p={32, 64, 128}, respectively. . When the dynamic range of each parameter value is different, as a result, when optimizing the cost function, each parameter has a different weighting value. To solve this, the predicted value calculator 230 and the DNN model selector 240 perform learning on the s, k, and p values substituted by the log scale, and select the optimal DNN model.

According to the above-described operation, the DNN model selection unit 240 includes n prediction values s1', k1', and p1' predicted by the first DNN model among m DNN models and the actual value storage unit 210. For the stored n actual values (s, k, p), the MSE values of s1' and s, the MSE values of k1' and k, and the MSE values of p1' and p are calculated, and the calculated MSE values are added. MSE1 for the first DNN model is calculated.

In this way, the DNN model selector 240 calculates MSEs for all m DNN models. [Table 2] shows the MSE performance in the Test Dataset for m DNN models in the DNN model selector 240.

Number of layers Per layer
Number of neurons MSE (testing data set) s MSE k MSE p MSE Ms of (s+k+p) 5 64-64-32-32 0.633. 0.005 0.552 1.191 5 32-32-16-16 0.634 0.006 0.548 1.189 5 16-16-8-8 0.577 0.139 0.580 1.295 5 8-8-4-4 0.625 0.020 0.609 1.254 7 32-32-32-16-16-16 0.640 0.006 0.575 1.220 3 16-8 0.592 0.122 0.555 1.269

Referring to [Table 2], it can be seen that the number of layers used in the DNN model is 5, and the number of neurons for each layer is 32, 32, 16, and 16, respectively, and the calculated MSE is the smallest. Therefore, the DNN model selector 240 may select the DNN structure as an optimal DNN model (32-32-16-16) to perform optimization of image quality and image transmission efficiency, and the selected optimal DNN model is an FEC variable controller It can be applied to 130 to be used to dynamically calculate FEC parameters. (32-32-16-16) As an example, the DNN structure may be changed according to a network state or an environment such as the number of samples.

3 is a diagram illustrating an optimal DNN model selected by the DNN model selector 240, that is, five fully connected neural network structures.

Referring to FIG. 3, input variables of the deep neural network include a packet loss rate (PLR) and a data rate (DR) corresponding to a network layer (or an IP layer) among OSI layers, and a video encoding rate (VER) corresponding to an application layer. It can be seen that the output variables are symbol size (s), number of symbols (k), and packet size (p), which are raptor encoding parameters.

The optimal DNN model selected according to an embodiment of the present invention is composed of five Fully Connected Layers, and the output of each Fully Connected layer passes through a ReLU (Rectified Liner Unit) to secure nonlinearity.

ReLU is a mapping function that allows negative input values to pass through, and non-negative input values to pass through. The hidden layer 1 and layer 2 are composed of 32 vectors (neurons), and the hidden layer 3 and layer 4 are composed of 16 vectors. The i-th complete connection layer that passes through the ReLU is expressed as [Equation 2].

및

은 i번째 완전 연결 계층의 입력 및 출력,

및

는 완전 연결 계층의 모델 파라미터로, 학습을 통해 결정된다. In [Equation 2],

And

Is the input and output of the i-th fully connected layer,

And

Is a model parameter of the fully connected layer and is determined through learning.

4 is a flowchart illustrating a method for transmitting ultra-narrowband image data using forward error correction according to an embodiment of the present invention.

The electronic device 100 for performing the ultra-narrowband image data transmission method of FIG. 4 may be the ultra-narrowband multimedia transmission device 100 described with reference to FIGS. 1 to 3.

Referring to FIG. 4, the electronic device 100 collects or calculates network state information generated during transmission and reception of image data (S410). The network status information includes various information such as packet loss rate (PLR), data rate (DR), delay, and bandwidth.

The electronic device 100 encodes the image data based on the video encoding rate (VER) (S420).

The electronic device 100 inputs the packet loss rate (PLR), the data rate (DR), and the video encoding rate (VER) used in step S420 into the optimal DNN model among the network state information acquired in step S410, and converts the image data. The FEC parameters to be applied during FEC encoding are dynamically varied according to a packet loss rate (PLR), a data rate (DR), and a video encoding rate (VER) (S430). The FEC parameters include symbol size (s), symbol number (k) and packet size (p).

The electronic device 100 FEC-encodes the video data encoded in step S420 using the FEC parameters variablely output in step S430 to generate a packet according to the packet size determined in step S430 (S440).

The electronic device 100 transmits the packet generated in step S440 to the target (S450).

5 is a flowchart illustrating a DNN model selection method according to an embodiment of the present invention.

Since the DNN model selection method of FIG. 5 may be performed by the DNN model selection device 200 described with reference to FIG. 2, detailed operation description is omitted.

The DNN model selection device 200 stores an actual value composed of (s, k, p) calculated using a multimedia transmission protocol algorithm in an electronic device (not shown) (S510). In step S510, the electronic device (not shown), using the multimedia transmission protocol algorithm, the network state information (PLR, DR) and video encoding rate (VER) is calculated while varying within a set test range, maximum throughput is possible The combination of symbol size (s), number of symbols (k), and packet size (p) is extracted as an actual value consisting of (s, k, p).

The DNN model selection device 200 generates m different DNN models in which at least one of the number of layers used for DNN learning and the number of neurons for each layer is different (S520).

The DNN model selection device 200 inputs the network state information (PLR, DR) and video encoding rate (VER) for each of the m DNN models generated in step S520 while varying within a set test range, thereby m m DNN models N prediction values of (s', k', p') are calculated for each field (S530).

The DNN model selection device 200 compares the predicted value and the actual value for each m DNN models and selects an optimal DNN model having a minimum cost function (S540).

Meanwhile, in the algorithm using the existing multimedia transmission protocol, transmission control is performed by applying a raptor FEC with a fixed packet size. However, there is a limitation in transmission efficiency when the packet size is fixed.

Accordingly, in order to overcome the disadvantages of the existing multimedia transmission protocol, the DNN structure is applied in the embodiment of the present invention, and an appropriate packet size is selected according to the packet loss rate (PLR), data rate (DR), and video encoding rate (VER). Throughput can be improved. That is, according to an embodiment of the present invention, more video data may be transmitted at a level similar to a peak signal-to-noise ratio (PSNR) based on the same network environment. This can improve video transmission efficiency when applied to a limited network resource or a congested traffic environment.

The ultra-narrow band image data transmission method of the ultra-narrow band image data transmission apparatus using forward error correction according to the present invention is included in a recording medium that can be read through a computer by tangibly implementing a program of instructions for implementing it. It may be readily understood by those skilled in the art.

Accordingly, the present invention provides a program stored in a computer-readable recording medium executed on a computer controlling the image data transmission device in order to implement a method for transmitting ultra narrow band image data using forward error correction.

100: ultra-narrowband multimedia transmission device
110: network information providing unit
120: video encoder
130: FEC variable control
140: FEC encoding and packet generator
200: DNN model selector
210: actual value storage unit
220: DNN model generator
230: predicted value calculation unit
240: DNN model selector

Claims (2)

In a method of transmitting ultra-narrowband multimedia using Forward Error Correction (FEC),
(A) the electronic device, collecting or calculating network status information that occurs while transmitting and receiving video data;
(B) the electronic device encoding the video data based on a video encoding rate;
(C) the electronic device dynamically inputs the network state information provided in step (A) into an optimal DNN model and dynamically outputs FEC parameters to be applied when encoding FEC of the image data according to the network state information. step;
(D) generating, by the electronic device, a packet by FEC encoding the encoded video data using FEC parameters that are variablely output in the step (C); And
(E) the electronic device, transmitting the packet generated in step (D); includes,
In step (B),
The electronic device provides video encoding rate information applied during the encoding,
In step (C),
The electronic device inputs the video encoding rate information input from the step (B) together with the network state information into the optimal DNN model to change the FEC parameters according to the network state information and the video encoding rate information. Output,
Before step (A),
(F) The symbol size (s) and the number of symbols (maximum throughput) that the DNN model selection device calculates while varying the network state information and the video encoding rate within a set test range using a multimedia transmission protocol algorithm ( storing the combination of k) and packet size (p) as an actual value consisting of (symbol size, number of symbols, packet size);
(G) generating, by the DNN model selection apparatus, m different DNN models in which at least one of the number of layers used for DNN learning and the number of neurons for each layer is different;
(H) The DNN model selection device inputs the network state information and the video encoding rate for each of the m DNN models while varying within a set test range, and then generates n (symbol sizes) for each of the m DNN models. Calculating a prediction value consisting of', the number of symbols' and the packet size'); And
(I) the DNN model selection device, comparing the predicted value and the actual value for each of the m DNN models to select the optimal DNN model; Ultra narrowband multimedia transmission method. According to claim 1,
The network status information includes a packet loss rate (PLR) and a data rate,
The FEC parameters include symbol size (s, symbol size), symbol number (k, symbol number), and packet size (p, packet size),
Step (C) is,
The electronic device dynamically calculates a packet size according to the packet loss rate and data rate,
Step (D) is,
The electronic device generates a packet after FEC encoding the encoded video data based on the packet size dynamically calculated in the step (C), and then generates a packet in the ultra-narrowband using forward error correction.

Images