KR102271371B1 - Super-Resolution Streaming Video Delivery System Based-on Mobile Edge Computing for Network Traffic Reduction - Google Patents

Abstract

The present invention relates to a video transmission system for upgrading and providing a low-resolution streaming video to a high-resolution streaming video. More particularly, the super-resolution streaming video transmission system based on mobile edge computing for reducing network traffic, is formed to: calculate in advance a super-revolution parameter capable of high-resolution upgrade streaming for a predicted video which a user is likely to watch by using an artificial intelligence-based super-resolution (SR) model in a mobile edge computing server; by providing the super-revolution parameter calculated if the user watches the video, upgrade the low-resolution streaming video into a high-resolution streaming video to watch; and share computing power by forming a community between the mobile edge computing server for reducing network traffic. According to the present invention, the super-resolution streaming video transmission system comprises a cloud server, an edge computing server and a user device.

Description

Super-Resolution Streaming Video Delivery System Based-on Mobile Edge Computing for Network Traffic Reduction

The present invention relates to a video transmission system that upgrades a low-resolution streaming video to a high-resolution streaming video and provides it, and more particularly, by using an artificial intelligence-based super-resolution (SR) model in a mobile edge computing server, the possibility of user viewing For the predicted video, the super-revolution parameter capable of high-resolution upgrade streaming is calculated in advance, and when the video is viewed, the calculated super-revolution parameter is provided so that the low-resolution streaming video can be upgraded to a high-resolution streaming video for viewing. However, it relates to a mobile edge computing-based super-resolution streaming video transmission system for reducing network traffic that is formed to share computing power by forming a community between mobile edge computing servers to reduce network traffic.

As of June 2019, released by the Ministry of Science and ICT, the number of domestic smartphone lines reached about 50.4 million, and the smartphone use rate of Korean adults was 93%. According to Pew Research, an American public opinion organization, the smartphone ownership rate in Korea is 95%, which is the highest in the 27 countries surveyed, and the use of smartphones in Korea is almost essential.

Among them, smartphones are widely used for online video viewing, and according to a recent survey, Netflix is experiencing a rapid increase in viewership by more than double compared to 2019, with YouTube being used for a total of 31.7 billion minutes per month. The use of smartphones for online video viewing is steadily increasing.

On the other hand, when watching the online video as described above, 140p, 240p, 360p, ... , 1024p, etc., are provided in various resolutions from low to high resolution. Of these, the more high-resolution video is provided, the more encoding and traffic costs are incurred from the content provider providing the service. Since preparations such as the need to prepare the video format for each terminal and browser must be made, it ultimately leads to a user's charge burden.

Accordingly, the super-resolution technology was developed, and the super-resolution technology is made to learn a weight that can minimize the difference between the original high-resolution image and the low-resolution image, so that the high-resolution image is viewed despite viewing the low-resolution image. It showed the advantage of being able to watch by minimizing the difference between the

However, if the super-resolution technology is applied only to minimize the difference between viewing images as described above, real-time transmission of images is impossible due to learning time, and the difference between high-resolution and low-resolution images for one image to be viewed Due to the characteristic of simply minimizing , there is a problem that an error occurs when the same weight is applied to other images.

On the other hand, the prior art related to the resolution improvement of moving pictures is Korean Patent No. 10-2130074 'High-definition VOD providing system in poor network environment', Korean Patent No. 10-1961177 'Image processing method and apparatus using a neural network' , Korean Patent Application Laid-Open No. 10-2019-0119550 'Method and apparatus for improving image resolution'.

The present invention has been devised to solve the above problems, and an object of the present invention is to utilize an artificial intelligence-based super-resolution (SR) model in a mobile edge computing server based on cache information, etc., an image that a user is likely to view By predicting and learning the weight in advance, and when the user views the corresponding image in low resolution, the super-resolution parameter for the corresponding image is transmitted together so that it can be viewed in high resolution, so that even with a small amount of data Super-resolution streaming video transmission based on mobile edge computing to reduce network traffic that allows users to view high-resolution video in real time and share computing power by forming a community between mobile edge computing servers to further reduce network traffic to provide a system.

A mobile edge computing-based super-resolution streaming video transmission system for reducing network traffic according to an embodiment of the present invention for solving the above problems stores a plurality of video contents and transmits the corresponding video contents in real time according to the video contents playback request A cloud server that supports streaming services by providing; An edge computing server that interworks with the cloud server to provide a super-resolution parameter capable of high-resolution for a low-resolution image among the image content provided by the cloud server, and the cloud server by sending a video content playback request to support the requested video content and a user device that receives a corresponding super-resolution parameter from the edge computing server and uses low-resolution image data and reproduces it as a high-resolution image, wherein the edge computing server is another edge computing server that has little effect on network traffic. It can be configured to perform learning by receiving computing power from other edge computing servers tied to the block chain by forming a block chain.

Here, the edge computing server may include: a learning data management unit for receiving the image content delivered from the cloud server in parallel as a low-resolution image and a high-resolution image, changing each of the super-resolution parameters into a format required for learning, and storing; a super-resolution model unit for generating super-resolution parameters of weight and bias capable of increasing the low resolution by learning to minimize the difference in resolution between the formatted low-resolution image and the high-resolution image from the learning data management unit; A super-resolution parameter storage unit for storing super-resolution parameters of weight and bias generated by learning from the super-resolution model unit, and a super-resolution parameter storage unit for weights and biases stored in the super-resolution parameter storage unit It may include a communication unit that transmits the resolution parameter to the user device when the user device reproduces a low-resolution image.

In addition, the learning data management unit may be configured to delete the low-resolution image data and the high-resolution image data used for the learning when the learning is finished in the super-resolution model unit.

In addition, the edge computing server further includes an image content analysis unit that analyzes image cache information stored in the user device, and classifies image content according to user preference image information derived according to the analysis by category, the super-resolution The model unit generates the super-resolution parameter by pre-learning the image content classified by category in the image content analysis unit, and the communication unit uses the super-resolution parameter generated when the user device is not in use. may be configured for delivery to a device.

In addition, the image content analysis unit may be configured to classify the popular images selected by the cloud server by category in addition to the image content according to the user preference image information.

In addition, the categories are formed in a hierarchical structure, and the super-resolution model unit is configured to perform learning to generate super-resolution parameters corresponding to all categories based on the highest category image among the categories of the hierarchical structure. can

In addition, the video content analysis unit, when a new category is added according to the change in the video cache information, connects to a subcategory of the category determined to be closest to the added video cache information, and when the closest category does not exist, the new top category It can be configured to create

In addition, the hierarchical category is composed of a graphic model in which each category is composed of a node and the super-resolution parameter is composed of a variable, so that the user device is a super - When searching for a resolution parameter, it can be configured to support streaming that converts a low-resolution image into a high-resolution image by searching for an array through a node and reading only the variable of the corresponding category. .

On the other hand, in the super-resolution streaming video transmission system, when there is no matching category for the video content requested to be reproduced from the user device and the network environment satisfies a set condition, the cloud server sends a high-resolution video to the user device Simultaneously with streaming, the video content analysis unit generates a category for the video content currently being played, and the edge computing server generates a super-resolution parameter and transmits it to a user device, wherein the user device includes the super-resolution A streaming video may be reproduced by receiving a low-resolution video from the cloud server from a point in time when the parameter exists, and converting it to a high-resolution.

In addition, in the super-resolution streaming video transmission system, when there is no matching category for the video content requested to be reproduced from the user device and the network environment does not satisfy the set condition, the user device provides information about the requested streaming video. At the same time, a low-resolution image is reproduced with a high resolution through the super-resolution parameter of a higher-level category, and the image content analysis unit creates a category for the currently played image content, and generates a super-resolution parameter in the edge computing server. to the user device, and the user device may receive the low-resolution image from the cloud server from the point in time when the super-resolution parameter is present, convert it to a high-resolution image, and reproduce the image content.

In addition, the edge computing server, after the video content reproduced by the user device is finished, proceeds with the feedback itself or is transmitted from the user device, and for video content that has received feedback below a set reference value, the super-resolution model Wealth can be transformed to increase learning precision.

Here, the feedback is a comparative feedback comparing a high-resolution image provided by a cloud server with respect to the reproduced image content and a high-resolution image transmitted from the edge computing server with a super-resolution parameter, and input to the user device. It may be one or more of the evaluated feedbacks.

In addition, the transformation of the super-resolution model unit may be configured to increase the learnable super-resolution parameter by adding more nodes and layers.

In addition, the edge computing server performs super-resolution parameter learning in a learning mode or a test mode, and the learning mode is updated by activating a gradient of a learnable super-resolution parameter, and the test mode is a learnable Learning to deactivate the gradient of the super-resolution parameter may be performed.

In addition, a plurality of edge computing servers tied to the block chain constitute a community mobile edge computing set as a committee node and a local node, and the edge computing server as a committee node selects an edge computing server that performs super-resolution parameter learning, A coin is issued to the edge computing server that has performed learning, and it is formed to record the ledger for coin issuance, and the edge computing server, which is a local node, supports computing power according to the instructions of the edge computing server, which is the committee node, and in return It may be formed to receive a coin payment.

The mobile edge computing-based super-resolution streaming video transmission system for reducing network traffic according to an embodiment of the present invention has the effect of enabling a mobile streaming video viewer to enjoy a high-resolution video at a lower communication cost than before.

In addition, since the mobile edge computing-based super-resolution streaming video transmission system for reducing network traffic according to an embodiment of the present invention can reproduce the entire streaming time only with low-resolution video, high-definition video can be reproduced even in areas with poor network infrastructure. there is an effect

In addition, the mobile edge computing-based super-resolution streaming video transmission system for reducing network traffic according to an embodiment of the present invention collects data necessary for learning based on user preference, so that an image is selected based on user preference. Various business models such as a recommendation service may be applicable.

In addition, the mobile edge computing-based super-resolution streaming video transmission system for reducing network traffic according to an embodiment of the present invention may advance high-definition video streaming technology.

In addition, the mobile edge computing-based super-resolution streaming video transmission system for reducing network traffic according to an embodiment of the present invention can reduce traffic and increase learning performance by forming a community between edge computing servers and sharing computing power. have.

In addition, the mobile edge computing-based super-resolution streaming video transmission system for reducing network traffic according to an embodiment of the present invention has the advantage of being applicable to blockchain-based services through blockchain-based distributed processing learning.

In addition, the above-mentioned effects according to the embodiments of the present invention are not limited to the described content, and may further include all effects predictable from the specification and drawings.

1 is a block diagram illustrating the configuration of a mobile edge computing-based super-resolution streaming video transmission system for reducing network traffic according to an embodiment of the present invention.
FIG. 2 is a block diagram of a detailed configuration of an edge computing server, which is a configuration of the super-resolution streaming video transmission system of FIG. 1 .
3 is a diagram illustrating a format process of low-resolution and high-resolution images required for super-resolution parameter learning of a mobile edge computing-based super-resolution streaming video transmission system for reducing network traffic according to an embodiment of the present invention.
4 is a diagram showing a flow of high-resolution low-resolution video of a mobile edge computing-based super-resolution streaming video transmission system for reducing network traffic according to an embodiment of the present invention.
5 is a diagram illustrating a hierarchical category structure.
6 is a diagram illustrating an example of a flow of a super-resolution parameter generation process using cache information of a user device according to an embodiment of the present invention.
7 is a diagram illustrating an example of a flow of a super-resolution parameter generation process using cache information of a cloud server according to an embodiment of the present invention.
8 is a diagram showing an example of a method of playing a streaming video when there is no category according to an embodiment of the present invention.
9 is a diagram showing a flow of a method of reproducing a streaming video when the category of FIG. 8 does not exist.
10 is a flowchart illustrating a super-resolution re-learning process through feedback according to an embodiment of the present invention.
11 is a diagram illustrating an overall form of improving a super-resolution model by using feedback according to an embodiment of the present invention.
12 is a diagram illustrating an example of sharing computing power by forming a community between edge computing servers according to an embodiment of the present invention and receiving compensation accordingly.

Hereinafter, the description of the present invention with reference to the drawings is not limited to specific embodiments, and various modifications may be made and various embodiments may be provided. In addition, it should be understood that the contents described below include all conversions, equivalents, and substitutes included in the spirit and scope of the present invention.

In the following description, terms such as first and second are terms used to describe various components, meanings are not limited thereto, and are used only for the purpose of distinguishing one component from other components.

Like reference numbers used throughout this specification refer to like elements.

As used herein, the singular expression includes the plural expression unless the context clearly dictates otherwise. In addition, terms such as "comprises", "comprising" or "have" described below are intended to designate the existence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification. It should be construed as not precluding the possibility of addition or existence of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

In addition, terms such as “…unit”, “…group”, and “…module” described in the specification mean a unit that processes at least one function or operation, which may be implemented by hardware or software or a combination of hardware and software. can

Hereinafter, a mobile edge computing-based super-resolution streaming video transmission system for reducing network traffic according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram of the configuration of a mobile edge computing-based super-resolution streaming video transmission system for reducing network traffic according to an embodiment of the present invention, and FIG. 2 is a configuration of the super-resolution streaming video transmission system of FIG. 1 It is a block diagram of the detailed configuration of an in-edge computing server, and FIG. 3 is a low-resolution and high-resolution required for super-resolution parameter learning of a mobile edge computing-based super-resolution streaming video transmission system for reducing network traffic according to an embodiment of the present invention. It is a diagram illustrating the process of formatting an image.

In addition, FIG. 4 is a view showing a flow of high-resolution a low-resolution video of a mobile edge computing-based super-resolution streaming video transmission system for reducing network traffic according to an embodiment of the present invention, and FIG. 5 illustrates a hierarchical category structure 6 is a diagram showing an example of the flow of the super-resolution parameter generation process using cache information of the user device according to an embodiment of the present invention.

In addition, FIG. 7 is a diagram showing an example of a flow of a super-resolution parameter generation process using cache information of a cloud server according to an embodiment of the present invention, and FIG. 8 is a case in which there is no category according to an embodiment of the present invention It is a diagram showing an example of a method of playing a streaming video, and FIG. 9 is a diagram showing a flow of a method of playing a streaming video when there is no category of FIG.

Also, FIG. 10 is a flowchart showing a super-resolution re-learning process through feedback according to an embodiment of the present invention, and FIG. 11 is an overall form of improving a super-resolution model using feedback according to an embodiment of the present invention. 12 is a diagram illustrating an example of forming a community between edge computing servers according to an embodiment of the present invention to share computing power and receive compensation accordingly.

First, referring to FIG. 1 , a mobile edge computing-based super-resolution streaming video transmission system for reducing network traffic according to an embodiment of the present invention is a cloud server 10 , an edge computing server 20 , and a user device 30 . It may be composed of

Specifically, the cloud server 10 may store a plurality of image contents. In addition, the cloud server 10 may support the streaming service by providing the requested image content to the user device 30 in real time according to the image content reproduction request of the user device 30 . In this case, the video content may be supported by various resolutions, and the resolution of the supported resolution is determined according to the video content reproduction request of the user device 30 .

The edge computing server 20 may provide a super-resolution parameter capable of high-resolution for a low-resolution image among the image contents provided from the cloud server 10 by interworking with the cloud server 10 .

Here, the edge computing server 20 is preferably a mobile edge computing server, and may be an edge computing server provided inside the user device 30 . However, the present invention is not necessarily limited thereto, and the edge computing server 20 may be provided outside the user device 30 , on the other hand, by connecting to the user device 30 in a state provided externally through an external hard drive or USB. It may be in the form used.

The user device 30 is a terminal or device that a user carries and can watch a video, such as a mobile phone or a tablet PC, and transmits a video content playback request to the cloud server 10 by receiving a video content playback signal from the user. A streaming service for the requested image content may be supported from the server 10 .

In this case, the user device 30 may be provided with the super-resolution parameter provided from the edge computing server 20 , so that even if the user requests to view the low-resolution image, the user can view the high-resolution image. Through this, image data such as traffic can be viewed with low communication cost by receiving a high-resolution service while using low-resolution image data, and it is possible to view the image even in an area where the network is poor.

On the other hand, the user device 30 receiving the high-resolution upgrade service as described above is configured to receive a low-resolution video viewing request and deliver it to the cloud server 10, as well as automatically view a low-resolution video when receiving a high-resolution video viewing request. It may also be configured to convert a request and forward it to the cloud server 10 .

However, this is not necessarily limited, and the above configuration may depend on the setting of the user device 30 .

In addition, the edge computing server 20 can share computing power by forming a community tied to a block chain with other edge computing servers, and compensation can be made accordingly. A detailed description thereof will be described later with reference to FIG. 9 . decide to do

2 to 7 , the edge computing server 20 includes a learning data management unit 21 , a super-resolution model unit 22 , a super-resolution parameter storage unit 23 , and a communication unit 24 . can be

Specifically, the learning data management unit 21 may receive the image content delivered from the cloud server 10 in parallel as a low-resolution image and a high-resolution image. In this case, the classification of the low-resolution image and the high-resolution image depends on the setting. Also, the learning data management unit 21 may change and store the received low-resolution image and high-resolution image into a format required for super-resolution parameter learning, respectively.

Referring to FIG. 3 illustrating the format process required for super-resolution parameter learning, an example of the format boxed above in the algorithm is a format (Lorf) example of a low-resolution image, and an example of the format boxed below is a format of a high-resolution image (Hirf) This is an example. Among them, the <model background=”background/nature/mountain.h5”/> part may be the model name to be used for training.

The super-resolution model unit 22 may learn to minimize the difference in resolution between the low-resolution image and the high-resolution image formatted from the learning data management unit 21 . Through this, it is possible to generate super-resolution parameters of a weight and a bias that can make a low-resolution image high-resolution.

Here, the generated weights and biases are transmitted to the user device 30 requesting to reproduce the low-resolution image content together with the super-resolution (SR) model to be used for high-resolution imaging.

Also, the super-resolution model unit 22 may perform super-resolution parameter learning in a learning mode or a test mode when performing super-resolution parameter learning as described above. Here, the learning mode is a mode in which the gradient of the learnable super-resolution parameter is activated and updated, and in the test mode, the gradient of the learnable super-resolution parameter is deactivated and the input image is present in the input image. It is a mode in which learning is performed according to the parameters to be used and the result is obtained.

The super-resolution parameter storage unit 23 may store super-resolution parameters of weights and biases generated by learning from the super-resolution model unit 22 , and the communication unit 24 includes the super-resolution parameter storage unit 23 . The super-resolution parameters stored in the user device 30 can be transmitted to play the high-resolution super-resolution streaming video when the user device 30 plays the low-resolution video, and the process of providing the super-resolution streaming video through the above configurations is shown in Fig. It may be as shown in 4 .

On the other hand, the learning and super-resolution parameter generation of the edge computing server 20 as described above are performed before the user watches the video, and at the same time the user starts watching the video, the user device 30 receives the super-resolution parameter. It is desirable to watch it in high resolution in real time. A detailed description thereof will be provided later.

In addition, the learning data management unit 21 transmits in parallel from the cloud server 10 and uses the low-resolution image data for learning when the learning of the image content for which the super-resolution model unit 22 has been trained is finished. and high-resolution image data can be deleted to free up space and improve processing speed.

In addition, the edge computing server 20 analyzes the image cache information stored in the user device 30, and further includes an image content analysis unit 25 that classifies image content according to user preference image information derived according to the analysis into categories. may be included.

Here, the image cache information is a record of the image content viewed by the user, and the image content analysis unit 25 may be configured to analyze the category, title, viewing time, etc. of the recorded image content, and user preferences related to the user. Image information can be derived.

In this way, when the user preference image information is derived, the image content analysis unit 25 classifies all the corresponding images by category such as drama, comic, movie, etc. Preferably, as shown in FIG. 4, the category is selected from the upper category. It can be classified to form a hierarchical structure having a branching structure and forming a vertical to sub-categories.

The image content analysis unit 25 as described above has a configuration in which the user predicts the image to be viewed in advance and prepares the low-resolution high-resolution image for the image with high viewing possibility, and the user-preferred image is derived through the image content analysis unit 25 According to the flow shown in FIGS. 2 and 6, the super-resolution model unit 22 learns in advance the image content classified by category in the image content analysis unit 25 and generates super-resolution parameters. In addition, the communication unit 24 transmits the pre-generated super-resolution parameter to the user device 30 so that when the user views the low-resolution image, the user can view the high-resolution image in real time.

In this case, the pre-generated super-resolution parameter may be transmitted during a time when the user device 30 is not being used, so that it may not be processed during an image viewing time.

On the other hand, the video content analysis unit 25 has only described about the prediction of viewing by analyzing the viewing record of the user to help understanding, but it is not necessarily limited thereto. On the other hand, as shown in FIG. 7 , the user preference In addition to the image content according to the image information, the cloud server 10 may be formed to classify the popular images selected daily, weekly or monthly by category. Through this, prediction accuracy of an image that can be viewed on the user device 30 may be increased.

The super-resolution model unit 22 may perform learning to generate super-resolution parameters corresponding to all categories based on the highest category image among the categories formed in the hierarchical structure classified by the image content analysis unit 25. have.

For example, assuming that upper categories such as Korean films, American films, Chinese films, and Japanese films are formed by country based on the top category of movies, all subcategories branching from Korean films, all subcategories branching from American films, etc. It is to perform learning to generate super-resolution parameters according to categories.

At this time, when the image content analysis unit 25 detects a change in the image cache information of the user device 30, when there is no category of the corresponding image cache information, it can be added as a new category. It can be linked to a sub-category of a category determined to be closest to the video cache information, and when the closest category does not exist, a new top-level category can be created.

For example, when a category called German movies is added, it is added and linked to the parent category of movies by country, but when a category called romantic comedy movies is added, if a category related to romantic comedy movies does not exist, it will be displayed next to movies by country. You can create separate categories for romantic comedy movies.

In this case, categories can be divided by natural language processing, and if the categories are the same, they can be overwritten and reset.

In the category of such a hierarchical structure, each category is composed of nodes, and super-resolution parameters according to the corresponding categories are composed of graphic models composed of variables, so that efficient data management can be performed, The user device 30 may be formed to receive faster high-resolution image support.

More specifically, the user device 30 may search for a super-resolution parameter in order to achieve high resolution for the video content supported by the cloud server 10 . At this time, the user device 30 may search for a super-resolution parameter through a nodeized category. You can search for and read only the variable of the category to receive support for faster resolution of low-resolution images.

On the other hand, when the video content requested to be reproduced from the user device 30 is video content that is not predicted in advance by the video content analysis unit 25 and does not have a category having a super-resolution parameter, the network environment satisfies the set condition In some cases, it may be configured to support high-resolution images.

Here, the case in which the network environment satisfies the set condition may be a case in which the network environment is good, and the case in which the network environment does not satisfy the set condition may be a case in which the network environment is poor.

Specifically, when there is no matching category for the video content requested to be reproduced from the user device 30 and the network environment satisfies the set condition, the cloud server 10 first converts the original high-resolution video to the user device 30 . At the same time supporting streaming, the video content analysis unit 25 generates a category for the currently played video content, and the edge computing server 20 generates a super-resolution parameter for the corresponding category so that the user device ( 30) can be transmitted.

Through this, as shown in FIGS. 8 and 9, when a user receives a super-resolution parameter in the middle of viewing an image, the user switches to receive a low-resolution image from the cloud server 10 from the time when the super-resolution parameter exists. , the user device 30 may display an image with a high resolution through the super-resolution parameter.

Here, the original high-resolution image transmitted from the cloud server 10 to the user device 30 may be supported as DASH streaming in which the resolution size is adjusted according to network conditions.

Conversely, if there is no matching category for the video content requested to be reproduced from the user device 30 , and the network environment does not satisfy the set condition, the edge computing server 20 once the streaming requested from the user device 30 . The super-resolution parameter of the upper-level category for the video is provided to the user device 30 so that the low-resolution video is reproduced in high-resolution, while the video content analysis unit 25 creates a category for the video content currently being played. Then, the edge computing server 20 may generate the super-resolution parameter and transmit it to the user device 30 .

In this case, using the super-resolution parameter transmitted to the user device 30 , a low-resolution image may be received from the cloud server 10 from the corresponding point in time, and the image content may be reproduced by making it high-resolution.

Through this, the present invention can support the high-resolution upgrade service so that the user can see the maximum effect by flexibly applying the super-resolution parameter according to the network situation even when the user's viewing image is not predicted.

In addition, after the video content reproduced by the user device 30 is finished, the edge computing server 20 may itself provide feedback or receive it from the user device 30 to provide better super-resolution streaming. You can upgrade.

Specifically, the feedback is a comparative feedback comparing the original high-resolution image provided by the cloud server 10 with the high-resolution image by which the super-resolution parameter is transmitted from the edge computing server 20 with respect to the reproduced image content, and the user; It may be one of the feedbacks input to the device 30 and evaluated, or a composite feedback thereof.

At this time, the feedback input and evaluated from the user device 30 is not limited, but may be a score or graded feedback such as 1 to 10 points or A to F grade as an example, or may be feedback written in writing. In the written feedback, positive and negative words can be recognized and the feedback can be reflected.

The edge computing server 20 determines whether or not the feedback is obtained below the set reference value when the feedback is obtained as described above. For video content for which the feedback is obtained below the set reference value, the super-resolution model unit 22 increases the learning precision. can be converted to

That is, in the case of comparison feedback comparing the original high-resolution image provided by the cloud server 10 with the high-resolution image transmitted from the edge computing server 20 with the super-resolution parameter for the reproduced image content, FIG. As shown, by selecting a representative image for each section, the application of the super-resolution parameter is judged as good and bad, and the super-resolution (SR) application part determined as bad is re-learning. can be done to do so.

Here, the transformation to increase the learning precision is by adding more nodes and layers, and the learnable super-resolution parameter can be increased, and the reliability and high-resolution accuracy can be improved while showing the flow as shown in FIG. 11 . It can be improved and improved.

In addition, as described above, the edge computing server 20 forms a community tied to another edge computing server and a block chain to share computing power with other edge computing servers, and thus can receive a reward in coins.

Referring to FIG. 12 , the edge computing server 20 may configure a community mobile edge computing by setting a zone for mobile edge computing servers within a range that does not preferably affect network traffic, and binding them with a block chain.

In addition, multiple edge computing servers tied to the blockchain can be configured as committee nodes and local nodes.

At this time, the edge computing server, which is a committee node, selects an edge computing server that performs super-resolution parameter learning, issues a coin to the edge computing server that has performed the learning, and records a ledger for issuing coins. . That is, committee nodes can be configured to play a decision-making role.

In addition, the edge computing server, which is a local node, may be formed to support computing power according to the instructions of the edge computing server, which is a committee node, and to receive coins in return. At this time, the payment of the coin may be paid as a reward in proportion to the share ratio according to how much computing power can be provided, but this is only an example of the payment method and is not necessarily limited thereto. may be performed.

By configuring the community mobile edge computing as described above, the limitation of the computing power of the edge computing server 20 can be overcome, and when learning is performed in the super-resolution model unit 22, several super- for each layer or at the same time It is possible to learn the resolution model, so the learning performance can be improved.

Although the embodiments of the present invention have been described above with reference to the accompanying drawings, those of ordinary skill in the art to which the present invention pertains can practice the present invention in other specific forms without changing the technical spirit or essential features of the present invention. you will be able to understand Accordingly, the embodiments described above are illustrative in all respects and not restrictive.

10: Cloud Server
20: Edge Computing Server
21: learning data management unit
22: super-resolution model part
23: super-resolution parameter storage unit
24: communication department
25: video content analysis unit
30: user device

Claims (15)

a cloud server that stores a plurality of video contents and provides the corresponding video contents in real time in response to a request to play the video contents to support a streaming service;
An edge computing server that works with the cloud server to learn and provide super-resolution parameters capable of high-resolution for low-resolution images among the image contents provided from the cloud server; and
A user device that transmits a video content playback request to the cloud server to receive the requested video content, receives a corresponding super-resolution parameter from the edge computing server, uses low-resolution video data, and reproduces a high-resolution video,
The edge computing server,
It is tied with other edge computing servers that have little effect on network traffic to form a block chain, and performs learning by receiving computing power from other edge computing servers tied to the block chain.
The edge computing server,
a learning data management unit for receiving the image content delivered from the cloud server in parallel as a low-resolution image and a high-resolution image, changing each of the super-resolution parameters into a format required for learning, and storing them;
a super-resolution model unit for generating super-resolution parameters of weight and bias capable of increasing the low resolution by learning to minimize the difference in resolution between the formatted low-resolution image and the high-resolution image from the learning data management unit;
a super-resolution parameter storage unit for storing super-resolution parameters of weight and bias generated by learning from the super-resolution model unit; and
and a communication unit for transmitting super-resolution parameters of weights and biases stored in the super-resolution parameter storage unit to the user device when the user device reproduces a low-resolution image;
Analyzes the image cache information stored in the user device, and further comprises an image content analysis unit for classifying image content according to user preference image information derived according to the analysis by category,
The super-resolution model unit,
The video content analysis unit generates the super-resolution parameter by learning in advance for the video content classified by category,
The communication unit,
transmits the super-resolution parameter generated when the user device is not in use to the user device;
When there is no matching category for the video content requested to be reproduced from the user device, and the network environment satisfies the set condition,
The cloud server streams a high-resolution video to the user device, and at the same time creates a category for the video content currently being played by the video content analysis unit, generates a super-resolution parameter in the edge computing server, and transmits it to the user device and
The user device is
From the point in time when the super-resolution parameter exists, a low-resolution image is received from the cloud server, and the streaming image is reproduced by high-resolution
When there is no matching category for the video content requested to be reproduced from the user device, and the network environment does not satisfy the set condition,
In the user device, a low-resolution video is reproduced in high resolution through a super-resolution parameter of a higher-level category for the requested streaming video,
The image content analysis unit generates a category for the currently played image content, generates a super-resolution parameter in the edge computing server, and transmits it to the user device,
The user device is
Mobile edge computing-based super-resolution streaming video transmission system, characterized in that from the point in time when the super-resolution parameter is present, the low-resolution video is received from the cloud server, and the video content is reproduced by making it high-resolution.
delete The method of claim 1,
The learning data management unit,
Mobile edge computing-based super-resolution streaming video transmission system, characterized in that when learning is finished in the super-resolution model unit, the low-resolution image data and high-resolution image data used for the learning are deleted.
delete The method of claim 1,
The video content analysis unit,
Mobile edge computing-based super-resolution streaming video transmission system, characterized in that in addition to the video content according to the user preference video information, the popular video selected from the cloud server is classified by category.
The method of claim 1,
The categories are formed in a hierarchical structure,
The super-resolution model unit,
Mobile edge computing-based super-resolution streaming video transmission system, characterized in that learning is performed to generate super-resolution parameters corresponding to all categories based on the highest category video among the categories of the hierarchical structure.
7. The method of claim 6,
The video content analysis unit,
Mobile characterized in that when a new category is added according to the change in the image cache information, a sub-category of the category determined to be closest to the added image cache information is connected, and when the closest category does not exist, a new top category is created. Edge computing-based super-resolution streaming video delivery system.
7. The method of claim 6,
The categories of the hierarchical structure are,
Each category is composed of a node and the super-resolution parameter is composed of a graphic model composed of variables,
the user device;
When searching for super-resolution parameters for supported video content, streaming that converts a low-resolution video to a high-resolution video by searching for an array through a node and reading only the variables of the corresponding category Mobile edge computing-based super-resolution streaming video transmission system, characterized in that it is supported.
delete delete The method of claim 1,
The edge computing server,
After the video content reproduced by the user device is finished, the feedback is performed by itself or is transmitted from the user device,
Mobile edge computing-based super-resolution streaming video transmission system, characterized in that for video content for which feedback is obtained below a set reference value, the super-resolution model unit is converted to increase learning precision.
12. The method of claim 11,
The feedback is
One of a comparative feedback comparing a high-resolution image provided by a cloud server with respect to the reproduced image content, a high-resolution image transmitted by a super-resolution parameter from the edge computing server, and a feedback input to the user device and evaluated Mobile edge computing-based super-resolution streaming video transmission system, characterized in that the above.
12. The method of claim 11,
The transformation of the super-resolution model part is,
A mobile edge computing-based super-resolution streaming video transmission system, characterized in that by adding more nodes and layers to increase the learnable super-resolution parameters.
The method of claim 1,
The edge computing server,
Perform super-resolution parameter learning in learning mode or test mode,
The learning mode is updated by activating the gradient of the learnable super-resolution parameter, and the test mode is mobile edge computing-based super-resolution, characterized in that the gradient of the learnable super-resolution parameter is deactivated. Streaming video transmission system.
The method of claim 1,
A number of edge computing servers tied to the block chain constitute a community mobile edge computing set as a committee node and a local node,
The edge computing server, which is a committee node, is formed to select an edge computing server that performs super-resolution parameter learning, issues coins to the edge computing server that performs learning, and records a ledger for coin issuance,
A mobile edge computing based super-resolution streaming video transmission system, characterized in that the edge computing server, which is a local node, is formed to support computing power according to the instructions of the edge computing server, which is the committee node, and to receive coins in return.

Images