KR102270818B1 - Super-Resolution Streaming Video Delivery System Based-on Mobile Edge Computing - Google Patents

Abstract

The present invention relates to an image transmission system for upgrading and providing a low-resolution streaming image to a high-resolution streaming image. In more detail, using an artificial intelligence-based super-resolution (SR) model in a mobile edge computing server, a super-revolution parameter which can enable high-resolution upgrade streaming for a predicted image likely to be viewed by a user is calculated in advance. By providing the calculated super-revolution parameter when viewing the corresponding image, the low-resolution streaming image can be upgraded to a high-resolution streaming image for viewing in the super-resolution streaming image transmission system based on mobile edge computing.

Description

Super-Resolution Streaming Video Delivery System Based-on Mobile Edge Computing

The present invention relates to a video transmission system that upgrades a low-resolution streaming video to a high-resolution streaming video and provides it, and more particularly, by using an artificial intelligence-based super-resolution (SR) model in a mobile edge computing server, the possibility of user viewing Mobile edge computing-based super that calculates in advance the super-revolution parameter that can enable high-resolution upgrade streaming for the predicted video, and provides a high-resolution streaming video by applying the calculated super-revolution parameter when viewing the video - It relates to a resolution streaming video transmission system.

As of June 2019, released by the Ministry of Science and ICT, the number of domestic smartphone lines reached about 50.4 million, and the smartphone use rate of Korean adults was 93%. According to Pew Research, an American public opinion organization, the smartphone ownership rate in Korea is 95%, which is the highest in the 27 countries surveyed, and the use of smartphones in Korea is almost essential.

Among them, smartphones are widely used for online video viewing, and according to a recent survey, Netflix is experiencing a rapid increase in viewership by more than double compared to 2019, with YouTube being used for a total of 31.7 billion minutes per month. The use of smartphones for online video viewing is steadily increasing.

On the other hand, when watching the online video as described above, 140p, 240p, 360p, ... , 1024p, etc., are provided in various resolutions from low to high resolution. Of these, the more high-resolution video is provided, the more encoding and traffic costs are incurred from the content provider providing the service. Since preparations such as the need to prepare the video format for each terminal and browser must be made, it ultimately leads to a user's charge burden.

Accordingly, the super-resolution technology was developed, and the super-resolution technology is made to learn a weight that can minimize the difference between the original high-resolution image and the low-resolution image, so that the high-resolution image is viewed despite viewing the low-resolution image. It showed the advantage of being able to watch by minimizing the difference between the

However, if the super-resolution technology is applied only to minimize the difference between viewing images as described above, real-time transmission of images is impossible due to learning time, and the difference between high-resolution and low-resolution images for one image to be viewed Due to the characteristic of simply minimizing , there is a problem that an error occurs when the same weight is applied to other images.

On the other hand, the prior art related to the resolution improvement of moving pictures is Korean Patent No. 10-2130074 'High-definition VOD providing system in poor network environment', Korean Patent No. 10-1961177 'Image processing method and apparatus using a neural network' , Korean Patent Application Laid-Open No. 10-2019-0119550 'Method and apparatus for improving image resolution'.

The present invention has been devised to solve the above problems, and an object of the present invention is to utilize an artificial intelligence-based super-resolution (SR) model in a mobile edge computing server to allow users to watch based on popular video information or cache information. Less data by predicting a possible image, learning the weight, etc. in advance, and providing a high-resolution image by applying the super-resolution parameter to the image when the user views the image in low resolution It is to provide a mobile edge computing-based super-resolution streaming video transmission system that allows users to view high-resolution videos in real time in both quantities.

A mobile edge computing-based super-resolution streaming video transmission system according to an embodiment of the present invention for solving the above problems stores a plurality of video content and provides the video content in real time in response to a video content playback request to provide a streaming service a cloud server that supports; An edge computing server and the edge computing server that form a long distance with the cloud server over a set standard and are linked with the cloud server, and provide a super-resolution image of a high resolution of a low-resolution image among the image contents provided from the cloud server; forms a short distance less than the set standard, and transmits a video content playback request to the cloud server, and receives the requested video content as a corresponding super-resolution video from the edge computing server. It may include a user device that plays it.

Here, the edge computing server may include: a learning data management unit for receiving the image content delivered from the cloud server in parallel as a low-resolution image and a high-resolution image, changing each of the super-resolution parameters into a format required for learning, and storing; a super-resolution model unit for generating super-resolution parameters of weight and bias capable of increasing the low resolution by learning to minimize the difference in resolution between the formatted low-resolution image and the high-resolution image from the learning data management unit; A super-resolution parameter storage unit for storing super-resolution parameters of weight and bias generated by learning from the super-resolution model unit, and a super-resolution parameter storage unit for weights and biases stored in the super-resolution parameter storage unit It may be configured to include a communication unit that transmits a super-resolution image obtained by converting a low-resolution image into a high-resolution using a resolution parameter to a user device.

In addition, the learning data management unit may be configured to delete the low-resolution image data and the high-resolution image data used for the learning when the learning is finished in the super-resolution model unit.

In addition, the edge computing server further includes a video content analysis unit for classifying the popular video content selected by the cloud server by category, and the super-resolution model unit is configured to provide image content classified by category in the video content analysis unit. It is possible to generate the super-resolution parameter by learning in advance.

In addition, the image content analysis unit may analyze the image cache information stored in the user device in addition to the popular image content selected from the cloud server, and classify the image content according to the user preference image information derived according to the analysis by category. have.

In addition, the categories are formed in a hierarchical structure, and the super-resolution model unit may perform learning to generate super-resolution parameters corresponding to all categories based on an image of the highest category among the categories of the hierarchical structure. .

In addition, the video content analysis unit, when a new category is added according to the change in the selection of popular video content of the cloud server, connects to a subcategory of the category determined to be closest to the selected popular video content, but when the closest category does not exist can be created as a new top-level category.

In addition, the hierarchical category consists of a graphic model in which each category is configured as a node and the super-resolution parameter is configured as a variable, so that the edge computing server When searching for a resolution parameter, it is possible to search for a super-resolution parameter by searching for an array through a node and reading only a variable of a corresponding category.

In addition, in the super-resolution streaming video transmission system, when there is no matching category for the video content requested to be reproduced from the user device and the network environment satisfies a set condition, the cloud server sends a high-resolution video to the user device Simultaneously with streaming, the video content analysis unit creates a category for the currently played video content, and the edge computing server performs super-resolution parameter generation, from the point in time when the super-resolution parameter exists, the edge computing server may be configured to receive a low-resolution image from the cloud server and transmit a high-resolution super-resolution image to the user device.

In addition, in the super-resolution streaming video transmission system, when there is no matching category for the video content requested to be reproduced from the user device and the network environment does not satisfy the set condition, the user device provides information about the requested streaming video. At the same time that the high-resolution image to which the super-resolution parameter of the higher-level category is applied is played, the image content analysis unit creates a category for the currently played image content, and generates a super-resolution parameter in the edge computing server, From a point in time when the super-resolution parameter exists, the edge computing server may be configured to receive a low-resolution image from the cloud server and transmit a high-resolution super-resolution image to the user device.

In addition, the edge computing server, after the video content reproduced by the user device is finished, proceeds with the feedback itself or is transmitted from the user device, and for video content that has received feedback below a set reference value, the super-resolution model It can be configured to transform wealth to increase learning precision.

Here, the feedback may include a comparative feedback comparing the high-resolution image provided by the cloud server with the high-resolution image provided by the edge computing server with respect to the reproduced image content, and feedback input and evaluated by the user device. There may be more than one.

In addition, the transformation of the super-resolution model unit may be configured to increase the learnable super-resolution parameter by adding more nodes and layers.

The mobile edge computing-based super-resolution streaming video transmission system according to an embodiment of the present invention has the effect of enabling a mobile streaming video viewer to enjoy a high-resolution video at a lower communication cost than before.

In addition, since the mobile edge computing-based super-resolution streaming video transmission system according to an embodiment of the present invention can reproduce the entire streaming time only with low-resolution images, there is an effect that high-quality images can be reproduced even in regions with poor network infrastructure.

In addition, since the mobile edge computing-based super-resolution streaming video transmission system according to an embodiment of the present invention collects data necessary for learning based on user preferences, various A business model may be applicable.

In addition, the mobile edge computing-based super-resolution streaming video transmission system according to an embodiment of the present invention may advance high-definition video streaming technology.

In addition, the above-mentioned effects according to the embodiments of the present invention are not limited to the described content, and may further include all effects predictable from the specification and drawings.

1 is a block diagram illustrating the configuration of a mobile edge computing-based super-resolution streaming video transmission system according to an embodiment of the present invention.
FIG. 2 is a block diagram of a detailed configuration of an edge computing server, which is a configuration of the super-resolution streaming video transmission system of FIG. 1 .
3 is a diagram illustrating a format process of low-resolution and high-resolution images required for super-resolution parameter learning of a mobile edge computing-based super-resolution streaming video transmission system according to an embodiment of the present invention.
4 is a diagram showing a flow of high-resolution a low-resolution image of a mobile edge computing-based super-resolution streaming image transmission system according to an embodiment of the present invention.
5 is a diagram illustrating a hierarchical category structure.
6 is a diagram showing an example of a flow of a super-resolution parameter generation process using cache information of a cloud server according to an embodiment of the present invention.
7 is a diagram illustrating an example of a flow of a super-resolution parameter generation process using cache information of a user device according to an embodiment of the present invention.
8 is a diagram showing an example of a method of playing a streaming video when there is no category according to an embodiment of the present invention.
9 is a diagram showing a flow of a method of reproducing a streaming video when the category of FIG. 8 does not exist.
10 is a flowchart illustrating a super-resolution re-learning process through feedback according to an embodiment of the present invention.
11 is a diagram illustrating a form of improving a super-resolution model by using feedback according to an embodiment of the present invention.

Hereinafter, the description of the present invention with reference to the drawings is not limited to specific embodiments, and various modifications may be made and various embodiments may be provided. In addition, it should be understood that the contents described below include all conversions, equivalents, and substitutes included in the spirit and scope of the present invention.

In the following description, terms such as first and second are terms used to describe various components, meanings are not limited thereto, and are used only for the purpose of distinguishing one component from other components.

Like reference numbers used throughout this specification refer to like elements.

As used herein, the singular expression includes the plural expression unless the context clearly dictates otherwise. In addition, terms such as "comprises", "comprising" or "have" described below are intended to designate the existence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification. It should be construed as not precluding the possibility of addition or existence of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

In addition, terms such as “…unit”, “…group”, and “…module” described in the specification mean a unit that processes at least one function or operation, which may be implemented by hardware or software or a combination of hardware and software. can

Hereinafter, a mobile edge computing-based super-resolution streaming video transmission system according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram of the configuration of a mobile edge computing-based super-resolution streaming video transmission system according to an embodiment of the present invention, and FIG. 2 is an edge computing server that is a configuration of the super-resolution streaming video transmission system of FIG. 1 It is a block diagram of a detailed configuration, and FIG. 3 is a diagram illustrating the format process of low-resolution and high-resolution images required for super-resolution parameter learning of a mobile edge computing-based super-resolution streaming video transmission system according to an embodiment of the present invention.

In addition, FIG. 4 is a diagram showing a flow of high-resolution a low-resolution video of a mobile edge computing-based super-resolution streaming video transmission system according to an embodiment of the present invention, and FIG. 5 is a diagram illustrating a hierarchical category structure, FIG. 6 is a diagram showing an example of a flow of a super-resolution parameter generation process using cache information of a cloud server according to an embodiment of the present invention.

In addition, FIG. 7 is a diagram showing an example of the flow of a super-resolution parameter generation process using cache information of a user device according to an embodiment of the present invention, and FIG. 8 is a case in which there is no category according to an embodiment of the present invention It is a diagram showing an example of a method of playing a streaming video, and FIG. 9 is a diagram showing a flow of a method of playing a streaming video when there is no category of FIG.

In addition, FIG. 10 is a flowchart showing a super-resolution re-learning process through feedback according to an embodiment of the present invention, and FIG. 11 is a flowchart showing a form of improving a super-resolution model using feedback according to an embodiment of the present invention It is a drawing.

First, referring to FIG. 1 , a mobile edge computing-based super-resolution streaming video transmission system according to an embodiment of the present invention is configured to include a cloud server 10 , an edge computing server 20 , and a user device 30 . can

Specifically, the cloud server 10 may store a plurality of image contents. In addition, the cloud server 10 may support the streaming service by providing the requested image content to the user device 30 in real time according to the image content reproduction request of the user device 30 . In this case, the video content may be supported by various resolutions, and the resolution of the supported resolution is determined according to the video content reproduction request of the user device 30 .

The edge computing server 20 may form a long-distance LT greater than or equal to a set standard with the cloud server 10 . Here, the set standard may be a standard according to a national unit or a distance unit.

For example, the national unit means the distance between countries or continents. If the national unit long distance means the cloud server 10 is located in the United States and the edge computing server 20 is located in Korea, it is a cloud server. (10) may be different from the country in which it is located.

In addition, the distance unit means a distance expressed in units such as m, km, and the like. If the standard is set to 10000 km, the distance unit distance is all distances belonging to 10000 km or more. That is, the edge computing server 20 located at a distance belonging to 10000 km or more from the cloud server 10 means that all of them belong to a long distance.

In the present invention, as a national unit for easy understanding, the description will be limited to a case where the cloud server 10 is located in the United States and the edge computing server 20 is located in Korea.

The edge computing server 20 is linked with the cloud server 10 in a state of forming a long-distance (LT) with the cloud server 10 as described above to achieve high resolution for low-resolution images among the image contents provided from the cloud server 10 . Thus, it can be provided to the user device 30 .

Here, the edge computing server 20 is preferably a mobile edge computing server, and may be an edge computing server provided inside the user device 30 . However, the present invention is not necessarily limited thereto, and the edge computing server 20 may be provided outside the user device 30 , on the other hand, by connecting to the user device 30 in a state provided externally through an external hard drive or USB. It may be in the form used.

The user device 30 is provided so that the edge computing server 20 is provided inside or interlocked from the outside as described above, and may form a short distance ST less than a set standard with the edge computing server 20 . Here, the short distance in the national unit basically means a distance within the same country, and in the case of a country forming a relatively large area, it may mean a regional distance within the country.

For example, if the edge computing server 20 is located in Korea, the user device 30 may also be in Korea. In addition, in the case of the edge computing server 20 belonging to the United States or China, in the case of the United States, if there is an edge computing server 20 in New York, the user device 30 is located in New York or Washington located in the corresponding local or adjacent local area In the case of China, if there is an edge computing server 20 in Hunan, the user device 30 may be in a local distance such as Hunan or adjacent Hubei.

On the other hand, as a distance unit, when the long distance is set to 10000 km or more, less than 10000 km may be a short distance, but more preferably, the distance between the edge computing server 20 and the user device 30 is the cloud server 10 and the edge It does not belong to the setting criteria between the computing servers 20, and the criteria may be set separately.

For example, the long distance is set to 10000 km or more, but the short distance standard may be separately set to less than 1000 km, which may be set by the administrator.

The short-distance will also be exemplified based on the national short-distance setting for better understanding, and accordingly, the edge computing server 20 and the user device 30 will be described by limiting them to being located in Korea.

The user device 30 is a terminal or device that a user carries and can watch a video, such as a mobile phone or a tablet PC, and transmits a video content playback request to the cloud server 10 by receiving a video content playback signal from the user. A streaming service for the requested image content may be supported from the server 10 .

In this case, the user device 30 may receive the image content to which the super-resolution parameter is applied from the edge computing server 20 , so that even if the user requests to view the low-resolution image, it can be viewed in a high-resolution image. Through this, image data such as traffic can be viewed with low communication cost by receiving a high-resolution service while using low-resolution image data, and it is possible to view the image even in an area where the network is poor.

On the other hand, as described above, the user device 30, which is provided with the service for upgrading the low-resolution video content to the high-resolution video content, is configured to receive the low-resolution video viewing request and deliver it to the cloud server 10 as well as receive the high-resolution video viewing request. It may also be configured to automatically switch to a low-resolution video viewing request and deliver it to the cloud server 10 .

However, this is not necessarily a limitation, and the configuration of automatic conversion into a request for viewing a low-resolution video as described above may depend on the setting of the user device 30 .

2 to 7 , the edge computing server 20 includes a learning data management unit 21 , a super-resolution model unit 22 , a super-resolution parameter storage unit 23 , and a communication unit 24 . can be

Specifically, the learning data management unit 21 may receive the image content delivered from the cloud server 10 in parallel as a low-resolution image and a high-resolution image. In this case, the classification of the low-resolution image and the high-resolution image depends on the setting. Also, the learning data management unit 21 may change and store the received low-resolution image and high-resolution image into a format required for super-resolution parameter learning, respectively.

Referring to FIG. 3 illustrating the format process required for super-resolution parameter learning, an example of the format boxed above in the algorithm is a format (Lorf) example of a low-resolution image, and an example of the format boxed below is a format of a high-resolution image (Hirf) This is an example. Among them, the <model background=”background/nature/mountain.h5”/> part may be the model name to be used for training.

The super-resolution model unit 22 may learn to minimize the difference in resolution between the low-resolution image and the high-resolution image formatted from the learning data management unit 21 . Through this, it is possible to generate super-resolution parameters of a weight and a bias that can make a low-resolution image high-resolution.

Here, the generated weights and biases are parameters for making a low-resolution image high-resolution, and when the image content requested to be reproduced is transmitted from the user device 30 through the communication unit 24 to be described later, the weights and biases are applied. It can be delivered in a high-resolution state.

The super-resolution parameter storage unit 23 may store the super-resolution parameters of weights and biases generated by learning from the super-resolution model unit 22 , and the communication unit 24 includes the super-resolution parameter storage unit 23 . By applying the super-resolution parameter stored in the user device 30 to the video content requested by the user device 30 , the high-resolution image may be transmitted to the user device 30 .

A process of providing a super-resolution streaming image, which is a streaming image with high resolution through the above-described configurations, to the user device 30 may be as shown in FIG. 3 .

On the other hand, the learning and super-resolution parameter generation of the edge computing server 20 as described above are performed before the user watches the video, so that the user can either view the video content in advance or at the same time so that the user device 30 is used in real time. It is desirable to transmit high-resolution images to A detailed description thereof will be provided later.

In addition, the learning data management unit 21 transmits in parallel from the cloud server 10 and uses the low-resolution image data for learning when the learning of the image content for which the super-resolution model unit 22 has been trained is finished. and high-resolution image data can be deleted to free up space and improve processing speed.

In addition, the edge computing server 20 may be configured to further include an image content analysis unit 25 that classifies the popular image content selected by the cloud server 10 by category.

Here, the popular video content may be selected on a daily, weekly or monthly basis, and may be selected based on the number of views or replays, etc., and the video content analysis unit 25 converts these daily, weekly or monthly popular video content into a drama. , comics, movies, etc. can be classified by category.

In this case, the categories may be classified so as to form a hierarchical structure having a branching structure from the upper category to the lower category.

The video content analysis unit 25 as described above is a configuration that prepares to make a low-resolution high-resolution image for an image that is likely to be viewed by a user, and classifies the popular video content selected through the image content analysis unit 25 by category, As shown in FIGS. 2 and 5 , the super-resolution model unit 22 may generate super-resolution parameters by learning in advance for the image content classified by category in the image content analysis unit 25, and the communication unit ( In 24), when a request to play popular video content with super-resolution parameters generated in advance from the user device 30 is transmitted to the cloud server 10, a low-resolution video is received from the cloud server 10 and the super-resolution parameters are set. By applying the high-resolution image can be delivered to the user device (30).

On the other hand, the pre-learned super-resolution parameter may be applied and stored before there is a request to play popular video content from the user device 30 , and the video content to which the super-resolution parameter has been applied in advance to the user device 30 can be directly accessed. can also be forwarded.

In addition, the video content analysis unit 25 predicts the video that the user is likely to watch from the user device 30 using the video cache information stored in the user device 30, not based on the popular video content as described above, and super - The resolution model unit 22 may be configured to learn the super-resolution parameters in advance.

Here, the image cache information is a record of the image content viewed by the user, and the image content analysis unit 25 may be configured to analyze the category, title, viewing time, etc. of the recorded image content, and user preferences related to the user. Image information can be derived.

In this way, when the user preference image information is derived, the image content analysis unit 25 classifies all the corresponding images by category, and viewing predicted by the user preference image information, such as applying the super-resolution parameter to the above-mentioned popular image content. A super-resolution parameter may be generated in advance for possible video content and applied to the user device 30 before or when a video content playback request is made.

By predicting the image to be viewed by the user in advance through the image content analysis unit 25 as described above, and preparing the high-resolution image for the high-viewability image, the high-resolution image can be reproduced more quickly.

Meanwhile, the super-resolution model unit 22 performs learning to generate super-resolution parameters corresponding to all categories based on the highest category image among the categories formed in the hierarchical structure classified by the image content analysis unit 25 . can do.

For example, assuming that upper categories such as Korean films, American films, Chinese films, and Japanese films are formed by country based on the top category of movies, Korean films, American films, Japanese films, as well as all subcategories branching from Korean films And, all of the subcategories branching from the American movie and all the subcategories branching from the Japanese movie are trained to generate super-resolution parameters along the categories.

At this time, when the image content analysis unit 25 detects a change in the selection of popular image content of the cloud server 10 or a change in image cache information of the user device 30, when there is no category of the corresponding popular image content or image cache information It can be added as a new category.

Here, when a new category is added, it can be connected to a sub-category of a category determined to be closest to the added popular video content or video cache information, and when the closest category does not exist, it can be configured to create a new top-level category. .

For example, when a category called German movies is added, it is added and linked to the parent category of movies by country, but when a category called romantic comedy movies is added, if a category related to romantic comedy movies does not exist, it will be displayed next to movies by country. You can create separate categories for romantic comedy movies.

In this case, categories can be divided by natural language processing, and if the categories are the same, they can be overwritten and reset.

In the category of such a hierarchical structure, each category is composed of nodes, and super-resolution parameters according to the corresponding categories are composed of graphic models composed of variables, so that efficient data management can be performed, The user device 30 may be formed to receive faster high-resolution image support.

More specifically, the edge computing server 20 may search for a super-resolution parameter for high-resolution image content supported by the cloud server 10, and at this time, an Array through the nodeized category. ), and by reading only the category variable (Variable) to search the super-resolution parameter, the low-resolution image can be made higher-resolution faster, and the high-resolution image can be delivered to the user device 30 . have.

On the other hand, when the video content requested to be reproduced from the user device 30 is video content that is not predicted in advance by the video content analysis unit 25 and does not have a category having a super-resolution parameter, the network environment satisfies the set condition In some cases, it may be configured to support high-resolution images.

Here, the case in which the network environment satisfies the set condition may be a case in which the network environment is good, and the case in which the network environment does not satisfy the set condition may be a case in which the network environment is poor.

Specifically, if there is no matching category for the video content requested to be reproduced from the user device 30 and the network environment satisfies the set condition, the cloud server 10 converts the original high-resolution image to the user device 30 once. At the same time supporting streaming, the video content analysis unit 25 generates a category for the currently played video content, and the edge computing server 20 generates a super-resolution parameter for the corresponding category, and super-resolution From the point in time when the parameter exists, the edge computing server 20 may receive a low-resolution image from the cloud server 10 and transmit a high-resolution super-resolution image to the user device 30 .

Through this, as shown in FIGS. 8 and 9 , the user device 30 displays the original high-resolution image at first, and displays the high-resolution image from the point in time when the super-resolution parameter exists in the middle of viewing the image to display a high-resolution image without interruption. You can play video.

Here, the original high-resolution image that is once transmitted from the cloud server 10 to the user device 30 may be supported as a DASH streaming image that automatically adjusts the resolution size according to network conditions.

Conversely, if there is no matching category for the video content requested to be reproduced from the user device 30 , and the network environment does not satisfy the set condition, the edge computing server 20 once the streaming requested from the user device 30 . The super-resolution parameter of the upper-level category for the video is provided to the user device 30 so that the low-resolution video is reproduced in high-resolution, while the video content analysis unit 25 creates a category for the video content currently being played. In addition, the edge computing server 20 may transmit a high-resolution image to the user device 30 by generating and applying a super-resolution parameter.

That is, as in the above case, a low-resolution image is transmitted from the cloud server 10 from the point in time when the super-resolution parameter exists, and the edge computing server 20 converts it to a high resolution and transmits it to the user device 30 .

Through this, the present invention provides a high-resolution image to which the super-resolution parameter is applied flexibly according to the network situation even in a state where the user's viewing image is not predicted, so that the high-resolution upgrade service can be supported so that the maximum effect can be seen. .

In addition, after the video content reproduced by the user device 30 is finished, the edge computing server 20 may itself provide feedback or receive it from the user device 30 to provide better super-resolution streaming. You can upgrade.

Specifically, the feedback is comparative feedback comparing the original high-resolution image provided by the cloud server 10 with the super-resolution parameter provided by the edge computing server 20 for the reproduced image content and the high-resolution image is applied, It may be one of the feedbacks input to the user device 30 and evaluated, or a composite feedback thereof.

At this time, the feedback input and evaluated from the user device 30 is not limited, but may be a score or graded feedback such as 1 to 10 points or A to F grade as an example, or may be feedback written in writing. In the written feedback, positive and negative words can be recognized and the feedback can be reflected.

The edge computing server 20 determines whether or not the feedback is obtained below the set reference value when the feedback is obtained as described above. For video content for which the feedback is obtained below the set reference value, the super-resolution model unit 22 increases the learning precision. can be converted to

That is, in the case of comparison feedback comparing the original high-resolution image provided by the cloud server 10 with the high-resolution image transmitted from the edge computing server 20 with the super-resolution parameter for the reproduced image content, FIG. As shown, by selecting a representative image for each section, the application of the super-resolution parameter is judged as good and bad, and the super-resolution (SR) application part determined as bad is re-learning. can be done to do so.

Here, the transformation to increase the learning precision is by adding more nodes and layers, and the learnable super-resolution parameter can be increased, and the reliability and high-resolution accuracy can be improved while showing the flow as shown in FIG. 11 . It can be improved and improved.

Although the embodiments of the present invention have been described above with reference to the accompanying drawings, those of ordinary skill in the art to which the present invention pertains can practice the present invention in other specific forms without changing the technical spirit or essential features of the present invention. you will be able to understand Accordingly, the embodiments described above are illustrative in all respects and not restrictive.

10: Cloud Server
20: Edge Computing Server
21: learning data management unit
22: super-resolution model part
23: super-resolution parameter storage unit
24: communication department
25: video content analysis unit
30: user device

Claims (13)

a cloud server that stores a plurality of video contents and provides the corresponding video contents in real time in response to a request to play the video contents to support a streaming service;
Edge computing that forms a long distance with the cloud server more than a set standard and interworks with the cloud server, and provides a super-resolution image obtained by applying a super-resolution parameter to a low-resolution image among the image contents provided from the cloud server. server and
A user device that forms a short distance less than a set standard with the edge computing server, transmits a video content playback request to the cloud server, and receives the requested video content as a corresponding super-resolution video from the edge computing server and plays it ,
The edge computing server,
a learning data management unit for receiving the image content delivered from the cloud server in parallel as a low-resolution image and a high-resolution image, changing each of the super-resolution parameters into a format required for learning, and storing them;
a super-resolution model unit for generating super-resolution parameters of weight and bias capable of increasing the low resolution by learning to minimize the difference in resolution between the formatted low-resolution image and the high-resolution image from the learning data management unit;
a super-resolution parameter storage unit for storing super-resolution parameters of weight and bias generated by learning from the super-resolution model unit; and
A communication unit for transmitting a super-resolution image obtained by converting a low-resolution image into a high resolution by using the super-resolution parameters of weights and biases stored in the super-resolution parameter storage unit to a user device,
Further comprising a video content analysis unit for classifying the popular video content selected in the cloud server by category,
The super-resolution model unit,
The video content analysis unit generates the super-resolution parameter by learning in advance for the video content classified by category,
The categories are formed in a hierarchical structure,
The super-resolution model unit,
learning to generate super-resolution parameters corresponding to all categories based on the highest category image among the categories of the hierarchical structure;
The video content analysis unit,
When a new category is added according to the change in the selection of popular video content in the cloud server, it is linked to a subcategory of the category judged to be closest to the selected popular video content, but if the closest category does not exist, creating a new top category A mobile edge computing-based super-resolution streaming video transmission system featuring.
delete The method of claim 1,
The learning data management unit,
Mobile edge computing-based super-resolution streaming video transmission system, characterized in that when learning is finished in the super-resolution model unit, the low-resolution image data and high-resolution image data used for the learning are deleted.
delete The method of claim 1,
The video content analysis unit,
Mobile edge computing-based super, characterized in that by analyzing the image cache information stored in the user device in addition to the popular image content selected from the cloud server, and classifying the image content according to the user preference image information derived according to the analysis by category -Resolution streaming video transmission system.
delete delete a cloud server that stores a plurality of video contents and provides the corresponding video contents in real time in response to a request to play the video contents to support a streaming service;
Edge computing that forms a long distance with the cloud server more than a set standard and interworks with the cloud server, and provides a super-resolution image obtained by applying a super-resolution parameter to a low-resolution image among the image contents provided from the cloud server. server and
A user device that forms a short distance less than a set standard with the edge computing server, transmits a video content playback request to the cloud server, and receives the requested video content as a corresponding super-resolution video from the edge computing server and plays it ,
The edge computing server,
a learning data management unit for receiving the image content delivered from the cloud server in parallel as a low-resolution image and a high-resolution image, changing each of the super-resolution parameters into a format required for learning, and storing them;
a super-resolution model unit for generating super-resolution parameters of weight and bias capable of increasing the low resolution by learning to minimize the difference in resolution between the formatted low-resolution image and the high-resolution image from the learning data management unit;
a super-resolution parameter storage unit for storing super-resolution parameters of weight and bias generated by learning from the super-resolution model unit; and
A communication unit for transmitting a super-resolution image obtained by converting a low-resolution image into a high resolution by using the super-resolution parameters of weights and biases stored in the super-resolution parameter storage unit to a user device,
Further comprising a video content analysis unit for classifying the popular video content selected in the cloud server by category,
The super-resolution model unit,
The video content analysis unit generates the super-resolution parameter by learning in advance for the video content classified by category,
The categories are formed in a hierarchical structure,
The super-resolution model unit,
learning to generate super-resolution parameters corresponding to all categories based on the highest category image among the categories of the hierarchical structure;
The categories of the hierarchical structure are,
Each category is composed of a node and the super-resolution parameter is composed of a graphic model composed of variables,
The edge computing server,
When searching for super-resolution parameters for video content, it is characterized in that the super-resolution parameters are searched by searching for an array through a node and reading only the variables of the corresponding category. Super-resolution streaming video transmission system based on mobile edge computing.
a cloud server that stores a plurality of video contents and provides the corresponding video contents in real time in response to a request to play the video contents to support a streaming service;
Edge computing that forms a long distance with the cloud server more than a set standard and interworks with the cloud server, and provides a super-resolution image obtained by applying a super-resolution parameter to a low-resolution image among the image contents provided from the cloud server. server and
A user device that forms a short distance less than a set standard with the edge computing server, transmits a video content playback request to the cloud server, and receives the requested video content as a corresponding super-resolution video from the edge computing server and plays it ,
The edge computing server,
a learning data management unit for receiving the image content delivered from the cloud server in parallel as a low-resolution image and a high-resolution image, changing each of the super-resolution parameters into a format required for learning, and storing them;
a super-resolution model unit for generating super-resolution parameters of weight and bias capable of increasing the low resolution by learning to minimize the difference in resolution between the formatted low-resolution image and the high-resolution image from the learning data management unit;
a super-resolution parameter storage unit for storing super-resolution parameters of weight and bias generated by learning from the super-resolution model unit; and
A communication unit for transmitting a super-resolution image obtained by converting a low-resolution image into a high resolution by using the super-resolution parameters of weights and biases stored in the super-resolution parameter storage unit to a user device,
Further comprising a video content analysis unit for classifying the popular video content selected in the cloud server by category,
The super-resolution model unit,
The video content analysis unit generates the super-resolution parameter by learning in advance for the video content classified by category,
When there is no matching category for the video content requested to be reproduced from the user device, and the network environment satisfies the set condition,
The cloud server streams a high-resolution image to the user device, at the same time creating a category for the image content currently being played in the image content analysis unit, and generating a super-resolution parameter in the edge computing server
From the point in time when the super-resolution parameter exists, the edge computing server receives the low-resolution image from the cloud server and transmits the high-resolution super-resolution image to the user device,
When there is no matching category for the video content requested to be reproduced from the user device, and the network environment does not satisfy the set condition,
In the user device, a high-resolution image to which a super-resolution parameter of a higher-level category for the requested streaming image is applied is played,
The video content analysis unit generates a category for the currently played video content, and performs super-resolution parameter generation in the edge computing server,
Mobile edge computing-based super-resolution streaming video, characterized in that the edge computing server receives the low-resolution video from the cloud server from the point in time when the super-resolution parameter exists and transmits the high-resolution super-resolution video to the user device. transmission system.
delete delete a cloud server that stores a plurality of video contents and provides the corresponding video contents in real time in response to a request to play the video contents to support a streaming service;
Edge computing that forms a long distance with the cloud server more than a set standard and interworks with the cloud server, and provides a super-resolution image obtained by applying a super-resolution parameter to a low-resolution image among the image contents provided from the cloud server. server and
A user device that forms a short distance less than a set standard with the edge computing server, transmits a video content playback request to the cloud server, and receives the requested video content as a corresponding super-resolution video from the edge computing server and plays it ,
The edge computing server,
a learning data management unit for receiving the image content delivered from the cloud server in parallel as a low-resolution image and a high-resolution image, changing each of the super-resolution parameters into a format required for learning, and storing them;
a super-resolution model unit for generating super-resolution parameters of weight and bias capable of increasing the low resolution by learning to minimize the difference in resolution between the formatted low-resolution image and the high-resolution image from the learning data management unit;
a super-resolution parameter storage unit for storing super-resolution parameters of weight and bias generated by learning from the super-resolution model unit; and
A communication unit for transmitting a super-resolution image obtained by converting a low-resolution image into a high resolution by using the super-resolution parameters of weights and biases stored in the super-resolution parameter storage unit to a user device,
After the video content reproduced by the user device is finished, the feedback is performed by itself or is transmitted from the user device,
For video content for which feedback is obtained below a set reference value, the super-resolution model unit is converted to increase learning precision,
The feedback is
It is characterized in that at least one of a comparison feedback comparing a high-resolution image provided by a cloud server with a high-resolution image provided by the edge computing server with respect to the reproduced image content, and a feedback input to the user device and evaluated Super-resolution streaming video transmission system based on mobile edge computing.
a cloud server that stores a plurality of video contents and provides the corresponding video contents in real time in response to a request to play the video contents to support a streaming service;
Edge computing that forms a long distance with the cloud server more than a set standard and interworks with the cloud server, and provides a super-resolution image obtained by applying a super-resolution parameter to a low-resolution image among the image contents provided from the cloud server. server and
A user device that forms a short distance less than a set standard with the edge computing server, transmits a video content playback request to the cloud server, and receives the requested video content as a corresponding super-resolution video from the edge computing server and plays it ,
The edge computing server,
a learning data management unit for receiving the image content delivered from the cloud server in parallel as a low-resolution image and a high-resolution image, changing each of the super-resolution parameters into a format required for learning, and storing them;
a super-resolution model unit for generating super-resolution parameters of weight and bias capable of increasing the low resolution by learning to minimize the difference in resolution between the formatted low-resolution image and the high-resolution image from the learning data management unit;
a super-resolution parameter storage unit for storing super-resolution parameters of weight and bias generated by learning from the super-resolution model unit; and
A communication unit for transmitting a super-resolution image obtained by converting a low-resolution image into a high resolution by using the super-resolution parameters of weights and biases stored in the super-resolution parameter storage unit to a user device,
After the video content reproduced by the user device is finished, the feedback is performed by itself or is transmitted from the user device,
For video content for which feedback is obtained below a set reference value, the super-resolution model unit is converted to increase learning precision,
The transformation of the super-resolution model part is,
Mobile edge computing-based super-resolution streaming video transmission system, characterized in that it is configured to increase the learnable super-resolution parameter by adding more nodes and layers.

Images