KR20210056733A - Electronic device for providing streaming data and method for operating thereof - Google Patents

Abstract

According to various embodiments, the electronic apparatus comprises: a display; a communication module; a processor operatively coupled to the display and the communication module; and a memory operatively coupled to the processor. The memory can store instructions for the processor, upon execution, to obtain the first streaming data corresponding to the first resolution of the original streaming data for the first domain from the external electronic apparatus through the communication module, select a first deep learning model corresponding to the first resolution of the original streaming data from among a plurality of deep learning models corresponding to a plurality of resolutions of the original streaming data, convert the first streaming data to the first restored streaming data using the first deep learning model, and display the first restored streaming data through the display. In addition, various embodiments are possible. Accordingly, the present invention is to provide streaming data similar to the original streaming data with ultra-high resolution.

Description

Electronic devices that provide streaming data and how they operate {ELECTRONIC DEVICE FOR PROVIDING STREAMING DATA AND METHOD FOR OPERATING THEREOF}

Various embodiments relate to a method and apparatus for selecting a deep learning model for restoring streaming data by an electronic device displaying multimedia streaming data.

Today, with the development of networks and popularization of mobile terminals, users who use multimedia streaming services through desktops, tablet PCs, and smart phones are increasing. Accordingly, service providers providing multimedia streaming services have placed importance on improving service quality and user experience quality in order to stably provide multimedia according to the needs of consumers.

On the other hand, deep learning super resolution technology is a neural network model that plays the role of changing the picture quality of low resolution to high resolution. Currently, the level of technology is quite high enough that TVs are commercialized, and the technology is being developed to a level similar to the original even images that cannot be fully recognized due to the recent grafting with GAN network.

In order to reproduce high-resolution streaming data in real time in an electronic device, the performance of a specific encoding technique for encoding streaming data was important. Recently, in order to reproduce streaming data of ultra high resolution beyond high resolution in real time, there is a problem in that streaming data is not sufficiently restored to raw data of ultra high resolution by simply using a specific encoding technique or a specific algorithm.

Various embodiments may adaptively select a deep learning model corresponding to a plurality of resolutions of the original streaming data for a plurality of domains to restore low-resolution streaming data acquired from an external electronic device as much as the original streaming data. .

According to various embodiments, the electronic device includes a display, a communication module, a processor operatively connected to the display and the communication module, and a memory operatively connected to the processor, and the memory, when executed, the The processor, through the communication module, obtains first streaming data corresponding to a first resolution of the original streaming data for a first domain from an external electronic device, and a plurality of Selecting a first deep learning model corresponding to the first resolution of the original streaming data from among deep learning models, and converting the first streaming data into first reconstructed streaming data using the first deep learning model, And instructions for displaying the first reconstructed streaming data through the display.

According to various embodiments, a method of providing streaming data includes an operation of acquiring first streaming data corresponding to a first resolution of the original streaming data for a first domain from an external electronic device through a communication module, the original Selecting a first deep learning model corresponding to the first resolution of the original streaming data from among a plurality of deep learning models corresponding to a plurality of resolutions of streaming data, the first deep learning model using the first deep learning model It may include an operation of converting the first streaming data into first reconstructed streaming data, and an operation of displaying the first reconstructed streaming data through a display.

According to various embodiments, the streaming service providing server includes a communication module, a processor operatively connected to the communication module, and a memory operatively connected to the processor, and the memory is, when executed, the processor And transmitting a plurality of streaming data corresponding to a plurality of resolutions of the original streaming data for a specific domain through the communication module to a plurality of electronic devices, and the plurality of resolutions of the original streaming data and the original streaming data Train a plurality of deep learning models corresponding to the plurality of resolutions of the original streaming data using the plurality of streaming data corresponding to, and included in the plurality of electronic devices through the communication module. The image difference degree for the specific domain or the resolution of the original streaming data is obtained from the electronic device, and the plurality of deep learning models corresponding to the plurality of resolutions of the original streaming data are obtained through the communication module. A deep learning model corresponding to the image difference degree for a specific domain or the resolution of the original streaming data may be transmitted to the electronic device.

According to various embodiments, the electronic device adaptively checks a deep learning model corresponding to the resolution of the original streaming data from among a plurality of deep learning models trained for a plurality of resolutions, and determines the confirmed deep learning model. Streaming data similar to the original streaming data of ultra-high resolution can be provided by restoring the streaming data by using.

1 is a block diagram of an electronic device in a network environment, according to various embodiments.
2 is a flowchart illustrating a method of displaying, by an electronic device, restored streaming data using a deep learning model, according to various embodiments.
3 is a diagram illustrating an embodiment in which an electronic device restores streaming data using a deep learning model and transmits the restored streaming data to an external electronic device, according to various embodiments.
4A is a diagram illustrating an embodiment in which an external electronic device transmits a plurality of deep learning models corresponding to a plurality of resolutions of original streaming data to a plurality of electronic devices, according to various embodiments.
FIG. 4B is a diagram for describing an embodiment in which an external electronic device checks an image difference degree for a deep learning model using reconstructed streaming data, according to various embodiments.
5 is a diagram for describing an embodiment in which an electronic device selects a specific sub deep learning model corresponding to a specific resolution of original streaming data for a specific sub domain, according to various embodiments.
6 is a flowchart illustrating a method of selecting, by an electronic device, a specific deep learning model based on a user input, and displaying restored streaming data using the selected specific deep learning model, according to various embodiments.
7A is a diagram illustrating a first embodiment in which an electronic device selects a specific deep learning model based on a user input for an image difference diagram, according to various embodiments.
7B is a diagram illustrating a second embodiment in which an electronic device selects a specific deep learning model based on a user input for an image difference diagram, according to various embodiments.
8 is, according to various embodiments, an electronic device selects a specific deep learning model based on a compression rate and an image difference corresponding to a plurality of resolutions, and displays the restored streaming data using the selected specific deep learning model. This is a flow chart explaining how to do it.
FIG. 9 is a diagram for describing a first embodiment in which an electronic device selects a specific deep learning model corresponding to a specific resolution based on a compression rate corresponding to a plurality of resolutions and an image difference degree according to various embodiments. .
FIG. 10 is a diagram illustrating a second embodiment in which an electronic device selects a specific deep learning model corresponding to a specific resolution based on a compression rate corresponding to a plurality of resolutions and an image difference degree according to various embodiments. .
11 is a diagram for describing an embodiment in which an external electronic device trains a specific deep learning model corresponding to a specific resolution range, according to various embodiments.
12 is a diagram for describing an embodiment in which an external electronic device trains a specific deep learning model using patched streaming data, according to various embodiments.
13 is a diagram for describing an embodiment in which an external electronic device trains a plurality of deep learning models corresponding to a plurality of resolutions of original streaming data for a plurality of domains, according to various embodiments.
14 is a flowchart illustrating a method of transmitting, by an external electronic device, a deep learning model corresponding to an image difference degree for a specific domain or a resolution of original streaming data for a specific domain to the electronic device, according to various embodiments .

1 is a block diagram of an electronic device 101 in a network environment 100 according to various embodiments. Referring to FIG. 1, in a network environment 100, the electronic device 101 communicates with the electronic device 102 through a first network 198 (for example, a short-range wireless communication network), or a second network 199 It is possible to communicate with the electronic device 104 or the server 108 through (eg, a long-distance wireless communication network). According to an embodiment, the electronic device 101 may communicate with the electronic device 104 through the server 108. According to an embodiment, the electronic device 101 includes a processor 120, a memory 130, an input device 150, an audio output device 155, a display device 160, an audio module 170, and a sensor module ( 176, interface 177, haptic module 179, camera module 180, power management module 188, battery 189, communication module 190, subscriber identification module 196, or antenna module 197 ) Can be included. In some embodiments, at least one of these components (eg, the display device 160 or the camera module 180) may be omitted or one or more other components may be added to the electronic device 101. In some embodiments, some of these components may be implemented as one integrated circuit. For example, the sensor module 176 (eg, a fingerprint sensor, an iris sensor, or an illuminance sensor) may be implemented while being embedded in the display device 160 (eg, a display).

The processor 120, for example, executes software (eg, a program 140) to implement at least one other component (eg, a hardware or software component) of the electronic device 101 connected to the processor 120. It can be controlled and can perform various data processing or operations. According to an embodiment, as at least a part of data processing or operation, the processor 120 may transfer commands or data received from other components (eg, the sensor module 176 or the communication module 190) to the volatile memory 132. It is loaded into, processes commands or data stored in the volatile memory 132, and the result data may be stored in the nonvolatile memory 134. According to an embodiment, the processor 120 includes a main processor 121 (eg, a central processing unit or an application processor), and a secondary processor 123 (eg, a graphic processing unit, an image signal processor) that can be operated independently or together. , A sensor hub processor, or a communication processor). Additionally or alternatively, the coprocessor 123 may be set to use lower power than the main processor 121 or to be specialized for a designated function. The secondary processor 123 may be implemented separately from the main processor 121 or as a part thereof.

The co-processor 123 is, for example, in place of the main processor 121 while the main processor 121 is in an inactive (eg, sleep) state, or the main processor 121 is active (eg, executing an application). ) While in the state, together with the main processor 121, at least one of the components of the electronic device 101 (for example, the display device 160, the sensor module 176, or the communication module 190) It is possible to control at least some of the functions or states associated with it. According to an embodiment, the coprocessor 123 (eg, an image signal processor or a communication processor) may be implemented as a part of other functionally related components (eg, the camera module 180 or the communication module 190). have.

The memory 130 may store various data used by at least one component of the electronic device 101 (eg, the processor 120 or the sensor module 176). The data may include, for example, software (eg, the program 140) and input data or output data for commands related thereto. The memory 130 may include a volatile memory 132 or a nonvolatile memory 134.

The program 140 may be stored as software in the memory 130, and may include, for example, an operating system 142, middleware 144, or an application 146.

The input device 150 may receive a command or data to be used for a component of the electronic device 101 (eg, the processor 120) from outside (eg, a user) of the electronic device 101. The input device 150 may include, for example, a microphone, a mouse, a keyboard, or a digital pen (eg, a stylus pen).

The sound output device 155 may output an sound signal to the outside of the electronic device 101. The sound output device 155 may include, for example, a speaker or a receiver. The speaker can be used for general purposes such as multimedia playback or recording playback, and the receiver can be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part of the speaker.

The display device 160 may visually provide information to the outside of the electronic device 101 (eg, a user). The display device 160 may include, for example, a display, a hologram device, or a projector and a control circuit for controlling the device. According to an embodiment, the display device 160 may include a touch circuitry set to sense a touch, or a sensor circuit (eg, a pressure sensor) set to measure the strength of a force generated by the touch. have.

The audio module 170 may convert sound into an electrical signal, or conversely, may convert an electrical signal into sound. According to an embodiment, the audio module 170 acquires sound through the input device 150, the sound output device 155, or an external electronic device (eg: Sound may be output through the electronic device 102 (for example, a speaker or headphones).

The sensor module 176 detects an operating state (eg, power or temperature) of the electronic device 101, or an external environmental state (eg, a user state), and generates an electrical signal or data value corresponding to the detected state. can do. According to an embodiment, the sensor module 176 is, for example, a gesture sensor, a gyro sensor, an atmospheric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an infrared (IR) sensor, a biometric sensor, It may include a temperature sensor, a humidity sensor, or an illuminance sensor.

The interface 177 may support one or more designated protocols that may be used for the electronic device 101 to directly or wirelessly connect with an external electronic device (eg, the electronic device 102). According to an embodiment, the interface 177 may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

The connection terminal 178 may include a connector through which the electronic device 101 can be physically connected to an external electronic device (eg, the electronic device 102). According to an embodiment, the connection terminal 178 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (eg, a headphone connector).

The haptic module 179 may convert an electrical signal into a mechanical stimulus (eg, vibration or movement) or an electrical stimulus that a user can perceive through tactile or motor sensations. According to an embodiment, the haptic module 179 may include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

The camera module 180 may capture a still image and a video. According to an embodiment, the camera module 180 may include one or more lenses, image sensors, image signal processors, or flashes.

The power management module 188 may manage power supplied to the electronic device 101. According to an embodiment, the power management module 188 may be implemented as at least a part of, for example, a power management integrated circuit (PMIC).

The battery 189 may supply power to at least one component of the electronic device 101. According to an embodiment, the battery 189 may include, for example, a non-rechargeable primary cell, a rechargeable secondary cell, or a fuel cell.

The communication module 190 includes a direct (eg, wired) communication channel or a wireless communication channel between the electronic device 101 and an external electronic device (eg, the electronic device 102, the electronic device 104, or the server 108). It is possible to support establishment and communication through the established communication channel. The communication module 190 operates independently of the processor 120 (eg, an application processor) and may include one or more communication processors supporting direct (eg, wired) communication or wireless communication. According to an embodiment, the communication module 190 is a wireless communication module 192 (eg, a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 194 (eg : A local area network (LAN) communication module, or a power line communication module) may be included. Among these communication modules, a corresponding communication module is a first network 198 (e.g., a short-range communication network such as Bluetooth, WiFi direct or IrDA (infrared data association)) or a second network 199 (e.g., a cellular network, the Internet, Alternatively, it may communicate with an external electronic device through a computer network (for example, a telecommunication network such as a LAN or WAN). These various types of communication modules may be integrated into one component (eg, a single chip), or may be implemented as a plurality of separate components (eg, multiple chips). The wireless communication module 192 uses subscriber information stored in the subscriber identification module 196 (eg, International Mobile Subscriber Identifier (IMSI)) in a communication network such as the first network 198 or the second network 199. The electronic device 101 can be checked and authenticated.

The antenna module 197 may transmit a signal or power to the outside (eg, an external electronic device) or receive from the outside. According to an embodiment, the antenna module may include one antenna including a conductor formed on a substrate (eg, a PCB) or a radiator formed of a conductive pattern. According to an embodiment, the antenna module 197 may include a plurality of antennas. In this case, at least one antenna suitable for a communication method used in a communication network such as the first network 198 or the second network 199 is, for example, provided by the communication module 190 from the plurality of antennas. Can be chosen. The signal or power may be transmitted or received between the communication module 190 and an external electronic device through the at least one selected antenna. According to some embodiments, other components (eg, RFIC) other than the radiator may be additionally formed as part of the antenna module 197.

At least some of the components are connected to each other through a communication method (e.g., a bus, general purpose input and output (GPIO), serial peripheral interface (SPI), or mobile industry processor interface (MIPI))) between peripheral devices and a signal ( E.g. commands or data) can be exchanged with each other.

According to an embodiment, commands or data may be transmitted or received between the electronic device 101 and the external electronic device 104 through the server 108 connected to the second network 199. Each of the external electronic devices 102 and 104 may be a device of the same or different type as the electronic device 101. According to an embodiment, all or part of the operations executed by the electronic device 101 may be executed by one or more of the external electronic devices 102, 104, or 108. For example, when the electronic device 101 needs to perform a function or service automatically or in response to a request from a user or another device, the electronic device 101 In addition or in addition, it is possible to request one or more external electronic devices to perform the function or at least part of the service. One or more external electronic devices receiving the request may execute at least a part of the requested function or service, or an additional function or service related to the request, and transmit a result of the execution to the electronic device 101. The electronic device 101 may process the result as it is or additionally and provide it as at least part of a response to the request. For this purpose, for example, cloud computing, distributed computing, or client-server computing technology Can be used.

FIG. 2 is a flowchart illustrating a method of displaying, by an electronic device (eg, the electronic device 101 of FIG. 1 ), reconstructed streaming data using a deep learning model, according to various embodiments.

3 is a diagram in which the electronic device 101 restores streaming data using a deep learning model and transmits the restored streaming data to an external electronic device (eg, the server 108 of FIG. 1) according to various embodiments. It is a figure showing an Example.

In operation 201, according to various embodiments, the electronic device 101 (for example, the processor 120 of FIG. 1) is through a communication module (for example, the communication module 190 of FIG. 1), and the external electronic device 108 ) From the first streaming data (eg, the first streaming data 311 of FIG. 3) corresponding to the first resolution of the original streaming data (eg, the original streaming data 301 of FIG. 3) for the first domain can do. Streaming data described herein may mean a plurality of frames or a single frame, and may include media streaming data (eg, video streaming data).

According to various embodiments, the external electronic device 108 may be a server that provides a streaming service for a plurality of domains. According to various embodiments, a domain may include a specific application (eg, a game application), a specific function, or a specific service, and includes various contents capable of providing an image frame. can do. According to an embodiment, the external electronic device 108 may convert the original streaming data 301 for the first domain into the first streaming data 311. For example, referring to FIG. 3, in order to transmit the streaming data to the electronic device 101, the external electronic device 108 sets the original resolution (eg, 100%) of the original streaming data 301 to the first resolution ( Example: The first streaming data 311 resized to 80% of the original resolution) may be generated. The external electronic device 108 may transmit the generated first streaming data 311 to the electronic device 101. As another example, the external electronic device 108 resizes the original resolution (eg, 100%) of the original streaming data 301 to a second resolution (eg, 60% of the original resolution). ) Can be created. According to an embodiment, the external electronic device 108 uses the original streaming data 301 and specific streaming data corresponding to a specific resolution of the original streaming data 301 to correspond to a specific resolution of the original streaming data 301. You can train specific deep learning models. For example, referring to FIG. 3, the external electronic device 108 corresponds to the first streaming data 311 as training data and the original streaming data 301 as correct answer data to a first resolution of the original streaming data 301 The first deep learning model 310 may be trained by providing it to a first deep learning model (eg, the first deep learning model 310 of FIG. 3) (eg, a training model). For another example, referring to FIG. 3, the external electronic device 108 converts the second streaming data 412 as training data and the original streaming data 301 as correct answer data to a second resolution of the original streaming data 301. The second deep learning model 320 may be trained by providing it to a second deep learning model (eg, the second deep learning model 320 of FIG. 3) corresponding to. According to an embodiment, the external electronic device 108 uses data obtained by encoding specific streaming data corresponding to a specific resolution of the original streaming data 301 or data that has undergone additional image processing as training data, and uses the original streaming data 301 ), a specific deep learning model corresponding to a specific resolution can be trained. According to various embodiments, there is no limitation on streaming data that becomes training data, and various streaming data image-processed from the original streaming data 301 by the external electronic device 108 are applied to a specific deep learning model corresponding to a specific resolution. Can be trained against. According to an embodiment, a plurality of deep learning models include a deep neural network, a convolutional neural network, a recurrent neural network, a restricted Boltzmann machine, and a deep trust neural network. Deep belief network) or deep Q-network, and the above-described algorithm is only an example, and is not limited to the above-described example, and various algorithms applicable by those skilled in the art Can be trained based on

According to various embodiments, the external electronic device 108 may include a streaming service providing server. According to an embodiment, a streaming data transmission server for transmitting streaming data and a streaming data training server for training streaming data may be included in the external electronic device 108 that is a streaming service providing server as one structure, or each The structure of can be divided and exist as a separate structure.

In operation 203, according to various embodiments, the electronic device 101 (for example, the processor 120 of FIG. 1) is an original among a plurality of deep learning models corresponding to a plurality of resolutions of the original streaming data 301. The first deep learning model 310 corresponding to the first resolution of the streaming data 301 may be selected. According to an embodiment, the electronic device 101 may acquire a plurality of deep learning models corresponding to a plurality of resolutions of the original streaming data 301 from the external electronic device 108 through the communication module 190. . According to an embodiment, the electronic device 101 may store a plurality of acquired deep learning models in a memory (eg, the memory 130 of FIG. 1 ). For example, referring to FIG. 3, the electronic device 101 corresponds to a first deep learning model 310 corresponding to a first resolution of the original streaming data 301 and a second resolution of the original streaming data 301. The second deep learning model 320 may be stored in the memory 130. According to an embodiment, the electronic device 101 may delete at least one of a plurality of deep learning models from the memory 130 based on a user's request, an external electronic device 108's request, or a specified period. According to an embodiment, the electronic device 101 can confirm that the first streaming data 311 acquired from the external electronic device 108 corresponds to the first resolution of the original streaming data 301, and a plurality of deep learning Among the models, a first deep learning model 310 corresponding to the first resolution of the original streaming data 301 may be selected. For example, referring to FIG. 3, the electronic device 101 includes a first deep learning model 310 and a second deep learning model 320 corresponding to a first resolution of the original streaming data 301. A deep learning model 310 may be selected.

According to various embodiments, the electronic device 101 (eg, the processor 120 of FIG. 1) corresponds to the first resolution of the original streaming data 301 among a plurality of deep learning models based on a user input. The first deep learning model 310 may be selected. According to an embodiment, the user input may include a user input for selecting an image difference degree between the original streaming data 301 and specific streaming data (eg, the first reconstructed streaming data 321 in FIG. 3 ). . According to an embodiment, the electronic device 101 is configured to perform a first program corresponding to a specific resolution (eg, the lowest resolution) of the original streaming data 301 among at least one deep learning model that satisfies the image difference degree selected by the user. A deep learning model 310 may be selected. The degree of video difference between the original streaming data 301 and the specific streaming data is an error rate (e.g., MSE (mean square error)), PSNR (peak signal-to-noise ratio), or SSIM (structural similarity index). )). For example, the error rate may represent a difference between a pixel value of the original streaming data 301 and a pixel value of the specific streaming data. The above-described example regarding the degree of difference between the original streaming data 301 and the specific streaming data is only an embodiment, is not limited to the above example, and may include various criteria for confirming the degree of similarity between the images. According to an embodiment, the user input may include a user input for selecting the first resolution of the original streaming data 301. According to an embodiment, the electronic device 101 may select the first deep learning model 310 corresponding to the first resolution selected by the user. According to an embodiment, the electronic device 101 may obtain a user input for selecting a network latency, and check the resolution of the original streaming data 301 corresponding to the network delay selected by the user. , A deep learning model corresponding to the resolution of the verified original streaming data 301 may be selected. A specific operation of selecting the first deep learning model 310 based on a user input will be described later in the description of FIGS.

According to various embodiments, the electronic device 101 (for example, the processor 120 of FIG. 1) may also provide a compression rate corresponding to a plurality of resolutions of the original streaming data 301 and an image difference corresponding to the plurality of resolutions. Based on, from among a plurality of deep learning models, a first deep learning model 310 corresponding to a first resolution of the original streaming data 301 may be selected. The compression rate corresponding to the specific resolution may mean a value (A/B) obtained by dividing the size (A) of the streaming data corresponding to the specific resolution of the original streaming data by the size (B) of the original streaming data. According to an embodiment, the electronic device 101 satisfies a specified condition among values obtained by applying an image difference degree corresponding to each of the plurality of resolutions to a compression rate corresponding to each of the plurality of resolutions (eg, the lowest value). A first resolution derived from) can be checked, and a first deep learning model 310 corresponding to the checked first resolution can be selected. Here, the image difference degree corresponding to a specific resolution may mean an image difference degree for a specific deep learning model corresponding to the specific resolution. A specific operation of selecting the first deep learning model 310 based on a compression rate corresponding to a plurality of resolutions and an image difference degree will be described later in the description of FIGS. 8 to 10.

According to various embodiments, the electronic device 101 (for example, the processor 120 of FIG. 1) is based on the network bandwidth of the electronic device 101, the original streaming data among a plurality of deep learning models. The first deep learning model 310 corresponding to the first resolution of 301 may be selected. According to an embodiment, the electronic device 101 may measure a network bandwidth for receiving streaming data, and may transmit the measured network bandwidth to the external electronic device 108. In this case, the external electronic device 108 may select a first resolution of the original streaming data 301 corresponding to the received network bandwidth, and the first streaming data corresponding to the first resolution of the original streaming data 301 ( 311) may be transmitted to the electronic device 101. The electronic device 101 may check the first resolution of the original streaming data 301 selected based on the network bandwidth, and select a first deep learning model 310 corresponding to the checked first resolution.

According to various embodiments, the electronic device (eg, the processor 120 of FIG. 1) may check usage statistics of image difference degrees for a plurality of deep learning models for a specified period, and the most used image difference A first deep learning model 310 corresponding to may be selected. According to an embodiment, the external electronic device 108 may check the usage statistics of the image difference degree for the plurality of deep learning models for a period specified by the plurality of electronic devices, and the most used image difference degree A corresponding first deep learning model 310 may be selected. According to an embodiment, the external electronic device 108 may check usage statistics of image difference degrees for a plurality of deep learning models for a period specified by the plurality of electronic devices, and determine the most used image difference degrees. A specific deep learning model corresponding to an optimal resolution may be selected by applying the equations described in FIGS. 8 to 10 to be described later.

Using the method described in operation 203, you can check the image difference for each deep learning model,

The electronic device 101 (for example, the processor 120 of FIG. 1) is based on the network bandwidth of the electronic device 101, the first resolution of the original streaming data 301 among a plurality of deep learning models. A first deep learning model 310 corresponding to may be selected.

In operation 205, according to various embodiments, the electronic device 101 (eg, the processor 120 of FIG. 1) first restores the first streaming data 311 using the first deep learning model 310. It can be converted into streaming data 321. According to an embodiment, the electronic device 101 may apply the first streaming data 311 as input data to the first deep learning model 310, and upscale the first streaming data 311 as output data. Upscaling first restored streaming data 321 may be obtained. For example, referring to FIG. 3, the electronic device 101 applies the first streaming data 311 to the first deep learning model 310 (for example, an inference model), thereby first reconstruction streaming. Data 321 may be obtained.

According to various embodiments, the electronic device 101 (eg, the processor 120 of FIG. 1) transmits the first restored streaming data 321 to the external electronic device 108 through the communication module 190. I can. According to an embodiment, the external electronic device 108 uses the first reconstructed streaming data 321 obtained from the electronic device 101 to provide a first deep learning model corresponding to the first resolution of the original streaming data 301. An image difference degree for 310 may be calculated. A specific operation of calculating an image difference degree for the first deep learning model 310 using the first reconstructed streaming data 321 will be described later in the description of FIG. 4B.

In operation 207, according to various embodiments, the electronic device 101 (for example, the processor 120 of FIG. 1) may transmit the first restored streaming data (eg, the display device 160 of FIG. 1). 321)) can be displayed.

4A illustrates a plurality of deep learning models corresponding to a plurality of resolutions of the original streaming data (eg, the original streaming data 301 of FIG. 3) by the external electronic device 108 according to various embodiments. It is a diagram for explaining an embodiment of transmitting to devices.

According to various embodiments, the external electronic device 108 may transmit specific streaming data corresponding to a specific resolution of the original streaming data 301 for the first domain to a plurality of electronic devices. For example, the external electronic device 108 transmits first streaming data (eg, the first streaming data 311 of FIG. 3) corresponding to the first resolution of the original streaming data 301 to a first electronic device (eg: Transmission may be made to a plurality of first electronic devices including the electronic device 101 of FIG. 1. As another example, the external electronic device 108 stores the second streaming data 412 corresponding to the second resolution of the original streaming data 301 as a plurality of second electronic devices including the second electronic device 401. Can be sent to. As another example, the external electronic device 108 stores third streaming data (not shown) corresponding to the third resolution of the original streaming data 301 and includes a plurality of third electronic devices (not shown). It can be transmitted to electronic devices. The second electronic device 401 and the third electronic device (not shown) may include the same components as the first electronic device 101.

According to various embodiments, the external electronic device 108 corresponds to a specific resolution of the original streaming data 301 by using the original streaming data 301 and specific streaming data corresponding to a specific resolution of the original streaming data 301 You can train specific deep learning models. For example, the external electronic device 108 uses first streaming data 311 as training data and original streaming data 301 as correct answer data as a first deep learning model corresponding to a first resolution of the original streaming data 301. The first deep learning model 310 may be trained by providing it to (eg, the first deep learning model 310 of FIG. 3) (eg, a training model). The external electronic device 108 may train the second deep learning model (eg, the second deep learning model 320 of FIG. 3) and the third deep learning model 430 using the above-described method.

According to various embodiments, the external electronic device 108 may transmit a specific deep learning model corresponding to a specific resolution of the original streaming data 301 to at least one electronic device. For example, the external electronic device 108 may transmit the first deep learning model 310 corresponding to the first resolution of the original streaming data 301 to the first electronic device 101, and the original streaming data ( The second deep learning model 320 corresponding to the second resolution of the 301 may be transmitted to the second electronic device 401. According to an embodiment, the external electronic device 108 may transmit at least one deep learning model to at least one electronic device according to a specified period. According to an embodiment, the external electronic device 108 converts the requested specific deep learning model to the first electronic device 101 based on obtaining a request for a specific deep learning model from the first electronic device 101. I can send it. According to an embodiment, after obtaining a request for transmission of specific streaming data corresponding to a specific resolution from the first electronic device 101, the external electronic device 108 transmits the specific streaming data to the first electronic device 101. Before transmission, a specific deep learning model corresponding to a specific resolution may be transmitted to the first electronic device 101 in advance. According to an embodiment, the external electronic device 108 may transmit a specific deep learning model to the first electronic device 101 at a first time point while training a specific deep learning model, and after the first time point. At the second time point, an updated version of a specific deep learning model may be transmitted to the first electronic device 101.

According to various embodiments, a plurality of electronic devices may store a plurality of deep learning models acquired from the external electronic device 108 in a memory (eg, the memory 130 of FIG. 1 ). For example, the first electronic device 101 may store the first deep learning model 310 and the third deep learning model 430 in the memory 130, and the second electronic device 401 The learning model 310 and the second deep learning model 320 may be stored in the memory 130. According to an embodiment, the first electronic device 101 may update a specific deep learning model stored in the memory 130 by using an updated version of a specific deep learning model obtained from the external electronic device 108. I can.

FIG. 4B is a diagram for describing an embodiment in which an external electronic device (eg, the server 108 of FIG. 1) checks an image difference diagram for a deep learning model by using reconstructed streaming data, according to various embodiments. .

According to various embodiments, the external electronic device 108 includes first streaming data corresponding to a first resolution of the original streaming data (eg, the original streaming data 301 of FIG. 3) (eg, the first streaming data of FIG. 3). The data 311 may be transmitted to a first electronic device (eg, the electronic device 101 of FIG. 1 ), and second streaming data 412 corresponding to the second resolution of the original streaming data 301 is provided. 2 Can be transmitted to the electronic device 401. The second electronic device 401 may include the same components as the first electronic device 101.

According to various embodiments, the first electronic device 101 (eg, the processor 120 of FIG. 1) is a first deep learning model corresponding to the first resolution of the original streaming data 301 (eg, the processor 120 of FIG. 3 ). The first streaming data 311 may be converted into first reconstructed streaming data (eg, the first reconstructed streaming 321 of FIG. 3) using the first deep learning model 310. According to various embodiments, the second electronic device 401 second reconstructs the second streaming data 412 using the second deep learning model 320 corresponding to the second resolution of the original streaming data 301. It can be converted into streaming data 422.

According to various embodiments, the first electronic device 101 (eg, the processor 120 of FIG. 1) may transmit the first restored streaming data 321 to the external electronic device 108. According to an embodiment, the first electronic device 101 may transmit some frames of the first restored streaming data 321 to the external electronic device 108 at a specified period or in real time. According to an embodiment, the first electronic device 101 transmits the first restored streaming data 321 in response to receiving a request for the first restored streaming data 321 from the external electronic device 108 or a user It can be transmitted to the external electronic device 108. According to an embodiment, the second electronic device 401 may transmit the second reconstructed streaming data 422 to the external electronic device 108.

According to various embodiments, the external electronic device 108 is converted from each of the plurality of first electronic devices including the first electronic device 101 by using the first deep learning model 310 An image of the first deep learning model 310 may be obtained by obtaining the data 321 and using the original streaming data 301 and the first reconstructed streaming data 321 obtained from each of the plurality of first electronic devices You can check the difference. According to an embodiment, the external electronic device 108 may calculate an image difference degree for the first deep learning model 310 by using frames of the first reconstructed streaming data 321 acquired during a specified period. The external electronic device 108 may calculate an image difference degree for each frame of the original streaming data 301 and the first reconstructed streaming data 321, as an image difference degree for the first deep learning model 310, It is possible to calculate the average of the image difference of each frame acquired during a specified period. According to an embodiment, the external electronic device 108 is an image difference diagram with respect to the first deep learning model 310, and the frame of the original streaming data 301 and the first reconstructed streaming data 321 It is possible to calculate the image difference degree of the frame. According to various embodiments, the external electronic device 108 is a second restoration streaming converted from each of a plurality of second electronic devices including the second electronic device 401 using the second deep learning model 320. The data 422 can be obtained, and an image of the second deep learning model 320 using the original streaming data 301 and the second reconstructed streaming data 422 obtained from each of a plurality of second electronic devices You can check the difference. According to an embodiment, the external electronic device 108 may store an image difference degree for each deep learning model in a memory in the form of a mapping table.

5 is a diagram for describing an embodiment in which the electronic device 101 selects a specific sub deep learning model corresponding to a specific resolution of original streaming data for a specific sub domain, according to various embodiments.

According to various embodiments, the electronic device (eg, the electronic device 101 of FIG. 1) corresponds to the first resolution of the original streaming data for the first domain (eg, the original streaming data 301 of FIG. 3). The first streaming data (eg, the first streaming data 311 of FIG. 3) may be obtained from an external electronic device (eg, the server 108 of FIG. 1 ). According to various embodiments, the first domain may include a plurality of designated areas or a plurality of sub-domains corresponding to at least one of a plurality of designated periods. According to an embodiment, the first domain may include a plurality of sub-domains corresponding to a plurality of designated areas. For example, referring to <501>, a first domain (eg, a first game application) includes a first sub-domain 510a and a second area (eg, a general field area) corresponding to a first area (eg, a general field area). A second sub-domain 510b corresponding to the dungeon area A) may be included. According to an embodiment, the first domain may include a plurality of sub-domains corresponding to a plurality of designated periods. For example, referring to <502>, a first domain (eg, a first game application) is a first sub-domain 520a corresponding to a first period (eg, day) and a second period (eg, night). It may include a second sub-domain 520b corresponding to. According to an embodiment, the electronic device 101 may obtain the first streaming data 311 corresponding to the first resolution of the original streaming data 301 for a specific sub-domain from the external electronic device 108.

According to various embodiments, the electronic device (eg, the processor 120 of FIG. 1) is a first deep learning model corresponding to the first resolution of the original streaming data 301 for the first domain (eg, the processor 120 of FIG. 3 ). The first deep learning model 310) may be selected. According to an embodiment, the first deep learning model 310 may include a plurality of sub deep learning models corresponding to a plurality of sub domains. For example, referring to <501>, the first deep learning model 310 corresponds to the first sub-deep learning model 511 and the second sub-domain 510b corresponding to the first sub-domain 510a. A second sub deep learning model 512 may be included. For another example, referring to <502>, the first deep learning model 310 is in the first sub-deep learning model 511 and the second sub-domain 520b corresponding to the first sub-domain 520a. A corresponding second sub deep learning model 512 may be included. The first sub-deep learning model 511 corresponding to the first sub-domain 510a is a first sub-deep learning model corresponding to the first resolution of the original streaming data 301 for the first sub-domain 510a ( 511). According to an embodiment, the electronic device 101 obtains the first streaming data 311 corresponding to the first resolution of the original streaming data 301 for the first sub-domain 510a from the external electronic device 108 Based on this, the first sub deep learning model 511 corresponding to the first resolution of the original streaming data 301 for the first sub domain 510a may be selected.

6 illustrates a streaming restored by using an electronic device (for example, the electronic device 101 of FIG. 1) selects a specific deep learning model based on a user input, and reconstructs the selected specific deep learning model according to various embodiments. This is a flow chart for explaining how to display data.

7A is a diagram illustrating a first embodiment in which the electronic device 101 selects a specific deep learning model based on a user input for an image difference diagram, according to various embodiments.

7B is a diagram illustrating a second embodiment in which the electronic device 101 selects a specific deep learning model based on a user input for an image difference diagram, according to various embodiments.

In operation 601, according to various embodiments, the electronic device 101 (for example, the processor 120 of FIG. 1) is through a communication module (for example, the communication module 190 of FIG. : It is possible to obtain specific streaming data corresponding to a specific resolution of the original streaming data for a specific domain from the server 108 of FIG. 1.

According to various embodiments, the electronic device 101 (for example, the processor 120 of FIG. 1) transmits specific streaming data corresponding to a specific resolution of the original streaming data to the external electronic device 108 based on a user input. The request may be made, and in response to the request, specific streaming data corresponding to a specific resolution may be obtained from the external electronic device 108. According to an embodiment, the user input is a user input for selecting an image difference degree for a specific domain, a user input for selecting a network delay for a specific domain, or a user input for selecting a specific resolution of the original streaming data for a specific domain. It may include. According to an embodiment, the electronic device 101 receives the first streaming data corresponding to the first resolution of the original streaming data from the external electronic device 108, based on acquiring a user input, Second streaming data corresponding to the second resolution may be received from 108. For example, while receiving the first streaming data corresponding to the first resolution (eg, 80% of the original resolution) of the original streaming data from the external electronic device 108, the electronic device 101 2 Based on obtaining a user input for selecting a resolution (eg, 60% of the original resolution), the external electronic device 108 may request second streaming data corresponding to the second resolution, and in response to the request, Second streaming data corresponding to the second resolution may be obtained from the external electronic device 108.

In operation 603, according to various embodiments, the electronic device 101 (for example, the processor 120 of FIG. 1) is a plurality of deep learning models corresponding to a plurality of resolutions of the original streaming data based on a user input. Among them, a specific deep learning model corresponding to a specific resolution of the original streaming data can be selected.

According to various embodiments, the electronic device 101 (for example, the processor 120 of FIG. 1) generates a specific deep learning model corresponding to a specific resolution of the original streaming data based on a user input for selecting an image difference degree. You can choose. According to an embodiment, the electronic device 101 is at least one deep learning model that satisfies an image difference degree selected by a user from among a plurality of deep learning models corresponding to a plurality of resolutions of original streaming data for a specific domain. can confirm. For example, referring to FIG. 7A, the electronic device 101 includes a first deep learning model 710 corresponding to a first resolution (eg, 80% of the original resolution) of the original streaming data for the first domain 701. ), a second deep learning model 720 corresponding to a second resolution (eg, 60% of the original resolution), a third deep learning model 730 corresponding to a third resolution (eg, 40% of the original resolution), Among the fourth deep learning model 740 corresponding to the fourth resolution (eg, 20% of the original resolution) and the fifth deep learning model (not shown) corresponding to the fifth resolution (eg, 30% of the original resolution) , A first deep learning model 710 to a third deep learning model 730 that satisfies an image difference degree (eg, an error rate of 3%) selected by the user at the first time point T1 (eg, a user selected time point) I can confirm. According to an embodiment, the electronic device 101 may select a specific deep learning model corresponding to a specific resolution of the original streaming data from among at least one deep learning model that satisfies the image difference degree selected by the user. For example, referring to FIG. 7A, the electronic device 101 includes first to third deep learning models 710 to 3 satisfying an image difference degree (eg, 3%) selected by a user at a first time point T1. Among the deep learning models 730, a third deep learning model 730 corresponding to the lowest resolution (eg, 40% of the original resolution) of the original streaming data may be selected. According to an embodiment, the electronic device 101 may select a specific deep learning model corresponding to a resolution having a specific network delay from among at least one deep learning model that satisfies an image difference degree selected by a user. For example, the electronic device 101 may include a first deep learning model 710 to a third deep learning model 730 that satisfies an image difference (eg, 3%) selected by a user at a first time point T1. Among them, you can check streaming data (eg, third streaming data) corresponding to the resolution (eg, 40% of the original resolution) with the lowest network delay (eg 5 ms), and A third deep learning model 730 may be selected. According to an embodiment, while at least one deep learning model that does not satisfy the image difference selected by the user at the first time point acquires updated versions of the plurality of deep learning models from the external electronic device 108 at a specified period, Based on the confirmation that the image difference degree is satisfied at the second view point, the electronic device 101 selects a resolution lower than the resolution selected at the first view point among at least one deep learning model that satisfies the image difference degree at the second view point. You can choose the corresponding deep learning model. For example, referring to FIG. 7A, while the electronic device 101 obtains an updated version of the first deep learning model 710 to the fifth deep learning model (not shown) from the external electronic device 108 at a specified period, , The fourth deep learning model 740 and the fifth deep learning model (not shown) that do not satisfy the image difference (eg, an error rate of 3%) selected by the user at the first time point T1 are the second time point ( Based on the confirmation that the image difference degree is satisfied in T2), the electronic device 101 provides a fourth deep learning model 740 and a fifth deep learning model ( 750), the fourth deep learning model 740 corresponding to the lowest resolution (eg, 20% of the original resolution) may be selected.

According to various embodiments, the electronic device 101 (eg, the processor 120 of FIG. 1) selects a specific deep learning model corresponding to a specific resolution of the original streaming data based on a user input for selecting a network delay. I can. According to an embodiment, the electronic device 101 may check a specific resolution of original streaming data corresponding to a network delay selected by a user input, and select a specific deep learning model corresponding to the identified specific resolution. For example, the first streaming data corresponding to the first resolution of the original streaming data has a network delay of 5 to 6 ms, the second streaming data corresponding to the second resolution has a network delay of 6 to 7 ms, and the third It is assumed that the third streaming data corresponding to the resolution has a network delay of 7 to 8 ms. In this case, the electronic device 101 may check the first streaming data corresponding to the first resolution of the original streaming data having a network delay (eg, 5.5 ms) selected by the user input, and A first deep learning model corresponding to the first resolution may be selected. According to various embodiments, the external electronic device 108 may obtain an image difference degree selected by the user from the electronic device 101, and correspond to a specific resolution among deep learning models that satisfy the selected image difference degree. The deep learning model may be transmitted to the electronic device 101. For example, referring to FIG. 7A, the external electronic device 108 may obtain an image difference degree selected by a user (eg, an error rate of 3%) from the electronic device 101, and satisfy the selected image difference degree. The third deep learning model 730 corresponding to the lowest resolution among the first to third deep learning models 710 to 730 may be transmitted to the electronic device 101. According to an embodiment, while training of a plurality of deep learning models is in progress, at least one deep learning model that does not satisfy the image difference selected by the user at the first time point is configured to calculate the image difference degree at the second time point. Based on the confirmation of satisfaction, the external electronic device 108 may transmit, to the electronic device 101, a deep learning model corresponding to a specific resolution from among at least one deep learning model that satisfies the image difference degree at the second view point. . For example, referring to FIG. 7A, while training of the first deep learning model 710 to the fifth deep learning model (not shown) is in progress, the external electronic device 108 The fourth deep learning model 740 and the fifth deep learning model (not shown) that do not satisfy the image difference degree selected by (eg, 3% error rate) satisfy the image difference degree at the second view point T2. Based on the confirmation of, the external electronic device 108 has the lowest resolution among the fourth deep learning model 740 and the fifth deep learning model (not shown) that satisfy the image difference degree at the second view point T2. Example: A fourth deep learning model 740 corresponding to 20% of the original resolution) may be transmitted to the electronic device 101.

According to various embodiments, the electronic device 101 (for example, the processor 120 of FIG. 1) is based on a user input for selecting an image difference degree for a specific domain, a specific resolution of the original streaming data for a specific domain. You can select a specific deep learning model corresponding to. According to an embodiment, the electronic device 101 is based on a user input for selecting an image difference degree for a specific domain, among at least one deep learning model that satisfies the selected image difference degree. You can select a specific deep learning model that corresponds to the lowest resolution. For example, referring to FIG. 7B, the electronic device 101 is a user who selects an image difference 751 (eg, a first error rate of 5%) for a first domain 701 (eg, A game app). Corresponds to the lowest resolution of the original streaming data for the first domain 701 among the first deep learning model 710 to the third deep learning model 730 that satisfies the selected image difference degree 751 based on the input The third deep learning model 730 may be selected. For another example, referring to FIG. 7B, the electronic device 101 selects an image difference degree 752 (eg, a second error rate of 10%) for the second domain 702 (eg, B game app). The lowest resolution of the original streaming data for the second domain 702 among the first deep learning model 711 to the fourth deep learning model 741 that satisfies the selected image difference 752 based on the user input. A fourth deep learning model 741 corresponding to may be selected.

According to various embodiments, the external electronic device 108 may obtain an image difference degree for a specific domain selected by the user from the electronic device 401, and is the most A deep learning model corresponding to a low resolution may be transmitted to the electronic device 101. For example, referring to FIG. 7B, the external electronic device 108 is a deep learning model that satisfies the image difference 761 (eg, a first error rate of 3%) for the first domain 701 selected by the user. Among them, the first deep learning model 710 corresponding to the lowest resolution of the original streaming data for the first domain 701 may be transmitted to the electronic device 401. As another example, referring to FIG. 7B, the external electronic device 108 satisfies the image difference 762 (eg, a second error rate of 5%) for the second domain 702 selected by the user. 1 Among the deep learning models 711 to the third deep learning models 731, the third deep learning model 731 corresponding to the lowest resolution of the original streaming data for the second domain 702 is used as the electronic device 401. I can send it.

In operation 605, according to various embodiments, the electronic device 101 (for example, the processor 120 of FIG. 1) uses a selected specific deep learning model to determine a specific resolution corresponding to a specific resolution of the original streaming data for a specific domain. Streaming data can be converted into restored streaming data. According to an embodiment, the electronic device 101 may apply specific streaming data as input data to a selected specific deep learning model, and obtain restored streaming data by upscaling specific streaming data as output data. have.

In operation 607, according to various embodiments, the electronic device 101 (eg, the processor 120 of FIG. 1) may display the restored streaming data through a display (eg, the display device 160 of FIG. 1). I can.

FIG. 8 shows, according to various embodiments, an electronic device (eg, the electronic device 101 of FIG. 1) selecting a specific deep learning model based on a compression rate corresponding to a plurality of resolutions and an image difference This is a flow chart for explaining a method of displaying the restored streaming data using a deep learning model.

9 illustrates a first embodiment in which the electronic device 101 selects a specific deep learning model corresponding to a specific resolution based on a compression rate corresponding to a plurality of resolutions and an image difference degree according to various embodiments It is a drawing for.

10 illustrates a second embodiment in which the electronic device 101 selects a specific deep learning model corresponding to a specific resolution based on a compression rate corresponding to a plurality of resolutions and an image difference degree according to various embodiments It is a drawing for.

In operation 801, according to various embodiments, the electronic device 101 (for example, the electronic device 101 of FIG. 1) is through a communication module (for example, the communication module 190 of FIG. Example: It is possible to obtain specific streaming data corresponding to a specific resolution of the original streaming data for a specific domain from the server 108 of FIG. 1.

According to various embodiments, the electronic device 101 (for example, the processor 120 of FIG. 1) is based on a compression rate corresponding to a plurality of resolutions of the original streaming data and an image difference degree corresponding to the plurality of resolutions. A specific resolution of the original streaming data for a specific domain can be checked, specific streaming data corresponding to a specific resolution of the original streaming data can be requested from the external electronic device 108, and in response to the request, the external electronic device 108 ), it is possible to obtain specific streaming data corresponding to a specific resolution. A method of checking a specific resolution of the original streaming data for a specific domain based on a compression rate corresponding to the plurality of resolutions and an image difference degree corresponding to the plurality of resolutions will be described later in operation 803.

According to an embodiment, while receiving the first streaming data corresponding to the first resolution of the original streaming data from the external electronic device 108, the electronic device 101 Second streaming data corresponding to the second resolution may be received from the external electronic device 108 based on the image difference corresponding to the data. For example, while receiving the first streaming data corresponding to the first resolution (eg, 80% of the original resolution) of the original streaming data from the external electronic device 108, the electronic device 101 A second resolution (eg, 60% of the original resolution) for deriving a value that satisfies a specified condition may be identified by using a corresponding compression rate and an image difference degree corresponding to a plurality of resolutions. In this case, the electronic device 101 may request the second streaming data corresponding to the second resolution from the external electronic device 108, and in response to the request, the external electronic device 108 It is possible to acquire second streaming data.

In operation 803, according to various embodiments, the electronic device 101 (for example, the processor 120 of FIG. 1) determines a compression rate corresponding to a plurality of resolutions of the original streaming data and an image difference corresponding to the plurality of resolutions Based on the diagram, a specific deep learning model corresponding to a specific resolution of the original streaming data may be selected from among a plurality of deep learning models corresponding to a plurality of resolutions of the original streaming data.

According to various embodiments, the electronic device 101 (for example, the processor 120 of FIG. 1) applies an image difference degree corresponding to each of the plurality of resolutions to a compression rate corresponding to each of the plurality of resolutions. A specific resolution that derives a value that satisfies a specified condition can be checked, and a specific deep learning model corresponding to the identified specific resolution can be selected. The compression rate corresponding to the specific resolution may mean a value (A/B) obtained by dividing the size (A) of the streaming data corresponding to the specific resolution of the original streaming data by the size (B) of the original streaming data. According to an embodiment, an image difference degree corresponding to a specific resolution may mean an image difference degree for a specific deep learning model corresponding to a specific resolution. For example, referring to <901> of FIG. 9, a plurality of deep learning models corresponding to a plurality of resolutions of the original streaming data may be initially stored in the electronic device 101 in an untrained state, and a plurality of The image difference degree (eg, error rate) for each of the deep learning models may be equal to '1'. In this case, the electronic device 101 is selected from among values 911 (for example, a value obtained by summing a compression rate and an error rate) to which an image difference degree corresponding to each of the plurality of resolutions is applied to a compression rate corresponding to each of the plurality of resolutions. You can check a specific resolution 931 (e.g., 100% original resolution) that derives a value that satisfies the condition (921) (e.g., the lowest value), and select a specific deep learning model corresponding to a specific resolution (931). I can. <901> In the graph, since the specific resolution 931 corresponding to 100% of the original resolution has a value with an image difference of 0, the value 921 (eg, the lowest value) that satisfies the specified condition is the specific resolution ( 931). For another example, referring to <902> of FIG. 9, a plurality of deep learning models corresponding to a plurality of resolutions of the original streaming data are stored in a trained state in the electronic device 101 after a certain period of time. The image difference degree for each of the plurality of deep learning models may be different from each other. In this case, the electronic device 101 is selected from among values 912 (eg, a value obtained by adding a compression rate and an error rate) to which an image difference degree corresponding to each of the plurality of resolutions is applied to a compression rate corresponding to each of the plurality of resolutions. You can check a specific resolution 932 (e.g. 80% of the original resolution) that derives a value 922 that satisfies the condition (e.g., the lowest value), and a specific deep learning model corresponding to a specific resolution 932 can be identified. You can choose.

According to various embodiments, the electronic device 101 (for example, the processor 120 of FIG. 1) may provide a compression rate corresponding to a plurality of resolutions of the original streaming data, an image difference degree corresponding to the plurality of resolutions, and an electronic device. Based on the network bandwidth of the device 101, a specific deep learning model corresponding to a specific resolution of the original streaming data may be selected from among a plurality of deep learning models corresponding to a plurality of resolutions of the original streaming data. . According to an embodiment, the electronic device 101 uses the following [Equation 1] and the following [Equation 2], from among a plurality of deep learning models corresponding to a plurality of resolutions of the original streaming data. You can select a specific deep learning model that corresponds to a specific resolution of the streaming data.

는 특정 해상도에 대응하는 스트리밍 데이터를 제공하기 위하여 미리 정해진 최대 네트워크 지연(network latency)을 나타내고,

는 원본 스트리밍 데이터를 송신하기 위하여 미리 정해진 최소 네트워크 지연을 나타낼 수 있다. g 함수는 원본 스트리밍 데이터를 송신해도 시간이 충분할 경우, 영상 차이도를 0으로 보낼 수 있다. 또한, g 함수는 특정 해상도에 대응하는 스트리밍 데이터의 송신 시간이 미리 정해진 최대 네트워크 지연(network latency)보다 지연될 경우, 특정 해상도에 대응하는 스트리밍 데이터의 제공이 적절하지 않기 때문에 임의의 큰 값을 출력하여서 특정 해상도보다 더 낮은 해상도를 탐색할 수 있도록 유도할 수 있다.

는 영상 차이도와 네트워크 지연 사이의 가중치를 의미할 수 있다. 일실시예에 따르면,

는 전자 장치(101)의 사용자에 의하여 선택될 수 있고 또는 외부 전자 장치(108)에 의하여 미리 설정될 수 있다. 일실시예에 따르면,

는 상술한 [수학식 1] 및 [수학식 2]에서 생략 할 수 있다. 일실시예에 따르면, f 함수 값이 0에 가까워 질수록 영상 차이도 및 압축률이 낮아지므로, 전자 장치(101)는 더 최적화 된 값으로 확인할 수 있다. 일실시예에 따르면, 전자 장치(101)는 가장 낮은 f값을 도출하는 특정 해상도를 확인할 수 있고, 확인된 특정 해상도에 대응하는 특정 딥러닝 모델을 선택할 수 있다. 예를 들어, 도 10의 <1001>을 참조하면, 전자 장치(101)는 제 1 시점(T1)에 가장 낮은 f값을 도출하는 원본 스트리밍 데이터의 제 2 해상도(예: 원본 해상도의 20%)에 대응하는 제 2 딥러닝 모델(1020)을 이용하여 복원 스트리밍 데이터를 생성한 후에, 원본 스트리밍 데이터의 제 2 해상도에 대응하는 스트리밍 데이터의 픽셀 변화량이 지정된 범위를 초과함을 확인한 것에 기반하여, 제 2 시점(T2)에 가장 낮은 f값을 도출하는 원본 스트리밍 데이터의 제 1 해상도(예: 원본 해상도의 80%)를 확인할 수 있고, 제 1 해상도에 대응하는 제 1 딥러닝 모델(1010)을 선택하여 복원 스트리밍 데이터를 생성할 수 있다. 또 다른 예를 들어, 도 10의 <1002>를 참조하면, 전자 장치(101)는 제 1 시점(T3)에 가장 낮은 f값을 도출하는 원본 스트리밍 데이터의 제 1 해상도(예: 원본 해상도의 80%)에 대응하는 제 1 딥러닝 모델(1010)을 이용하여 복원 스트리밍 데이터를 생성한 후에, 전자 장치(101)의 네트워크 대역폭이 낮아짐을 확인한 것에 기반하여, 제 2 시점(T4)에 가장 낮은 f값을 도출하는 원본 스트리밍 데이터의 제 2 해상도(예: 원본 해상도의 20%)를 확인할 수 있고, 제 2 해상도에 대응하는 제 2 딥러닝 모델(1020)을 선택하여 복원 스트리밍 데이터를 생성할 수 있다.E denotes the image difference degree corresponding to the specific resolution of the original streaming data, and D/O denotes the specific resolution obtained by dividing the size (D) of the streaming data corresponding to the specific resolution of the original streaming data by the size (O) of the original streaming data. It may represent a compression rate corresponding to. L represents the network bandwidth measured by the electronic device 101 at a specific time point, D/L represents the time it takes to transmit the streaming data corresponding to the specific resolution, and O/L represents the time it takes to transmit the original streaming data. Can be indicated.

Denotes a predetermined maximum network latency to provide streaming data corresponding to a specific resolution,

May represent a predetermined minimum network delay in order to transmit the original streaming data. If there is enough time to transmit the original streaming data, the g function can send the video difference as 0. Also, when the transmission time of streaming data corresponding to a specific resolution is delayed than a predetermined maximum network latency, the g function outputs an arbitrary large value because it is not appropriate to provide streaming data corresponding to a specific resolution. Thus, it can be induced to search for a lower resolution than a specific resolution.

May mean a weight between the image difference and the network delay. According to one embodiment,

May be selected by the user of the electronic device 101 or may be preset by the external electronic device 108. According to one embodiment,

Can be omitted in [Equation 1] and [Equation 2] described above. According to an embodiment, as the f function value approaches 0, the image difference and compression rate decrease, so that the electronic device 101 may check the value as a more optimized value. According to an embodiment, the electronic device 101 may check a specific resolution for deriving the lowest f-value, and select a specific deep learning model corresponding to the identified specific resolution. For example, referring to <1001> of FIG. 10, the electronic device 101 provides a second resolution (eg, 20% of the original resolution) of the original streaming data that derives the lowest f value at the first time point T1. After generating the reconstructed streaming data using the second deep learning model 1020 corresponding to, based on confirming that the pixel change amount of the streaming data corresponding to the second resolution of the original streaming data exceeds the specified range, the first 2 It is possible to check the first resolution (eg, 80% of the original resolution) of the original streaming data that derives the lowest f-value at the time point T2, and select the first deep learning model 1010 corresponding to the first resolution. Thus, the restored streaming data can be generated. For another example, referring to <1002> of FIG. 10, the electronic device 101 determines the first resolution of the original streaming data (e.g., 80 of the original resolution) for deriving the lowest f value at the first time point T3. %) after generating the restored streaming data using the first deep learning model 1010 corresponding to), based on the confirmation that the network bandwidth of the electronic device 101 is lowered, the lowest f at the second time point T4 The second resolution (eg, 20% of the original resolution) of the original streaming data from which the value is derived can be checked, and the reconstructed streaming data can be generated by selecting a second deep learning model 1020 corresponding to the second resolution. .

According to various embodiments, operations for checking a specific resolution described in FIGS. 8 to 10 may be performed by the external electronic device 108, and the external electronic device 108 is The running model may be transmitted to the electronic device 101.

In operation 805, according to various embodiments, the electronic device 101 (for example, the processor 120 of FIG. 1) uses a selected specific deep learning model to determine a specific resolution corresponding to a specific resolution of the original streaming data for a specific domain. Streaming data can be converted into restored streaming data. According to an embodiment, the electronic device 101 may apply specific streaming data as input data to a selected specific deep learning model, and obtain restored streaming data by upscaling specific streaming data as output data. have.

In operation 807, according to various embodiments, the electronic device 101 (eg, the processor 120 of FIG. 1) may display the restored streaming data through a display (eg, the display device 160 of FIG. 1). I can.

FIG. 11 is a diagram for describing an embodiment in which an external electronic device (eg, the server 108 of FIG. 1) trains a specific deep learning model corresponding to a specific resolution range, according to various embodiments.

According to various embodiments, the electronic device 101 (for example, the processor 120 of FIG. 1) is a network for the electronic device 101 based on a specified period, a user's request, or an external electronic device 108's request. You can check the bandwidth. According to an embodiment, the electronic device 101 may transmit the identified network bandwidth to the external electronic device 108.

According to various embodiments, the external electronic device 108 includes a compression rate corresponding to a plurality of resolutions of the original streaming data, an image difference degree corresponding to the plurality of resolutions, and a network bandwidth of the electronic device 101. ), a specific deep learning model corresponding to a specific resolution of the original streaming data may be selected from among a plurality of deep learning models corresponding to a plurality of resolutions of the original streaming data. A method for the external electronic device 108 to check a specific resolution based on a compression rate, an image difference, and a network bandwidth may use the method for checking by the electronic device 101 described in operation 805 of FIG. 8. According to an embodiment, the external electronic device 108 may transmit a specific deep learning model corresponding to the identified specific resolution to the electronic device 101. For example, the external electronic device 108 is a specific deep learning model corresponding to a specific resolution identified among the first deep learning model 1110, the second deep learning model 1120, and the third deep learning model 1130. May be transmitted to the electronic device 101.

According to various embodiments, the external electronic device 108 may train a specific deep learning model corresponding to a specific resolution range of original streaming data for a specific domain. For example, the external electronic device 108 may train a first deep learning model 1110 corresponding to a first resolution range of the original streaming data (eg, 100% or less than 80% of the original resolution), and the first 2 Can train a second deep learning model 1120 corresponding to a resolution range (e.g., less than or equal to 80% and greater than 60% of the original resolution), and a third resolution range (e.g., less than or equal to 60% and greater than 40% of the original resolution) A third deep learning model 1130 corresponding to may be trained. According to an embodiment, the external electronic device 108 may transmit a specific deep learning model corresponding to a specific resolution range including the identified specific resolution to the electronic device 101. For example, the external electronic device 108 may transmit the second deep learning model 1120 corresponding to the second resolution range including the identified specific resolution (eg, 70%) to the electronic device 101. .

FIG. 12 is a diagram for describing an embodiment in which an external electronic device (eg, the server 108 of FIG. 1) trains a specific deep learning model using patched streaming data, according to various embodiments.

According to various embodiments, the external electronic device 108 stores original streaming data (eg, original streaming data 301 of FIG. 3) for the first domain based on a deep learning algorithm or software algorithm related to image conversion. Can be converted. For example, the external electronic device 108 applies a software algorithm (eg, a graphic patch) to the original streaming data 301, without changing the program code constituting the original streaming data 301, 301 may be converted into patched streaming data 1201. According to an embodiment, the external electronic device 108 may convert the patched streaming data 1201 into first streaming data 1211 corresponding to a first resolution of the patched streaming data 1201. For example, the external electronic device 108 resizes the original resolution (eg, 100%) of the patched streaming data 1201 to a first resolution (eg, 80% of the original resolution). Can be created. The external electronic device 108 may transmit the generated first streaming data 1211 to the electronic device 101. According to an embodiment, the external electronic device 108 uses patched streaming data 1201 and first streaming data 1211 corresponding to a first resolution of the patched streaming data 1201. The first deep learning model 1210 corresponding to the first resolution of 1201) may be trained. For example, the external electronic device 108 may apply the first streaming data 1211 as training data and the patched streaming data 1201 as correct answer data to a first dip corresponding to the first resolution of the patched streaming data 1201. By providing to the learning model 1210, the first deep learning model 1210 may be trained. According to an embodiment, the external electronic device 108 may transmit the first deep learning model 1210 corresponding to the first resolution of the patched streaming data 1201 to the electronic device 101. According to an embodiment, the patched streaming data 1201 may be included in the original streaming data 301, and in this case, the first deep learning model 1210 corresponding to the first resolution of the patched streaming data 1201 May be included in a first deep learning model (eg, the first deep learning model 310 of FIG. 3) corresponding to the first resolution of the original streaming data 301.

13 is a diagram in which an external electronic device (eg, the server 108 of FIG. 1) trains a plurality of deep learning models corresponding to a plurality of resolutions of original streaming data for a plurality of domains, according to various embodiments. It is a figure for explaining an embodiment.

According to various embodiments, the external electronic device 108 may generate a plurality of streaming data corresponding to a plurality of resolutions of the original streaming data for a plurality of domains. For example, the external electronic device 108 may generate a plurality of streaming data corresponding to a plurality of resolutions of the original streaming data for the first domain 1301 (eg, movie A as the first content). As another example, the external electronic device 108 may generate a plurality of streaming data corresponding to a plurality of resolutions of the original streaming data for the second domain 1302 (eg, drama B as second content). have. For another example, the external electronic device 108 may generate a plurality of streaming data corresponding to a plurality of resolutions of the original streaming data for the third domain 1303 (eg, live broadcast news C as third content). I can.

According to various embodiments, the external electronic device 108 corresponds to a plurality of resolutions of the original streaming data by using original streaming data for a specific domain and a plurality of streaming data corresponding to a plurality of resolutions of the original streaming data. It is possible to train a plurality of deep learning models.

According to an embodiment, the external electronic device 108 may train a plurality of deep learning models corresponding to a plurality of resolutions of the original streaming data for the first domain 1301. For example, the external electronic device 108 may provide first streaming data corresponding to a first resolution of the original streaming data for the first domain 1301 as training data and original streaming data for the first domain as correct answer data. The first deep learning model 1310 corresponding to the first resolution of the original streaming data for the first domain is provided to the first deep learning model 1310 corresponding to the first resolution of the original streaming data for the first domain. You can train. For another example, the external electronic device 108 includes second streaming data corresponding to a second resolution of the original streaming data for the first domain 1301 as training data and original streaming data for the first domain as correct answer data. Is provided to a second deep learning model (not shown) corresponding to the second resolution of the original streaming data for the first domain, and a second deep learning model corresponding to the second resolution of the original streaming data for the first domain ( Not shown) can be trained.

According to an embodiment, the external electronic device 108 may train a plurality of deep learning models corresponding to a plurality of resolutions of the original streaming data for the second domain 1302. For example, the external electronic device 108 may provide first streaming data corresponding to a first resolution of the original streaming data for the second domain 1302 as training data and original streaming data for the second domain as correct answer data. A first deep learning model (not shown) corresponding to the first resolution of the original streaming data for the second domain by providing it to a first deep learning model (not shown) corresponding to the first resolution of the original streaming data for the second domain. ) Can be trained. For another example, the external electronic device 108 includes second streaming data corresponding to a second resolution of the original streaming data for the second domain 1302 as training data and original streaming data for the second domain as correct answer data. Is provided to the second deep learning model 1320 corresponding to the second resolution of the original streaming data for the second domain, and a second deep learning model 1320 corresponding to the second resolution of the original streaming data for the second domain. ) Can be trained.

According to an embodiment, the external electronic device 108 may train a plurality of deep learning models corresponding to a plurality of resolutions of the original streaming data for the third domain 1303. For example, the external electronic device 108 provides third streaming data corresponding to a third resolution of the original streaming data for the third domain 1303 as training data and original streaming data for the third domain as correct answer data. The third deep learning model 1330 corresponding to the third resolution of the original streaming data for the third domain is provided to the third deep learning model 1330 corresponding to the third resolution of the original streaming data for the 3 domain. You can train.

FIG. 14 is an electronic diagram of a deep learning model corresponding to an image difference diagram for a specific domain or a resolution of original streaming data for a specific domain by an external electronic device (eg, the server 108 of FIG. 1) according to various embodiments. This is a flowchart illustrating a method of transmitting to a device (eg, the electronic device 101 of FIG. 1 ).

In operation 1401, according to various embodiments, the external electronic device 108 corresponds to a plurality of resolutions of the original streaming data for a specific domain through a communication module (eg, the communication module 190 of FIG. 1). A plurality of streaming data may be transmitted to a plurality of electronic devices. For example, the external electronic device 108 includes a first electronic device (eg, the electronic device 101 of FIG. 1) with a plurality of streaming data corresponding to a plurality of resolutions of the original streaming data for the first domain. It may be transmitted to a plurality of first electronic devices. As another example, the external electronic device 108 may transmit a plurality of streaming data corresponding to a plurality of resolutions of the original streaming data for the second domain to a second electronic device (for example, the second electronic device 401 of FIG. 4 ). )).

According to various embodiments, the external electronic device 108 (eg, the processor 120 of FIG. 1) may include a streaming service providing server. According to an embodiment, a streaming data transmission server for transmitting streaming data and a streaming data training server for training streaming data may be included in the external electronic device 108 that is a streaming service providing server as one structure, or each The structure of can be divided and exist as a separate structure.

In operation 1403, according to various embodiments, the external electronic device 108 (eg, the processor 120 of FIG. 1) uses the original streaming data and a plurality of streaming data corresponding to a plurality of resolutions of the original streaming data. Thus, it is possible to train a plurality of deep learning models corresponding to a plurality of resolutions of the original streaming data. For example, the external electronic device 108 stores a plurality of streaming data corresponding to a plurality of resolutions of the original streaming data for the first domain as training data and the original streaming data for the first domain as correct answer data in the first domain. Training a plurality of first deep learning models corresponding to a plurality of resolutions of the original streaming data for a first domain by providing them to a plurality of first deep learning models corresponding to a plurality of resolutions of the original streaming data for can do. As another example, the external electronic device 108 provides a plurality of streaming data corresponding to a plurality of resolutions of the original streaming data for the second domain as training data and original streaming data for the second domain as correct answer data. A plurality of second deep learning models corresponding to a plurality of resolutions of the original streaming data for the second domain by providing to a plurality of second deep learning models corresponding to a plurality of resolutions of the original streaming data for the 2 domain Can train them.

In operation 1405, according to various embodiments, the external electronic device 108 (for example, the processor 120 of FIG. 1) is transmitted from the electronic device 101 included in the plurality of electronic devices through the communication module 190. An image difference degree for a specific domain or a resolution of the original streaming data for a specific domain may be obtained. According to an embodiment, the electronic device 101 obtains a corresponding image difference based on a user input for selecting an image difference degree for a specific domain or a user input for selecting a resolution of the original streaming data for a specific domain Alternatively, the resolution of the original streaming data may be transmitted to the external electronic device 108.

In operation 1407, according to various embodiments, the external electronic device 108 (eg, the processor 120 of FIG. 1) corresponds to a plurality of resolutions of the original streaming data for a specific domain through the communication module 190. A deep learning model corresponding to an image difference degree for a specific domain or a resolution of the original streaming data for a specific domain among the plurality of deep learning models to be transmitted may be transmitted to the electronic device 101.

According to various embodiments, the external electronic device 108 (eg, the processor 120 of FIG. 1) may perform operation 1407 described above using the method described in FIGS. 6, 7A, and 7B. .

An electronic device according to various embodiments disclosed in this document may be a device of various types. The electronic device may include, for example, a portable communication device (eg, a smart phone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a home appliance. The electronic device according to the embodiment of the present document is not limited to the above-described devices.

Various embodiments of the present document and terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutes of the corresponding embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or a plurality of the items unless clearly indicated otherwise in a related context. In this document, “A or B”, “at least one of A and B”, “at least one of A or B”, “A, B or C”, “at least one of A, B and C” and “A, Each of phrases such as "at least one of B or C" may include any one of the items listed together in the corresponding one of the phrases, or all possible combinations thereof. Terms such as "first", "second", or "first" or "second" may be used simply to distinguish the component from other Order) is not limited. Some (eg, a first) component is referred to as “coupled” or “connected” to another (eg, a second) component, with or without the terms “functionally” or “communicatively”. When mentioned, it means that any of the above components may be connected to the other components directly (eg by wire), wirelessly, or via a third component.

The term "module" used in this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic blocks, parts, or circuits. The module may be an integrally configured component or a minimum unit of the component or a part thereof that performs one or more functions. For example, according to an embodiment, the module may be implemented in the form of an application-specific integrated circuit (ASIC).

Various embodiments of the present document include one or more instructions stored in a storage medium (eg, internal memory 136 or external memory 138) that can be read by a machine (eg, electronic device 101). It may be implemented as software (for example, the program 140) including them. For example, the processor (eg, the processor 120) of the device (eg, the electronic device 101) may call and execute at least one command among one or more commands stored from a storage medium. This enables the device to be operated to perform at least one function according to the at least one command invoked. The one or more instructions may include code generated by a compiler or code executable by an interpreter. A storage medium that can be read by a device may be provided in the form of a non-transitory storage medium. Here,'non-transient' only means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and this term refers to the case where data is semi-permanently stored in the storage medium. It does not distinguish between temporary storage cases.

According to an embodiment, a method according to various embodiments disclosed in the present document may be provided by being included in a computer program product. Computer program products can be traded between sellers and buyers as commodities. The computer program product is distributed in the form of a device-readable storage medium (e.g. compact disc read only memory (CD-ROM)), or through an application store (e.g. Play Store ^TM ) or two user devices (e.g. It can be distributed (e.g., downloaded or uploaded) directly between, e.g. smartphones), online. In the case of online distribution, at least a part of the computer program product may be temporarily stored or temporarily generated in a storage medium that can be read by a device such as a server of a manufacturer, a server of an application store, or a memory of a relay server.

According to various embodiments, each component (eg, module or program) of the above-described components may include a singular number or a plurality of entities. According to various embodiments, one or more components or operations among the above-described corresponding components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (eg, a module or program) may be integrated into one component. In this case, the integrated component may perform one or more functions of each component of the plurality of components in the same or similar to that performed by the corresponding component among the plurality of components prior to the integration. . According to various embodiments, operations performed by a module, program, or other component may be sequentially, parallel, repeatedly, or heuristically executed, or one or more of the operations may be executed in a different order or omitted. Or one or more other actions may be added.

According to various embodiments, the electronic device 101 includes a display 160, a communication module 190, a processor 120 operatively connected to the display 160 and the communication module 190, and the processor ( 120) and a memory 130 operatively connected to the memory 130, and the memory 130, when executed, the processor, through the communication module 190, streams the original from an external electronic device to the first domain Acquiring first streaming data corresponding to a first resolution of data, and a first dip corresponding to the first resolution of the original streaming data from among a plurality of deep learning models corresponding to a plurality of resolutions of the original streaming data Instructions for selecting a learning model, converting the first streaming data into first restored streaming data using the first deep learning model, and displaying the first restored streaming data through the display 160 You can store instructions.

According to various embodiments, the instructions may cause the processor to transmit the first restored streaming data to the external electronic device through the communication module.

According to various embodiments, the first domain may include a plurality of designated areas or a plurality of sub-domains corresponding to at least one of a plurality of designated periods.

According to various embodiments, the instructions may cause the processor to select the first deep learning model corresponding to the first resolution based on a user input.

According to various embodiments, the user input includes a first user input for selecting an image difference degree, and the instructions include, by the processor, the plurality of deep learning models based on the first user input. Among the at least one deep learning model that satisfies the image difference degree is identified, and the first deep learning model corresponding to the first resolution, which is the lowest resolution of the original streaming data, is selected from among the at least one deep learning model. You can choose.

According to various embodiments, the user input includes a second user input for selecting a network delay, and the instructions include, by the processor, the original corresponding to the network delay, based on the second user input. The first resolution of streaming data may be checked, and the first deep learning model corresponding to the first resolution may be selected.

According to various embodiments, the instructions, the processor, based on a compression rate corresponding to the plurality of resolutions of the original streaming data and an image difference degree corresponding to the plurality of resolutions, the first resolution. The corresponding first deep learning model may be selected.

According to various embodiments, the instructions include, by the processor, a value that satisfies a specified condition among values obtained by applying the image difference degree corresponding to the plurality of resolutions to the compression rate corresponding to the plurality of resolutions. The first resolution to be derived may be checked, and the first deep learning model corresponding to the first resolution may be selected.

According to various embodiments, the instructions include, by the processor, the compression rate corresponding to the plurality of resolutions of the original streaming data, the image difference degree corresponding to the plurality of resolutions, and the network of the electronic device. Based on the bandwidth, the first deep learning model corresponding to the first resolution may be selected.

According to various embodiments, the first deep learning model may correspond to a resolution range including the first resolution.

According to various embodiments, a method of providing streaming data includes an operation of acquiring first streaming data corresponding to a first resolution of the original streaming data for a first domain from an external electronic device through a communication module, the original Selecting a first deep learning model corresponding to the first resolution of the original streaming data from among a plurality of deep learning models corresponding to a plurality of resolutions of streaming data, the first deep learning model using the first deep learning model It may include an operation of converting the first streaming data into first reconstructed streaming data, and an operation of displaying the first reconstructed streaming data through a display.

According to various embodiments, the method of providing streaming data may further include transmitting the first restored streaming data to the external electronic device through the communication module.

According to various embodiments, the operation of selecting the first deep learning model corresponding to the first resolution includes an operation of selecting the first deep learning model corresponding to the first resolution based on a user input. can do.

According to various embodiments, the user input includes a first user input for selecting an image difference degree, and the operation of selecting a first deep learning model corresponding to the first resolution may be performed based on the first user input. Based on the operation of checking at least one deep learning model that satisfies the image difference among the plurality of deep learning models, and the first, the lowest resolution of the original streaming data among the at least one deep learning model It may include an operation of selecting the first deep learning model corresponding to the resolution.

According to various embodiments, the user input includes a second user input for selecting a network delay, and the operation of selecting a first deep learning model corresponding to the first resolution is based on the second user input. Thus, it may include an operation of checking the first resolution of the original streaming data corresponding to the network delay, and an operation of selecting the first deep learning model corresponding to the first resolution.

According to various embodiments, the operation of selecting the first deep learning model corresponding to the first resolution includes a compression rate corresponding to the plurality of resolutions of the original streaming data and an image difference corresponding to the plurality of resolutions. Based on the diagram, it may include an operation of selecting the first deep learning model corresponding to the first resolution.

According to various embodiments, the operation of selecting the first deep learning model corresponding to the first resolution includes calculating the image difference degree corresponding to the plurality of resolutions with respect to the compression rate corresponding to the plurality of resolutions. An operation of checking the first resolution for deriving a value that satisfies a specified condition among applied values, and an operation of selecting the first deep learning model corresponding to the first resolution.

According to various embodiments, the operation of selecting the first deep learning model corresponding to the first resolution includes the compression rate corresponding to the plurality of resolutions of the original streaming data, and the compression rate corresponding to the plurality of resolutions. An operation of selecting the first deep learning model corresponding to the first resolution based on an image difference degree and a network bandwidth of the electronic device may be included.

According to various embodiments, the streaming service providing server includes a communication module, a processor operatively connected to the communication module, and a memory operatively connected to the processor, and the memory is, when executed, the processor And transmitting a plurality of streaming data corresponding to a plurality of resolutions of the original streaming data for a specific domain through the communication module to a plurality of electronic devices, and the plurality of resolutions of the original streaming data and the original streaming data Train a plurality of deep learning models corresponding to the plurality of resolutions of the original streaming data using the plurality of streaming data corresponding to, and included in the plurality of electronic devices through the communication module. The image difference degree for the specific domain or the resolution of the original streaming data is obtained from the electronic device, and the plurality of deep learning models corresponding to the plurality of resolutions of the original streaming data are obtained through the communication module. A deep learning model corresponding to the image difference degree for a specific domain or the resolution of the original streaming data may be transmitted to the electronic device.

Claims (20)

In the electronic device,
display;
Communication module;
A processor operatively connected to the display and the communication module; And
And a memory operatively connected to the processor,
The memory, when executed, the processor,
Obtaining first streaming data corresponding to a first resolution of the original streaming data for a first domain from an external electronic device through the communication module,
Selecting a first deep learning model corresponding to the first resolution of the original streaming data from among a plurality of deep learning models corresponding to a plurality of resolutions of the original streaming data,
Converting the first streaming data into first reconstructed streaming data using the first deep learning model, and
An electronic device that stores instructions for displaying the first reconstructed streaming data through the display.
The method of claim 1,
The instructions, the processor,
An electronic device configured to transmit the first restored streaming data to the external electronic device through the communication module.
The method of claim 1,
The first domain includes a plurality of sub-domains corresponding to at least one of a plurality of designated areas or a plurality of designated periods.
The method of claim 1,
The instructions, the processor,
An electronic device configured to select the first deep learning model corresponding to the first resolution based on a user input.
The method of claim 4,
The user input includes a first user input for selecting an image difference degree,
The instructions, the processor,
Based on the first user input, check at least one deep learning model that satisfies the image difference degree among the plurality of deep learning models, and
An electronic device configured to select the first deep learning model corresponding to the first resolution, which is the lowest resolution of the original streaming data from among the at least one deep learning model.
The method of claim 4,
The user input includes a second user input for selecting a network delay,
The instructions, the processor,
Based on the second user input, confirming the first resolution of the original streaming data corresponding to the network delay, and
An electronic device configured to select the first deep learning model corresponding to the first resolution.
The method of claim 1,
The instructions, the processor,
An electronic device configured to select the first deep learning model corresponding to the first resolution based on a compression rate corresponding to the plurality of resolutions of the original streaming data and an image difference degree corresponding to the plurality of resolutions.
The method of claim 7,
The instructions, the processor,
Checking the first resolution for deriving a value that satisfies a specified condition among values to which the image difference degree corresponding to the plurality of resolutions is applied with respect to the compression rate corresponding to the plurality of resolutions, and
An electronic device configured to select the first deep learning model corresponding to the first resolution.
The method of claim 7,
The instructions, the processor,
The first corresponding to the first resolution based on the compression rate corresponding to the plurality of resolutions of the original streaming data, the image difference corresponding to the plurality of resolutions, and a network bandwidth of the electronic device. An electronic device that lets you choose a deep learning model.
The method of claim 1,
The first deep learning model corresponds to a resolution range including the first resolution.
In the method of providing streaming data,
Acquiring first streaming data corresponding to a first resolution of the original streaming data for the first domain from an external electronic device through a communication module,
Selecting a first deep learning model corresponding to the first resolution of the original streaming data from among a plurality of deep learning models corresponding to a plurality of resolutions of the original streaming data,
Converting the first streaming data into first reconstructed streaming data using the first deep learning model, and
And displaying the first restored streaming data through a display.
The method of claim 11,
The method of operating an electronic device further comprising transmitting the first restored streaming data to the external electronic device through the communication module.
The method of claim 11,
The first domain is a method of operating an electronic device including a plurality of sub-domains corresponding to at least one of a plurality of designated areas or a plurality of designated periods.
The method of claim 11,
Selecting the first deep learning model corresponding to the first resolution,
And selecting the first deep learning model corresponding to the first resolution based on a user input.
The method of claim 14,
The user input includes a first user input for selecting an image difference degree,
Selecting the first deep learning model corresponding to the first resolution,
Checking at least one deep learning model that satisfies the image difference degree among the plurality of deep learning models based on the first user input, and
And selecting the first deep learning model corresponding to the first resolution, which is the lowest resolution of the original streaming data from among the at least one deep learning model.
The method of claim 14,
The user input includes a second user input for selecting a network delay,
Selecting the first deep learning model corresponding to the first resolution,
Checking the first resolution of the original streaming data corresponding to the network delay based on the second user input, and
And selecting the first deep learning model corresponding to the first resolution.
The method of claim 11,
Selecting the first deep learning model corresponding to the first resolution,
And selecting the first deep learning model corresponding to the first resolution based on a compression rate corresponding to the plurality of resolutions of the original streaming data and an image difference degree corresponding to the plurality of resolutions. How to operate an electronic device.
The method of claim 17,
Selecting the first deep learning model corresponding to the first resolution,
Checking the first resolution for deriving a value that satisfies a specified condition among values to which the image difference degree corresponding to the plurality of resolutions is applied to the compression rate corresponding to the plurality of resolutions, and
And selecting the first deep learning model corresponding to the first resolution.
The method of claim 17,
Selecting the first deep learning model corresponding to the first resolution,
The first corresponding to the first resolution based on the compression rate corresponding to the plurality of resolutions of the original streaming data, the image difference corresponding to the plurality of resolutions, and a network bandwidth of the electronic device. A method of operating an electronic device including an operation of selecting a deep learning model.
In the streaming service providing server,
Communication module;
A processor operatively connected to the communication module; And
And a memory operatively connected to the processor,
The memory, when executed, the processor,
Transmitting a plurality of streaming data corresponding to a plurality of resolutions of the original streaming data for a specific domain to a plurality of electronic devices through the communication module,
Train a plurality of deep learning models corresponding to the plurality of resolutions of the original streaming data using the original streaming data and the plurality of streaming data corresponding to the plurality of resolutions of the original streaming data, and ,
Obtaining an image difference degree for the specific domain or a resolution of the original streaming data from an electronic device included in the plurality of electronic devices through the communication module, and
From the plurality of deep learning models corresponding to the plurality of resolutions of the original streaming data through the communication module, a deep learning model corresponding to the image difference degree for the specific domain or the resolution of the original streaming data A streaming service providing server to transmit to the electronic device.

Images