KR20200114707A - Electronic device and method for processing a streaming application in electronic device - Google Patents

Abstract

Disclosed are an electronic device and a method for processing a streaming application of the electronic device. According to various embodiments of the present invention, an electronic device comprises: a touch screen; a first processor which executes a streaming application and controls streaming data received from a server to be output through the touch screen according to the execution of the streaming application; and a second processor which process user input data inputted through the touch screen to be transmitted to the server using a first protocol data unit (PDU) during execution of the streaming application, and receives the streaming data processed using a second PDU session based on the user input data from the server and provides it to the first processor. Further, various other embodiments of the present invention may be possible.

Description

Electronic devices and methods of processing streaming applications in electronic devices {ELECTRONIC DEVICE AND METHOD FOR PROCESSING A STREAMING APPLICATION IN ELECTRONIC DEVICE}

Various embodiments relate to an electronic device and a method of processing a streaming application in the electronic device.

For example, in the case of executing a service or function requiring large-capacity image processing, there may be a limit to processing within a user terminal owned by each individual. To compensate for this, the service or function may be executed through a cloud computing-based server, and the executed result screen may be transmitted to a user terminal using a streaming technology. In this way, it is possible to use services or functions that require high specifications or high functions through the user terminal.

As described above, a technology that executes a specific service or function according to the request of the user terminal and transmits the result to the user terminal so that various services or functions can be used in the user terminal is called a cloud streaming service.

For example, screen virtualization technologies for virtualizing an application (eg, a PC game) are being provided by executing a program in a cloud server and streaming it to an electronic device without a separate program installation process in the electronic device.

The screen virtualization technology delivers control information inputted by a user through an electronic device to a cloud server, and the cloud server provides streaming data (eg, graphic streaming data) processed based on the control information to the user's electronic device. have.

The screen virtualization technology may be very sensitive to providing a stable network quality (QoS) because it must generate graphic streaming data in the server according to the control information input by the user and then transmit it to the user's electronic device. In addition, the graphic effect (resolution) of the cloud streaming service may be affected by the network transmission speed (bandwidth, throughput).

In various embodiments, in providing a cloud streaming service using a screen virtualization technology, an electronic device capable of seamlessly supporting a high-end streaming service as in 5G communication, and a streaming application processing method in the electronic device may be provided.

The electronic device according to various embodiments of the present disclosure includes a first processor that executes a touch screen and a streaming application, and controls the streaming data received from the server to be output through the touch screen according to the execution of the streaming application, and the streaming application. During execution, user input data input through the touch screen is processed to be transmitted to the server using a first protocol data unit (PDU) session, and processed using a second PDU session based on the transmitted user input data. It may include a second processor that receives the generated streaming data from the server and provides it to the first processor.

An operation method of an electronic device according to various embodiments includes: executing a streaming application in a first processor, receiving user input data for the streaming application through a touch screen, and receiving the user input data in a second processor. 1 An operation of processing to be transmitted to a server using a protocol data unit (PDU) session, an operation of receiving the streaming data processed by the server using a second PDU session based on the user input data from the server, and the reception It may include an operation of displaying the generated streaming data through the touch screen.

According to various embodiments, it is possible to minimize a processing operation for user input data and transmit it to a server through a 5G communication-based URLLC PDU session, thereby minimizing time for generating streaming data through a user input value. In addition, according to various embodiments, QoS of an application (eg, service or game) that supports high resolution and high level of frame per second (FPS) may be guaranteed.

1 is a diagram illustrating a network environment according to various embodiments.
2 is a block diagram illustrating a structure of an electronic device according to various embodiments.
3 is a block diagram illustrating a structure of an electronic device according to various embodiments.
4 is a diagram illustrating PDU types in a 4G communication scheme according to various embodiments.
5 is a diagram illustrating PDU types of a 5G communication scheme according to various embodiments.
6 is a block diagram illustrating a structure of an electronic device according to various embodiments.
7 is a block diagram illustrating a structure of an electronic device according to various embodiments.
8 is a diagram illustrating a processing method between an AP and a CP according to various embodiments.
9 is a diagram for comparing processing methods according to types of applications according to various embodiments.
10 is a flowchart illustrating a method of processing a streaming application in an electronic device according to various embodiments of the present disclosure.
11 is a flowchart illustrating a method of processing a streaming application in an electronic device according to various embodiments of the present disclosure.
12 is a signal flow diagram illustrating a method of processing a streaming application in an electronic device according to various embodiments of the present disclosure.
13 is a block diagram illustrating a program according to various embodiments.

Hereinafter, various embodiments of the present document will be described with reference to the accompanying drawings. The examples and terms used therein are not intended to limit the technology described in this document to a specific embodiment, and should be understood to include various modifications, equivalents, and/or substitutes of the corresponding embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar elements. Singular expressions may include plural expressions unless the context clearly indicates otherwise.

In various embodiments to be described later, the input device may include a touch panel included in the touch screen or various sensors. In embodiments to be described later, a touch screen panel (TSP) or a touch screen display may be understood as the same term as a touch screen or a concept included in the touch screen. The display device may include at least one display. In various embodiments to be described later, the touch screen may include a touch panel and a display panel, and the touch panel may function as an input device, and the display panel may function as a display device. The input device or display device of various embodiments to be described later is not limited to a touch screen (or a touch screen panel), and any device that receives a user input from an electronic device may be replaced with an input device. Any device that displays may be replaced with a display device.

In various embodiments to be described later, a touch event through a touch screen (or touch screen panel) is described as an example of an input event, but various input events other than the touch event may be processed in the same or similar method. For example, through various sensors (e.g., a gyro sensor, a gravity sensor, a geomagnetic sensor, an image sensor), input data about a user's motion can be processed as an input event, and an external input device (e.g., a joystick or VR (virtual reality)) can be processed. Input data input through a wired/wireless interface between) an input device) and an electronic device may be processed as an input event.

1 is a block diagram of an electronic device 101 in a network environment 100 according to various embodiments. Referring to FIG. 1, in a network environment 100, the electronic device 101 communicates with the electronic device 102 through a first network 198 (for example, a short-range wireless communication network), or a second network 199 It is possible to communicate with the electronic device 104 or the server 108 through (eg, a long-distance wireless communication network). According to an embodiment, the electronic device 101 may communicate with the electronic device 104 through the server 108. According to an embodiment, the electronic device 101 includes a processor 120, a memory 130, an input device 150, an audio output device 155, a display device 160, an audio module 170, and a sensor module ( 176, interface 177, haptic module 179, camera module 180, power management module 188, battery 189, communication module 190, subscriber identification module 196, or antenna module 197 ) Can be included. In some embodiments, at least one of these components (eg, the display device 160 or the camera module 180) may be omitted or one or more other components may be added to the electronic device 101. In some embodiments, some of these components may be implemented as one integrated circuit. For example, the sensor module 176 (eg, a fingerprint sensor, an iris sensor, or an illuminance sensor) may be implemented while being embedded in the display device 160 (eg, a display).

The processor 120, for example, executes software (eg, a program 140) to implement at least one other component (eg, a hardware or software component) of the electronic device 101 connected to the processor 120. It can be controlled and can perform various data processing or operations. According to an embodiment, as at least part of data processing or operation, the processor 120 may store commands or data received from other components (eg, the sensor module 176 or the communication module 190) to the volatile memory 132. The command or data stored in the volatile memory 132 may be processed, and result data may be stored in the nonvolatile memory 134. According to an embodiment, the processor 120 includes a main processor 121 (eg, a central processing unit or an application processor), and a secondary processor 123 (eg, a graphic processing unit, an image signal processor) that can be operated independently or together with the main processor 121 (eg, a central processing unit or an application processor). , A sensor hub processor, or a communication processor). Additionally or alternatively, the coprocessor 123 may be set to use lower power than the main processor 121 or to be specialized for a designated function. The secondary processor 123 may be implemented separately from the main processor 121 or as a part thereof.

The coprocessor 123 is, for example, on behalf of the main processor 121 while the main processor 121 is in an inactive (eg, sleep) state, or the main processor 121 is active (eg, an application is executed). ) While in the state, together with the main processor 121, at least one of the components of the electronic device 101 (for example, the display device 160, the sensor module 176, or the communication module 190) It is possible to control at least some of the functions or states related to. According to an embodiment, the coprocessor 123 (eg, an image signal processor or a communication processor) may be implemented as part of another functionally related component (eg, the camera module 180 or the communication module 190). have.

The memory 130 may store various data used by at least one component of the electronic device 101 (eg, the processor 120 or the sensor module 176). The data may include, for example, software (eg, the program 140) and input data or output data for commands related thereto. The memory 130 may include a volatile memory 132 or a nonvolatile memory 134.

The program 140 may be stored as software in the memory 130, and may include, for example, an operating system 142, middleware 144, or an application 146.

The input device 150 may receive a command or data to be used for a component of the electronic device 101 (eg, the processor 120) from an outside (eg, a user) of the electronic device 101. The input device 150 may include, for example, a microphone, a mouse, a keyboard, or a digital pen (eg, a stylus pen).

The sound output device 155 may output an sound signal to the outside of the electronic device 101. The sound output device 155 may include, for example, a speaker or a receiver. The speaker can be used for general purposes such as multimedia playback or recording playback, and the receiver can be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part of it.

The display device 160 may visually provide information to the outside of the electronic device 101 (eg, a user). The display device 160 may include, for example, a display, a hologram device, or a projector and a control circuit for controlling the device. According to an embodiment, the display device 160 may include a touch circuitry set to sense a touch, or a sensor circuit (eg, a pressure sensor) set to measure the strength of a force generated by the touch. have.

The audio module 170 may convert sound into an electric signal or, conversely, convert an electric signal into sound. According to an embodiment, the audio module 170 acquires sound through the input device 150, the sound output device 155, or an external electronic device (for example, an external electronic device directly or wirelessly connected to the electronic device 101). Sound may be output through the electronic device 102) (for example, a speaker or headphones).

The sensor module 176 detects an operating state (eg, power or temperature) of the electronic device 101, or an external environmental state (eg, a user state), and generates an electrical signal or data value corresponding to the detected state. can do. According to an embodiment, the sensor module 176 may include, for example, a gesture sensor, a gyro sensor, an atmospheric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, It may include a temperature sensor, a humidity sensor, or an illuminance sensor.

The interface 177 may support one or more designated protocols that may be used for the electronic device 101 to directly or wirelessly connect with an external electronic device (eg, the electronic device 102 ). According to an embodiment, the interface 177 may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

The connection terminal 178 may include a connector through which the electronic device 101 can be physically connected to an external electronic device (eg, the electronic device 102 ). According to an embodiment, the connection terminal 178 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (eg, a headphone connector).

The haptic module 179 may convert an electrical signal into a mechanical stimulus (eg, vibration or movement) or an electrical stimulus that a user can perceive through a tactile or motor sense. According to an embodiment, the haptic module 179 may include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

The camera module 180 may capture a still image and a video. According to an embodiment, the camera module 180 may include one or more lenses, image sensors, image signal processors, or flashes.

The power management module 188 may manage power supplied to the electronic device 101. According to an embodiment, the power management module 388 may be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

The battery 189 may supply power to at least one component of the electronic device 101. According to an embodiment, the battery 189 may include, for example, a non-rechargeable primary cell, a rechargeable secondary cell, or a fuel cell.

The communication module 190 is a direct (eg, wired) communication channel or a wireless communication channel between the electronic device 101 and an external electronic device (eg, electronic device 102, electronic device 104, or server 108). It is possible to support establishment and communication through the established communication channel. The communication module 190 operates independently of the processor 120 (eg, an application processor), and may include one or more communication processors that support direct (eg, wired) communication or wireless communication. According to an embodiment, the communication module 190 is a wireless communication module 192 (eg, a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 194 (eg : A LAN (local area network) communication module, or a power line communication module) may be included. Among these communication modules, a corresponding communication module is a first network 198 (for example, a short-range communication network such as Bluetooth, WiFi direct or IrDA (infrared data association)) or a second network 199 (for example, a cellular network, the Internet, or It can communicate with external electronic devices through a computer network (for example, a telecommunication network such as a LAN or WAN). These various types of communication modules may be integrated into one component (eg, a single chip), or may be implemented as a plurality of separate components (eg, multiple chips). The wireless communication module 192 uses subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module 196 in a communication network such as the first network 198 or the second network 199. The electronic device 101 can be checked and authenticated.

The antenna module 197 may transmit a signal or power to the outside (eg, an external electronic device) or receive from the outside. According to an embodiment, the antenna module may include one antenna including a conductor formed on a substrate (eg, a PCB) or a radiator formed of a conductive pattern. According to an embodiment, the antenna module 197 may include a plurality of antennas. In this case, at least one antenna suitable for a communication method used in a communication network such as the first network 198 or the second network 199 is, for example, provided by the communication module 190 from the plurality of antennas. Can be chosen. The signal or power may be transmitted or received between the communication module 190 and an external electronic device through the at least one selected antenna. According to some embodiments, other components (eg, RFIC) other than the radiator may be additionally formed as part of the antenna module 197.

At least some of the components are connected to each other through a communication method (e.g., a bus, general purpose input and output (GPIO), serial peripheral interface (SPI), or mobile industry processor interface (MIPI))) between peripheral devices and signals ( E.g. commands or data) can be exchanged with each other.

According to an embodiment, commands or data may be transmitted or received between the electronic device 101 and the external electronic device 104 through the server 108 connected to the second network 199. Each of the electronic devices 102 and 104 may be a device of the same or different type as the electronic device 101. According to an embodiment, all or part of the operations executed by the electronic device 101 may be executed by one or more of the external electronic devices 102, 104, or 108. For example, when the electronic device 101 needs to perform a function or service automatically or in response to a request from a user or another device, the electronic device 101 does not execute the function or service by itself. In addition or in addition, it is possible to request one or more external electronic devices to perform the function or at least part of the service. One or more external electronic devices receiving the request may execute at least a part of the requested function or service, or an additional function or service related to the request, and transmit the execution result to the electronic device 101. The electronic device 101 may process the result as it is or additionally and provide it as at least a part of a response to the request. To this end, for example, cloud computing, distributed computing, or client-server computing technology may be used.

Electronic devices according to various embodiments disclosed in this document may be devices of various types. The electronic device may include, for example, a portable communication device (eg, a smartphone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a home appliance. The electronic device according to the embodiment of the present document is not limited to the above-described devices.

Various embodiments of the present document and terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutes of the corresponding embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or a plurality of the items unless clearly indicated otherwise in a related context. In this document, “A or B”, “at least one of A and B”, “at least one of A or B,” “A, B or C,” “at least one of A, B and C,” and “A Each of phrases such as "at least one of, B, or C" may include any one of the items listed together in the corresponding one of the phrases, or all possible combinations thereof. Terms such as "first", "second", or "first" or "second" may be used simply to distinguish the component from other corresponding components, and the components may be referred to in other aspects (eg, importance or Order) is not limited. Some (eg, a first) component is referred to as “coupled” or “connected” with or without the terms “functionally” or “communicatively” to another (eg, second) component. When mentioned, it means that any of the above components can be connected to the other components directly (eg by wire), wirelessly, or via a third component.

The term "module" used in this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic blocks, parts, or circuits. The module may be an integrally configured component or a minimum unit of the component or a part thereof that performs one or more functions. For example, according to an embodiment, the module may be implemented in the form of an application-specific integrated circuit (ASIC).

Various embodiments of the present document include one or more stored in a storage medium (eg, internal memory 136 or external memory 138) readable by a machine (eg, electronic device 101). It may be implemented as software (eg, program 140) including instructions. For example, the processor (eg, the processor 120) of the device (eg, the electronic device 101) may call and execute at least one command among one or more commands stored from a storage medium. This makes it possible for the device to be operated to perform at least one function according to the at least one command invoked. The one or more instructions may include code generated by a compiler or code executable by an interpreter. A storage medium that can be read by a device may be provided in the form of a non-transitory storage medium. Here,'non-transient' only means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic wave), and this term refers to the case where data is semi-permanently stored in the storage medium. It does not distinguish between temporary storage cases.

According to an embodiment, a method according to various embodiments disclosed in the present document may be provided by being included in a computer program product. Computer program products can be traded between sellers and buyers as commodities. The computer program product is distributed in the form of a device-readable storage medium (e.g. compact disc read only memory (CD-ROM)), or through an application store (e.g. Play Store ^TM ) or two user devices ( It can be distributed (e.g., downloaded or uploaded) directly between, e.g. smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a storage medium that can be read by a device such as a server of a manufacturer, a server of an application store, or a memory of a relay server.

According to various embodiments, each component (eg, module or program) of the above-described components may include a singular number or a plurality of entities. According to various embodiments, one or more components or operations among the above-described corresponding components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (eg, a module or a program) may be integrated into one component. In this case, the integrated component may perform one or more functions of each component of the plurality of components in the same or similar to that performed by the corresponding component among the plurality of components prior to the integration. . According to various embodiments, operations performed by a module, program, or other component are sequentially, parallel, repeatedly, or heuristically executed, or one or more of the above operations are executed in a different order or omitted. Or one or more other actions may be added.

In the following embodiments, a streaming application is described as an example of an application, but the present disclosure is not limited to a specific application. For example, various applications such as game applications, video applications, and virtual reality applications may be applied. In addition, in the embodiments to be described later, detailed embodiments according to the 4G or 5G communication protocol are described, but the present disclosure is not limited to the communication protocol.

2 and 3 are block diagrams illustrating a structure of an electronic device according to various embodiments. 2 and 3, according to various embodiments, the electronic device 101 may include some or all of the electronic device 101 shown in FIG. 1. The electronic device 101 includes a processor 210 (eg, the processor 120 of FIG. 1), a touch screen panel (TSP) 240 (eg, the input device 150 of FIG. 1), or a display device ( 160)), sensor 250 (for example, the sensor module 176 in FIG. 1), communication module 260 (for example, the communication module 190 in FIG. 1), and memory 270 (for example, the memory of FIG. (130)). The processor 210 may include at least one processor, and communicates with, for example, an application processor (AP) 220 (eg, the main processor 121 in FIG. 1) and the AP 220 A communication processor (CP) (for example, the coprocessor 123 of FIG. 1) 230 may be included to perform a function. The AP 220, CP 230, touch screen panel 240, sensor 250, communication module 260, and memory 270 included in the electronic device 101 are electrically and/or operatively They can be connected to each other to exchange signals (eg commands or data) with each other.

According to various embodiments, the AP 220 or CP 230 of the electronic device 101 is a network 280 (eg, the first network 198 or the second network 199 in FIG. 1) or other configuration It is possible to communicate with a server 290 (eg, server 108 of FIG. 1) via an element (eg, communication module 190, interface 177, or connection terminal 178 of FIG. 1 ).

The AP 220 includes, for example, components of the electronic device 101 including the CP 230, the touch screen panel 240, the sensor 250, and the communication module 260, and the electronic device 101 using them. You can control the overall operation of The AP 220 may control access to the network 280 through the CP 230 or communication through the network 280. According to various embodiments, the AP 220 and the CP 230 may be manufactured as separate chips or may be physically manufactured as a single chip.

According to various embodiments, the CP 230 may perform a communication function in conjunction with the AP 220. The CP 230 may access the network 280 or communicate with an external electronic device (eg, the server 290 or a base station or an upper node) through the network 280. CP 230 is for access to the network 280 or communication through the network 280, the AP 220 or other components (for example, the communication module 190 of Figure 1, the interface 177, a connection terminal ( 178) or a combination thereof). According to various embodiments, the touch screen panel 240 may include a display panel and a touch panel as the same as or included in the touch screen, and the display panel is a display device 160, and the touch panel is an input device. Can function as 150.

According to various embodiments, the AP 220 is a TSP driver 221, a sensor driver 222, an input framework 223, a streaming app 224, a connectivity manager 225, an AP/CP interface for software processing. 226, may include a display I/F (interface) 227. According to various embodiments, the CP 230 may include an AP/CP interface 231, a protocol stack 232, and an RF driver 233 for software processing.

According to various embodiments, when a specific application (eg, a streaming application (hereinafter, referred to as'streaming app 224' for convenience)) is executed in the AP 220, the streaming app 224 is the server 290 The streaming data received through) may be provided to the touch screen panel 240 through the display I/F 227. The touch screen panel 240 may display the received streaming data on the screen.

While the streaming app is being executed, touch input data input through the touch screen panel 240 may be transmitted to the input framework 223 through the TSP driver 221. The input framework 223 may generate a touch event based on coordinate information included in the transmitted touch input data, and may transmit the generated touch event to the streaming app 224. The touch event may include various gestures related to a touch such as a touch at a specific point, a long touch, a force touch, and a drag. The input framework 223 may transmit the generated touch event to the streaming app 224.

The streaming app 224 may transmit a touch event including the coordinate information to the CP 230 through the connectivity manager 225 and the AP/CP interface 226. For example, the streaming app 224 transmits the touch event to the CP 230 through transmission control protocol (TCP) or user datagram protocol (UDP) communication based on the Internet PDN (protocol data network) IP (internet protocol). Can be passed on.

According to various embodiments, the CP 230 may receive a touch event from the AP 220 through the AP/CP interface 231 and transmit the received touch event to the protocol stack 232. The protocol stack 232 may transmit the touch event to the server 290 through the network 280 through the RF driver 233.

According to various embodiments, the server 290 generates streaming data (eg, image data) based on a touch event received from the electronic device 101, and transmits the generated data to an electronic device through the network 280. It can be transmitted to the device 101.

According to various embodiments, the electronic device 101 may receive the streaming data through the communication module 260 and provide it to the CP 230. The CP 230 may transmit the received streaming data to the AP 220 through the AP/CP interfaces 231 and 226 through the protocol stack 232. The AP 220 may check the IP and port number of the received streaming data and provide it to a related app (eg, streaming app 224). The streaming app 224 provides the received streaming data to the touch screen panel 240 through the display I/F 227, so that a screen corresponding to the streaming data may be displayed on the touch screen.

4 is a diagram illustrating PDU types of a 4G communication method according to various embodiments, and FIG. 5 is a diagram illustrating a PDU type of a 5G communication method according to various embodiments. Referring to FIG. 4, in the electronic device of the 4G communication method 400, an IP multimedia subsystem packet data unit (IMS PDU) session 410, which is a communication interface related to a service of a communication service provider, and a communication interface related to other services or applications. Data may be transmitted and received through an Internet PDU session 420. For example, the third app can transmit and receive data as a first PDU through the IMS PDU session 410, and the first app, the second app, and the fourth app are second PDUs, and data through the Internet PDU session 420 Can send and receive.

According to various embodiments, referring to FIG. 5, in an electronic device of a 5G communication scheme 500, an IMS PDU session 510, an enhanced mobile broadband (eMBB) PDU session 520 according to characteristics of a service or application, Multiple, such as URLLC (Utra-reliable and low latency communication) PDU session 530, mMTC (massive machine-type communications, hyper connection) PDU session 540, LADN (local area data network) PDU session 550 PDU session can be operated. For example, in the case of eMBB, it is a method that can be used in a service or application that requires a high throughput such as an Internet PDN in an existing 4G communication-based terminal, and in the case of the URLLC PDU session 530, low-latency ) Can be used in required services or applications.

According to various embodiments, in the 5G communication method 500, each PDU session 510, 520, 530, 540, 550 may be operated with a different IP (internet protocol) address, as shown in FIG. Likewise, a specific application may communicate with the server through a plurality of PDU sessions according to the operation requirements of the service or application. Each of the plurality of PDU sessions may have different source IP addresses. For example, the source IP address of the first PDU may be different from the source IP address of the second PDU. According to various embodiments, the URLLC PDU session 530 may have a lower latency characteristic than the eMBB PDU session 520, and the eMBB PDU session 520 has an ultra-wideband characteristic than the URLLC PDU session 530. Can have. The mMTC PDU session 540 may target hyper-connectivity for the purpose of smooth communication between very many devices such as a smart city and a smart building. The LADN PDU session 550 may connect a plurality of devices distributedly installed in a specific limited area to each other to communicate data with each other at high speed.

6 and 7 are block diagrams illustrating a structure of an electronic device according to various embodiments. 6 and 7, according to various embodiments, the electronic device 101 may include some or all of the electronic device 101 illustrated in FIG. 1. The electronic device 101 includes a processor 610 (eg, the processor 120 of FIG. 1), a touch screen panel (TSP) (eg, the input device 150 or the display device 160 of FIG. 1). 640, sensor 650 (for example, sensor module 176 in FIG. 1), communication module 660 (for example, communication module 190 in FIG. 1), memory 670 (for example, memory in FIG. (130)). The processor 610 may include at least one processor, and communicates with, for example, an application processor (AP) 620 (eg, the main processor 121 in FIG. 1) and the AP 620 A communication processor (CP) (for example, the coprocessor 123 of FIG. 1) 630 may be included to perform a function. The AP 620, the CP 630, the touch screen panel 640, the sensor 650, the communication module 660, and the memory 670 included in the electronic device 101 are electrically and/or operatively They can be connected to each other to exchange signals (eg commands or data) with each other.

According to various embodiments, the AP 620 or CP 630 of the electronic device 101 is a network 680 (eg, the first network 198 or the second network 199 in FIG. 1) or other configuration It is possible to communicate with a server 690 (eg, server 108 of FIG. 1) via an element (eg, communication module 190, interface 177, or connection terminal 178 of FIG. 1 ).

The AP 620 includes, for example, components of the electronic device 101 including the CP 630, the touch screen panel 640, the sensor 650, and the communication module 660, and the electronic device 101 using them. You can control the overall operation of The AP 620 may control access to the network 680 through the CP 630 or communication through the network 680. According to various embodiments, the AP 620 and the CP 630 may be manufactured as separate chips or may be physically manufactured as one chip.

According to various embodiments, the CP 630 may perform a communication function in conjunction with the AP 620. The CP 630 may access the network 680 or communicate with an external electronic device (eg, the server 690 or a base station or an upper node) through the network 680. CP 630 is for access to the network 680 or communication through the network 680, the AP 620 or other components (for example, the communication module 190 of Figure 1, the interface 177, a connection terminal ( 178) or a combination thereof). According to various embodiments, the touch screen panel 640 may include a display panel and a touch panel as the same as or included in the touch screen, and the display panel is the display device 160 and the touch panel is an input device. Can function as 150.

According to various embodiments, the AP 620 includes a first TSP driver 621, a first sensor driver 622, an input framework 623, a streaming app 624, a connectivity manager 625, for software processing. It may include an AP/CP interface 626 and a display I/F 627. According to various embodiments, the CP 630 includes an AP/CP interface 631, a protocol stack 632, an RF driver 633, a second TSP driver 635, and a second sensor driver 636 for software processing. It may include.

According to various embodiments, when a specific application (eg, a streaming application (hereinafter, referred to as “streaming app 624” for convenience)) is executed in the AP 620, the streaming app 624 is the server 690 The streaming data received through) may be provided to the touch screen panel 640 through the display I/F 627. The touch screen panel 640 may display the received streaming data on the screen.

According to various embodiments, touch input data input through the touch screen panel 640 while the streaming app is being executed may be transmitted to the input processing module 634 through the second TSP driver 635. According to various embodiments, data input through the sensor 650 while the streaming app is being executed may be transmitted to the input processing module 634 through the second sensor driver 636. The input processing module 634 transfers user input data transmitted through the second TSP driver 635 or the second sensor driver 636 or data converted from the user input data to a first PDU session (e.g., URLLC PDU). Session) through the protocol stack 632. According to various embodiments, the URLLC PDU session may be set up in advance, and the input processing module 634 may perform communication through a URLLC PDU session. The protocol stack 632 may transmit the user input data to the server 690 through the network 680 through the RF driver 633 and the communication module 660.

According to various embodiments, the user input data input through the touch screen panel 640 or the sensor 650 is not processed through the AP 620 but is directly processed by the CP 630, thereby delaying the processing time. Can be reduced. For example, the user input data input through the touch screen panel 640 or the sensor 650 is the input framework 623 through the first TSP driver 621 or the first sensor driver 622 of the AP 620 It may be processed through the input processing module 634 through the second TSP driver 635 of the CP 630 or the second sensor driver 636 without being transmitted to the CP 630.

The input processing module 634 of the CP 630 may perform the same or similar function as the input framework 623 of the AP 620. According to various embodiments, the input processing module 634 generates control data (e.g., a touch event) based on coordinate information included in the transmitted touch input data, and a streaming app 624 It can be transmitted to the server 690 through the protocol stack 632 without going through. The protocol stack 632 may transmit the touch event to the server 690 through the network 680 through the RF driver 633. The touch event may include various gestures related to a touch such as a touch at a specific point, a long touch, a force touch, and a drag.

According to various embodiments, the input processing module 634 may convert the transmitted touch input data into a streaming app or a type of input data requested by the server 690 corresponding to the streaming app. For example, if the type of streaming app currently running is an app such as an internet browser or a video streaming service, it can be converted into data of a type or format requested by the service provider's server and delivered. . According to various embodiments, when the server 690 directly receives the touch input data and processes the touch input data, the input processing module 634 converts the type or format of the transmitted touch input data. It can be delivered to the server 690 without having to do so.

According to various embodiments, the server 690 generates streaming data (eg, image data) based on control data (eg, a touch event) received from the electronic device 101, and transmits the generated data to a network. It may be transmitted to the electronic device 101 through 680. According to various embodiments, the server 690 generates streaming data corresponding to image information based on control data (eg, touch event) received through a URLLC PDU session, and transmits the streaming data through an eMBB PDU session. It can be transmitted to the electronic device 101. The eMBB PDU session may be set up in advance, and the streaming app 624 may perform communication through the eMBB PDU session.

According to various embodiments, the electronic device 101 may receive the streaming data through the communication module 660 and provide it to the CP 630. The CP 630 may transmit the received streaming data to the AP 620 through the AP/CP interfaces 631 and 626 through the protocol stack 632. The AP 620 may check the IP and port number of the received streaming data and provide it to a related app (eg, the streaming app 624). The streaming app 624 provides the received streaming data to the touch screen panel 640 through the display I/F 627, so that a screen corresponding to the streaming data may be displayed on the touch screen.

According to various embodiments, the streaming app 624 is a URLLC PDU session in the input processing module 634 of the CP 630 when a user input (Touch, Sensor) through the touch screen panel 640 or the sensor 650 occurs. It can be delivered to the server 690 through. According to various embodiments, when a touch user input occurs, the touch data corresponding to the X, Y coordinate values (eg, 300, 500) of the user input is transmitted from the touch screen panel 640 to the first TSP driver 621 or 2 It can be obtained through the TSP driver 635. The acquired touch data may be transmitted to the input processing module 634 of the CP 630 as it is, and the input processing module 634 may perform a touch event (e.g., press/release, drag) according to the pattern of the transmitted touch data. , gesture) may be determined and transmitted to the server 690 through a URLLC-based network interface. According to various embodiments, a method of determining a touch event by the input processing module 634 may be the same as or similar to the input framework 623 of the AP 620.

According to various embodiments, the server 690 may generate streaming data (eg, video information) based on the transmitted user input information and transmit it to the streaming app 624 through an eMBB PDU session. . According to various embodiments, the server 690 is electronically operated in a 4G communication base (eg, one IP communication) or a 5G communication base (eg, two IP communications (eMBB, URLLC)). It can communicate with the streaming app 624 of the device 101.

According to various embodiments, when an image processing operation based on user input information in the server 690 is slowed down or a problem occurs in an eMBB session-based network state between the server 690 and the electronic device 101, the electronic device The reception of image information may be slow compared to the user input information input at (101). Accordingly, when the image information reception is slow (throughput), the sampling time of user input information may be adjusted and transmitted to the server 690. For example, when the image information reception speed is reduced by 1/2 compared to the normal speed, the sampling time of user input information is also reduced by 1/2 and transmitted to the server 690 to prevent a framedrop phenomenon.

The electronic device according to any one of various embodiments includes a first processor that executes a touch screen and a streaming application, and controls the streaming data received from the server to be output through the touch screen according to the execution of the streaming application, and the During the execution of the streaming application, user input data input through the touch screen is processed to be transmitted to the server using a first protocol data unit (PDU) session, and a second PDU session is performed based on the transmitted user input data. It may include a second processor that receives the processed streaming data from the server and provides it to the first processor.

According to various embodiments, the first processor may be an application processor, and the second processor may be a communication processor.

According to various embodiments, the first PDU session may have a lower delay characteristic than the second PDU session.

According to various embodiments, the first PDU session may be an ultra-reliable and low latency communication (URLLC) PDU session.

According to various embodiments, the second PDU session may have a higher transmission rate than the first PDU session.

According to various embodiments, the second PDU session may be an enhanced mobile broadband (eMBB) PDU session.

According to various embodiments, the source IP (internet protocol) address of the first PDU may be different from the source IP address of the second PDU.

According to various embodiments, the second processor receives the user input data, determines a touch event based on a pattern of touch data from touch input data included in the received user input data, and determines the determined touch event Control data can be generated based on.

According to various embodiments, the second processor may receive the user input data and transmit touch input data included in the received user input data to the server.

According to various embodiments, the first processor determines whether the running streaming application is an application operating in a foreground, and as a result of the determination, if the application is operating in the foreground, the first processor is included in the user input data. When the touch input data is transmitted to the second processor, and as a result of the determination, if the application is not operating in the foreground, a touch event is determined based on a pattern of touch data from the touch input data included in the user input data, and the Control data may be generated based on the determined touch event.

According to various embodiments, the first processor may determine whether the streaming application being executed is an application operating in a foreground, and transmit the determination result to the second processor.

8 is a diagram illustrating a processing method between an AP and a CP according to various embodiments. 6 and 8, the input framework 623 of the AP 620 checks whether the application currently operating in the foreground is a streaming app in addition to the operation of determining input touch data as a touch event. This fact may be transmitted to the input processing module 634 of the CP 630. According to various embodiments, the input framework 623 may refer to a streaming app-related whitelist stored in the memory 670. As a result of checking, when the streaming app operates in the foreground, the user input information delivery path may be changed so that the server 690 can directly process it as shown.

According to various embodiments, the input framework 623 does not convert the touch or sensor data received from the first TSP driver 621 or the first sensor driver 622 into a touch event or a sensor event, May not be delivered to (624). According to various embodiments, the input framework 623 releases the interrupt handler operation of the first TSP driver 621 or the first sensor driver 622 so that the AP 620 no longer receives an interrupt for a user input. You can avoid receiving it.

According to various embodiments, the touch data or sensor data acquired through the first TSP driver 621 or the first sensor driver 622 is transferred to the second TSP driver 635 or the second sensor driver 635 of the CP 630. ), or the touch event or sensor event value determined by the input framework 623 may be transmitted to the input processing module 634 of the CP 630 without being transmitted to the streaming app 624.

According to various embodiments, the foreground operation status of the streaming app 624 may be transmitted to the input processing module 634 of the CP 630 through the AP/CP interfaces 631 and 626. The input processing module 634 converts the touch data or sensor data received from the TSP drivers 621 and 635 or the sensor drivers 622 and 636 into a touch event or a sensor event, and the conversion The generated value can be directly transferred to the server 690 through the URLLC network interface. According to another embodiment, the input processing module 634 activates an interrupt handler operation to receive user input data through the TSP drivers 621 and 635 or the sensor drivers 622 and 636 to provide touch data or sensor data. Can be delivered. According to various embodiments, the second TSP driver 635 or the second sensor driver 636 of the CP 630 may transmit touch data or sensor data values transmitted from the AP 620 to the streaming app 624. . Alternatively, the input processing module 634 of the CP 630 may directly transmit the touch event or sensor event value received from the input framework 623 of the AP 620 to the server 690 through the URLLC network interface. According to various embodiments, when the streaming app 624 is switched from the foreground to the background, user input information may be changed again so that the AP 620 can process it.

9 is a diagram for comparing processing methods according to types of applications according to various embodiments. Referring to FIG. 9, it is possible to determine whether the first app 910 is a streaming app while operating in the foreground. When the first app 910 is not a streaming app, user input information is processed by the AP 620 Can be. For example, when the first app 910 is not a streaming app, the AP 620 may perform various actions of the first app 910 based on user input information (eg, input of text, selection of an object, movement of a cursor). ) Can be controlled. According to various embodiments, when the app operating in the foreground is changed to the second app 920 and the second app 920 is a streaming app, the user input information is CP 630 as detailed in FIG. 6. Can be handled in

10 is a flowchart illustrating a method of processing a streaming application in an electronic device according to various embodiments of the present disclosure. Referring to FIG. 10, in operation 1010, the electronic device 101 may execute a streaming application in a first processor (eg, the AP 620 of FIG. 6 ).

In operation 1020, the electronic device 101 may receive user input data for the streaming application. For example, when a user touch input is generated through a touch screen panel (eg, the touch screen panel 640 in FIG. 6), the electronic device 101 may use user input data (eg, data including X and Y coordinates). May be processed through an AP (eg, the AP 620 of FIG. 6) or a CP (eg, the CP 630 of FIG. 6 ).

In operation 1030, the electronic device 101 may transmit the user input data to a server (eg, server 690 of FIG. 6) using a first PDU session (eg, URLLC PDU session). In various embodiments, the user input data may be processed by a CP (eg, CP 630 of FIG. 6) and transmitted to a server (eg, server 690 of FIG. 6 ).

In operation 1040, the electronic device 101 may receive the processed streaming data from the server 690 using a second PDU session (eg, eMBB PDU session) based on user input data.

In operation 1050, the electronic device 101 may display the received streaming data.

11 is a flowchart illustrating a method of processing a streaming application in an electronic device according to various embodiments of the present disclosure. Referring to FIG. 11, in operation 1110, the electronic device 101 may check an application currently operating in the foreground by a first processor (eg, the AP 620 of FIG. 6 ).

In operation 1120, as a result of the check, if the application running in the foreground is not a streaming application, user input data may be processed by the first processor in operation 1170.

In operation 1120, as a result of the confirmation, if the application running in the foreground is a streaming application, in operation 1130, the electronic device 101 processes user input data in a second processor (eg, CP 630 in FIG. 6). Can be switched to

In operation 1140, when receiving the user input data, the electronic device 101 may generate control data based on the user input data in the second processor (eg, the CP 630 of FIG. 6) according to the conversion.

In operation 1150, the electronic device 101 may transmit control data processed using a first PDU session (eg, a URLLC PDU session) to a server (eg, the server 690 of FIG. 6).

In operation 1160, the electronic device 101 receives streaming data processed using a second PDU session (eg, eMBB PDU session) based on user input data from the server 690, and uses the second PDU session. Thus, the processed streaming data can be displayed.

12 is a signal flow diagram illustrating a method of processing a streaming application in an electronic device according to various embodiments of the present disclosure. Referring to FIG. 12, in operation 1202, when a user touch input occurs through the touch screen panel 640, the electronic device 101 stores user input data (eg, data including X and Y coordinates) in operation 1204. It can be transmitted to the AP (620). In operation 1206, the AP 620 may process user input data.

According to various embodiments, in operation 1208, when the streaming application is executed, the AP 620 may notify the CP 630 of a change in the input information delivery path in operation 1210.

In operation 1212, when a user touch input is generated through the touch screen panel 640, in operation 1214, the electronic device 101 changes user input data (eg, data including X and Y coordinates) to the input information delivery path. It can be transmitted to the CP (630) according to.

In operation 1216, the CP 630 may generate control data by determining a touch event. In operation 1218, the CP 630 may process the generated control data using a first PDU session (eg, a URLLC PDU session) and then transmit it to the server 690.

In operation 1220, the server 690 may generate streaming data based on the received control data. In operation 1222, the server 690 may process the streaming data using a second PDU session (eg, eMBB PDU session) and transmit the processed data to the electronic device 101.

In operation 1224, the CP 630 may transmit streaming data to the AP 620, and in operation 1226, the AP 620 may transmit screen data to the touch screen panel 640. In operation 1228, the touch screen panel 640 may display a screen.

An operation method of the electronic device 101 according to any one of various embodiments includes an operation of executing a streaming application in a first processor, an operation of receiving user input data for the streaming application through a touch screen, and in a second processor. Processing the user input data to be transmitted to the server using a first protocol data unit (PDU) session, and the server sends streaming data processed using a second PDU session based on the user input data from the server It may include an operation of receiving and an operation of displaying the received streaming data through the touch screen.

According to various embodiments, the first processor may be an application processor, and the second processor may be a communication processor.

According to various embodiments, the first PDU session may have a lower delay characteristic than the second PDU session.

According to various embodiments, the first PDU session may be an ultra-reliable and low latency communication (URLLC) PDU session.

According to various embodiments, the second PDU session may have a higher transmission rate than the first PDU session.

According to various embodiments, the second PDU session may be an enhanced mobile broadband (eMBB) PDU session.

According to various embodiments, the source IP address of the first PDU may be different from the source IP address of the second PDU.

According to various embodiments, the second processor receives the user input data, determines a touch event based on a pattern of touch data from touch input data included in the received user input data, and determines the determined touch event Control data can be generated based on.

According to various embodiments, the second processor may receive the user input data and transmit touch input data included in the received user input data to the server.

13 is a block diagram 1300 illustrating a program 140 according to various embodiments. According to an embodiment, the program 140 includes an operating system 142 for controlling one or more resources of the electronic device 101, a middleware 144, or an application 146 executable in the operating system 142. Can include. The operating system 142 may include, for example, Android ^TM , iOS ^TM , Windows ^TM , Symbian ^TM , Tizen ^TM , or Bada ^TM . At least some of the programs 140 are, for example, preloaded on the electronic device 101 at the time of manufacture, or when used by a user, an external electronic device (eg, electronic device 102 or 104), or a server ( 108)) or can be updated.

The operating system 142 may control management (eg, allocation or collection) of one or more system resources (eg, process, memory, or power) of the electronic device 101. The operating system 142 is, additionally or alternatively, other hardware devices of the electronic device 101, for example, the input device 150, the sound output device 155, the display device 160, the audio module 170 , Sensor module 176, interface 177, haptic module 179, camera module 180, power management module 188, battery 189, communication module 190, subscriber identification module 196, or One or more driver programs for driving the antenna module 197 may be included.

The middleware 144 may provide various functions to the application 146 so that a function or information provided from one or more resources of the electronic device 101 can be used by the application 146. The middleware 144 is, for example, an application manager 1301, a window manager 1303, a multimedia manager 1305, a resource manager 1307, a power manager 1309, a database manager 1311, and a package manager 1313. ), a connectivity manager 1315, a notification manager 1317, a location manager 1319, a graphic manager 1321, a security manager 1323, a call manager 1325, or a voice recognition manager 1327. I can.

The application manager 1301 may manage the life cycle of the application 146, for example. The window manager 1303 may manage one or more GUI resources used on a screen, for example. The multimedia manager 1305, for example, identifies one or more formats required for playback of media files, and performs encoding or decoding of a corresponding media file among the media files by using a codec suitable for the selected corresponding format. Can be done. The resource manager 1307 may manage the source code of the application 146 or a memory space of the memory 130, for example. The power manager 1309 may manage the capacity, temperature, or power of the battery 189, for example, and determine or provide related information necessary for the operation of the electronic device 101 using the corresponding information. . According to an embodiment, the power manager 1309 may interwork with a basic input/output system (BIOS) (not shown) of the electronic device 101.

The database manager 1311 may create, search, or change a database to be used by the application 146, for example. The package manager 1313 may manage installation or update of an application distributed in the form of, for example, a package file. The connectivity manager 1315 may manage a wireless connection or a direct connection between the electronic device 101 and the external electronic device, for example. The notification manager 1317 may provide, for example, a function for notifying the user of the occurrence of a designated event (eg, incoming call, message, or alarm). The location manager 1319 may manage location information of the electronic device 101, for example. The graphic manager 1321 may manage, for example, one or more graphic effects to be provided to a user or a user interface related thereto.

The security manager 1323 may provide system security or user authentication, for example. The telephony manager 1325 may manage a voice call function or a video call function provided by the electronic device 101, for example. The voice recognition manager 1328 transmits, for example, a user's voice data to the server 108, and a command corresponding to a function to be performed in the electronic device 101 based at least in part on the voice data, Alternatively, text data converted based at least in part on the voice data may be received from the server 108. According to an embodiment, the middleware 1344 may dynamically delete some of the existing components or add new components. According to an embodiment, at least a portion of the middleware 144 may be included as a part of the operating system 142 or implemented as separate software different from the operating system 142.

The application 146 is, for example, a home 1351, a dialer 1353, an SMS/MMS 1355, an instant message (IM) 1357, a browser 1359, a camera 1361, and an alarm 1363. , Contact (1365), voice recognition (1367), email (1369), calendar (1371), media player (1373), album (1375), watch (1377), health (1379) (e.g., exercise or blood sugar) Biometric information measurement), or environmental information 1381 (eg, air pressure, humidity, or temperature information measurement) application may be included. According to an embodiment, the application 146 may further include an information exchange application (not shown) capable of supporting information exchange between the electronic device 101 and an external electronic device. The information exchange application may include, for example, a notification relay application configured to deliver specified information (eg, a call, a message, or an alarm) to an external electronic device, or a device management application configured to manage an external electronic device have. The notification relay application, for example, transmits notification information corresponding to a specified event (eg, mail reception) generated by another application (eg, e-mail application 1369) of the electronic device 101 to an external electronic device. I can. Additionally or alternatively, the notification relay application may receive notification information from an external electronic device and provide it to the user of the electronic device 101.

The device management application is, for example, an external electronic device that communicates with the electronic device 101 or some components thereof (for example, the display device 160 or the camera module 180). -Off) or a function (eg, brightness, resolution, or focus of the display device 160 or the camera module 180) may be controlled. The device management application may additionally or alternatively support installation, deletion, or update of an application operating in an external electronic device.

Each of the constituent elements described in various embodiments of the present invention may be composed of one or more components, and the name of the constituent element may vary according to the type of electronic device. In various embodiments, the electronic device may include at least one of the elements described in this document, and some elements may be omitted or additional other elements may be further included. In addition, some of the components of the electronic device according to various embodiments of the present disclosure are combined to form a single entity, so that functions of the corresponding components before the combination may be performed in the same manner.

The term "module" used in this document may mean, for example, a unit including one or a combination of two or more of hardware, software, or firmware. "Module" may be used interchangeably with terms such as, for example, unit, logic, logical block, component, or circuit. The "module" may be the smallest unit of integrally configured parts or a part thereof. The "module" may be a minimum unit or a part of one or more functions. The "module" can be implemented mechanically or electronically. For example, a "module" is one of known or future developed application-specific integrated circuit (ASIC) chips, field-programmable gate arrays (FPGAs), or programmable-logic devices that perform certain operations. It may include at least one.

At least a part of a device (eg, modules or functions thereof) or a method (eg, operations) according to various embodiments is, for example, a computer-readable storage media in the form of a program module. It can be implemented as a command stored in. When the command is executed by a processor (for example, the processor 120), the one or more processors may perform a function corresponding to the command. The computer-readable storage medium may be, for example, the memory 130.

Computer-readable recording media include hard disks, floppy disks, magnetic media (e.g. magnetic tape), optical media (e.g. compact disc read only memory (CD-ROM)), DVD ( digital versatile disc), magnetic-optical media (e.g. floptical disk), hardware device (e.g. read only memory (ROM), random access memory (RAM), or flash memory) In addition, the program instruction may include not only machine language codes such as those produced by a compiler but also high-level language codes that can be executed by a computer using an interpreter. It can be configured to operate as one or more software modules to perform the task, and vice versa.

A module or a program module according to various embodiments may include at least one or more of the above-described components, some of which are omitted, or additional other components. Operations performed by a module, a program module, or other components according to various embodiments may be executed sequentially, in parallel, repeatedly, or in a heuristic manner. Also, some operations may be executed in a different order, omitted, or other operations may be added.

According to various embodiments, in a storage medium storing instructions, the instructions are set to cause the at least one processor to perform at least one operation when executed by at least one processor, and the at least one operation is , An operation of executing a streaming application in the first processor, receiving user input data for the streaming application through a touch screen, the user input data in the second processor using a first protocol data unit (PDU) session Processing to be transmitted to a server, receiving streaming data processed by the server using a second PDU session based on the user input data from the server, and displaying the received streaming data through the touch screen May include actions.

In addition, the embodiments disclosed in this document are presented for description and understanding of the disclosed and technical content, and do not limit the scope of the technology described in this document. Accordingly, the scope of this document should be construed as including all changes or various other embodiments based on the technical idea of this document.

101: electronic device 610: processor
620: AP 621: first TSP driver
622: first sensor driver 623: input framework
624: Streaming App 625: Connectivity Manager
626: AP/CP interface 627: Display I/F
630: CP 631: AP/CP interface
632: protocol stack 633: RF driver
634: input processing module 635: second TCP driver
636: second sensor driver
640: touch screen panel 650: sensor
660: communication module 670: memory
680: network 690: server

Claims (20)

In the electronic device,
touch screen;
A first processor that executes a streaming application and controls the streaming data received from the server to be output through the touch screen according to the execution of the streaming application; And
Processes so that user input data input through the touch screen during execution of the streaming application is transmitted to the server using a first protocol data unit (PDU) session, and a second PDU session based on the transmitted user input data And a second processor for receiving the streaming data processed by using the server from the server and providing it to the first processor.
The method of claim 1,
The first processor is an application processor, and the second processor is a communication processor.
The method of claim 1,
The electronic device, wherein the first PDU session has a lower delay characteristic than the second PDU session.
The method of claim 1,
The first PDU session is a URLLC (Utra-reliable and low latency communication) PDU session.
The method of claim 1,
The electronic device, wherein the second PDU session has a higher transmission rate than the first PDU session.
The method of claim 1,
The second PDU session is an enhanced mobile broadband (eMBB) PDU session.
The method of claim 1,
The electronic device, wherein the source IP (internet protocol) address of the first PDU is different from the source IP address of the second PDU.
The method of claim 1, wherein the second processor,
The electronic device comprising: receiving the user input data, determining a touch event based on a pattern of touch data from the touch input data included in the received user input data, and generating control data based on the determined touch event.
The method of claim 1, wherein the second processor,
Receiving the user input data, and transmitting the touch input data included in the received user input data to the server.
The method of claim 1, wherein the first processor,
Determine whether the running streaming application is an application operating in the foreground,
As a result of the determination, if the application operates in the foreground, touch input data included in the user input data is transferred to the second processor,
As a result of the determination, if the application is not operating in the foreground, determining a touch event based on a pattern of touch data from touch input data included in the user input data, and generating control data based on the determined touch event, Electronic device.
The method of claim 1, wherein the first processor,
Determine whether the running streaming application is an application operating in the foreground,
The electronic device transmitting the determination result to the second processor.
In the method of operating an electronic device,
Executing a streaming application in the first processor;
Receiving user input data for the streaming application through a touch screen;
Processing the user input data to be transmitted to a server using a first protocol data unit (PDU) session by a second processor;
Receiving streaming data processed by the server using a second PDU session based on the user input data from the server; And
And displaying the received streaming data through the touch screen.
The method of claim 12,
The first processor is an application processor, and the second processor is a communication processor.
The method of claim 12,
The method of operating an electronic device, wherein the first PDU session has a lower delay characteristic than the second PDU session.
The method of claim 12,
The first PDU session is a URLLC (Utra-reliable and low latency communication) PDU session.
The method of claim 12,
The method of operating an electronic device, wherein the second PDU session has a higher transmission rate than the first PDU session.
The method of claim 12,
The second PDU session is an enhanced mobile broadband (eMBB) PDU session, a method of operating an electronic device.
The method of claim 12,
A method of operating an electronic device, wherein the source IP address of the first PDU is different from the source IP address of the second PDU.
The method of claim 12, wherein the second processor,
The electronic device receives the user input data, determines a touch event based on a pattern of touch data from the touch input data included in the received user input data, and generates control data based on the determined touch event. How it works.
The method of claim 12, wherein the second processor,
A method of operating an electronic device, receiving the user input data and transmitting touch input data included in the received user input data to the server.

Images