WO2024034851A1

WO2024034851A1 - Electronic device for displaying screen on external electronic device, and operating method therefor

Info

Publication number: WO2024034851A1
Application number: PCT/KR2023/009048
Authority: WO
Inventors: 정병석; 손보균; 조지현; 김상헌; 임연욱
Original assignee: 삼성전자 주식회사
Priority date: 2022-08-09
Filing date: 2023-06-28
Publication date: 2024-02-15

Abstract

An electronic device according to one embodiment of the present disclosure may comprise a display, a communication interface, a memory and at least one processor operatively connected to the display, the communication interface and the memory. The at least one processor can be configured to: confirm a connection to an external electronic device through the communication interface on the basis of the execution of an application; generate a virtual display on the basis of attributes related to the display; display, on the basis of the attributes of the generated virtual display, on the generated virtual display, a screen at least partially different from a display screen of the display; and transmit, through the communication interface, to the external electronic device, display data related to the screen displayed on the virtual display.

Description

Electronic device that displays a screen on an external electronic device and method of operating the same

This disclosure relates to a technology for displaying an application screen on an external electronic device through a virtual display in the electronic device.

With the development of wireless communication technology and processor technology, portable electronic devices (hereinafter referred to as electronic devices), such as smartphones, can run various applications such as phone calls, galleries, or games, and provide application screens through the display of the electronic device. You can.

Electronic devices may have relatively small displays to make them portable to users. Accordingly, the electronic device may be equipped with a mirroring function that transmits the application screen executed on the electronic device to an external device with a larger display and provides the application screen on the display of the external device. In addition, in addition to simply transmitting the execution screen of the application to the display of an external device, it provides a user interface for the application of the electronic device in a PC (personal computer) environment and provides functions such as running the application of the electronic device with user input in the PC environment. Services that control operations are also being developed.

An electronic device according to an embodiment of the present disclosure may include a display, a communication interface, a memory, and at least one processor operatively connected to the display, the communication interface, and the memory. The at least one processor may be set to check a connection with an external electronic device through the communication interface based on execution of an application. The at least one processor may be set to generate a virtual display based on attributes related to the display. The at least one processor may be set to display a screen on the generated virtual display that is at least partially different from the display screen of the display, based on properties of the generated virtual display. The at least one processor may be set to transmit display data related to a screen displayed on the virtual display to the external electronic device through the communication interface.

A method of operating an electronic device according to an embodiment of the present disclosure may include confirming a connection with an external electronic device through a communication interface based on execution of an application. A method of operating an electronic device according to an embodiment may include generating a virtual display based on display-related properties of the electronic device. A method of operating an electronic device according to an embodiment may include displaying a screen that is at least partially different from the display screen of the display on the generated virtual display, based on the properties of the generated virtual display. there is. A method of operating an electronic device according to an embodiment may include transmitting display data related to a screen displayed on the virtual display to the external electronic device through the communication interface.

A non-transitory computer-readable storage medium storing one or more programs according to an embodiment of the present disclosure may include an operation of confirming a connection with an external electronic device through a communication interface based on execution of an application. The storage medium may include an operation of generating a virtual display based on display-related properties of the electronic device. The storage medium may include an operation of displaying a screen that is at least partially different from a display screen of the display on the generated virtual display, based on properties of the generated virtual display. The storage medium may include an operation of transmitting display data related to a screen displayed on the virtual display to the external electronic device through the communication interface.

1 is a block diagram of an electronic device in a network environment, according to embodiments of the present disclosure.

Figure 2 is a configuration diagram of an electronic device according to an embodiment of the present disclosure.

Figure 3 is a configuration diagram of an external electronic device according to an embodiment of the present disclosure.

FIG. 4 illustrates a state in which a screen displayed on a virtual display generated by an electronic device is displayed on an external electronic device according to an embodiment of the present disclosure.

FIG. 5 illustrates a connection between a first display of an electronic device and a second display of an external electronic device through a virtual display according to an embodiment of the present disclosure.

FIG. 6 illustrates a flowchart for generating a virtual display of an electronic device (eg, processor) according to an embodiment of the present disclosure.

FIG. 7 illustrates a screen for connecting to an external electronic device according to execution of an application on an electronic device according to an embodiment of the present disclosure.

FIG. 8 illustrates a screen of the first display when connected to an external electronic device according to an embodiment of the present disclosure.

FIG. 9 illustrates a screen of a first display on which a cast view is displayed according to an embodiment of the present disclosure.

FIG. 10A illustrates an operation of hiding or displaying a cast view on the first display according to a user's input according to an embodiment of the present disclosure.

FIG. 10B illustrates an operation of hiding the cast view from the first display by displaying a keyboard according to an embodiment of the present disclosure.

FIG. 11 illustrates a screen related to detailed settings of an indicator displayed on the first display according to an embodiment of the present disclosure.

FIG. 12 illustrates a button for executing or setting an application related to transmitting display data to an external electronic device displayed on an executable application running on an electronic device according to an embodiment of the present disclosure.

FIG. 13 illustrates a screen of an electronic device in the tutorial mode of an application according to an embodiment of the present disclosure.

FIG. 14A illustrates a screen of an electronic device when moving to a virtual display of an executing application that is not supported by the application according to an embodiment of the present disclosure.

FIG. 14B illustrates a guide screen for executable applications that are not supported by the application according to an embodiment of the present disclosure.

Figure 15 shows a perspective view of the folding of an electronic device according to an embodiment of the present disclosure.

16A, 16B, and 16C illustrate screens of a virtual display and a second display according to changes to various states of an electronic device according to an embodiment of the present disclosure.

17 is a flowchart of a method of operating an electronic device according to various embodiments of the present disclosure.

1 is a block diagram of an electronic device 101 in a network environment 100, according to embodiments of the present disclosure. Referring to FIG. 1, in the network environment 100, the electronic device 101 communicates with the electronic device 102 through a first network 198 (e.g., a short-range wireless communication network) or a second network 199. It is possible to communicate with at least one of the electronic device 104 or the server 108 through (e.g., a long-distance wireless communication network). According to one embodiment, the electronic device 101 may communicate with the electronic device 104 through the server 108. According to one embodiment, the electronic device 101 includes a processor 120, a memory 130, an input module 150, an audio output module 155, a display module 160, an audio module 170, and a sensor module ( 176), interface 177, connection terminal 178, haptic module 179, camera module 180, power management module 188, battery 189, communication module 190, subscriber identification module 196 , or may include an antenna module 197. In some embodiments, at least one of these components (eg, the connection terminal 178) may be omitted or one or more other components may be added to the electronic device 101. In some embodiments, some of these components (e.g., sensor module 176, camera module 180, or antenna module 197) are integrated into one component (e.g., display module 160). It can be.

The processor 120, for example, executes software (e.g., program 140) to operate at least one other component (e.g., hardware or software component) of the electronic device 101 connected to the processor 120. It can be controlled and various data processing or calculations can be performed. According to one embodiment, as at least part of data processing or computation, the processor 120 stores commands or data received from another component (e.g., sensor module 176 or communication module 190) in volatile memory 132. The commands or data stored in the volatile memory 132 can be processed, and the resulting data can be stored in the non-volatile memory 134. According to one embodiment, the processor 120 includes a main processor 121 (e.g., a central processing unit or an application processor) or an auxiliary processor 123 that can operate independently or together (e.g., a graphics processing unit, a neural network processing unit ( It may include a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor). For example, if the electronic device 101 includes a main processor 121 and a secondary processor 123, the secondary processor 123 may be set to use lower power than the main processor 121 or be specialized for a designated function. You can. The auxiliary processor 123 may be implemented separately from the main processor 121 or as part of it.

The auxiliary processor 123 may, for example, act on behalf of the main processor 121 while the main processor 121 is in an inactive (e.g., sleep) state, or while the main processor 121 is in an active (e.g., application execution) state. ), together with the main processor 121, at least one of the components of the electronic device 101 (e.g., the display module 160, the sensor module 176, or the communication module 190) At least some of the functions or states related to can be controlled. According to one embodiment, co-processor 123 (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module 180 or communication module 190). there is. According to one embodiment, the auxiliary processor 123 (eg, neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. Artificial intelligence models can be created through machine learning. For example, such learning may be performed in the electronic device 101 itself on which the artificial intelligence model is performed, or may be performed through a separate server (e.g., server 108). Learning algorithms may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but It is not limited. An artificial intelligence model may include multiple artificial neural network layers. Artificial neural networks include deep neural network (DNN), convolutional neural network (CNN), recurrent neural network (RNN), restricted boltzmann machine (RBM), belief deep network (DBN), bidirectional recurrent deep neural network (BRDNN), It may be one of deep Q-networks or a combination of two or more of the above, but is not limited to the examples described above. In addition to hardware structures, artificial intelligence models may additionally or alternatively include software structures.

The memory 130 may store various data used by at least one component (eg, the processor 120 or the sensor module 176) of the electronic device 101. Data may include, for example, input data or output data for software (e.g., program 140) and instructions related thereto. Memory 130 may include volatile memory 132 or non-volatile memory 134.

The program 140 may be stored as software in the memory 130 and may include, for example, an operating system 142, middleware 144, or application 146.

The input module 150 may receive commands or data to be used in a component of the electronic device 101 (e.g., the processor 120) from outside the electronic device 101 (e.g., a user). The input module 150 may include, for example, a microphone, mouse, keyboard, keys (eg, buttons), or digital pen (eg, stylus pen).

The sound output module 155 may output sound signals to the outside of the electronic device 101. The sound output module 155 may include, for example, a speaker or a receiver. Speakers can be used for general purposes such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part of it.

The display module 160 can visually provide information to the outside of the electronic device 101 (eg, a user). The display module 160 may include, for example, a display, a hologram device, or a projector, and a control circuit for controlling the device. According to one embodiment, the display module 160 may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of force generated by the touch.

The audio module 170 can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module 170 acquires sound through the input module 150, the sound output module 155, or an external electronic device (e.g., directly or wirelessly connected to the electronic device 101). Sound may be output through the electronic device 102 (e.g., speaker or headphone).

The sensor module 176 detects the operating state (e.g., power or temperature) of the electronic device 101 or the external environmental state (e.g., user state) and generates an electrical signal or data value corresponding to the detected state. can do. According to one embodiment, the sensor module 176 includes, for example, a gesture sensor, a gyro sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, It may include a temperature sensor, humidity sensor, or light sensor.

The interface 177 may support one or more designated protocols that can be used to connect the electronic device 101 directly or wirelessly with an external electronic device (eg, the electronic device 102). According to one embodiment, the interface 177 may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

The connection terminal 178 may include a connector through which the electronic device 101 can be physically connected to an external electronic device (eg, the electronic device 102). According to one embodiment, the connection terminal 178 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (eg, a headphone connector).

The haptic module 179 can convert electrical signals into mechanical stimulation (e.g., vibration or movement) or electrical stimulation that the user can perceive through tactile or kinesthetic senses. According to one embodiment, the haptic module 179 may include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

The camera module 180 can capture still images and moving images. According to one embodiment, the camera module 180 may include one or more lenses, image sensors, image signal processors, or flashes.

The power management module 188 can manage power supplied to the electronic device 101. According to one embodiment, the power management module 188 may be implemented as at least a part of, for example, a power management integrated circuit (PMIC).

The battery 189 may supply power to at least one component of the electronic device 101. According to one embodiment, the battery 189 may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

Communication module 190 is configured to provide a direct (e.g., wired) communication channel or wireless communication channel between electronic device 101 and an external electronic device (e.g., electronic device 102, electronic device 104, or server 108). It can support establishment and communication through established communication channels. Communication module 190 operates independently of processor 120 (e.g., an application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module 190 is a wireless communication module 192 (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 194 (e.g., : LAN (local area network) communication module, or power line communication module) may be included. Among these communication modules, the corresponding communication module is a first network 198 (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network 199 (e.g., legacy It may communicate with an external electronic device 104 through a telecommunication network such as a cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., LAN or WAN). These various types of communication modules may be integrated into one component (e.g., a single chip) or may be implemented as a plurality of separate components (e.g., multiple chips). The wireless communication module 192 uses subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module 196 within a communication network such as the first network 198 or the second network 199. The electronic device 101 can be confirmed or authenticated.

The wireless communication module 192 may support 5G networks after 4G networks and next-generation communication technologies, for example, NR access technology (new radio access technology). NR access technology provides high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and access to multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low latency). -latency communications)) can be supported. The wireless communication module 192 may support high frequency bands (eg, mmWave bands), for example, to achieve high data rates. The wireless communication module 192 uses various technologies to secure performance in high frequency bands, for example, beamforming, massive array multiple-input and multiple-output (MIMO), and full-dimensional multiplexing. It can support technologies such as input/output (FD-MIMO: full dimensional MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module 192 may support various requirements specified in the electronic device 101, an external electronic device (e.g., electronic device 104), or a network system (e.g., second network 199). According to one embodiment, the wireless communication module 192 supports Peak data rate (e.g., 20 Gbps or more) for realizing eMBB, loss coverage (e.g., 164 dB or less) for realizing mmTC, or U-plane latency (e.g., 164 dB or less) for realizing URLLC. Example: Downlink (DL) and uplink (UL) each of 0.5 ms or less, or round trip 1 ms or less) can be supported.

The antenna module 197 may transmit or receive signals or power to or from the outside (eg, an external electronic device). According to one embodiment, the antenna module 197 may include an antenna including a radiator made of a conductor or a conductive pattern formed on a substrate (eg, PCB). According to one embodiment, the antenna module 197 may include a plurality of antennas (eg, an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network such as the first network 198 or the second network 199 is connected to the plurality of antennas by, for example, the communication module 190. can be selected. Signals or power may be transmitted or received between the communication module 190 and an external electronic device through the at least one selected antenna. According to some embodiments, in addition to the radiator, other components (eg, radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module 197.

According to one embodiment, the antenna module 197 may form a mmWave antenna module. According to one embodiment, a mmWave antenna module includes: a printed circuit board, an RFIC disposed on or adjacent to a first side (e.g., bottom side) of the printed circuit board and capable of supporting a designated high frequency band (e.g., mmWave band); And a plurality of antennas (e.g., array antennas) disposed on or adjacent to the second side (e.g., top or side) of the printed circuit board and capable of transmitting or receiving signals in the designated high frequency band. can do.

At least some of the components are connected to each other through a communication method between peripheral devices (e.g., bus, general purpose input and output (GPIO), serial peripheral interface (SPI), or mobile industry processor interface (MIPI)) and signal ( (e.g. commands or data) can be exchanged with each other.

According to one embodiment, commands or data may be transmitted or received between the electronic device 101 and the external electronic device 104 through the server 108 connected to the second network 199. Each of the external

electronic devices

102 or 104 may be of the same or different type as the electronic device 101. According to one embodiment, all or part of the operations performed in the electronic device 101 may be executed in one or more of the external

electronic devices

102, 104, or 108. For example, when the electronic device 101 needs to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device 101 may perform the function or service instead of executing the function or service on its own. Alternatively, or additionally, one or more external electronic devices may be requested to perform at least part of the function or service. One or more external electronic devices that have received the request may execute at least part of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device 101. The electronic device 101 may process the result as is or additionally and provide it as at least part of a response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology can be used. The electronic device 101 may provide an ultra-low latency service using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device 104 may include an Internet of Things (IoT) device. Server 108 may be an intelligent server using machine learning and/or neural networks. According to one embodiment, the external electronic device 104 or server 108 may be included in the second network 199. The electronic device 101 may be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

Electronic devices according to embodiments disclosed in this document may be of various types. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, electronic devices, or home appliances. Electronic devices according to embodiments of this document are not limited to the above-described devices.

The embodiments of this document and the terms used herein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various changes, equivalents, or replacements of the embodiments. In connection with the description of the drawings, similar reference numbers may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the above items, unless the relevant context clearly indicates otherwise. As used herein, “A or B”, “at least one of A and B”, “at least one of A or B”, “A, B or C”, “at least one of A, B and C”, and “A Each of phrases such as “at least one of , B, or C” may include any one of the items listed together in the corresponding phrase, or any possible combination thereof. Terms such as "first", "second", or "first" or "second" may be used simply to distinguish one component from another, and to refer to that component in other respects (e.g., importance or order) is not limited. One (e.g., first) component is said to be “coupled” or “connected” to another (e.g., second) component, with or without the terms “functionally” or “communicatively.” When mentioned, it means that any of the components can be connected to the other components directly (e.g. wired), wirelessly, or through a third component.

The term "module" used in embodiments of this document may include a unit implemented in hardware, software, or firmware, and is interchangeable with terms such as logic, logic block, component, or circuit, for example. It can be used as A module may be an integrated part or a minimum unit of the parts or a part thereof that performs one or more functions. For example, according to one embodiment, the module may be implemented in the form of an application-specific integrated circuit (ASIC).

One embodiment of the present document is one or more instructions stored in a storage medium (e.g., built-in memory 136 or external memory 138) that can be read by a machine (e.g., electronic device 101). It may be implemented as software (e.g., program 140) including these. For example, a processor (e.g., processor 120) of a device (e.g., electronic device 101) may call at least one command among one or more commands stored from a storage medium and execute it. This allows the device to be operated to perform at least one function according to the at least one instruction called. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. A storage medium that can be read by a device may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' only means that the storage medium is a tangible device and does not contain signals (e.g. electromagnetic waves), and this term refers to cases where data is semi-permanently stored in the storage medium. There is no distinction between temporary storage cases.

According to one embodiment, the method according to the embodiments disclosed in this document may be provided and included in a computer program product. Computer program products are commodities and can be traded between sellers and buyers. The computer program product may be distributed in the form of a machine-readable storage medium (e.g. compact disc read only memory (CD-ROM)) or through an application store (e.g. Play StoreTM) or on two user devices (e.g. It can be distributed (e.g. downloaded or uploaded) directly between smart phones) or online. In the case of online distribution, at least a portion of the computer program product may be at least temporarily stored or temporarily created in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

According to one embodiment, each component (e.g., module or program) of the above-described components may include a single or plural entity, and some of the plurality of entities may be separately placed in other components. there is. According to one embodiment, one or more of the above-described corresponding components or operations may be omitted, or one or more other components or operations may be added. Alternatively or additionally, multiple components (eg, modules or programs) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each component of the plurality of components in the same or similar manner as those performed by the corresponding component of the plurality of components prior to the integration. . According to embodiments, operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, or omitted. Alternatively, one or more other operations may be added.

FIG. 2 is a configuration diagram of an electronic device 200 according to an embodiment of the present disclosure.

Referring to FIG. 2, in one embodiment, the electronic device 200 (eg, the electronic device 101 of FIG. 1) may be implemented as a portable electronic device such as a smart phone or tablet PC. The electronic device 200 according to one embodiment includes a first processor 210, a first display 220, a first communication interface 230, a first memory 240, and/or a sensor module that are operatively connected to each other. Includes (250).

In one embodiment, the first processor 210 (e.g., processor 120 of FIG. 1) may perform processing operations stored in the first memory 240, execute an application to be described later, and control other components. You can. For example, the first processor 210 may be an application processor (AP) that can run various applications such as an Internet browser, gallery, message, and game, and provide the execution screen of the application to the first display 220. there is.

In one embodiment, the first display 220 may display a screen that visually provides information to the user based on display data received from the first processor 210. For example, the first display 220 may be composed of one fixed panel, the area of which may be varied by rolling, sliding, or folding, as described later, or the first display 220 may be independent of each other. It may be composed of a plurality of panels.

In one embodiment, the first communication interface 230 may be an interface capable of data communication with the external electronic device 200. For example, the first communication interface 230 may enable data communication with the external electronic device 200 wirelessly or wired, for example, a wired interface such as high definition multimedia interface (HDMI) or universal serial bus (USB). They can be connected to each other or to an external electronic device 200 through a short-range wireless communication network such as Wi-Fi direct or Bluetooth.

In one embodiment, the first memory 240 may be used to temporarily or permanently store digital data. For example, the first memory 240 includes one time programmable ROM (OTPROM), programmable ROM (PROM), erasable and programmable ROM (EPROM), electrically erasable and programmable ROM (EEPROM), mask ROM, flash ROM, and flash memory ( Examples: non-volatile memory, such as NAND flash or NOR flash, hard drives, or solid state drives (SSD), and dynamic RAM (DRAM), static RAM (SRAM), or synchronous dynamic RAM (SDRAM). It may contain the same volatile memory.

In one embodiment, the first memory 240 may store various instructions that can be executed by the first processor 210. These instructions may include various control instructions including arithmetic and logical operations, data movement, and/or input/output that can be recognized by the first processor 210.

In one embodiment, the sensor module 250 may detect a change in the state of the electronic device or a change in properties related to the first display. As an example, the sensor module 250 may include an acceleration sensor, a gyro sensor, a touch sensor, a motion sensor, a GPS sensor, and/or an angle sensor that detects the folding angle of the first display.

FIG. 3 is a configuration diagram of an external electronic device 300 according to an embodiment of the present disclosure.

Referring to FIG. 3, according to one embodiment, the external electronic device 300 (e.g., the electronic device 102 of FIG. 1) is made of components independent of the electronic device (e.g., the electronic device 101 of FIG. 1). It is configured and may be a display device such as a monitor or TV to display input display data.

The external electronic device 300 according to one embodiment includes a second processor 310, a second display 320, a second communication interface 330, and/or a second memory 340 that are operatively connected to each other. do.

In one embodiment, the second processor 310 may perform a processing operation stored in the second memory 340 and control a screen to be displayed on the second display 320 based on received data input.

In one embodiment, the second display 320 may display a screen that visually provides information to the user based on display data received from the electronic device through the second communication interface 330. For example, the second display 320 may be placed at a location spaced apart from the electronic device and provide the user with a screen that is different from the first display.

In one embodiment, the second communication interface 330 may be an interface capable of data communication with an electronic device (eg, a first communication interface). For example, the second communication interface 330 may enable data communication with an electronic device wirelessly or wired, and may be connected to each other through a wired interface such as high definition multimedia interface (HDMI) or universal serial bus (USB). Alternatively, it can be connected to an electronic device through a short-range wireless communication network such as Wi-Fi direct or Bluetooth.

FIG. 4 illustrates a state in which a screen displayed on the virtual display 260 generated by the electronic device 200 is displayed on the external electronic device 300 according to an embodiment of the present disclosure. FIG. 5 illustrates a connection between the first display 220 of the electronic device 200 and the second display 320 of the external electronic device 300 through the virtual display 260 according to an embodiment of the present disclosure. did it,

4 and 5, the electronic device 200 according to one embodiment transmits display data to the external electronic device 300 based on the connection with the external electronic device 300, and the external electronic device 300 may display a screen on the second display 320 based on the received display data.

In one embodiment, the electronic device 200 is connected to the external electronic device 300 based on the execution of an application (e.g., smart view), and performs mirroring to transmit the screen of the running application to the external electronic device 300. (mirroring) function was possible. For example, the electronic device 200 may duplicate the screen displayed on the first display 220 and display it on the second display 320 of the external electronic device 300. However, according to this mirroring function, the operation of the executing application may stop due to a difference in environmental settings between the first display 220 and the second display 320 and/or a change in the display mode of the first display 220. There was a problem with continuity not being secured and not working properly, such as re-running, moving to the initial screen, or terminating. Additionally, there was a problem in which the voice of the executing application was output overlappingly to the electronic device 200 and the external electronic device 300, or the voice that was supposed to be output from the electronic device 200 was output to the external electronic device 300.

The electronic device 200 according to an embodiment generates a virtual display 260 based on attributes related to the first display 220 included in the electronic device 200 and displays the virtual display 260 on the virtual display 260. Display data related to the screen can be transmitted to the external electronic device 300 and displayed on the second display 320.

Accordingly, even if the properties of the first display 220 of the electronic device 200 are changed after the execution application moves to the virtual display 260, the screen displayed on the second display 320 is not affected. , the continuity of running applications can be guaranteed. In particular, as will be described later, even if an event that changes the properties of the first display 220 (e.g., display mode change, display folding/unfolding, etc.) occurs, the execution application displayed on the second display 320 The screen can be maintained continuously.

Additionally, the electronic device 200 according to one embodiment moves the execution application to the virtual display 260 and displays it on the external electronic device 300, thereby displaying the execution application on the second display 320 of the external electronic device 300. An executable application may be executed on the first display 220 independently of at least one of the executable applications, allowing independent simultaneous use of the executable applications.

Specifically, the electronic device 200 according to one embodiment may display one or more execution applications displayed on the physical first display 220 through a concept display 250 (default display) displayed on the first display 220. At least one display screen can be displayed, and properties related to the first display 220 can be transmitted to the virtual display 260 as a snapshot through the concept display 250. Here, the concept display 250 refers to a software screen displayed on the first display 220, which is hardware, and may be a screen displayed based on properties related to the first display 200.

In one embodiment, the virtual display 260 is a software screen that is created and operated independently of the display screens of the first display 220 and the concept display 250, and the screen displayed on the virtual display 260 is It may be displayed on the physical second display 320 of the external electronic device 300. The electronic device 200 may transmit display data related to the screen displayed on the virtual display 260 to the external electronic device 300.

In one embodiment, a cast view 221 that displays the screen displayed on the virtual display 260 and the second display 320 may be displayed in at least a portion of the first display 220.

In one embodiment, when creating the virtual display 260, the electronic device 200 may set the properties of the virtual display 260 to correspond to the properties of the first display 220. In one embodiment, the properties related to the display include the resolution of the display, the density of the display, the screen layout of the display, the color mode of the display, and the HDR mode of the display (high dynamic). range mode), the properties of the input device, or a change in form factor related to the display.

For example, the first display 220 may have a resolution including HD, FHD, WQHD, or a customized resolution. Additionally, the first display 220 may have a density including mdpi, xdpi, xxdpi, xxxdpi, or a customized density. Additionally, the first display 220 may have a screen layout including large mode, long mode, small mode, xlarge mode, normal mode, LTR mode, or RTL mode. Additionally, the first display 220 may have a color mode including a wide color gamut or custom color mode. Additionally, the first display 220 may have an HDR mode including HDR enable or HDR disable. Additionally, the first display 220 may be connected to a mouse, touch screen, keyboard, or track ball as an input device.

In one embodiment, the electronic device 200 may generate the virtual display 260 based on properties related to at least one of the running applications moving from the first display 220 to the virtual display 260. . The electronic device 200 may set the properties of the virtual display 260 to have properties related to at least one of the running applications moving from the first display 220 to the virtual display 260. As an example, if at least one of the running applications moving to the virtual display 260 supports only a screen layout according to the vertical screen, the virtual display 260 is virtual so that at least one of the running applications is displayed in the ratio of the vertical screen. The properties of the display 260 can be set.

In one embodiment, even if the properties related to the first display 220 of the electronic device 200 are changed, after the execution application moves from the first display 220 to the virtual display 260, the virtual display ( Screens displayed on the

second display

260 and 320 may not be affected by changes in properties related to the first display 220 . Accordingly, even if properties related to the first display 220 are changed, it is possible to prevent an operation error in an executing application displayed on the virtual display 260 or the second display 320 from occurring.

FIG. 6 illustrates a flowchart 600 for generating a virtual display of an electronic device (e.g., the processor 120 of FIG. 1 or the first processor 210 of FIG. 2) according to an embodiment of the present disclosure. .

Referring further to FIG. 6, the application layer 610, framework layer 620, and graphic layer 630 according to one embodiment are processors of an electronic device (e.g., the processor 120 of FIG. 1 or the first processor of FIG. 2). It may be a software configuration included in the processor 210).

In one embodiment, the electronic device may generate a virtual display based on a user's input. As an example, the electronic device may execute an application (eg, smart view) that creates a connection with an external electronic device (eg, the external electronic device 300 of FIG. 3) based on the user's input. Based on the execution of the application, the electronic device can confirm the connection with the external electronic device through the communication interface.

In one embodiment, the electronic device may receive a user's input through a service 611 at the application layer 610, and the service 611 may transmit a command to create a virtual display to the display manager 612.

In one embodiment, in operation 640, the display manager 612 of the application layer 610 generates a virtual display based on properties related to the first display with the display manager service 621 of the framework layer 620. Commands can be sent. As an example, the display manager 612 may transmit a command to create a virtual display with the same resolution as the first display to the display manager service 621. The display manager service (621), which has received the command, requests properties (e.g., resolution) related to the first display from the logical display manager (622), and requests properties (e.g., resolution) related to the first display from the logical display manager (622). ) can be obtained, and properties (e.g. resolution) related to the acquired first display can be transmitted to the virtual display adapter 623. In one embodiment, the virtual display adapter 623 may create a virtual display device 624 having the same resolution and density as those of the first display. Accordingly, it is possible to prevent compatibility problems (eg, operation stoppage or re-execution) caused by moving an executing application from the first display to the virtual display.

In one embodiment, in operation 650, the virtual display device 624 obtains and creates properties related to the first display (e.g., resolution, density, screen layout, color mode, HDR mode, etc.) from the display manager service 621. You can override the properties of a virtual display. In particular, the properties related to the first display are the properties before the running application is moved to the virtual display, and the electronic device captures the properties related to the first display before moving the running application to the virtual display and stores the captured properties. It can be transmitted to a virtual display through snapshots. Properties of the virtual display may be set to correspond to properties related to the first display. Accordingly, operational errors in the executed application being moved can be prevented.

In one embodiment, in operation 660, the virtual display device 624 may transmit a command to the surface control 631 of the graphic layer 630 to display the screen displayed on the virtual display on the second display of the external electronic device. there is. Additionally, the surface control 631 of the graphic layer 630 may convert (composing) display data corresponding to the screen displayed on the virtual display on the surface flinger 632 to transmit it to an external electronic device.

In one embodiment, in operation 670, the service transmits a command to the activity task manager service 625 to move and display at least one of the executing applications to the virtual display, and the activity task manager service 625 controls the control of the electronic device. 1 At least one of the executing applications may be displayed on a virtual display created based on display-related properties. Additionally, the screen displayed on the virtual display by at least one of the execution applications moved to the virtual display can be transmitted from the virtual display device 624 to an external electronic device.

In one embodiment, the screen displayed on the virtual display may be at least partially different from the screen displayed on the first display. As an example, at least one of one or more executing applications running on the electronic device may be moved to a virtual display, and the at least one executing application may be displayed on the virtual display rather than the first display. For example, at least one of the one or more running applications that moves to the virtual display is selected by user input or under preset conditions (e.g., the highest running application, the most recently running application, etc.) It can be selected by .

Referring to FIG. 7, an electronic device (e.g., the electronic device 200 of FIG. 2) according to an embodiment transmits display data to the external electronic device (e.g., the external electronic device 300) based on connection with the external electronic device (e.g., the external electronic device 300). Based on the execution of the application being transmitted, the connection with an external electronic device can be confirmed through the communication interface.

In one embodiment, the electronic device may execute the application by receiving a user's application execution input. The electronic device according to one embodiment may display a list of a plurality of connectable external electronic devices on the first display 220 and allow the user to select an external electronic device to connect.

In one embodiment, when a preset execution application (e.g., a gallery application) that can be moved to the virtual display or the second display 320 of the external electronic device by the application is executed, and a connectable external electronic device is detected, the electronic device The device may display a notification that induces execution of the application on the first display 220.

FIG. 8 illustrates a screen of the first display 220 when connected to an external electronic device according to an embodiment of the present disclosure. FIG. 9 illustrates a screen of the first display 220 on which the cast view 221 according to an embodiment of the present disclosure is displayed.

Referring to FIGS. 8 and 9 , the electronic device according to one embodiment may display a cast view 221 that displays a screen displayed on a virtual display on at least a partial area of the first display 220. In one embodiment, the electronic device displays the screen displayed on the virtual display and/or the second display 320 of the external electronic device on the first display 220 by displaying a cast view 221 in the form of a pop-up window. ) can be displayed. The cast view 221 may be a pop-up window placed at a preset size and/or a preset position on the first display 220, and the preset size and position may be set in various ways depending on the properties of the virtual display. .

In one embodiment, the cast view 221 displayed on at least a portion of the first display 220 that displays the screen on the virtual display has a button to minimize, maximize, reduce, enlarge, or end the window screen. It may be in the form of a pop-up window displayed.

In one embodiment, the screen displayed on the first display 220 of the electronic device and the screen displayed on the second display 320 of the external electronic device may be a landscape screen or a portrait screen, respectively. In one embodiment, the properties of the first display 220 and/or the second display 320 are changed by a user's manipulation input or rotation of the first display 220 and/or the second display 320. ) Depending on the properties of the execution application displayed on each, a horizontal screen (landscape) or a vertical screen (portrait) may be displayed on the first display 220.

Here, the cast view 221 displayed on at least a portion of the first display 220 corresponds to the virtual display and/or the screen displayed on the second display 320, and is the background or the background of the first display 220. It can be independent of the screen displayed as a whole. In one embodiment, the cast view 221 displays the screen based on the properties of the virtual display and/or the second display 320 or the properties of the executing application displayed on the virtual display and/or the second display 320. Layout can be set.

That is, while the horizontal screen is displayed on the first display 220, the cast view 221 displayed on at least a portion of the first display 220 may be a horizontal screen and/or a vertical screen, and may also be displayed on the first display 220. When a vertical screen is displayed on 220, the cast view 221 displayed on at least a portion of the first display 220 may be a horizontal screen and/or a vertical screen.

In one embodiment, the electronic device may receive a touch input from the user through the cast view 221 displayed on at least a partial area of the first display 220. Specifically, the electronic device may display the virtual display and the screen displayed on the second display 320 on a partial area of the first display 220 while receiving a touch input from the user on the partial area. That is, the user can operate the virtual display and the executing application displayed on the second display 320 with a touch input through the first display 220.

FIG. 10A illustrates an operation of hiding or displaying the cast view 221 on the first display 220 according to a user's input according to an embodiment of the present disclosure. FIG. 10B illustrates an operation of hiding the cast view 221 on the first display 220 by displaying a keyboard according to an embodiment of the present disclosure.

Referring further to FIGS. 10A and 10B , the electronic device according to one embodiment displays the display in a partial area of the first display 220 when a preset condition is satisfied or a user input is received during execution of an application. The cast view 221 can be reduced or hidden.

In one embodiment, the electronic device may hide the cast view 221 displayed in a partial area of the first display 220 by a user's touch input. At the same time, the electronic device can hide the cast view 221 and the indicator 222, which will be described later. When the cast view 221 is hidden, a hint is displayed on the first display 220, and when a user's touch input for the hint occurs, the cast view 221 can be displayed again on the first display 220. there is. In one embodiment, the hint may be displayed at a position before displaying the cast view 221 on the first display 220.

In one embodiment, when a preset condition is satisfied, the electronic device may hide the cast view 221 displayed in a partial area of the first display 220. Here, the preset condition includes, if no touch input to the cast view 221 occurs for a preset time (e.g., 10 seconds), excluding a partial area of the first display 220 where the cast view 221 is displayed. This may include a case where a double tap touch input occurs for the remaining area, and/or a case where a keyboard (K, e.g., on-screen keyboard) is displayed on the first display 220.

FIG. 11 illustrates a screen related to detailed settings of the indicator 222 displayed on the first display 220 according to an embodiment of the present disclosure.

Referring further to FIG. 11 along with FIGS. 8 and 9, the indicator 222 along with the cast view 221 can be displayed on the first display 220. For example, the indicator 222 may be displayed fixedly at a preset position on the left or right side of the first display 220. In one embodiment, the indicator 222 may be hidden simultaneously with the cast view 221 or may be displayed on the first display 220 at the same time. The cast view 221 and the indicator 222 can be simultaneously hidden or displayed on the first display 220 by the user's swipe input on the cast view 221 and/or the indicator 222. there is.

The electronic device according to one embodiment is configured to set at least one of the following: whether to create a virtual display, whether to display the cast view 221 in at least a partial area of the first display 220, or properties related to the virtual display. A related indicator 222 may be displayed on at least a partial area of the first display 220. For example, the electronic device may display the indicator 222 adjacent to at least a partial area of the first display 220 that displays the cast view 221.

In one embodiment, the indicator 222 may be displayed on at least a portion of the first display 220 together with the cast view 221. The indicator 222 may include at least one button selectable by touch input. As an example, the indicator 222 may include at least one shortcut button.

For example, the indicator 222 may include a button to reduce or hide the display of the indicator 222. Additionally, the indicator 222 may include a button (S) for executing or setting an application related to transmitting display data to an external electronic device. The button S for executing or setting an application executes an application related to transmitting display data to an external electronic device to create a virtual display, or displays a cast view 221 on at least a partial area of the first display 220. Properties related to the setting of at least one of the properties related to the display or virtual display can be changed. In one embodiment, properties related to the cast view 221 and/or the screen displayed on the second display can be changed through the indicator 222 displayed on the first display 220.

In one embodiment, the indicator 222 includes zero configuration (easy connection), which is a function of searching for and connecting external electronic devices located nearby, app cast virtual screen management, which is a function of creating a virtual display according to an application, and a virtual screen management function of executing an application. cast app, a function that runs on the display; AL management, a function that utilizes manifest metadata or AL; app launch policy, a function that forces a cast view to run in full screen or removes the header of a cast view when running an application; and external electronic devices (e.g. 2 display) on an electronic device (e.g., a first display) in a cast view format, or a mirrored touch pad and/or a function that transmits a touch input through a cast view to an executing application and/or an executing application on an external electronic device (e.g., : You can display a shortcut related to the Appcast deep shortcut or Appcast context menu, which is a function that provides UX displayed on the second display.

FIG. 12 illustrates a button S for executing or setting an application related to transmitting display data to an external electronic device displayed on an executable application running on an electronic device according to an embodiment of the present disclosure.

Referring to FIG. 12, the electronic device according to one embodiment may display a button (S) for executing or setting an application related to transmitting display data to an external electronic device on an executing application running on the electronic device. there is.

In one embodiment, when an executable application that can be moved to a virtual display for display on a second display of an external electronic device is executed, the electronic device executes an application related to transmitting display data to the external electronic device in a part of the executable application. Alternatively, a button (S) for settings can be displayed. In one embodiment, when an executable application that can be moved to a virtual display is executed and a connectable external electronic device exists, the electronic device executes an application related to transmitting display data to the external electronic device in a part of the executable application. Alternatively, a button S for settings may be displayed to indicate to the user that the execution application can be moved to the second display of the external electronic device.

Accordingly, the user can execute the application for connection to the second display of the external electronic device through the button (S) for executing or setting the application related to the transmission of display data to the external electronic device displayed on the running application. . Based on the execution of an application related to the transmission of display data to an external electronic device, the electronic device may move the executing application to the virtual display.

In one embodiment, the electronic device may display the indicator 222 in a partial area of the first display 220 when an executable application that can be moved to a virtual display for display on the second display of the external electronic device is executed. there is.

Referring to FIG. 13, in one embodiment, the tutorial mode of the application may be executed when the application is first run or when there is no execution record, and when an execution input (touching the icon) of the application occurs, it may not be displayed again thereafter. . In one embodiment, the electronic device may execute the tutorial mode of the application when the executing application supports movement to the virtual display, and may execute the tutorial mode when the executable application is not supported or is executed in multi-window mode. It may not run.

In one embodiment, the electronic device may apply a highlight effect to the icon of the application displayed on the first display 220 in the tutorial mode, thereby inducing the user to execute the application (touch the icon). there is.

In one embodiment, the electronic device may display the cast view 221 in full screen on the first display in the tutorial mode. Additionally, as the application is executed, the electronic device may display a pop-up message containing information that the execution application can be moved on a virtual display and an external electronic device (eg, a second display).

In one embodiment, the electronic device may release the tutorial mode when the user inputs an application execution input or touches a confirmation button. In one embodiment, when a user's application execution input is input, the electronic device may end the tutorial mode and display the cast view 221 and the indicator 222 in a partial area of the first display.

FIG. 14A illustrates a screen of an electronic device when moving to a virtual display of an executing application that is not supported by the application according to an embodiment of the present disclosure. FIG. 14B illustrates a guide screen for executable applications that are not supported by the application according to an embodiment of the present disclosure.

Referring to FIGS. 14A and 14B, in one embodiment, the electronic device may be set to generate a virtual display for only a preset portion of a plurality of executing applications and support applications that move the executing application, and may be configured to support an application that moves the executing application. Application support may not be available for some of the remaining applications.

In one embodiment, the electronic device executes the user's application or displays it as a virtual display for an executable application that is not supported by the application, an executable application that is not on the home screen of the electronic device, or an executable application that is not executed on the entire screen of the electronic device. When receiving input of movement, a virtual display may not be created. The electronic device can create a virtual display and move the running application to the virtual display only when it receives input from the user to execute the application or move to the virtual display for the execution application supported by the application.

In one embodiment, the electronic device executes the user's application or displays it as a virtual display for an executable application that is not supported by the application, an executable application that is not on the home screen of the electronic device, or an executable application that is not executed on the entire screen of the electronic device. When a movement input is received, the screen displayed on the first display 220 of the electronic device can be moved to the virtual display, the second display, and/or the cast view 221 of the external electronic device to be displayed, as shown in FIG. 14a. You can. Additionally, the electronic device may display a home screen on the first display 220.

In one embodiment, the electronic device executes the user's application or displays it on a virtual display for an executable application that is not supported by the application, an executable application that is not on the home screen of the electronic device, or an executable application that is not executed on the entire screen of the electronic device. When a movement input is received, a guide screen may be displayed on the first display 220 as shown in FIG. 14B. For example, the guide screen may display a list of executable applications supported by the application or indicate that settings for all executable applications can be changed to be moved to a virtual display. The list of executable applications may include 15 to 30 preset executable applications among executable applications stored in the electronic device.

FIG. 15 illustrates a perspective view of the folding of the electronic device 200 according to an embodiment of the present disclosure.

Referring to FIG. 15, the electronic device 200 according to one embodiment may have a form factor in which the first display 220 is deformed by sliding or rolling. The electronic device 200 according to one embodiment may have a form factor that allows the first display 220 to be folded based on a hinge. The first display 220 includes an internal display 223 that is located inside the electronic device 200 and unfolds by unfolding based on the hinge, and an internal display 223 that is located outside the electronic device 200 and unfolds by unfolding based on the hinge. An external display 224 displayed by may be included.

In one embodiment, the electronic device 200 may undergo geometric deformation by folding or unfolding, and may be configured in an unfold mode (e.g., the folded angle exceeds 130°), a half-fold flex mode (e.g., the folded angle is 80°), ˚ ~ 130˚), can operate in half-fold cover mode (e.g. folded angle less than 80˚).

In one embodiment, the electronic device 200 includes an acceleration sensor, a gyro sensor, a geomagnetic field sensor, a grip sensor, a hinge angle sensor, a Hall sensor, etc. that contain information capable of determining the state of the electronic device 200. A sensor module (eg, sensor module 250 of FIG. 2) may be included. For example, the sensor module may receive status information through a grip sensor if the user grips it with one or both hands, and may use an acceleration sensor and/or gyro sensor to determine whether the electronic device 200 is resting on a table. Information can be received, and state information corresponding to the characteristics of the geometric form factor, such as the flex mode, of the electronic device 200 can be received.

FIGS. 16A, 16B, and 16C illustrate screens of the cast view 221 and the second display 320 according to changes in various states of the electronic device 200 according to an embodiment of the present disclosure.

Referring to FIGS. 16A, 16B, and 16C, the electronic device 200 according to one embodiment may be configured in an unfold mode, a half-fold flex mode (vertical), a half-fold flex mode (horizontal), a half-fold cover mode, and/or a half-fold mode. It can operate in fold flex panel mode. As the electronic device 200 changes to various modes, the properties related to the first display 220 may change, and accordingly, the properties of the virtual display may not be affected.

In one embodiment, the electronic device 200 may operate in the unfold mode 200A in which the first display 220 is unfolded. The electronic device 200 may display a screen on the entire area of the first display 220 in the unfolded mode 200A, and may be in portrait mode as shown or may be changed to landscape mode by rotation, accordingly. The screen layout of the first display 220 may be changed. Even if the properties (e.g., screen layout) related to the first display 220 are changed, the electronic device 200 displays the virtual display generated based on the properties related to the first display 220 before moving to the virtual display. The display properties can be maintained and the display screens of the second display 320 and the cast view 221 can be maintained.

In one embodiment, the electronic device 200, based on the changed properties related to the first display 220, displays properties (e.g., position, size, or direction) can be changed. As an example, when the screen layout of the first display 220 changes, the electronic device 200 may change the display position, size, or direction of the cast view 221 displayed on the first display 220.

In one embodiment, the electronic device 200 operates in a half-fold flex mode (200B, 200C) in which the first display 220 is folded within a preset angle range based on the folding angle of the first display 220. You can. Additionally, the electronic device 200 may operate in half-fold flex mode-vertical (200B) or half-fold flex mode-horizontal (200C) by rotation. Accordingly, the size, resolution, and/or screen layout of the first display 220 may be changed. Even if the properties related to the first display 220 are changed, the electronic device 200 may maintain the properties of the virtual display created based on the properties related to the first display 220 before moving to the virtual display. there is.

In one embodiment, the electronic device 200 may change properties related to the cast view 221 displayed on at least a portion of the first display 220 based on changed properties related to the first display 220. As an example, when the electronic device 200 operates in the half-fold flex mode (200B, 200C), the cast view 221 on the first display 220 is positioned at a preset position (e.g., on the left or right) from the portion folded by the hinge. The cast view 221 can be moved to the right or hidden by reducing or minimizing the cast view 221 on the first display 220 so that the user can perform another task.

In one embodiment, when the size or screen layout of the first display 220 changes, the electronic device 200 may change the size or screen layout of the cast view 221 displayed on the first display 220. . As an example, the electronic device 200 may move the cast view 221 displayed on the first display 220 in half-fold flex mode-portrait 200B so that it does not overlap the hinge of the first display 220. .

In one embodiment, the electronic device 200 may display an execution application or a home screen only on a portion of the first display 220 in half-fold flex mode-landscape 200C, and display a preset screen or an input screen on the remaining portion. can be displayed. The electronic device 200 may reduce or minimize the cast view 221 displayed on the first display 220 in half-fold flex mode-landscape 200C and hide it to the left or right.

In one embodiment, the electronic device 200 is in a half-fold cover mode in which the first display 220 is folded below a preset angle based on the folding angle of the first display 220, as shown in FIG. 16B. It can operate at (200D). The electronic device 200 can display a screen on the external display 224 included in the first display 220 in the half-fold cover mode 200D, and accordingly, the size, resolution, and/or Or the screen layout may change. Even if the properties related to the first display 220 are changed, the electronic device 200 may maintain the properties of the virtual display created based on the properties related to the first display 220 before moving to the virtual display. there is. Additionally, the electronic device 200 may reduce or minimize the cast view 221 displayed on the first display 220 in the half-fold cover mode 200D and hide it to the left or right.

In one embodiment, the electronic device 200 displays the first display 220 based on the folding angle of the first display 220 and/or whether the electronic device 200 is held, as shown in FIG. 16C. It can be operated in half-fold flex panel mode (200E), folded within a preset angle range and placed on a table. As an example, when the electronic device 200 operates in the half-fold flex panel mode 200E, it moves the cast view 221 on the first display 220 to a preset position (e.g., left or right), and 1 A user's touch input may be received through some areas of the display 220. The electronic device 200 displays a virtual display and/or an external electronic device based on the user's touch input pattern (e.g., tap, double tap, pinch zoom-in, pinch zoom-out, sliding in the up, down, left, and right directions, etc.) or the touch input of the keyboard. A manipulation for an executing application moved to the second display 320 of 300 may be received. For example, an operation for an executing application may be to adjust the volume of content, and the screen display for volume adjustment may be displayed on both the first display 220 and the second display 320, or may be selectively displayed on one of the displays. You can.

FIG. 17 is a flowchart 1700 of a method of operating an electronic device according to various embodiments of the present disclosure.

Referring to FIG. 17 , in operation 1710, an electronic device according to various embodiments may execute an application based on a user's input. The application may have a function that supports displaying an executable application running on the electronic device on the second display of the external electronic device.

In operation 1720, the electronic device according to various embodiments may confirm the connection with the external electronic device through the first communication interface based on the execution of the application. The first communication interface can transmit and receive data with the second communication interface of the external electronic device through a preset communication method.

The electronic device according to various embodiments may generate a virtual display based on properties related to the first display of the electronic device in

operations

1731 and 1732.

In one embodiment, the electronic device may generate a virtual display based on the resolution of the first display in operation 1731 and override properties related to the first display to the virtual display created in operation 1732.

In one embodiment, in operation 1740, the electronic device may display a screen that is at least partially different from the display screen of the first display on the generated virtual display, based on the properties of the generated virtual display. As an example, the electronic device moves and displays the display screen of at least one of one or more execution applications running on the electronic device on a virtual display, and accordingly, the virtual display may have at least a portion different from the display screen of the first display. there is.

In one embodiment, in operation 1750, the electronic device may transmit display data related to a screen displayed on a virtual display to an external electronic device through a communication interface. The external electronic device may display the screen displayed on the virtual display on the second display based on the received display data.

In one embodiment, in operation 1760, the electronic device displays a cast view on at least a partial area of the first display, determines whether a virtual display is generated, whether a cast view is displayed in at least a partial area of the first display, or creates a virtual display. An indicator related to settings for at least one of the display-related properties may be displayed in at least a partial area of the first display.

In one embodiment, the electronic device may reduce or hide the cast view displayed on the first display when a preset condition is satisfied or a user's input is received.

An electronic device (101 in FIG. 1; 200 in FIG. 2) according to an embodiment of the present disclosure includes a display 220, a communication interface 230, a memory 240, and the display 220 and the communication interface ( 230) and at least one processor 210 operatively connected to the memory 240. The at least one processor 210 may be set to check the connection with the external electronic device 102 (300) through the communication interface 230 based on execution of the application. The at least one processor 210 may be set to generate a virtual display 260 based on properties related to the display 220. The at least one processor 210 displays a screen on the generated virtual display 260 that is at least partially different from the display screen of the display 220, based on the properties of the generated virtual display 260. It can be set to display. The at least one processor 210 may be set to transmit display data related to the screen displayed on the virtual display 260 to the external electronic device 102 (300) through the communication interface 230. there is.

In the electronic device (101 in FIG. 1; 200 in FIG. 2) according to an embodiment of the present disclosure, the at least one processor 210 is configured to, at least as part of an operation of generating the virtual display 260, generate the display. A virtual display 260 may be created based on the resolution of 220 and may be set to override properties related to the display 220 on the created virtual display 260.

In the electronic device (101 in FIG. 1; 200 in FIG. 2) according to an embodiment of the present disclosure, the at least one processor 210 displays a display screen of the display 220 on the virtual display 260. As part of the operation of displaying at least some different screens, moving at least one display screen among one or more execution applications running on the electronic device (101 in FIG. 1; 200 in FIG. 2) to the virtual display 260. It can be set to display.

In an electronic device (101 in FIG. 1; 200 in FIG. 2) according to an embodiment of the present disclosure, an attribute related to the display 220 is configured to display the at least one display screen among the one or more executing applications as the virtual display. This may be an attribute before moving to (260).

In an electronic device (101 in FIG. 1; 200 in FIG. 2) according to an embodiment of the present disclosure, properties related to the display 220 may be changeable. The at least one processor 210 is configured to, when properties related to the display 220 are changed after moving the display screen of the at least one of the one or more execution applications to the virtual display 260, It may be set to maintain the properties of the virtual display 260 created based on properties related to the display 220 before moving to the display 260.

In an electronic device (101 in FIG. 1; 200 in FIG. 2) according to an embodiment of the present disclosure, the at least one processor 210 performs the execution as at least part of an operation for generating the virtual display 260. It may be set to generate the virtual display 260 based on properties related to at least one of the applications.

In the electronic device (101 in FIG. 1; 200 in FIG. 2) according to an embodiment of the present disclosure, the at least one processor 210 displays a cast view 221 that displays the screen of the virtual display 260. may be further set to be displayed in at least some areas of the display 220.

In an electronic device (101 in FIG. 1; 200 in FIG. 2) according to an embodiment of the present disclosure, properties related to the display 220 may be changeable. When the properties related to the display 220 are changed after the creation of the virtual display 260, the at least one processor 210 operates the display ( 220) may be set to change properties related to the display of the cast view 221.

In an electronic device (101 in FIG. 1; 200 in FIG. 2) according to an embodiment of the present disclosure, the at least one processor 210, through the cast view 221 displayed on the display 220, It may be further configured to receive a touch input from the user.

In an electronic device (101 in FIG. 1; 200 in FIG. 2) according to an embodiment of the present disclosure, the at least one processor 210 displays the cast view 221 on at least a partial area of the display 220. As at least part of the display operation, when a preset condition is satisfied or a user's input is received, the cast view 221 displayed on the display 220 may be further set to be reduced or hidden.

In the electronic device (101 in FIG. 1; 200 in FIG. 2) according to an embodiment of the present disclosure, the at least one processor 210 determines whether to generate the virtual display 260 and determines the An indicator 222 related to whether to display the cast view 221 in at least some areas or a setting for at least one of the properties related to the virtual display 260 is displayed in at least some areas of the display 220. It can be set to display.

A method 1700 of operating an electronic device (101 in FIG. 1; 200 in FIG. 2) according to an embodiment of the present disclosure is to operate an external electronic device (102; 300) through the communication interface 230 based on execution of an application. ) may include an operation 1720 to check the connection. A method 1700 of operating an electronic device (101 in FIG. 1; 200 in FIG. 2) according to an embodiment is based on properties related to the display 220 of the electronic device (101 in FIG. 1; 200 in FIG. 2). This may include

operations

1731 and 1732 of generating the virtual display 260. A method 1700 of operating an electronic device (101 in FIG. 1; 200 in FIG. 2) according to an embodiment includes operating the generated virtual display 260 based on the properties of the generated virtual display 260. may include an operation 1740 of displaying a screen that is at least partially different from the display screen of the display 220. A method 1700 of operating an electronic device (101 in FIG. 1; 200 in FIG. 2) according to an embodiment includes display data related to a screen displayed on the virtual display 260 through the communication interface 230. It may include an operation 1750 of transmitting to the external electronic device 102 (300).

In the method 1700 of operating an electronic device (101 in FIG. 1; 200 in FIG. 2) according to an embodiment of the present disclosure,

operations

1731 and 1732 of generating the virtual display 260 include the display ( An operation 1731 of generating a virtual display 260 based on the resolution of 220), and an operation 1732 of overriding properties related to the display 220 on the created virtual display 260. You can.

In the method 1700 of operating an electronic device (101 in FIG. 1; 200 in FIG. 2) according to an embodiment of the present disclosure, the virtual display 260 is at least partially different from the display screen of the display 220. The screen display operation 1740 involves moving and displaying at least one display screen among one or more execution applications running on the electronic device (101 in FIG. 1; 200 in FIG. 2) on the virtual display 260. You can.

In the method 1700 of operating an electronic device (101 in FIG. 1; 200 in FIG. 2) according to an embodiment of the present disclosure, the attribute related to the display 220 includes the display of the at least one of the one or more running applications. This may be an attribute before moving the screen to the virtual display 260.

In a method 1700 of operating an electronic device (101 in FIG. 1; 200 in FIG. 2) according to an embodiment of the present disclosure, properties related to the display 220 may be changed. When properties related to the display 220 are changed after moving the display screen of at least one of the one or more running applications to the virtual display 260, the display 220 is displayed on the virtual display 260. The operation 1740 of displaying a screen that is at least partially different from the display screen of ) includes the virtual display 260 generated based on properties related to the display 220 before moving to the virtual display 260. ) properties can be maintained.

A method 1700 of operating an electronic device (101 in FIG. 1; 200 in FIG. 2) according to an embodiment of the present disclosure includes casting a cast view 221 that displays the screen of the virtual display 260 to the display ( An operation 1750 of displaying at least a portion of the area 220 may be further included.

In a method 1700 of operating an electronic device (101 in FIG. 1; 200 in FIG. 2) according to an embodiment of the present disclosure, properties related to the display 220 may be changed. When properties related to the display 220 are changed after the creation of the virtual display 260, the operation 1750 of displaying the cast view 221 on at least a partial area of the display 220 includes the above Based on the changed properties related to the display 220, the properties related to the display of the cast view 221 of the display 220 can be changed.

In an operation method 1700 of an electronic device (101 in FIG. 1; 200 in FIG. 2) according to an embodiment of the present disclosure, the operation of displaying the cast view 221 on at least a partial area of the display 220 ( 1750) may reduce or hide the cast view 221 displayed on the display 220 when a preset condition is satisfied or a user's input is received.

A method 1700 of operating an electronic device (101 in FIG. 1; 200 in FIG. 2) according to an embodiment of the present disclosure includes determining whether to generate the virtual display 260, and determining the at least a portion of the display 220. An operation of displaying an indicator 222 related to whether to display the cast view 221 or a setting for at least one of the properties related to the virtual display 260 in at least a partial area of the display 220 ( 1750) may further be included.

A non-transitory computer-readable storage medium storing one or more programs according to an embodiment of the present disclosure confirms the connection with the external electronic device (102; 300) through the communication interface (230) based on the execution of the application. Can include actions. The storage medium may include an operation of generating a virtual display 260 based on properties related to the display 220 of the electronic device (101 in FIG. 1; 200 in FIG. 2). The storage medium includes an operation of displaying a screen that is at least partially different from the display screen of the display 220 on the generated virtual display 260, based on the properties of the generated virtual display 260. can do. The storage medium may include an operation of transmitting display data related to a screen displayed on the virtual display 260 to the external electronic device 102 (300) through the communication interface 230.

Claims

In an electronic device (101 in FIG. 1; 200 in FIG. 2),

display 220;

communication interface 230;

memory(240); and

At least one processor 210 operatively connected to the display 220, the communication interface 230, and the memory 240, wherein the at least one processor 210:

Based on the execution of the application, the connection with the external electronic device (102; 300) is confirmed through the communication interface (230),

Creating a virtual display 260 based on properties related to the display 220,

Based on the properties of the generated virtual display 260, displaying a screen on the generated virtual display 260 that is at least partially different from the display screen of the display 220,

Set to transmit display data related to the screen displayed on the virtual display 260 to the external electronic device 102 (300) through the communication interface 230,

Electronic device (101 in FIG. 1; 200 in FIG. 2).
According to claim 1,

The at least one processor 210:

At least as part of the operation of generating the virtual display 260,

Creating a virtual display 260 based on the resolution of the display 220,

Set to override properties related to the display 220 on the created virtual display 260,

Electronic device (101 in FIG. 1; 200 in FIG. 2).
According to at least one of claims 1 and 2,

The at least one processor 210:

One or more executions executed by the electronic device (101 in FIG. 1; 200 in FIG. 2) as at least part of an operation of displaying a screen on the virtual display 260 that is at least partially different from the display screen of the display 220 Set to move and display at least one display screen of the application on the virtual display 260,

Electronic device (101 in FIG. 1; 200 in FIG. 2).
According to claim 3,

The properties related to the display 220 are properties before moving the display screen of the at least one of the one or more running applications to the virtual display 260,

Electronic device (101 in FIG. 1; 200 in FIG. 2).
According to at least one of claims 3 to 4,

Properties related to the display 220 can be changed,

The at least one processor 210:

When properties related to the display 220 are changed after moving the display screen of at least one of the one or more executing applications to the virtual display 260, the display screen before moving to the virtual display 260 Set to maintain the properties of the virtual display 260 created based on properties related to the display 220,

Electronic device (101 in FIG. 1; 200 in FIG. 2).
According to at least one of claims 3 to 5,

The at least one processor 210:

At least as part of the operation of generating the virtual display 260, set to generate the virtual display 260 based on properties related to the at least one of the executing applications,

Electronic device (101 in FIG. 1; 200 in FIG. 2).
According to at least one of claims 1 to 6,

The at least one processor 210:

Further set to display the cast view 221, which displays the screen of the virtual display 260, on at least a partial area of the display 220,

Electronic device (101 in FIG. 1; 200 in FIG. 2).
According to claim 7,

Properties related to the display 220 can be changed,

The at least one processor 210:

If the properties related to the display 220 are changed after the creation of the virtual display 260, the cast view 221 of the display 220 is changed based on the changed properties related to the display 220. set to change properties related to the display,

Electronic device (101 in FIG. 1; 200 in FIG. 2).
According to at least one of claims 7 to 8,

The at least one processor 210:

Further set to receive a touch input from the user through the cast view 221 displayed on the display 220,

Electronic device (101 in FIG. 1; 200 in FIG. 2).
According to at least one of claims 7 to 9,

The at least one processor 210:

The cast view displayed on the display 220 when a preset condition is satisfied or a user input is received as at least part of the operation of displaying the cast view 221 on at least a partial area of the display 220. further set to reduce or hide (221),

Electronic device (101 in FIG. 1; 200 in FIG. 2).
According to at least one of claims 7 to 10,

The at least one processor 210:

Setting of at least one of the following: whether to create the virtual display 260, whether to display the cast view 221 in at least some area of the display 220, or properties related to the virtual display 260 Set to display an indicator 222 related to at least a portion of the display 220,

Electronic device (101 in FIG. 1; 200 in FIG. 2).
In a method 1700 of operating an electronic device (101 in FIG. 1; 200 in FIG. 2),

Based on the execution of the application, an operation 1720 of confirming the connection with the external electronic device 102 (300) through the communication interface 230;

Operations 1731 and 1732 of generating a virtual display 260 based on properties related to the display 220 of the electronic device (101 in FIG. 1; 200 in FIG. 2);

An operation 1740 of displaying a screen that is at least partially different from the display screen of the display 220 on the generated virtual display 260, based on the properties of the generated virtual display 260; and

Comprising an operation (1750) of transmitting display data related to a screen displayed on the virtual display (260) to the external electronic device (102; 300) through the communication interface (230).

How it works.
According to claim 12,

The operations 1731 and 1732 of generating the virtual display 260 are,

An operation 1731 of generating a virtual display 260 based on the resolution of the display 220; and

Including an operation 1732 of overriding properties related to the display 220 on the created virtual display 260,

How it works.
According to at least one of claims 12 to 13,

The operation 1740 of displaying a screen that is at least partially different from the display screen of the display 220 on the virtual display 260 is performed by one or more steps executed by the electronic device (101 in FIG. 1; 200 in FIG. 2). Moving and displaying at least one display screen of an executing application on the virtual display 260,

How it works.
According to at least one of claims 12 to 14,

An operation 1750 of displaying a cast view 221 displaying a screen of the virtual display 260 on at least a partial area of the display 220; and

Setting of at least one of the following: whether to create the virtual display 260, whether to display the cast view 221 in at least some area of the display 220, or properties related to the virtual display 260 Further comprising an operation 1750 of displaying an indicator 222 related to at least a portion of the display 220.

How it works.