KR101612549B1 - A mobile terminal having a plurality of virtual screen and a controlling method thereof - Google Patents

Abstract

The present invention provides a method for overcoming the constraints of a mobile terminal such as a mobile phone in which a display screen is small and user input / output means is limited. The present invention relates to a mobile terminal capable of forming a plurality of virtual screens in a mobile terminal such as a mobile phone and switching among a plurality of virtual screens according to an intuitive operation of the user, And a control method thereof.

Multi-desktop, multi-screen, virtual screen, mobile terminal, mobile phone, tilt sensor, gyro sensor, acceleration sensor,

Description

Technical Field The present invention relates to a mobile terminal having a plurality of virtual screens and a control method thereof.

The present invention relates to a mobile terminal and a control method thereof.

More specifically, the present invention forms a plurality of virtual screens in a mobile terminal such as a mobile phone in which a display screen is small and a user input / output means is limited, and can switch among a plurality of virtual screens according to an intuitive operation of a user And a mobile terminal capable of allowing a user to have a wider working space and a control method thereof.

A mobile terminal such as a mobile phone, PDA, or MP3 can be configured to perform various functions. Examples of such various functions include a data and voice communication function, a function of photographing a video or a moving image through a camera, a voice storage function, a music file playback function through a speaker system, and an image or video display function. Some mobile terminals include additional functions to execute games, and some other mobile terminals are also implemented as multimedia devices. Moreover, modern mobile terminals can receive broadcast or multicast signals to watch video or television programs.

Further, efforts for supporting and increasing the functions of the mobile terminal continue. Such efforts include not only changes and improvements in structural components that form the mobile terminal, but also improvements in software or hardware.

Since the mobile terminal must guarantee its mobility, the means by which a user can input a command is limited, and the size of the display is limited. In addition, when the multitasking is performed, It is impossible to implement a dual monitor such as a PC.

Even if dual monitor implementation is possible, movement between monitors like a normal PC is very inconvenient and slow.

There is a need for a way to overcome the limitations of such mobile terminals.

It is an object of the present invention to provide a user interface useful for a multitasking operation in a mobile terminal.

It is an object of the present invention to provide a mobile terminal capable of setting a plurality of virtual screens having different output images and performing a desired operation while the user switches between virtual screens using an intuitive gesture and a control method thereof .

The method and structure proposed by the present invention can be applied not only to mobile terminals, but also to all kinds of electronic devices equipped with a display.

A display device control method according to an embodiment of the present invention includes: generating a plurality of virtual screens having independent output images; Displaying one of the plurality of virtual screens; Detecting an input of a user; And switching and displaying a different virtual screen according to the input of the user.

A method of controlling a display device according to an embodiment of the present invention includes: generating a plurality of virtual screens having independent output images; Creating a text editing screen on one of the virtual screens; Detecting a gesture for selecting an object in one of the other virtual screens; Detecting a screen switching gesture into a virtual screen including the text editing screen; Displaying a screen including the text editing screen; Detecting a gesture by which the user releases the touch; And attaching the selected object to a position where the touch of the text editing screen is released.

A method of controlling a display device according to an embodiment of the present invention includes: generating a plurality of virtual screens having independent output images; Creating a text editing screen on one of the virtual screens; Detecting a gesture of selecting and dragging a media file in one of the other virtual screens; Detecting a screen switching gesture; Displaying a screen on which the text editing screen is created; Detecting a gesture by which the user releases the touch; And attaching the selected media file as an attachment to the text being created.

A mobile terminal according to an embodiment of the present invention includes a display unit; And a control unit for generating a plurality of virtual screens having separate output images independent of each other and controlling at least one of the plurality of virtual screens to be output to the display unit according to a user's input.

According to the present invention, it is possible to provide a user interface useful for a multitasking operation in a mobile terminal.

Further, according to the present invention, it is possible to easily move between a plurality of virtual screens having separate output images using an intuitive gesture.

In addition, according to the present invention, when a screen other than a main screen is displayed, the screen of the main screen is displayed in a semi-transparent overlapping manner, so that events occurring on the two screens can be confirmed in one screen.

According to the present invention, a plurality of virtual screens can be set on a mobile terminal, programs can be independently executed on each virtual screen, and multitasking can be performed on the main screen and the left or right screen quickly.

According to the present invention, a copy and paste function and a media file attachment function by dragging, which are difficult to implement in a mobile terminal, can be easily implemented.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

1 is a block diagram of a mobile terminal 100 according to an embodiment of the present invention.

The mobile terminal may be implemented in various forms. For example, the mobile terminal described in this specification may be a mobile phone, a smart phone, a notebook computer, a digital broadcasting terminal, a PDA (Personal Digital Assistants), a PMP (Portable Multimedia Player) have.

The mobile terminal 100 includes a wireless communication unit 110, an A / V input unit 120, a user input unit 130, a sensing unit 140, an output unit 150, a memory 160, An interface unit 170, a control unit 180, a power supply unit 190, and the like. 1 shows a mobile terminal having various components. However, not all illustrated components are required. A mobile terminal may be implemented by more components than the illustrated components, or a mobile terminal may be implemented by fewer components.

Hereinafter, the components will be described in order.

The wireless communication unit 110 may include one or more components that allow wireless communication between the mobile terminal 100 and the wireless communication system or wireless communication between the mobile terminal 100 and a network on which the mobile terminal 100 is located. For example, the wireless communication unit 110 may include a broadcast receiving module 111, a mobile communication module 112, a wireless Internet module 113, a short range communication module 114, and a location information module 115 .

The broadcast receiving module 111 receives broadcast signals and / or broadcast-related information from an external broadcast management server through a broadcast channel. The broadcast channel may include a satellite channel and a terrestrial channel. The broadcast management server may refer to a server for generating and transmitting broadcast signals and / or broadcast related information, or a server for receiving broadcast signals and / or broadcast related information generated by the broadcast management server and transmitting the generated broadcast signals and / or broadcast related information. The broadcast-related information may refer to a broadcast channel, a broadcast program, or information related to a broadcast service provider. The broadcast signal may include a TV broadcast signal, a radio broadcast signal, a data broadcast signal, and a broadcast signal in which a data broadcast signal is combined with a TV broadcast signal or a radio broadcast signal.

Meanwhile, the broadcast related information may be provided through a mobile communication network, and in this case, it may be received by the mobile communication module 112.

The broadcast-related information may exist in various forms. For example, an EPG (Electronic Program Guide) of DMB (Digital Multimedia Broadcasting) or an ESG (Electronic Service Guide) of Digital Video Broadcast-Handheld (DVB-H).

The broadcast receiving module 111 receives broadcasting signals using various broadcasting systems. In particular, the broadcasting receiving module 111 may be a Digital Multimedia Broadcasting-Terrestrial (DMB-T), a Digital Multimedia Broadcasting-Satellite (DMB-S) Only Digital Broadcast-Handheld (DVB-H), Integrated Services Digital Broadcast-Terrestrial (ISDB-T), and the like. Of course, the broadcast receiving module 111 is configured to be suitable for all broadcasting systems that provide broadcasting signals as well as the digital broadcasting system described above.

The broadcast signal and / or broadcast related information received through the broadcast receiving module 111 may be stored in the memory 160.

In addition, the mobile communication module 112 transmits and receives radio signals to at least one of a base station, an external terminal, and a server on a mobile communication network. Here, the wireless signal may include various types of data depending on a voice call signal, a video call signal, or a text / multimedia message transmission / reception.

The wireless Internet module 113 is a module for wireless Internet access, and the wireless Internet module 113 can be built in or externally.

The short-range communication module 114 refers to a module for short-range communication. Bluetooth, radio frequency identification (RFID), infrared data association (IrDA), ultra wideband (UWB), ZigBee, etc. may be used as a short distance communication technology.

The location information module 115 is a module for confirming or obtaining the location of the mobile terminal. One example is the GPS (Global Position System) module. The GPS module receives position information from a plurality of satellites. Here, the location information may include coordinate information indicated by latitude and longitude. For example, the GPS module can accurately calculate the current location according to the triangulation method by measuring the precise time and distance from three or more satellites and measuring three different distances. A method of obtaining distance and time information from three satellites and correcting the error by one satellite may be used. In particular, the GPS module can acquire latitude, longitude and altitude as well as three-dimensional velocity information and accurate time from the position information received from the satellite.

An audio / video (A / V) input unit 120 is for inputting an audio signal or a video signal, and may include a camera 121 and a microphone 122. The camera 121 processes an image screen such as a still image or a moving image obtained by the image sensor in the video communication mode or the photographing mode. Then, the processed image screen can be displayed on the display unit 151. [

The image screen processed by the camera 121 can be stored in the memory 160 or transmitted to the outside through the wireless communication unit 110. [ The camera 121 may be equipped with two or more cameras according to the configuration of the terminal.

The microphone 122 receives an external sound signal through a microphone in a communication mode, a recording mode, a voice recognition mode, or the like, and processes it as electrical voice data. The processed voice data can be converted into a form that can be transmitted to the mobile communication base station through the mobile communication module 112 when the voice data is in the call mode, and output. The microphone 122 may be implemented with various noise reduction algorithms for eliminating noise generated in the process of receiving an external sound signal.

The user input unit 130 generates input data for a user to control the operation of the terminal. The user input unit 130 may include a touch panel 131 and a tilt sensor 132. The user input unit 130 may also include a key pad, a dome switch, a jog wheel, a jog switch, . In particular, when the touch panel 131 is mounted on the display unit 151 described later, it can be called a touch screen.

A tilt sensor or an accelerometer may be used as the tilt sensor, and a user's face image is captured using a camera 121 described later, and the relative inclination of the user of the mobile terminal 100 with respect to the user .

The state sensing unit 140 senses the current state of the mobile terminal 100 such as the open / close state of the mobile terminal 100, the position of the mobile terminal 100, the presence of the user, the orientation of the mobile terminal, And generates a signal for controlling the operation of the mobile terminal 100. For example, when the mobile terminal 100 is in the form of a slide phone, it is possible to sense whether the slide phone is opened or closed. In the case of a touch screen phone, it can be sensed whether the lock of the touch screen is set or released. Also, it is possible to sense whether the power supply unit 190 is powered on, whether the interface unit 170 is connected to an external device, and the like.

The interface unit 170 serves as an interface with all external devices connected to the mobile terminal 100. For example, a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, a port for connecting a device having an identification module, an audio I / O port, Video input / output (I / O) ports, earphone ports, and the like.

Here, the identification module is a chip that stores various information for authenticating the use right of the mobile terminal 100, and includes a user identification module (UIM), a subscriber identity module (SIM) ), A Universal Subscriber Identity Module (" USIM "), and the like. In addition, an apparatus having an identification module (hereinafter referred to as 'identification device') can be manufactured in a smart card format. Accordingly, the identification device can be connected to the terminal 100 through the port. The interface unit 170 receives data from an external device or receives power from the external device to transfer the data to each component in the mobile terminal 100 or to transmit data in the mobile terminal 100 to an external device.

The output unit 150 is for outputting an audio signal, a video signal, or an alarm signal. The output unit 150 may include a display unit 151, an audio output module 152, an alarm unit 153, and the like.

The display unit 151 displays and outputs information processed by the mobile terminal 100. For example, when the mobile terminal is in the call mode, a UI (User Interface) or a GUI (Graphic User Interface) associated with a call is displayed. If the mobile terminal 100 is in the image communication mode or the photographing mode, the image or UI or GUI that is photographed and / or received is displayed.

Meanwhile, as described above, when the display unit 13 and the touch panel 131 have a mutual layer structure to constitute a touch screen, the display unit 151 can also be used as an input device. The display unit 151 may be a liquid crystal display, a thin film transistor-liquid crystal display, an organic light-emitting diode, a flexible display, a three-dimensional display 3D display). In addition, there may be two or more display units 151 according to the embodiment of the mobile terminal 100. For example, the mobile terminal 100 may be provided with an external display unit (not shown) and an internal display unit (not shown) at the same time.

The audio output module 152 outputs audio data received from the wireless communication unit 110 or stored in the memory 160 in a call signal reception mode, a call mode or a recording mode, a voice recognition mode, a broadcast reception mode, The sound output module 152 outputs an acoustic signal related to functions (e.g., call signal reception sound, message reception sound, etc.) performed in the mobile terminal 100. [ The sound output module 152 may include a speaker, a buzzer, and the like.

The alarm unit 153 outputs a signal for notifying the occurrence of an event of the mobile terminal 100. Examples of events that occur in the mobile terminal include receiving a call signal, receiving a message, and inputting a key signal. The alarm unit 153 may output a signal for notifying the occurrence of an event in a form other than an audio signal or a video signal. For example, it is possible to output a signal in a vibration mode. When the call signal is received or a message is received, the alarm unit 153 can output a vibration to notify it. Alternatively, when the key signal is input, the alarm unit 153 can output the vibration by the feedback to the key signal input. The user can recognize the occurrence of an event through the vibration output as described above. Of course, a signal for notifying the occurrence of an event may also be output through the display unit 151 or the audio output module 152.

The memory 160 may store a program for processing and controlling the control unit 180 and may store a function for temporarily storing input / output data (e.g., a phone book, a message, a still image, .

The memory 160 may be a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, SD or XD memory), a RAM (Random Access Memory) SRAM (Static Random Access Memory), ROM (Read Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read- And an optical disc. In addition, the mobile terminal 100 may operate a web storage for storing the memory 150 on the Internet.

The controller 180 typically controls the overall operation of the mobile terminal. For example, voice communication, data communication, video communication, and the like. In addition, the control unit 180 may include a multimedia module 181 for multimedia playback. The multimedia module 181 may be implemented in the control unit 180 or may be implemented separately from the control unit 180. [

The power supply unit 190 receives external power and internal power under the control of the controller 180 and supplies power necessary for operation of the respective components.

The various embodiments described herein may be embodied in a computer-readable recording medium using, for example, software, hardware, or a combination thereof.

According to a hardware implementation, the embodiments described herein may be implemented as application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays May be implemented using at least one of processors, controllers, micro-controllers, microprocessors, and electrical units for performing functions. In some cases, And may be implemented by the control unit 180.

According to a software implementation, embodiments such as procedures or functions may be implemented with separate software modules that perform at least one function or operation. The software code may be implemented by a software application written in a suitable programming language. Also, the software codes may be stored in the memory 160 and executed by the control unit 180. [

Further, according to an embodiment of the present invention, the above-described method can be implemented as computer-readable code on a medium on which a program is recorded. The computer readable medium includes all kinds of recording devices in which data that can be read by a computer system is stored. Examples of the computer-readable medium include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage, and the like, and also implemented in the form of a carrier wave (for example, transmission over the Internet) . In addition, the computer may include a control unit 180 of the terminal.

2 illustrates an operation of the mobile terminal 100 capable of implementing a plurality of virtual screens according to an embodiment of the present invention.

The mobile terminal 100 shown in Fig. 2 is a so-called "touch screen phone ", which is a cellular phone in which the display 151 occupies most of the front side and the display 151 comprises a touch screen. Although the touch screen phone is described as an example in the present embodiment, as described above, the features of the present invention can be applied to various display devices.

The display 151 of the mobile terminal 100 may be a touch screen. Various icons 208 are displayed on the display 151 and the wireless signal sensitivity 212a, the current time 212b, the battery remaining amount 212c, and the like may be displayed. Also, the right switch icon 204a, the left switch icon 204b, and the switch icon 206 can be displayed.

The physical buttons 210 may be disposed at the lower end of the display 151 of the mobile terminal 100 and the buttons 210 may be call, end, and cancel buttons, respectively. In addition, additional buttons such as a camera button and a jog wheel may be mounted on the front surface or the side surface of the mobile terminal 100 according to an embodiment.

Although the mobile terminal 100 according to the embodiment of the present invention has only one display 151, a plurality of virtual screens can be displayed on the display 151. [ In this embodiment, it is assumed that the main screen 200, the sub-screens, i.e., the left screen 300 and the right screen 400 are driven. Each of the virtual screens 200, 300, and 400 has independent images, and the mobile terminal 100 can support multitasking in which separate programs are simultaneously executed for each screen.

In FIG. 2, a screen of the main screen 200 is displayed on the display 151. FIG. The main screen 200 displays icons 208 indicating functions of a message, a calendar, a calculator, a camera, an electronic book, and the like. Although not shown on the display 151, the left screen 300 includes various icons L1 to L15 and 213 and the right screen 400 also includes various icons R1 to R14 and 211 have.

The user can execute the corresponding icon by selecting an icon included in the virtual screen displayed on the display 151 and then touching or double-touching the icon.

In FIG. 2, each of the main screen, the left and right screens displays menu icons in the standby state of the mobile phone. However, if a separate program is operated, an image suitable for each program can be displayed.

When the user tilts the mobile terminal to the right as shown in FIG. 3A in the state where the main screen 200 is shown in FIG. 2, the right screen 200 can display the display 151 as shown in FIG. 3B. At this time, the main screen 200 may be blurred as a background screen of the right screen 400. That is, the translucent images of the right screen 400 and the main screen 200 may overlap and be displayed. In the following description and drawings, if there is an indication such as box 217 at the bottom, it is assumed that this indicates that the screen displayed on the box 217 as the background screen overlaps with the background screen.

This operation can be similarly applied even when the mobile terminal 100 is tilted to the left. That is, when the user tilts the mobile terminal 100 to the left, the left screen 300 may be displayed, and the main screen 200 may be displayed as a background screen.

That is, when the subscreen is displayed, the main screen is overlapped with the background screen so that the user can recognize that the screen currently displayed corresponds to the subscreen. In addition, since the main screen overlaps with the background screen in a translucent state, events occurring on the main screen as well as the subscreen can be checked through one screen.

In some embodiments, the output image of the subscreens 300 and 400 may be displayed in a translucent manner while the main screen 200 is being displayed. It is also possible to display all the screens other than the currently displayed virtual screen in a translucent manner on the background screen.

At this time, the tilting gesture of the mobile terminal 100 may be detected by the tilt sensor 132 mounted on the mobile terminal 100. When the tilt of the mobile terminal 100 is detected by the tilt sensor 132, the control unit can switch the screen displayed on the display 151 according to the tilted direction of the terminal 100, as described above.

As the tilt sensor 132, a geomagnetic sensor, an acceleration sensor, a gyro sensor, or the like can be used. 4, a head tracking technique for processing a face image of a user photographed by a camera is applied using a camera 121 mounted on the front surface of the mobile terminal 100 The left-right tilt can be measured.

In FIG. 4, the user face 221 faces the display 151 of the mobile terminal 100. At this time, the camera 121 photographs the user's face 221. When the user tilts the mobile terminal 100 to the left or right, the proportion of the user's face in the image in the image taken by the camera 121 becomes smaller than when the mobile terminal 100 faces the user's face. Further, it is possible to judge which portion of the user's face is photographed in large quantities, and to determine which portion is inclined.

Thus, according to an embodiment of the present invention, three independent output images can be allocated to three virtual screens, and a virtual screen displayed according to a user's input can be switched and displayed. With such a configuration, the user can have the effect of using three displays even in a single small display, and can easily carry out necessary work while moving between the screens.

As described above, the virtual screens according to the present invention may be composed of one main screen and several sub-screens, or may be composed of several virtual screens without distinguishing between main and sub.

The output images displayed by the virtual screen according to the invention are all executed by independent processes and are not affected by the work being performed on the other screens. The virtual screens may execute instructions independently from each other similar to a multitasking task window or a multi-monitor (dual monitor, triple monitor, etc.) in a general PC, and display the results to the user.

As shown in FIGS. 5A to 5C, according to an embodiment of the present invention, an icon belonging to a screen displayed on the current display can be configured to be executed in another virtual screen.

5A, when the user selects one of the icons included in the output image of the main screen 200, for example, the calculator icon 220, and then, in a state of touching and holding, The right screen 400 is displayed as shown in FIG. 5B. Then, when the touch 222 is released in the state of Fig. 5B, the calculator function can be executed on the right screen 400 as shown in Fig. 5C.

5B and 5C, the right screen 400 is displayed. As described above, the main screen can be overlapped with the background screen, as described above.

The user may run the calculator on the right screen as shown in FIG. 5C and then return to the main screen 200 to execute another task. With this configuration, the user can specify a virtual screen on which the icon will be executed, regardless of which virtual screen the icon belongs to.

Similarly, when the mobile terminal 100 is tilted to the left in the state that the address book icon 223 is touched and held in FIG. 5A, a screen on which the address book function is executed can be displayed on the left screen 300 as shown in FIG. have. Similarly, the background screen of the left screen 300 may be overlapped with the main screen screen.

7A and 7B illustrate how to use the switch icons 204a, 204b, and 206 to execute an icon belonging to the currently displayed virtual screen on another screen.

7A, when the user selects one of the icons, for example, the calculator icon 220, and then drags and drops the icon on the right switch icon 204b, the right screen 400 The calculator can be executed. The main screen image can be displayed overlapping with the background image.

Similarly, by dragging and dropping to the left switch icon 204a, the calculator can be executed on the left screen. By dragging and dropping to the switch icon 206, the calculator can be executed on the right or left screen.

8 is a flowchart illustrating a method of executing an icon included in an output screen of a currently displayed virtual screen in another virtual screen according to an embodiment of the present invention.

In step S101, a touch on the icon included in the currently displayed virtual screen is detected.

In step S102, it is confirmed whether a gesture tilting the mobile terminal is detected. It may be ascertained whether any screen switching command has been detected instead of a tilting gesture. In step S103 without the screen switching command, if the touch release is detected, the icon touched in step S104 can be executed on the currently displayed virtual screen.

If the inclination of the mobile terminal is detected while the icon is touched, the mobile terminal switches to another virtual screen in step S105. If the touch is released in step S106, that is, when the finger is released, And the corresponding icon is executed in the virtual screen.

9A to 9C illustrate a method of moving an icon included in an output screen of a currently displayed virtual screen to another virtual screen according to an embodiment of the present invention.

9A, in a state where the main screen 200 is displayed, the user can select and drag an icon, for example, the map icon 220. [ After dragging and releasing the finger as it is, the map icon 220 can be moved to the selected position. If the position where the finger is released is the switch icon 204b as shown in FIG. 7A, the displayed screen is switched, and the corresponding map function will be executed on the other screen.

9B, the right screen 400 is displayed and the map icon 220 is also moved to the right screen 400 when the mobile terminal 100 is tilted to the right while the finger is not dragged . When the user releases the finger, i.e., releases the finger, in the state where the right screen 400 has been changed, the map icon 220 is moved to that position. That is, the map icon 220 is now included in the output image of the right screen 400.

In the state of FIG. 9B, the map icon 220 can be further dragged and moved to a desired position as shown in FIG. 9C. The icon can be moved to a virtual screen desired by the user even among a plurality of virtual screens in which independent processes are executed through the above operation.

Fig. 10 is a flowchart showing the method described with reference to Figs. 9A to 9C.

In step S201, an icon is selected, and a gesture for dragging the icon can be detected. In step S202, it is determined whether a command for switching the virtual screen has been input. That is, it is determined whether the mobile terminal is inclined in one direction. If a touch cancel is detected in step S203 while no switching command is input, the position of the touched icon is moved within the currently displayed virtual screen, and the process is terminated.

In step S202, if an instruction to switch the virtual screen has been input, in step S205, switching to another virtual screen is performed. In step S206, if the touch release is detected, in step S207, Moves the selected icon to the virtual screen. If you drag further in the moved state, the position of the icon within the switched virtual screen is shifted.

In accordance with one embodiment of the present invention, it is possible to synchronize the tasks being executed in a plurality of virtual screens, copy the text displayed on one virtual screen, attach it to another virtual screen, or display the image displayed on one virtual screen on another virtual screen Attaching it to the text being edited, and so on.

11 illustrates a method of attaching a preview image to a text being edited while performing a text editing operation in one virtual screen and an image preview function in another virtual screen according to an embodiment of the present invention.

On the left side of Fig. 11, a screen for performing a text editing operation using the mobile terminal 100, for example, a text message creation job, on the main screen 200 is shown. On the right side of Fig. 11, a screen on which the image preview function is being executed is displayed on the right screen 400.

In the state where the right screen 400 is displayed on the display, the image file 310 is selected and the image file 310 is moved to the main screen 200 in the same manner as the icon described with reference to FIGS. 9A to 9C . That is, when the image file 310 is touched and dragged and then the mobile terminal 100 is tilted to the left, the screen is switched to the main screen 200. When the image file 310 is dropped into the text message editing window, The image file 310 may be attached to the message being transmitted.

The above embodiment can be applied not only to image files but also to all kinds of multimedia files such as moving picture files and music files.

In addition, the text to which the multimedia file is attached may be any text capable of attaching a file or inserting a file such as a text message, e-mail, or other word file.

That is, an e-mail may be written on the main screen 200, then switched to the right or left screen, a media file icon to be attached may be displayed, and a media file may be dragged and attached to the e-mail being created on the main screen 200.

In addition, if a word program is being executed in the main screen 200, a picture, an audio file, or a moving picture file can be inserted into the word file being created by dragging the image file of the right or left screen.

According to one embodiment of the present invention, text contained in an output image running on another virtual screen can be selected and pasted to the currently displayed virtual screen. In other words, copy and paste functions can be executed on a general PC between different task windows.

The left side of FIG. 12 shows a screen on which the text editing function, for example, a text message creating function, is being executed on the main screen 200, and the right screen 400 shows a screen on which the Internet browser is running.

By dragging the desired portion on the right screen 400, the text 320 can be selected. The selected text 320 can be displayed in blocks or inverted to be distinguished from other parts. Then, when the selected text 320 is dragged and then the screen is switched, it is moved to the main screen 200. When the user touches the desired position of the text message creation window and then releases the touch, ) May be copied.

As in the embodiment of Fig. 11, the embodiment of Fig. 12 can be applied not only to text message creation but also to arbitrary text editing windows such as e-mail creation, word processing, and the like.

13 is a flowchart illustrating a method of copying and pasting text or media files of FIG. 11 or 12, according to an embodiment of the present invention.

In step S301, a text editing function such as a text message, e-mail, or word is executed in one of the plurality of virtual screens. In step S302, a text or a media file is selected from a screen including a certain text such as an Internet browser or a list of media files or a screen including an icon on another virtual screen.

The text can be selected by the block setting command by dragging, and the media file can be selected by touch or click.

If the selected text is the text in step S303, the process proceeds to step S304, and if it is the media file, the process proceeds to step S309.

If the text selected in the other virtual screen is text, in step S304, a gesture that dragged the selected text in an arbitrary direction in the virtual screen on which the current text is displayed is detected. In step S305, it is determined whether a screen switching command has been input. If no screen switching command is input, the process is terminated.

If a screen switching command has been input, in step S306, the text editing function switches to the running virtual screen. If the touch on the selected text is canceled in step S307, the text selected at the cursor of the currently edited text is attached in step S308. In some embodiments, if the user further dragged within the virtual screen in which the text editing function is running, the selected text may be pasted at that location.

If the selected file is a media file in step S303, then in step S309, dragging of the media file is detected in the virtual screen to which the media file belongs. In step S310, it is determined whether a screen switching command has been input. If no screen switching command has been input, the media file icon is moved to the dragged position in step S311 and the process is terminated. If a screen switching command has been input, then in step S312 the text editing is switched to the running virtual screen and in step S313 a touch release is detected and in step S314 a media file is added to the text file being edited You can attach it. That is, if the text being edited is a text message or an email, a media file can be attached. If the text file being edited is a word file, you can insert the media file at the current cursor position of the text or at a location specified by the user.

Figures 14 and 15 illustrate an embodiment for performing different tasks on a plurality of virtual screens, in accordance with one embodiment of the present invention.

In Fig. 14, it is possible to execute an internet browser on the main screen 200 and execute a dictionary function on another sub screen, for example, the right screen 400. [ When the user is surfing the Internet through the main screen 200 and finds an unknown word, the user can switch to the right screen 400 and search for the word.

In FIG. 15, a video call can be executed on the main screen 200, and an Internet browser can be executed on another sub screen, for example, the right screen 400. When the user wants to know information, for example, when the movie screening schedule information is needed, the user switches to the Internet browser screen of the right screen 400 to search for necessary information, Can be determined.

The mobile terminal and its control method of the present invention described above can be applied to a mobile terminal and a control method of the mobile terminal according to embodiments of the present invention. Some of which may be selectively combined.

1 is a block diagram of a mobile terminal 100 according to an embodiment of the present invention.

2 illustrates an operation of the mobile terminal 100 capable of implementing a plurality of virtual screens according to an embodiment of the present invention.

3A and 3B show an example in which, when the user tilts the mobile terminal in one direction, another virtual screen is displayed according to an embodiment of the present invention.

4 shows an example of detecting a slope of a mobile terminal using a camera according to an embodiment of the present invention.

Figures 5A through 5C and 6 illustrate a method of executing an icon belonging to a screen currently displayed on a different virtual screen according to an embodiment of the present invention.

7A and 7B illustrate a method of executing an icon belonging to a currently displayed virtual screen on another screen using a switch icon, according to an embodiment of the present invention.

8 is a flowchart illustrating a method of executing an icon included in an output screen of a currently displayed virtual screen in another virtual screen according to an embodiment of the present invention.

9A to 9C illustrate a method of moving an icon included in an output screen of a currently displayed virtual screen to another virtual screen according to an embodiment of the present invention.

Fig. 10 is a flowchart showing the method described with reference to Figs. 9A to 9C.

11 illustrates a method of attaching a preview image to a text being edited while performing a text editing operation in one virtual screen and an image preview function in another virtual screen according to an embodiment of the present invention.

12 shows a screen on which a text editing function is being executed on the main screen and a screen on which the Internet browser is running on the right screen, according to an embodiment of the present invention.

13 is a flowchart illustrating a method of copying and pasting text or media files of FIG. 11 or 12, according to an embodiment of the present invention.

Figures 14 and 15 illustrate an embodiment for performing different tasks on a plurality of virtual screens, in accordance with one embodiment of the present invention.

Claims (23)

delete delete delete delete delete delete delete delete delete delete delete Creating a plurality of virtual screens having independent output images; Creating a text editing screen on one of the virtual screens; Detecting a gesture for selecting an object in one of the other virtual screens; Detecting a screen switching gesture into a virtual screen including the text editing screen; Displaying a screen including the text editing screen; Detecting a gesture by which the user releases the touch; And attaching the selected object to a position where the touch of the text editing screen is released. 13. The method of claim 12, Wherein the text is a text message or an e-mail. 13. The method of claim 12, Wherein the object is one of a text, an emoticon, and an icon. Creating a plurality of virtual screens having independent output images; Creating an edit screen of text on one of the virtual screens; Detecting a gesture of selecting and dragging a media file in one of the other virtual screens; Detecting a screen switching gesture; Displaying a screen on which the editing screen of the text is created; Detecting a gesture by which the user releases the touch; And attaching the selected media file as an attachment to the text. 16. The method of claim 15, Wherein the text is a text message or an e-mail. 16. The method of claim 15, Wherein the media file is any one of an image file, a moving picture file, and a text file. delete delete delete delete delete delete

Images