KR20160102835A - Terminal device and operating method thereof - Google Patents

Abstract

An operating method of a terminal capable of providing information on other programs or external inputs connected to a terminal according to an embodiment of the present invention comprises: a step of outputting an execution screen of a program on the entire screen of the terminal; a step of receiving a screen division request to divide the entire screen into a first area and a second area; and a step of outputting an execution screen of the program on the first area and a multi-tasking screen to provide a multi-tasking environment on the second area at the screen division request, wherein the multi-tasking screen includes a program list containing at least one other program except for the program and an external input list containing at least one external input connected to the terminal in a wired or wireless manner.

Description

[0001] TERMINAL DEVICE AND OPERATING METHOD THEREOF [0002]

The present invention relates to a terminal and a method of operation thereof.

The spread of mobile devices such as smart phones and smart pads makes simple tasks on mobile devices, but tasks such as detailing tasks and multitasking are performed in desktop or laptop environments.

Thus, in the case of a job having a high degree of complexity and difficulty, interworking with a B or C job and data application may be required for an operation A However, it is not easy and simple to arrange windows for a plurality of applications on a single screen in a suitable manner. Generally, it is necessary to adjust the size of the window with the input device such as a mouse, move the position, and arrange the window corresponding to the other application to the remaining area. Because of this inconvenience, After switching the program and executing the necessary tasks, it is going to return to the existing program by typing alt + Tab again.

However, there is a disadvantage in the simultaneous task type, not the cross type task, that the conversion of the full screen is complicated.

An object of the present invention is to provide a terminal capable of providing information on another program or external input connected to the terminal in a simple and fast way while executing a single program, and a method of operating the terminal.

A method of operating a terminal according to an exemplary embodiment of the present invention includes the steps of outputting a program execution screen to a full screen of the terminal, receiving a screen division request for dividing the entire screen into a first area and a second area, And outputting an execution screen of the program to the first area and a multitasking screen for providing a multitasking environment to the second area in response to a screen division request, A list of programs including one or more other programs, and an external input list including one or more external inputs that are wired or wirelessly connected to the terminal.

A terminal according to an embodiment of the present invention includes a user input interface unit for receiving a request for screen control of the terminal and an execution screen of a program on the entire screen of the terminal, And a controller for outputting an execution screen of the program in the first area and a multitasking screen for providing a multitasking environment in the second area in response to the screen division request And the multitasking screen includes at least one of a program list including one or more other programs other than the program and an external input list including one or more external inputs connected to the terminal in a wired or wireless manner.

According to various embodiments of the present invention, a user can quickly and easily arrange a screen of another program or a screen of an external input connected to the terminal on one screen during execution of one program.

In addition, high usability can be provided by changing the display mode according to the work situation.

1 is a block diagram illustrating a configuration of a terminal according to an embodiment of the present invention.
2 is a block diagram of a remote control device according to an embodiment of the present invention.
3 shows an example of an actual configuration of a remote control apparatus according to an embodiment of the present invention.
4 shows an example of utilizing a remote control device according to an embodiment of the present invention.
5 is a flowchart illustrating an operation method of a terminal according to an embodiment of the present invention.
6 is a view for explaining an embodiment of outputting an execution screen of a first program on a full screen according to an embodiment of the present invention.
7 is a view for explaining an embodiment in which a second window is additionally output by dividing the entire screen into a first area and a second area according to an embodiment of the present invention.
8 to 9 show examples of screens provided when one or more program items are selected in the program list on the second window according to an embodiment of the present invention.
FIGS. 10 to 12 are diagrams for explaining an embodiment in which an execution screen of another program is additionally output through a second window for providing a multitasking environment according to an embodiment of the present invention.
13 to 14 are views for explaining an embodiment in which a layout constituting an entire screen is changed according to another embodiment of the present invention, and a multitasking operation can be performed through a changed layout.
15 to 16 show an example of a screen provided when an external input icon for screen mirroring is selected from an external input list on the second window according to an embodiment of the present invention.
17 to 18 show an example of a screen provided when an external input icon representing an HDMI input is selected from an external input list on the second window according to an embodiment of the present invention.
FIG. 19 shows an example of a screen provided when an external input icon representing a TV input among the external input list on the second window is selected according to an embodiment of the present invention.
FIGS. 20 to 29 are views for explaining an embodiment of moving information output on an execution screen of a program to a screen of an external input according to various embodiments of the present invention and transmitting the information to the mobile terminal.
30 is a flowchart illustrating an operation method of a terminal according to another embodiment of the present invention.
31 to 32 are views for explaining a process of operating an external input screen according to another embodiment of the present invention.
33 to 34 are views for explaining an embodiment of moving information output from an external input screen to a program execution screen and receiving corresponding information from a mobile terminal according to various embodiments of the present invention.
35 to 36 are diagrams for explaining an embodiment for providing a divided screen according to receiving a screen mirroring request from a mobile terminal.
37 to 38 are diagrams illustrating a process of capturing a screen of a terminal through a screen capture function of the mobile terminal according to an embodiment of the present invention.
FIG. 39 is a diagram illustrating an example in which a mobile terminal is controlled using a wireless touch pad according to an embodiment of the present invention. Referring to FIG.
40 is a diagram illustrating an example of controlling a mobile terminal using a mobile terminal as a wireless keyboard according to an embodiment of the present invention.
41 is a diagram illustrating an example in which a mobile terminal is controlled using a remote controller according to an embodiment of the present invention.
42 to 43 are diagrams for explaining an embodiment for providing a screen mode corresponding to a file type based on a file type displayed on a screen of a terminal according to an embodiment of the present invention.

Hereinafter, embodiments related to the present invention will be described in detail with reference to the drawings. The suffix "module" and " part "for the components used in the following description are given or mixed in consideration of ease of specification, and do not have their own meaning or role.

A terminal according to an embodiment of the present invention is an intelligent terminal that adds a computer support function to a broadcast reception function, for example, and is equipped with an Internet function while being faithful to a broadcast reception function, A remote controller and the like can be provided. Also, it can be connected to the Internet and a computer by the support of a wired or wireless Internet function, and can perform functions such as e-mail, web browsing, banking or game. A standardized general-purpose OS can be used for these various functions.

Accordingly, the terminal described in the present invention can perform various user-friendly functions because various applications can be freely added or deleted on a general-purpose OS kernel, for example. More specifically, the terminal may be, for example, a network TV, an HBBTV, a smart TV, an LED TV, an OLED TV, or the like, and may be applied to a smartphone in some cases.

In another embodiment of the present invention, the terminal may be a computer, a notebook, or a laptop.

In another embodiment of the present invention, the terminal may be a digital signage device providing digital signage services. Digital signage device is a communication tool that can induce marketing, advertisement, training effect and customer experience of companies. It is a digital image device that provides specific information as well as broadcast programs in public places such as airports, hotels, and hospitals.

1 is a block diagram illustrating a configuration of a terminal according to an embodiment of the present invention.

1, a terminal 100 includes a broadcast receiving unit 130, an external device interface unit 135, a storage unit 140, a user input interface unit 150, a controller 170, a display unit 180, An audio output unit 185, and a power supply unit 190.

The broadcast receiving unit 130 may include a tuner 131, a demodulation unit 132, and a network interface unit 133.

The tuner 131 can tune to a specific broadcast channel according to the channel tuning command. The tuner 131 may receive a broadcast signal for the selected broadcast channel.

The demodulator 132 can separate the received broadcast signal into a video signal, an audio signal, and a data signal related to a broadcast program, and can restore the separated video signal, audio signal, and data signal into a form capable of outputting.

The external device interface unit 135 may receive an application or application list in an adjacent external device and may transmit the received application or application list to the control unit 170 or the storage unit 140.

The network interface unit 133 may provide an interface for connecting the terminal 100 to a wired / wireless network including the Internet network. The network interface unit 133 can transmit or receive data to another user or another electronic device via the network connected thereto or another network linked to the connected network.

In addition, some content data stored in the terminal 100 can be transmitted to another user registered in the terminal 100 or a selected user or selected one of the other electronic devices.

The network interface unit 133 can access a predetermined web page through the connected network or another network linked to the connected network. That is, it is possible to access a predetermined web page through a network and transmit or receive data with the server.

The network interface unit 133 may receive content or data provided by the content provider or the network operator. That is, the network interface unit 133 can receive contents such as a cinema, an advertisement, a game, a VOD, a broadcasting signal, and the related information provided from a content provider or a network provider through a network.

In addition, the network interface unit 133 can receive update information and an update file of the firmware provided by the network operator, and can transmit data to the Internet or a content provider or a network operator.

The network interface unit 133 can select and receive a desired application among applications open to the public via the network.

The storage unit 140 stores programs for signal processing and control in the controller 170, and can store image-processed video, audio, or data signals.

The storage unit 140 may perform a function for temporarily storing video, audio, or data signals input from the external device interface unit 135 or the network interface unit 133, It may also store information about the image.

The storage unit 140 may store a list of applications or applications input from the external device interface unit 135 or the network interface unit 133. [

The terminal 100 may reproduce and provide a content file (a moving image file, a still image file, a music file, a document file, an application file, etc.) stored in the storage unit 140 to a user.

The user input interface unit 150 may transmit a signal input from any one of a mouse, a keyboard, and the remote controller 200 to the controller 170 or a signal from the controller 170 to the user. For example, the user input interface unit 150 may be implemented by various communication methods such as Bluetooth, Ultra Wideband (WB), ZigBee, RF (Radio Frequency) communication, and infrared (IR) It is possible to receive and process control signals such as power on / off, channel selection, screen setting, and the like from the remote control device 200, or transmit control signals from the control unit 170 to the remote control device 200. [

The user input interface unit 150 may transmit a control signal input from a local key (not shown) such as a power key, a channel key, a volume key, and a set value to the controller 170.

The video signal processed by the controller 170 may be input to the display unit 180 and displayed as an image corresponding to the video signal. The video signal processed by the controller 170 may be input to the external output device through the external device interface 135. [

The audio signal processed by the control unit 170 may be output to the audio output unit 185 through audio. The voice signal processed by the control unit 170 may be input to the external output device through the external device interface unit 135.

In addition, the control unit 170 can control the overall operation in the terminal 100. [

The control unit 170 can control the terminal 100 through a user command or an internal program input through the user input interface unit 150 and accesses a network and displays a list of applications or applications desired by the user in the terminal 100 ) To be downloaded.

The control unit 170 can output the video or audio signal processed through the channel information selected by the user through the display unit 180 or the audio output unit 185.

The control unit 170 controls the operation of the external device through the external device interface unit 135 according to an external device video playback command received through the user input interface unit 150. For example, So that a video signal or a voice signal can be output through the display unit 180 or the audio output unit 185.

The control unit 170 may control the display unit 180 to display an image and may output the broadcast image input through the tuner 131 or the external input through the external device interface unit 135, The image input through the network interface unit or the image stored in the storage unit 140 may be displayed on the display unit 180. [ In this case, the image displayed on the display unit 180 may be a still image or a moving image, and may be a 2D image or a 3D image.

In addition, the controller 170 may control the content stored in the terminal 100, the received broadcast content, and the external input content input from the outside to be reproduced. The content may be a broadcast image, an external input image, A still image, a connected web screen, and a document file.

The display unit 180 converts the video signal, the data signal, the OSD signal, the video signal, the data signal, and the like received from the external device interface unit 135 into R, G, and B signals, respectively, Signal can be generated.

Meanwhile, the terminal 100 shown in FIG. 1 is only an embodiment of the present invention. Some of the illustrated components may be integrated, added, or omitted depending on the specifications of the terminal 100 actually implemented.

That is, two or more constituent elements may be combined into one constituent element, or one constituent element may be constituted by two or more constituent elements, if necessary. In addition, the functions performed in each block are intended to illustrate the embodiments of the present invention, and the specific operations and apparatuses do not limit the scope of the present invention.

1, the terminal 100 does not include the tuner 131 and the demodulation unit 132, but may be connected to the network interface unit 133 or the external device interface unit 135 And the like.

For example, the terminal 100 may be divided into an image processing apparatus such as a set-top box for receiving broadcast signals or contents according to various network services, and a content reproducing apparatus for reproducing content input from the image processing apparatus .

In this case, the operation method of the terminal according to the embodiment of the present invention to be described below is not limited to the terminal 100 as described with reference to FIG. 1, but also to the image processing apparatus such as the set-top box or the display unit 180, And an audio output unit 185. The audio output unit 185 may be connected to the audio output unit 185,

The terminal may receive an input signal through an input device such as a mouse or a keyboard.

Next, a remote control apparatus according to an embodiment of the present invention will be described with reference to Figs. 2 to 3. Fig.

FIG. 2 is a block diagram of a remote control apparatus according to an embodiment of the present invention, and FIG. 3 shows an example of an actual configuration of a remote control apparatus 200 according to an embodiment of the present invention.

2, the remote control apparatus 200 includes a fingerprint recognition unit 210, a wireless communication unit 220, a user input unit 230, a sensor unit 240, an output unit 250, a power supply unit 260 A storage unit 270, a control unit 280, and a voice acquiring unit 290. [0034]

Referring to FIG. 2, the wireless communication unit 225 transmits / receives signals to / from any one of the terminals according to the embodiments of the present invention described above.

The remote control apparatus 200 includes an RF module 221 capable of transmitting and receiving signals to and from the terminal 100 according to an RF communication standard and includes an IR module (223). In addition, the remote control device 200 may include a Bluetooth module 225 capable of transmitting and receiving signals with the terminal 100 according to the Bluetooth communication standard. The remote control apparatus 200 includes an NFC module 227 capable of transmitting and receiving signals to and from the terminal 100 in accordance with a NFC (Near Field Communication) communication standard. 100 and a WLAN module 229 capable of transmitting and receiving signals.

The remote control device 200 transmits a signal including information on the motion of the remote control device 200 to the terminal 100 through the wireless communication unit 220.

The remote control device 200 can receive a signal transmitted from the terminal 100 through the RF module 221 and can turn on and off the power to the terminal 100 through the IR module 223, Channel change, volume change, and the like.

The user input unit 230 may include a keypad, a button, a touch pad, or a touch screen. The user can input a command related to the terminal 100 to the remote control device 200 by operating the user input unit 230. [ When the user input unit 230 includes a hard key button, the user can input a command related to the terminal 100 to the remote control device 200 through the push operation of the hard key button. This will be described with reference to FIG.

Referring to FIG. 3, the remote control apparatus 200 may include a plurality of buttons. The plurality of buttons include a fingerprint recognition button 212, a power button 231, a home button 232, a live button 233, an external input button 234, a volume control button 235, a voice recognition button 236, A channel change button 237, an OK button 238, and a back button 239. [

The fingerprint recognition button 212 may be a button for recognizing the fingerprint of the user. In one embodiment, the fingerprint recognition button 212 is capable of a push operation and may receive a push operation and a fingerprint recognition operation. The power button 231 may be a button for turning on / off the power of the terminal 100. The home button 232 may be a button for moving to the home screen of the terminal 100. The live button 233 may be a button for displaying a real time broadcast program. The external input button 234 may be a button for receiving an external input connected to the terminal 100. The volume control button 235 may be a button for controlling the magnitude of the volume output by the terminal 100. The voice recognition button 236 may be a button for receiving the user's voice and recognizing the received voice. The channel change button 237 may be a button for receiving a broadcast signal of a specific broadcast channel. The confirmation button 238 may be a button for selecting a specific function, and the back button 239 may be a button for returning to a previous screen.

Fig. 2 will be described again.

When the user input unit 230 includes a touch screen, the user may touch a soft key of the touch screen to input a command related to the terminal 100 to the remote control device 200. [ In addition, the user input unit 230 may include various types of input means such as a scroll key, a jog key, and the like that can be operated by the user, and the present invention does not limit the scope of the present invention.

The sensor unit 240 may include a gyro sensor 241 or an acceleration sensor 243 and the gyro sensor 241 may sense information on the motion of the remote control device 200. [

For example, the gyro sensor 241 may sense information about the operation of the remote control device 200 on the basis of the x, y, and z axes, and the acceleration sensor 243 may sense the movement speed of the remote control device 200 And the like can be sensed. The remote control device 200 may further include a distance measuring sensor to sense the distance to the display unit 180 of the terminal 100. [

The output unit 250 may output an image or a voice signal corresponding to an operation of the user input unit 235 or a signal transmitted from the terminal 100. The user can recognize whether the user input unit 235 is operated or whether the terminal 100 is controlled through the output unit 250.

For example, the output unit 250 may include an LED module 251 that is turned on when a user input unit 235 is operated or a signal is transmitted to or received from the terminal 100 through the wireless communication unit 225, a vibration module 253 An audio output module 255 for outputting sound, or a display module 257 for outputting an image.

In addition, the power supply unit 260 supplies power to the remote control device 200, and if the remote control device 200 is not moved for a predetermined time, the power supply is interrupted to reduce power consumption. The power supply unit 260 may resume power supply when a predetermined key provided in the remote control device 200 is operated.

The storage unit 270 may store various types of programs, application data, and the like necessary for the control or operation of the remote control apparatus 200. [ If the remote control apparatus 200 wirelessly transmits and receives a signal through the terminal 100 and the RF module 221, the remote control apparatus 200 and the terminal 100 transmit and receive signals through a predetermined frequency band.

The control unit 280 of the remote control device 200 stores information on a frequency band or the like capable of wirelessly transmitting and receiving a signal with the terminal 100 paired with the remote control device 200 in the storage unit 270, .

The control unit 280 controls various matters related to the control of the remote control device 200. [ The control unit 280 transmits a signal corresponding to a predetermined key operation of the user input unit 235 or a signal corresponding to the motion of the remote control device 200 sensed by the sensor unit 240 to the terminal 100 ).

Also, the voice acquisition unit 290 of the remote control device 200 can acquire voice.

The voice acquisition unit 290 may include at least one microphone 291 and may acquire voice through the microphone 291.

Next, Fig. 4 will be described.

4 shows an example of utilizing a remote control device according to an embodiment of the present invention.

4A illustrates that the pointer 205 corresponding to the remote control device 200 is displayed on the display unit 180. In FIG. Hereinafter, the pointer 205 corresponds to the remote control device 200, but is not limited thereto, and may correspond to an input device such as a mouse.

The user can move or rotate the remote control device 200 up, down, left or right. The pointer 205 displayed on the display unit 180 of the terminal 100 corresponds to the movement of the remote control device 200. [ In this remote control device 200, as shown in the figure, since the pointer 205 is moved according to the movement in the 3D space, it can be called a space remote controller.

4B illustrates that when the user moves the remote controller 200 to the left, the pointer 205 displayed on the display unit 180 of the terminal 100 moves to the left correspondingly.

Information on the motion of the remote control device 200 sensed through the sensor of the remote control device 200 is transmitted to the terminal 100. The terminal 100 can calculate the coordinates of the pointer 205 from the information on the motion of the remote control device 200. [ The terminal 100 may display the pointer 205 so as to correspond to the calculated coordinates.

4C illustrates a case where the user moves the remote control device 200 away from the display unit 180 while the specific button in the remote control device 200 is depressed. Thereby, the selected area in the display unit 180 corresponding to the pointer 205 can be zoomed in and displayed.

Conversely, when the user moves the remote control apparatus 200 close to the display unit 180, the selection area within the display unit 180 corresponding to the pointer 205 may be zoomed out and displayed.

On the other hand, when the remote control device 200 moves away from the display unit 180, the selected area is zoomed out, and when the remote control device 200 approaches the display unit 180, the selected area may be zoomed in.

Further, the recognition of the upward, downward, and leftward / rightward movement can be excluded when a specific button in the remote control device 200 is pressed. That is, when the remote control device 200 moves away from or approaches the display unit 180, it is not recognized that the up, down, left, and right movements are recognized, and only the forward and backward movements are recognized. Only the pointer 205 is moved in accordance with the upward, downward, leftward, and rightward movement of the remote control device 200 in a state where the specific button in the remote control device 200 is not pressed.

On the other hand, the moving speed and moving direction of the pointer 205 may correspond to the moving speed and moving direction of the remote control device 200.

The pointer in the present specification means an object displayed on the display unit 180 in correspondence with the operation of the remote control device 200. [ Therefore, various shapes of objects other than the arrow shapes shown in the figure are possible with the pointer 205. [ For example, it can be a concept that includes points, cursors, prompts, thick outlines, and so on. The pointer 205 may be displayed corresponding to a point on a horizontal axis or a vertical axis on the display unit 180 or may be displayed corresponding to a plurality of points such as a line or a surface Do.

Next, a method of operating a terminal according to an embodiment of the present invention will be described with reference to FIG.

5 is a flowchart illustrating an operation method of a terminal according to an embodiment of the present invention.

The control unit 170 of the terminal 100 outputs a first window indicating the execution screen of the first program to the entire screen of the display unit 180 (S101). In one embodiment, the program may correspond to any one of an application program and an application installed in the terminal 100.

The control unit 170 of the terminal 100 receives a screen division request for dividing the entire screen into the first area and the second area in step S103, and in response to the received screen division request, And outputs the second window to the second area (S105).

In one embodiment, the screen division request may be a request to divide the entire screen of the display unit 180 into a first area and a second area having the same area, but the present invention is not limited thereto. That is, the area of the first area and the area of the second area may be different from each other. The second window may be a multitasking screen for providing a multitasking environment to the user.

In one embodiment, the second window may include a program list indicating programs other than the first program and an external input list indicating an external input connected to the terminal 100 in a wired or wireless manner.

In another embodiment, a screen of a peripheral device connected to the terminal 100 via a wired or wireless connection may be output on the second window. This will be described later.

This will be described later.

Steps S101 to S105 will be described below.

6 is a view for explaining an embodiment of outputting an execution screen of a first program on a full screen according to an embodiment of the present invention.

Referring to FIG. 6, the controller 170 may control the display unit 180 to display a first window 400 corresponding to an execution screen of the first program on the entire screen 300. For example, the first window 400 may be provided with a web site connection screen.

Next, Fig. 7 will be described.

7 is a view for explaining an embodiment in which a second window is additionally output by dividing the entire screen into a first area and a second area according to an embodiment of the present invention.

6, when the screen division request is received, the controller 170 may divide the entire screen 300 into the first area 310 and the second area 330 as shown in FIG. 7 have.

In one embodiment, when the terminal 100 is a PC or a notebook, a screen division request may be generated by an input of simultaneously pressing a control key and a tab key provided on the keyboard. Here, the screen division request is generated by inputting a control key and a tab key at the same time, which is an example only, and can be changed according to the setting of the user.

In another embodiment, when the terminal 100 is a TV such as a smart TV, a screen division request may be generated by an input of pressing a specific button provided on the remote control device 200. [

The controller 170 displays a first window 400 indicating the execution window of the first program in the first area 310 and a second window window 500 for providing a multitasking environment in the second area 320 The display unit 180 may be controlled. The size of the first window 400 may be reduced according to the size of the first area 310.

Here, the area of the first area 310 and the area of the second area 330 are described as being the same, but this is merely an example, and the areas may be different from each other.

On the second window 500, a program list 510 and an external input list 530 can be displayed. The program list 510 may include one or more program items 511-516.

In one embodiment, each program item may represent a recent execution picture corresponding to a program recently executed in the terminal 100 or a thumbnail image corresponding to a recently executed program. The one or more program items 511 to 516 may be arranged in order according to the latest execution time. That is, the first program item 511 is the item corresponding to the most recently executed program, and the second program item 512 to the sixth program item 516 may follow.

In another embodiment, each program item may represent a thumbnail image corresponding to a program registered as a favorite or a favorite execution screen corresponding to programs registered as favorites.

The external input list 530 may include one or more external input icons. Each external input icon may represent an external input wirelessly or wired to the terminal 100. For example, the first external input icon 531 may be an icon representing an external input connected through a screen mirroring technique. The screen mirroring technology is a technology for sharing data between terminals by transmitting data through a short distance wireless communication technology such as Wi-Fi and Bluetooth. The second external input icon 533 may be an icon representing an external input connected through the HDMI (High Definition Multimedia Interface) standard. The third external input icon 535 may be an icon representing a wired or wireless connected TV. Although not shown in FIG. 7, the external input list 530 may further include peripheral devices such as a DVR, a Blu-ray player, a USB memory, a terminal for transmitting media content using a Widi (wireless display) standard, And may further include external input icons indicating the external input icons.

On each external input icon, one or more of a name identifying the external input, a thumbnail image of the media content recently played through the external input, and a name of an externally input peripheral device may be displayed.

The external input icons 531 to 535 may be arranged in order according to the point in time when the terminal 100 is recently connected. For example, if the first external input icon 531 is the icon of the external input most recently connected to the terminal 100, and the second external input icon 533 and the third external input icon 535 can follow have.

In yet another embodiment of the present invention, the external input list 530 may display thumbnail images of media content reproduced through an external input instead of an external input icon.

In yet another embodiment of the present invention, thumbnail images of the media content reproduced through the external input may be displayed on each external input icon.

5 is described again.

The control unit 170 of the terminal 100 transmits a selection request for selecting one of one or more program items representing a program contained on the output second window and one or more external input icons representing an external input (S107), and outputs one or more windows of a first window, which is an execution window corresponding to the selected program item, and a second window, which is an external input window, corresponding to the selected external input icon according to the received selection request (S109).

Hereinafter, steps S107 to S109 will be described in detail.

8 to 9 show examples of screens provided when one or more program items are selected in the program list on the second window according to an embodiment of the present invention.

Especially. FIG. 8 shows an embodiment of a screen provided when one program item is selected in the program list, and FIG. 9 shows an embodiment provided when two program items are selected in the program list.

First, Fig. 8 will be described.

When a request to select the first program item 511 in the program list 510 of FIG. 7 is received, the control unit 170 controls the third window 600 (FIG. 7), which is the execution window of the program corresponding to the first program item 511 May be displayed on the second area 330. [0051] In this case, the second window 500 that has been displayed in the second area 330 may not be displayed due to the third window 600. In one embodiment, when the terminal 100 is a PC or a notebook, a request to select the first program item 511 may be made by pressing a mouse click or a control key on the keyboard and a Tab Key Lt; / RTI > In another embodiment, when the terminal 100 is a TV, a request to select the first program item 511 may be performed via the remote control device 200. [

The user can easily execute another program through the information of the programs displayed on the second window 500 to perform multitasking work.

Next, Fig. 9 will be described.

When a request to select the first program item 511 and the second program item 513 is received from the program list 510 of FIG. 7, the control unit 170 determines whether the program corresponding to the first program item 511 It is possible to control the display unit 180 to display the fourth window 700 as the execution window of the program corresponding to the third window 600 and the second program item 513 in the second area 330 have. The second region 330 may be divided into a first sub-region 331 and a second sub-region 333 when a request to select a first program item 511 and a second program item 513 is received have. The third window 600 may be displayed on the first sub region 331 and the fourth window 700 may be displayed on the second sub region 700. [ In this case, the second window 500 that has been displayed in the second area 330 may not be displayed due to the third window 600 and the fourth window 700.

In one embodiment, when the terminal 100 is a PC or a notebook, a request to select a first program item 511 and a second program item 513 may be made by pressing a control key on the keyboard, May be generated by an input selecting the first program item 511 and the second program item 513. In another embodiment, when the terminal 100 is a TV, a request to select the first program item 511 and the second program item 513 may be performed via the remote control device 200. [

The user can easily execute another program through the information of the programs displayed on the second window 500 to perform multitasking work.

According to another embodiment of the present invention, a second window for providing a multitasking environment may be output again while displaying execution screens of a program. In addition, an execution screen of another program can be additionally output through the output second window.

This will be described with reference to FIGS. 10 to 12. FIG.

FIGS. 10 to 12 are diagrams for explaining an embodiment in which an execution screen of another program is additionally output through a second window for providing a multitasking environment according to an embodiment of the present invention.

10 (a) is the same as the view of Fig. 10 (a), when the second window output request for outputting the second window 500 for providing the multitasking environment is received, the controller 180, as shown in FIG. 10 (b) The display unit 180 may be controlled to additionally display the second window 500 on the full screen 300. [ Here, it is assumed that the full screen 300 includes a fourth area 351, a fifth area 353, and a sixth area 355. The control unit 180 displays the first window 400 as the execution window of the program corresponding to the first program in the fourth area 351 of the full screen 300 according to the second window output request, The display unit 180 displays the third window 600 as the execution window of the program corresponding to the first window 351 in the fifth area 353 and displays the second window 500 in the sixth area 355 Can be controlled. If the third program item is selected on the second window 500, the control unit 180 can control the display unit 180 to display the execution window of the program corresponding to the third program item in the sixth area 355 have.

In one embodiment, when the terminal 100 is a PC or a notebook, a request to output the second window 500 in FIG. 10 may include a control key and a tab key provided on the keyboard Can be generated by an input that is pressed simultaneously.

In another embodiment, when the terminal 100 is a TV, such as a smart TV, a request to output the second window 500 may be generated by an input of pressing a specific button provided on the remote control device 200. [

Next, Fig. 11 will be described.

11 (a) is the same view as FIG. 10 (b).

11 (a), when the layout change request for changing the layout of the screens constituting the entire screen 300 is received, the control unit 180 controls the first window 400, the second window 500, and the third window 600 can be changed. That is, in response to the layout change request, the controller 180 sets the first window 400 to the first area 310, the third window 600 to the first sub-area 331 of the second area 330, It is possible to control the display unit 180 to display the second window 500 in the second sub-area 333 of the second area 300. In one embodiment, if the terminal 100 is a PC or a notebook, a layout change request may be generated by an input that simultaneously presses the control key and direction key of the keyboard. In another embodiment, when the terminal 100 is a TV, a layout change request may be received via the remote control device 200. [

Next, Fig. 12 will be described.

When the third program item 513 is selected on the second window 500 of FIG. 11B, the controller 180 determines that the third program item 513 is selected in the second sub-area 330 of the second area 300, It is possible to control the display unit 180 to display the fourth window 700, which is an execution window of the program corresponding to the second window 513.

Next, with reference to FIGS. 13 to 14, a process of changing the layout of the entire screen 300 according to another embodiment of the present invention and performing a multitasking operation through the changed layout will be described.

13 to 14 are views for explaining an embodiment in which a layout constituting the entire screen 300 is changed according to another embodiment of the present invention, and a multitasking operation can be performed through the changed layout.

First, Fig. 13 will be described.

Fig. 13 (a) is the same view as Fig. 13 (b), the controller 180 enlarges the size of the second area 330, and when the layout enlargement request for expanding the layout is received in the state of FIG. 13 (a), the controller 180 enlarges the size of the second area 330, The size of the first area 310 can be reduced. Accordingly, the size of the first window 400 can be reduced by the size of the reduced first region 310, and the size of the second window 500 can be enlarged by the size of the expanded second region 330 . In one embodiment, when the terminal 100 is a PC or a notebook, a layout enlargement request may be generated by an input that simultaneously presses a control key, a shift key, and a left directional key of the keyboard. In another embodiment, when the terminal 100 is a TV, a layout enlargement request can be received via the remote control device 200. [

On the other hand, when an input for simultaneously pressing the control key, the shift key, and the right direction key of the keyboard is received, the control unit 180 can reduce the length of the second window 500. The control unit 180 may increase the vertical length of the second window 500 when an input that simultaneously presses the control key, shift key and up key of the keyboard is received. The control unit 180 can reduce the vertical length of the second window 500 when an input for simultaneously pressing the control key, the shift key, and the downward key of the keyboard is received.

Next, Fig. 14 will be described.

Fig. 14 (a) is the same view as Fig. 13 (b). 14A, when a request to select the first program item 511 included in the program list 510 is received, the control unit 180 executes the program corresponding to the selected first program item 511 It is possible to control the display unit 180 to display the third window 600, which is a screen, in the enlarged second area 330. [

Next, an example of a screen provided when an external input icon for screen mirroring is selected from the external input list on the second window will be described.

15 to 16 show an example of a screen provided when an external input icon for screen mirroring is selected from an external input list on the second window according to an embodiment of the present invention.

When a request to select the first external input icon 531 for screen mirroring is received from the external input list 530 output to the second window 500 in Figure 15A, It is possible to control the display unit 180 to display the screen mirroring list screen 800 in the second area 330 of the full screen 300 as shown in (b) of FIG.

In one embodiment, the scroll mirroring list screen 800 may include a mobile terminal list 810 that is currently connected to the terminal 100 and can share a screen. The mobile terminal list 810 may include at least one of a name of each mobile terminal and a user name of each mobile terminal.

In another embodiment, the scroll mirror list screen 800 may include a mobile terminal list 810 including a mobile terminal connected to the terminal 100 and a mobile terminal that has been connected.

The terminal 100 may perform screen mirroring with the selected one or more mobile terminals when one or more mobile terminals included on the mobile terminal list 810 are selected. This will be described later.

Next, Fig. 16 will be described.

If a request to select the first external input icon 531 for screen mirroring is received from the external input list 530 output to the second window 500 in FIG. The display unit 180 may be controlled to display the screen mirroring list screen 800 in the second area 330 of the full screen 300 as shown in FIG.

Referring to FIG. 16, the screen mirroring list screen 800 may include a mobile terminal list 830 that is currently connected to the terminal 100 and can share a screen. The mobile terminal list 830 may include at least one of a screen of each mobile terminal, a name of each mobile terminal, and a user name of each mobile terminal. For example, the mobile terminal list 830 may include a screen 831 of a first mobile terminal, a screen 833 of a second mobile terminal, and a screen 835 of a third mobile terminal. In one embodiment, the screen of each mobile terminal may be the current screen of the mobile terminal connected to the terminal 100. In another embodiment, the screen of each mobile terminal may be a screen of the mobile terminal at the time when the connection with the mobile terminal 100 is terminated. When one of the screens is selected in the mobile terminal list 830, the controller 180 may output the screen of the selected mobile terminal to the second area 330 of the full screen 300. Accordingly, the terminal 100 may perform screen mirroring with the mobile terminal corresponding to the screen of the selected mobile terminal.

Next, an example of a screen provided when an external input icon indicating HDMI input is selected from the external input list is shown.

17 to 18 show an example of a screen provided when an external input icon representing an HDMI input is selected from an external input list on the second window according to an embodiment of the present invention.

17A, when a request to select the second external input icon 533 included in the external input list 530 output to the second window 500 is received, the display unit 180 is controlled to display the current screen 900 of another terminal connected to the terminal 100 through the HDMI standard in the second area 330 of the full screen 300 as shown in FIG. can do. The other terminal may be any one of a PC and a notebook. The screen 900 of the other terminal

Next, Fig. 18 will be described.

17B, when a request to output the second window 500 is received on the first area 310, the controller 180 controls the first area 310 to display the first window 310, A second window 500 for displaying the first window 400 in the first sub region 311 of the first region 310 and providing a multitasking environment to the second sub region 313 of the first region 310, It is possible to control the display unit 180 to display it. A request to output the second window 500 on the first area 310 may be a screen division request to divide the first area 310 into two sub-areas.

In the second window 500, a screen division icon (PBP, Picture By Picture icon, 550) indicating a screen division function may be displayed. 18 (a), when the request to select the screen division icon 550 is received, the controller 180 ends the screen division function and displays the full screen 300 The display unit 180 may be controlled such that only the first window 400 is displayed.

Next, an example of a screen provided when an external input icon representing TV input is selected from the external input list is shown.

FIG. 19 shows an example of a screen provided when an external input icon representing a TV input among the external input list on the second window is selected according to an embodiment of the present invention.

When a request to select a third external input icon 535 indicating TV input is received from the external input list 530 output to the second window 500 in FIG. 19 (a) It is possible to control the display unit 180 to display the TV screen 1000 in the second area 330 of the full screen 300 as shown in (b) of FIG. That is, the controller 180 controls the display unit 180 to display the TV screen 1000 connected to the terminal 100 in the second area 330 according to a request to select the third external input icon 535 . The TV program 1000 may display a program currently being reproduced on the TV.

5 is described again.

The control unit 180 of the terminal 100 receives the information movement input for moving the information output to the first window, which is the execution window corresponding to the program item, to the second window as the external input screen (S113) The information output in the first window according to the input is transmitted to the peripheral device corresponding to the external input (S115).

According to the embodiment of the present invention, the terminal 100 and the peripheral device can be interlocked with each other in accordance with the information movement input for moving the information output on the execution screen of the program to the external input screen connected to the terminal 100. This interlocking process will be described in detail below.

FIGS. 20 to 29 are views for explaining an embodiment of moving information output on an execution screen of a program to a screen of an external input according to various embodiments of the present invention and transmitting the information to the mobile terminal.

Hereinafter, it is assumed that a web site screen is displayed in the first area 310 of the full screen 300 and a mirroring screen of the mobile terminal 10 connected to the terminal 100 is displayed in the second area 330.

When a screen mirroring request for screen mirroring is received from the mobile terminal 10, steps S103 to S109 described in Fig. 5 may be omitted. That is, the control unit 180 divides the entire screen 300 into the first area 310 and the second area 330 upon receiving the screen mirroring request from the mobile terminal 10, The display unit 180 may be controlled to display the mirroring screen of the mobile terminal 10 on the first area 310 and display the mirroring screen of the mobile terminal 10 on the second area 330. [

In another embodiment, when a screen mirroring request for screen mirroring is received from the mobile terminal 10, only steps S107 to S109 may be omitted. That is, in a state where the entire screen 300 is divided into the first area 310 and the second area 330, the controller 180 receives the screen mirroring request from the mobile terminal 10, 1 display area 310 and display the mirroring screen of the mobile terminal 10 in the second area 330. [

First, Fig. 20 will be described.

20, the terminal 100 may display the website screen 1100 in the first area 310 of the entire screen 300 through the display unit 180, The mirroring screen 1200 of the mobile terminal 10 connected to the terminal 100 may be displayed in the area 330 through screen mirroring. The controller 180 may arrange the mirroring screen 1200 on the entire screen 300 according to the screen resolution of the mobile terminal 10. [ The control unit 180 receives information on the screen resolution from the mobile terminal 10 upon receiving the screen mirroring request and transmits the mirroring screen 1200 to the full screen 300 ). ≪ / RTI > The controller 180 may adjust the size of the second area 330 using information on the screen resolution of the mobile terminal 10. [ Accordingly, the controller 180 may provide a split screen on the entire screen 300 so that the screen of the mobile terminal 10 is implemented as it is. The resolution of the web site screen 1100 and the resolution of the mirroring screen 1200 may be different from each other.

When an information movement request for moving the selected text 1101 to the mirroring screen 1200 is received from the information displayed on the website screen 1100 of FIG. 20, (180) may transmit the text (1101) to the mobile terminal (10) according to the received information movement request. Here, the information movement request may be a drag-and-drop input through the pointer 205. The pointer 205 may be moved to the mirroring screen 1200. That is, the terminal 100 receives a command to move the pointer 205 from the website screen 1100 to the mirroring screen 1200, and displays the pointer 205 on the mirroring screen 1200 according to the received command . The user can also control the mirroring screen 1200 via the pointer 205. [ That is, the user may select an icon corresponding to the application displayed on the mirroring screen 1200 through the pointer 205 and execute the corresponding application. This will be described later.

The mobile terminal 10 may store the received text 1101 in the memory of the mobile terminal 10. [ The mobile terminal 10 may output an alarm indicating that the text 1101 has been received in the form of a message, voice, or vibration.

In another embodiment, as shown in FIG. 20, a text 1101 is selected from the information output on the web site screen 1100, and the selected text 1101 is displayed on a memo representing the memo application on the mirroring screen 1200 When the information movement request to move to the icon 1201 is received, the controller 180 may transmit the text 1101 to the mobile terminal 10 according to the received information movement request. The mobile terminal 10 can receive the text 1101 from the terminal 100 and execute the memo application at the same time. This will be described with reference to FIG.

21, the control unit 180 displays the web site screen 1100 and the memo information in the first area 310 of the full screen 300 according to an information movement request to move the text 1101 to the memo icon 1201. [ It is possible to control the display unit 180 to display the application execution screen 1200. That is, the information movement request for moving the text 1101 to the memo icon 1201 may be a trigger for executing the memo application installed in the mobile terminal 10. The mobile terminal 10 can also display an execution screen of the memo application. The execution screen 1200 of the memo application may include the text 1101 moved from the web site screen 1100. [

20 to 21 exemplify the memo application, but the present invention is not limited thereto. For example, when an information movement request to move the text 1101 to the SNS icon corresponding to the social network service (SNS) application is received, the control unit 180 can execute the SNS application. The user can transmit the text 1101 to another user's mobile terminal through the executed SNS application. In addition, when an information movement request for moving the text 1101 to a text message icon corresponding to the text message application is received, the control unit 180 can execute the text message application. The user can transmit the text 1101 to another user's mobile terminal through the executed text message application.

Next, Figs. 22 to 23 will be described.

22, the terminal 100 can display the website screen 1100 in the first area 310 of the entire screen 300 through the display unit 180, The mirroring screen 1200 of the mobile terminal 10 connected to the terminal 100 may be displayed in the area 330 through screen mirroring.

In one embodiment, when a phone number 1103 is selected from information output on the web site screen 1100 of FIG. 22 and an information movement request for moving the selected phone number 1103 to the mirroring screen 1200 is received , The control unit 180 may transmit the telephone number 1103 to the mobile terminal 10 according to the received information movement request. Here, the information movement request may be a drag-and-drop input through a pointer.

The mobile terminal 10 may store the received telephone number 1103 in the memory of the mobile terminal 10. [ The mobile terminal 10 may output an alarm indicating that the telephone number 1103 has been received in the form of a message, voice, or vibration.

In another embodiment, as shown in FIG. 22, the telephone number 1103 of the information output on the web site screen 1100 is selected, and the selected telephone number 1103 is transmitted to the calling application of the mirroring screen 1200 The control unit 180 may transmit the telephone number 1103 to the mobile terminal 10 in response to the received information movement request. The mobile terminal 10 can execute the communication application at the same time as receiving the telephone number 1103 from the terminal 100. [ This will be described with reference to FIG.

23, in response to an information movement request to move the telephone number 1103 to the call icon 1203, the control unit 180 displays the web site screen 1100 and the web page screen 1110 in the first area 310 of the full screen 300, The display unit 180 can be controlled to display the execution screen 1300 of the call application. That is, the information movement request for moving the telephone number 1103 to the call icon 1203 may be a trigger for executing the call application installed in the mobile terminal 10. [ The mobile terminal 10 can also display an execution screen of the call application. The execution screen 1300 of the call application may include the telephone number 1103 moved from the website screen 1100. [

The control unit 180 controls the telephone number 1103 in accordance with the information movement request to move the telephone number 1103 on the web site screen 1100 to the call icon 1203 of the mirroring screen 1200. In this case, The mobile terminal 10 can control the mobile terminal 10 to transmit a call signal to the mobile terminal corresponding to the mobile terminal 10. In this case, the screen of the mirroring screen 1200 and the screen of the mobile terminal 10 may be changed to a screen showing a process of transmitting a call signal.

Next, Fig. 24 to Fig. 25 will be described.

24, the terminal 100 displays the media content screen 1400 stored in the storage unit 140 of the terminal 100 in the first area 310 of the entire screen 300 through the display unit 180 And the terminal 100 may display the mirroring screen 1200 of the mobile terminal 10 connected to the terminal 100 through the screen mirroring in the second area 330. [

When an information movement request for selecting an image 1401 out of the information output on the media content screen 1400 of FIG. 24 and moving the selected image 1401 to the mirroring screen 1200 is received, The mobile terminal 180 may transmit the image 1401 to the mobile terminal 10 according to the received information movement request. Here, the information movement request may be a drag-and-drop input through a pointer.

The mobile terminal 10 may store the received image 1401 in the memory of the mobile terminal 10. [ The mobile terminal 10 may output an alarm indicating that the image 1401 has been received in the form of a message, voice, or vibration.

In another embodiment, as shown in FIG. 24, the image 1401 is selected from the information output on the media content screen 1400, and the selected image 1401 is displayed on a gallery (not shown) When the information movement request to move to the icon 1205 is received, the controller 180 may transmit the image 1401 to the mobile terminal 10 according to the received information movement request. The mobile terminal 10 can execute the gallery application upon receiving the image 1401 from the terminal 100. [ The gallery application may be an application that provides images or moving pictures stored in the memory of the mobile terminal 10. [ Next, Fig. 23 will be described.

25, the control unit 180 displays the media content screen 1400 and the gallery 1402 in the first area 310 of the full screen 300 according to an information movement request to move the image 1401 to the gallery icon 1205. [ The display unit 180 can be controlled to display the application execution screen 1500. [ That is, the information movement request for moving the image 1401 to the gallery icon 1205 may be a trigger for executing the gallery application installed in the mobile terminal 10. The mobile terminal 10 can also display an execution screen of the gallery application. The execution image 1500 of the gallery application may include the moved image 1401 and pre-stored images (image 2, image 3) from the media content screen 1400.

Next, Fig. 26 to Fig. 27 will be described.

26, the terminal 100 displays a media content screen 1400 stored in the storage unit 140 of the terminal 100 in the first area 310 of the entire screen 300 through the display unit 180 And the terminal 100 may display the mirroring screen 1200 of the mobile terminal 10 connected to the terminal 100 through the screen mirroring in the second area 330. [

When an information movement request for selecting the music file 1403 out of the information output on the media content screen 1400 of FIG. 26 and moving the selected music file 1403 to the mirroring screen 1200 is received , The control unit 180 may transmit the music file 1403 to the mobile terminal 10 according to the received information movement request. Here, the information movement request may be a drag-and-drop input through a pointer.

The mobile terminal 10 may store the received music file 1403 in the memory of the mobile terminal 10. [ The mobile terminal 10 may output an alarm indicating that the music file 1403 has been received in the form of a message, voice, or vibration.

26, the music file 1403 is selected from the information output on the media content screen 1400, and the selected music file 1403 is displayed on the music playback application 1200 of the mirroring screen 1200 The control unit 180 may transmit the music file 1403 to the mobile terminal 10 in response to the received information movement request. The mobile terminal 10 can receive the music file 1403 from the terminal 100 and execute the music reproduction application at the same time. The music application may be an application for playing music files stored in the memory of the mobile terminal 10. [ Next, Fig. 27 will be described.

27, the control unit 180 displays a media content screen 1400 and a media content screen 140 in the first area 310 of the entire screen 300 according to an information movement request to move the music file 1403 to the music icon 1207, The display unit 180 can be controlled to display the execution screen 1600 of the music playback application. That is, the information movement request for moving the music file 1403 to the music icon 1207 may be a trigger for executing the music playback application installed in the mobile terminal 10. The mobile terminal 10 can also display an execution screen of the music playback application. The execution screen 1600 of the music playback application includes a music playback screen 1610 and a music file 1403 for playing back the music file 1403 moved from the media content screen 1400 and pre-stored music files music 2, music 3 And a music file list 1630 including the music file list 1630. [

Next, Figs. 28 to 29 will be described.

27, the terminal 100 displays a media content screen 1400 stored in the storage unit 140 of the terminal 100 in the first area 310 of the entire screen 300 through the display unit 180 And the terminal 100 may display the mirroring screen 1200 of the mobile terminal 10 connected to the terminal 100 through the screen mirroring in the second area 330. [

In one embodiment, when an information movement request for moving the selected video file 1405 to the mirroring screen 1200 is received, the video file 1405 is selected from the information output on the media content screen 1400 of FIG. 27 , The control unit 180 may transmit the video file 1405 to the mobile terminal 10 according to the received information movement request. Here, the information movement request may be a drag-and-drop input through a pointer.

The mobile terminal 10 may store the received video file 1405 in the memory of the mobile terminal 10. The mobile terminal 10 can output an alarm indicating that the video file 1405 has been received in the form of a message, voice, or vibration.

In another embodiment, as shown in FIG. 27, a video file 1405 is selected from the information output on the media content screen 1400, and the selected video file 1405 is displayed on the video playback application 1200 of the mirroring screen 1200 The control unit 180 may transmit the video file 1405 to the mobile terminal 10 in response to the received information movement request. The mobile terminal 10 can execute the video playback application upon receiving the video file 1405 from the terminal 100. [ The video application may be an application for playing a video file stored in the memory of the mobile terminal 10. [ Next, Fig. 29 will be described.

29, the control unit 180 displays a media content screen 1400 and a media content screen 140 in the first area 310 of the entire screen 300 according to an information movement request to move the video file 1405 to the video icon 1209, The display unit 180 can be controlled to display the execution screen 1700 of the video playback application. That is, the information movement request for moving the video file 1405 to the video icon 1209 may be a trigger for executing the video playback application installed in the mobile terminal 10. [ The mobile terminal 10 can also display an execution screen of the video playback application. The execution screen 1700 of the video playback application includes a video playback screen 1710 and a video playback screen 1710 for playing back the moved video file 1405 from the media content screen 1400 and a video file 1405 and pre- 3). ≪ / RTI >

According to another embodiment of the present invention, a corresponding function may be performed upon receiving a control command on the external input screen constituting the full screen 300 described in step S109. This will be described with reference to FIG.

30 is a flowchart illustrating an operation method of a terminal according to another embodiment of the present invention.

After step S109, the control unit 180 receives a control command for controlling the output external input screen (S201), and performs a function corresponding to the control command on the external input screen according to the received control command (S203).

In one embodiment, the control command may be an instruction to select information output on the external input screen through a pointer or to execute an application. This will be described with reference to FIGS. 31 to 32. FIG.

31 to 32 are views for explaining a process of operating an external input screen according to another embodiment of the present invention.

Hereinafter, it is assumed that the external input screen displayed in the second area 330 of the full screen 300 is a mirror screen of the mobile terminal 10.

Referring to FIG. 31, the controller 180 of the terminal 100 may receive a pointer movement command to move the pointer 205 through a device such as a mouse or a remote control device 200. The control unit 180 may move the pointer 205 from the website screen 1100 to the mirroring screen 1200 according to the pointer movement command. The control unit 180 receives an icon selection command for selecting the memo icon 1201 through the pointer 205 and displays the memo icon 1201 on the mirroring screen 1200 in accordance with the received icon selection command The display unit 180 can be controlled to display an execution screen. In this case, the execution screen of the memo application can also be displayed on the screen of the mobile terminal 10.

The control unit 180 of the terminal 100 may perform screen mirroring with the mobile terminal 10 using a user input back channel (UIBC) technique. The user input back channel technique is a technique that can transmit the user input received by the terminal 100 to the mobile terminal 10. The controller 180 may extract the coordinates of the pointer 205 controlled by the remote controller 200 using a user input back channel (UIBC) technique. The control unit 180 may transmit the input to the pointer 205 to the mobile terminal 10 based on the coordinates of the extracted pointer 205. [ The mobile terminal 10 may receive an input to the pointer 205 received from the terminal 100 and perform a function corresponding to the received input. The result of the function performed in the mobile terminal 10 may be displayed on the mirroring screen 1200 as it is. The input to the pointer 205 may include an input to move the pointer 205, an input to select an icon corresponding to a particular application, and an input to move specific information.

According to various embodiments of the present invention, information output on the program execution screen of the terminal 100 can be easily transferred to the mobile terminal 10, thereby enhancing the intuitiveness and convenience of the user.

30 will be described again.

The control unit 180 receives the information movement input for moving the information output on the external input screen to the execution screen of the program (S205), and receives the information output from the peripheral device according to the received information movement input (S207).

According to the embodiment of the present invention, the terminal 100 and the peripheral device are interlocked with each other according to the information movement input for moving the information output to the external input screen (for example, the mirroring screen) to the program execution screen of the terminal 100 . This interlocking process will be described in detail later.

33 to 34 are views for explaining an embodiment of moving information output from an external input screen to a program execution screen and receiving corresponding information from a mobile terminal according to various embodiments of the present invention.

Here, it is assumed that the external input screen is a mirroring screen of the mobile terminal 10 connected to the terminal 100, and that the information movement input is a drag-and-drop input through the pointer 205. [

33, an execution screen 1800 of a mail application capable of sending and receiving mail is displayed in the first area 310 of the full screen 300, and the execution area 1800 of the mail application 10 A mirroring screen 1200 is displayed. On the mirroring screen 1200, an execution screen of the memo application installed in the mobile terminal 10 is displayed. When the information movement input for moving the text 1211 on the mirroring screen 1200 to the execution screen 1800 of the mail application is received via the pointer 205 from the mobile terminal 10, And may control the display unit 180 to display the received text 1211 on the execution screen 1800 of the mail application. Here, the information movement input may be a trigger for the terminal 100 to request information from the mobile terminal 10, and the mobile terminal 10 may transmit information to the terminal 100 according to the received trigger.

Next, Fig. 34 will be described.

34, a media content screen 1400 stored in the storage unit 140 of the terminal 100 is displayed in the first area 310 of the entire screen 300, A mirroring screen which is an execution screen 1500 of the gallery application of the mobile terminal 10 is displayed. When an information movement input for moving the image 1501 on the execution screen 1500 of the gallery application to the media content screen 1400 is received via the pointer 205 from the mobile terminal 10, 1501 and may control the display unit 180 to display the received image 1501 on the media content screen 1400. Here, the information movement input may be a trigger for the terminal 100 to request information from the mobile terminal 10, and the mobile terminal 10 may transmit information to the terminal 100 according to the received trigger.

According to another embodiment of the present invention, a request for screen mirroring may be received from the mobile terminal 10. This will be described with reference to FIG. 35 to FIG.

35 to 36 are diagrams for explaining an embodiment for providing a divided screen according to receiving a screen mirroring request from a mobile terminal.

Referring to FIG. 35, the mobile terminal 10 can display a screen control menu 20 by executing a remote control application installed in the mobile terminal 10. The remote control application may be an application for the mobile terminal 10 to remotely control the function of the terminal 100. [ The screen control menu 20 may include a screen mirroring menu 21 for providing a screen mirroring function and a screen capture menu 23 for providing a screen capture function.

When a request to select the screen mirroring menu 21 is received, the mobile terminal 10 may send a screen mirroring request to the terminal 100. [ The control unit 180 of the terminal 100 may divide the entire screen 300 into the first area 310 and the second area 330 in response to the screen mirroring request received from the mobile terminal 10, It may try to connect with the terminal 10. The controller 180 displays the program execution screen 1100 in the divided first area 310 and displays the program execution screen 1100 in the second area 330 in the mirroring screen of the mobile terminal 10, It is possible to control the display unit 180 to display the display unit 1200.

Next, Figs. 37 to 38 will be described.

37 to 38 are diagrams illustrating a process of capturing a screen of a terminal through a screen capture function of the mobile terminal according to an embodiment of the present invention.

37, when a request to select the screen capture menu 23 included in the screen control menu 20 shown in FIG. 35 is received, the mobile terminal 10 displays the screen capture menu 23, To operate in the camera mode. When the mobile terminal 10 captures a part of the screen 1910 on the entire screen 300 of the terminal 100 through the mobile terminal 10, Lt; / RTI > The information on the partial screen 1910 may include an image of the partial screen 1910 taken by the mobile terminal 10. The terminal 100 may extract a part of the screen 1910 using the information about the partial screen 1910 received from the mobile terminal 10 and may store the image corresponding to the extracted part of the screen 1910. [ 100 may transmit information on the screen provided in the full screen 300 to the mobile terminal 10. For example, the terminal 100 may transmit information on the screen provided in the full screen 300, Address and the like to the mobile terminal 10.

Next, Fig. 39 will be described.

FIG. 39 is a diagram illustrating an example in which a mobile terminal is controlled using a wireless touch pad according to an embodiment of the present invention. Referring to FIG.

Referring to FIG. 39, the mobile terminal 10 can control the touch pad area 30 to be displayed on the screen according to the execution of the remote control application. The touch pad area 30 may be an area for controlling the function of the terminal 100 through the user's touch input. For example, when the mobile terminal 10 receives an instruction to move a touch input on the touch pad area 30, the mobile terminal 10 can transmit the received command to the terminal 100, The pointer 205 can be moved in accordance with the moving direction of the touch input.

Next, Fig. 40 will be described.

40 is a diagram illustrating an example of controlling a mobile terminal using a mobile terminal as a wireless keyboard according to an embodiment of the present invention.

Referring to FIG. 40, the mobile terminal 10 can control to display the touch pad area 30 and the key pad area 50 on the screen according to the execution of the remote control application. The keypad area 50 may be an area for inputting characters or symbols. When the mobile terminal 10 receives a command to input a specific character on the keypad area 50, the mobile terminal 10 can transmit the received command to the terminal 100 while displaying the input character on the touch pad area 303. [ The terminal 100 may display the inputted character according to a command received from the mobile terminal 10. [ The user can easily perform a process of inputting a search word on the screen of the terminal 100 by using the mobile terminal 10 as a wireless keyboard.

Next, Fig. 41 will be described.

41 is a diagram illustrating an example in which a mobile terminal is controlled using a remote controller according to an embodiment of the present invention.

Referring to FIG. 41, the mobile terminal 10 can control to display the touch pad area 30 and the media content playback control area 60 on the screen according to the execution of the remote control application. The media content playback control area 60 may be an area for controlling functions related to playback of media content such as stop, play, fast forward, and backward of the media content being played back on the screen 300 of the terminal 100. When the mobile terminal 10 receives the reproduction control command on the media content reproduction control area 60, the mobile terminal 10 can transmit the received reproduction control command to the terminal 100. [ The terminal 100 may perform a function related to playback of media content according to a playback control command received from the mobile terminal 10. [ The user can easily control the functions of the terminal 100 by using the mobile terminal 10 together with the remote control device 200. [

According to another embodiment of the present invention, a screen mode corresponding to the file type may be provided based on the type of the file output on the screen of the terminal 100. [ This will be described with reference to FIGS. 42 to 43. FIG.

42 to 43 are diagrams for explaining an embodiment for providing a screen mode corresponding to a file type based on a file type displayed on a screen of a terminal according to an embodiment of the present invention.

42 (a), a medical image screen 2000 is displayed in a first area 310 of a full screen 300 of the terminal 100, and a text area A screen 2100 is displayed. The control unit 180 may control the display unit 180 to display the smart picture mode setting menu 2200 as shown in FIG. 42 (b) according to the execution request of the remote control application. The smart picture mode may be a mode for allowing a user to easily utilize each screen by providing a program suitable for the type of content displayed on each screen. The file type may include medical image file type, text file type, cinema file type, and photo file type.

The smart picture mode setting menu 2200 may include a text reader program list 2210, a medical image program list 2220, a cinema program list 2230 and a photograph program list 2240.

The text reader program list 2210 may include program icons corresponding to programs for providing a program that can utilize a text file and a program addition icon 2211. [ The medical image program list 2220 may include program icons corresponding to programs for providing a program that can utilize a medical image file and a program addition icon 2221. [ The cinema program list 2230 may include program icons and program addition icons corresponding to programs for providing a program that can utilize a cinema file. The photograph program list 2240 may include program icons corresponding to programs for providing a program that can utilize a photo file and a program addition icon.

When the medical image program corresponding to the medical image screen 2000 is selected in the medical image program list 2220, the control unit 180 executes the selected medical image program to display the medical image screen 2000, Can be displayed. 43, when the text reader program corresponding to the text screen 2200 is selected in the text reader program list 2210, the control unit 180 executes the selected text reader program to display the text screen 2000, Can be displayed. The control unit 180 can execute the medical image file with the selected medical image program when the medical image file is executed and can execute the text file with the selected text reader program when the text file is executed. The control unit 180 may display indicators 2001 and 2101 indicating that the smart picture mode is executed on each screen. Each indicator may contain information about the executed program.

According to an embodiment of the present invention, the above-described method can be implemented as a code readable by a processor on a medium on which a program is recorded. Examples of the medium that can be read by the processor include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage, etc., and may be implemented in the form of a carrier wave (e.g., transmission over the Internet) .

The above-described terminal may be configured such that the configuration and method of the embodiments described above are not limitedly applied, but the embodiments may be configured such that all or some of the embodiments are selectively combined so that various modifications may be made. have.

Claims (20)

A method of operating a terminal,
Outputting an execution screen of the program to the entire screen of the terminal;
Receiving a screen division request for dividing the entire screen into a first area and a second area; And
Outputting an execution screen of the program in the first area and a multitasking screen for providing a multitasking environment in the second area in response to the screen division request,
The multitasking screen
A list of programs including one or more other programs other than the program and an external input list including one or more external inputs connected to the terminal in a wired or wireless manner
A method of operating a terminal. The method according to claim 1,
The program list
Comprising one or more program items,
Each program item including at least one of a program recently executed in the terminal and a program registered as a favorite
A method of operating a terminal. 3. The method of claim 2,
Each program item
Indicating a recent execution screen of each program or a thumbnail image corresponding to each program
A method of operating a terminal. 3. The method of claim 2,
Receiving a request to select any one of the one or more program items; And
And outputting an execution screen of the program corresponding to the selected program item to the second area in response to the received request
A method of operating a terminal. 3. The method of claim 2,
The program list
Comprising a plurality of program items,
Receiving a request to select two of the plurality of program items; And
And outputting the execution screens of the program corresponding to the two program items selected according to the received request to the first sub-area and the second sub-area of the second area, respectively
A method of operating a terminal. The method according to claim 1,
The external input list
One or more external input icons,
Each external input icon
A first external input icon connected to the terminal for screen mirroring, a second external input icon indicating that the terminal is connected to the terminal through an HDMI (High Definition Multimedia Interface) standard, and a third external input icon for inputting a TV Containing
A method of operating a terminal. The method according to claim 6,
Each external input icon includes information about the corresponding external input,
Information about external inputs
A name identifying the external input, a name of the external input peripheral device, and a thumbnail image of the media content reproduced through the external input
A method of operating a terminal. The method according to claim 6,
Receiving a request to select an external input icon of one or more external input icons; And
And outputting an external input corresponding to the selected external input icon to the second area according to the received request
A method of operating a terminal. The method according to claim 6,
Receiving a request to select the first external input icon; And
And outputting to the second area a screen mirroring list including information on the mobile terminals capable of screen mirroring according to the received request
A method of operating a terminal. 10. The method of claim 9,
The information for each mobile terminal
A name of each mobile terminal connected to the terminal, a current screen of each mobile terminal, and a user name of each mobile terminal
A method of operating a terminal. In the terminal,
A user input interface for receiving a request for screen control of the terminal; And
The method comprising the steps of: outputting a program execution screen to the entire screen of the terminal, receiving a screen division request for dividing the entire screen into a first area and a second area, and executing the program in the first area And a controller for outputting a multitasking screen for providing a multitasking environment to the second area,
The multitasking screen
A list of programs including one or more other programs other than the program and an external input list including one or more external inputs connected to the terminal in a wired or wireless manner
terminal. 12. The method of claim 11,
The program list
Comprising one or more program items,
Each program item including at least one of a program recently executed in the terminal and a program registered as a favorite
terminal. 13. The method of claim 12,
Each program item
Indicating a recent execution screen of each program or a thumbnail image corresponding to each program
terminal. 13. The method of claim 12,
The control unit
Receiving a request to select one of the one or more program items and outputting an execution screen of the program corresponding to the selected program item to the second area in accordance with the received request
terminal. 13. The method of claim 12,
The program list
Comprising a plurality of program items,
The control unit
Receiving a request to select two program items of the plurality of program items, and executing screens of programs corresponding to the two program items selected according to the received request in a first sub-area and a second sub- Respectively.
terminal. 12. The method of claim 11,
The external input list
One or more external input icons,
Each external input icon
A first external input icon connected to the terminal for screen mirroring, a second external input icon indicating that the terminal is connected to the terminal through an HDMI (High Definition Multimedia Interface) standard, and a third external input icon for inputting a TV Containing
terminal. 17. The method of claim 16,
Each external input icon includes information about the corresponding external input,
Information about external inputs
A name identifying the external input, a name of the external input peripheral device, and a thumbnail image of the media content reproduced through the external input
terminal. 17. The method of claim 16,
The control unit
Receiving a request for selecting one of the one or more external input icons and outputting an external input corresponding to the selected one of the plurality of external input icons according to the received request to the second region
terminal. 17. The method of claim 16,
The control unit
Receiving a request to select the first external input icon and outputting to the second area a screen mirroring list including information on the mobile terminals capable of screen mirroring according to the received request
terminal. 20. The method of claim 19,
The information for each mobile terminal
A name of each mobile terminal connected to the terminal, a current screen of each mobile terminal, and a user name of each mobile terminal
terminal.

Images