KR20150050883A - Terminal and method for controlling the same - Google Patents

Abstract

A terminal capable of mirroring with an external device includes: a display unit for displaying multiple sets of content; a wireless communications unit for receiving display information from an external device; a user input unit for receiving a user input; and a control unit which adjusts a scale and a display mode of multiple sets of content considering display information of an external device, controls the wireless communications unit so as to display the adjusted content via an external device, determines, if a user input is received through the user input unit while an external device displays the adjusted content, an action mapped to the user input considering the type of content which is subject to the user input, and controls the wireless communications unit to allow an external device to perform the operations described above.

Description

[0001] TERMINAL AND METHOD FOR CONTROLLING THE SAME [0002]

The present invention relates to a mobile terminal and a control method thereof, in which usage of a terminal can be realized by further considering convenience of a user.

A terminal can be divided into a mobile terminal (mobile / portable terminal) and a stationary terminal according to whether the terminal can be moved. The mobile terminal can be divided into a handheld terminal and a vehicle mount terminal according to whether the user can directly carry the mobile terminal.

Such a terminal has various functions, for example, in the form of a multimedia device having multiple functions such as photographing and photographing of a moving picture, reproduction of a music or video file, reception of a game and broadcasting, etc. .

In order to support and enhance the functionality of such terminals, it may be considered to improve the structural and / or software parts of the terminal.

Recently, it has become possible to connect the mobile terminal and the external video display terminal to share images and contents displayed on each device. In particular, a video display terminal such as a smart TV can be connected to a mobile terminal possessed by a user using a wired or wireless network to perform data communication. Such a function can be referred to as "mirroring ".

When the mirroring function is implemented, each terminal can not provide an optimized screen to the user when the aspect ratio of the bar display sharing the same screen frame is different.

In addition, when the mirroring function is implemented, there is a problem that the user input received from the one-way terminal can be applied to the other terminal as it is.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a terminal providing a mirroring screen optimized for a user in consideration of characteristics of a display and a control method thereof.

It is another object of the present invention to provide a terminal and a control method thereof for mapping and operating user input according to characteristics of a display and a type of a content to be mirrored.

According to an aspect of the present invention, there is provided a terminal capable of performing mirroring with an external device according to an embodiment of the present invention includes: a display unit displaying a plurality of contents; A wireless communication unit for receiving display information from the external device; A user input for receiving user input; And controlling the scaling and display mode of the plurality of contents in consideration of display information of the external apparatus and controlling the wireless communication unit so that the adjusted plurality of contents can be displayed through the external apparatus, Determines the operation to be mapped to the user input in consideration of the type of the content to which the user input is applied when the user input is received through the user input unit while displaying the adjusted plurality of contents, And a controller for remotely controlling the external device so that the external device performs the operation.

According to the terminal and the control method therefor according to an embodiment of the present invention, an optimized mirroring screen according to the ratio of the display can be provided.

In addition, the user input can be automatically converted according to the characteristics of the display and the type of content to provide an intuitive mirroring operation.

The effects obtained by the present invention are not limited to the above-mentioned effects, and other effects not mentioned can be clearly understood by those skilled in the art from the following description will be.

1 is a block diagram of a mobile terminal according to an embodiment of the present invention.
2 is a perspective view of a mobile terminal according to an embodiment of the present invention.
3 is a rear perspective view of the mobile terminal shown in FIG.
4 is a diagram showing a concept of a mirroring system according to an embodiment of the present invention.
5 and 6 are views illustrating a method of performing mirroring by adjusting a scale and a display mode of contents according to an embodiment of the present invention.
7 to 9 are views illustrating a method of mirroring user input to content according to an embodiment of the present invention.
10 is a diagram illustrating a method of mirroring an input to zoom in / out for a lower level list according to an embodiment of the present invention.
11 is a diagram illustrating a method of mirroring text content according to an embodiment of the present invention.
12 to 15 are diagrams illustrating a method of mirroring video content according to an embodiment of the present invention.
16 to 19 are diagrams illustrating a method of mirroring image content according to an embodiment of the present invention. And,
20 is a flowchart illustrating a method of controlling a terminal according to an embodiment of the present invention.

Hereinafter, a mobile terminal related to the present invention will be described in detail with reference to the drawings. The suffix "module" and " part "for the components used in the following description are given or mixed in consideration of ease of specification, and do not have their own meaning or role.

The mobile terminal described in this specification may include a mobile phone, a smart phone, a laptop computer, a digital broadcasting terminal, a PDA (Personal Digital Assistants), a PMP (Portable Multimedia Player), and navigation. However, it will be understood by those skilled in the art that the configuration according to the embodiments described herein may be applied to a fixed terminal such as a digital TV, a desktop computer, and the like, unless the configuration is applicable only to a mobile terminal.

In the present specification, a technique for transferring / displaying content from one terminal to another is referred to as "mirroring ". In other words, the current screen of the mobile terminal is directly transmitted to another terminal using the mirroring function, and the user can perform a touch operation or the like using at least one of the other terminal and the mobile terminal. In addition, the mobile terminal can control another terminal in synchronization with the service use in the mobile terminal and the application driving.

On the other hand, the mirroring function may include user mirroring for displaying on other terminals of the display screen, and user input mirroring for allowing user terminals such as a touch operation to be performed on other terminals.

The synchronization interface for this mirroring function is composed of Virtual Network Computing (VNC), Real Time Transport Protocol (RTP) and Short Range Wireless Communication BT (Bluetooth) for audio and video content transmission, Universal Plug and Play UPnP Universal Plug and Play).

1 is a block diagram of a mobile terminal according to an embodiment of the present invention.

The mobile terminal 100 includes a wireless communication unit 110, an audio / video input unit 120, a user input unit 130, a sensing unit 140, an output unit 150, a memory 160, A controller 170, a controller 180, a power supply 190, and the like. The components shown in FIG. 1 are not essential, and a mobile terminal having more or fewer components may be implemented.

Hereinafter, the components will be described in order.

The wireless communication unit 110 may include one or more modules for enabling wireless communication between the mobile terminal 100 and the wireless communication system or between the mobile terminal 100 and the network in which the mobile terminal 100 is located. For example, the wireless communication unit 110 may include a broadcast receiving module 111, a mobile communication module 112, a wireless Internet module 113, a short range communication module 114, and a location information module 115 .

The broadcast receiving module 111 receives broadcast signals and / or broadcast-related information from an external broadcast management server through a broadcast channel. The broadcast channel may include a satellite channel and a terrestrial channel. Two or more broadcast receiving modules 1100 may be provided to the mobile terminal 100 for simultaneous broadcast reception or broadcast channel switching for at least two broadcast channels.

The broadcast management server may refer to a server for generating and transmitting broadcast signals and / or broadcast related information, or a server for receiving broadcast signals and / or broadcast related information generated by the broadcast management server and transmitting the generated broadcast signals and / or broadcast related information. The broadcast signal may include a TV broadcast signal, a radio broadcast signal, a data broadcast signal, and a broadcast signal in which a data broadcast signal is combined with a TV broadcast signal or a radio broadcast signal.

The broadcast-related information means information related to a broadcast channel, a broadcast program, or a broadcast service provider. The broadcast-related information may also be provided through a mobile communication network. In this case, it may be received by the mobile communication module 112.

The broadcast-related information may exist in various forms. For example, an EPG (Electronic Program Guide) of DMB (Digital Multimedia Broadcasting) or an ESG (Electronic Service Guide) of Digital Video Broadcast-Handheld (DVB-H).

For example, the broadcast receiving module 111 may be a Digital Multimedia Broadcasting-Terrestrial (DMB-T), a Digital Multimedia Broadcasting-Satellite (DMB-S), a Media Forward Link Only (Mobile Broadcasting Business Management System), ISDB-T (Integrated Services Digital Broadcasting (ISDB-T)), Digital Multimedia Broadcasting (MBMS) Digital Broadcast-Terrestrial) or the like. Of course, the broadcast receiving module 111 may be adapted to other broadcasting systems as well as the digital broadcasting system described above.

The broadcast signal and / or broadcast related information received through the broadcast receiving module 111 may be stored in the memory 160.

The mobile communication module 112 may comprise at least one of a base station, an external terminal, and a server on a mobile communication network such as Global System for Mobile communications (GSM), Code Division Multiple Access (CDMA), Wideband CDMA Transmits and receives wireless signals with one. The wireless signal may include various types of data depending on a voice call signal, a video call signal or a text / multimedia message transmission / reception.

The wireless Internet module 113 is a module for wireless Internet access, and may be built in or externally attached to the mobile terminal 100. Wireless Internet technologies include WLAN (Wi-Fi), Wibro (Wireless broadband), Wimax (World Interoperability for Microwave Access), HSDPA (High Speed Downlink Packet Access), GSM, CDMA, WCDMA, LTE Evolution (but not limited to) may be used.

The wireless Internet module 113, which performs wireless Internet access through the mobile communication network, is connected to the mobile communication module 110 through the mobile communication network, for example, from the viewpoint that the wireless Internet access by Wibro, HSDPA, GSM, CDMA, WCDMA, LTE, (112).

The short-range communication module 114 refers to a module for short-range communication. Bluetooth, Radio Frequency Identification (RFID), infrared data association (IrDA), Ultra Wideband (UWB), ZigBee, and the like can be used as a short range communication technology.

The position information module 115 is a module for obtaining the position of the mobile terminal, and a representative example thereof is a Global Position System (GPS) module. According to the current technology, the position information module 115 calculates distance information and accurate time information from three or more satellites, and then applies a trigonometric method to the calculated information to obtain three-dimensional (3D) information according to latitude, longitude, The current position information can be accurately calculated. At present, a method of calculating position and time information using three satellites and correcting an error of the calculated position and time information using another satellite is widely used. In addition, the GPS module 115 can calculate speed information by continuously calculating the current position in real time.

Referring to FIG. 1, an A / V (Audio / Video) input unit 120 is for inputting an audio signal or a video signal, and may include a camera 121 and a microphone 122. The camera 121 processes image frames such as still images or moving images obtained by the image sensor in the video communication mode or the photographing mode. The processed image frame can be displayed on the display unit 151. [

The image frame processed by the camera 121 may be stored in the memory 160 or transmitted to the outside through the wireless communication unit 110. [ Two or more cameras 121 may be provided depending on the use environment.

The microphone 122 receives an external sound signal through a microphone in a communication mode, a recording mode, a voice recognition mode, or the like, and processes it as electrical voice data. The processed voice data can be converted into a form that can be transmitted to the mobile communication base station through the mobile communication module 112 when the voice data is in the call mode, and output. Various noise reduction algorithms may be implemented in the microphone 122 to remove noise generated in receiving an external sound signal.

The user input unit 130 generates input data for a user to control the operation of the terminal. The user input unit 130 may include a button 136 and a touch sensor (static pressure / static electricity) 137 located on the front and rear sides or side surfaces of the mobile terminal 100, and may include a keypad, A dome switch, a jog wheel, a jog switch, and the like.

The sensing unit 140 senses the current state of the mobile terminal 100 such as the open / close state of the mobile terminal 100, the position of the mobile terminal 100, the presence or absence of user contact, the orientation of the mobile terminal, And generates a sensing signal for controlling the operation of the mobile terminal 100. Examples of the sensing unit 140 include a gyroscope sensor, an acceleration sensor, and a geomagnetic sensor.

For example, when the mobile terminal 100 is in the form of a slide phone, it is possible to sense whether the slide phone is opened or closed. It is also possible to sense whether the power supply unit 190 is powered on, whether the interface unit 170 is connected to an external device, and the like. Meanwhile, the sensing unit 140 may include a proximity sensor 141.

The output unit 150 is for generating output related to the visual, auditory or tactile sense and includes a display unit 151, an audio output module 152, an alarm unit 153, and a haptic module 154 .

The display unit 151 displays (outputs) information processed by the mobile terminal 100. For example, when the mobile terminal is in the call mode, a UI (User Interface) or a GUI (Graphic User Interface) associated with a call is displayed. When the mobile terminal 100 is in the video communication mode or the photographing mode, the photographed and / or received video or UI and GUI are displayed.

The display unit 151 may be a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), a flexible display display, and a 3D display.

Some of these displays may be transparent or light transmissive so that they can be seen through. This can be referred to as a transparent display, and a typical example of the transparent display is TOLED (Transparent OLED) and the like. The rear structure of the display unit 151 may also be of a light transmission type. With this structure, the user can see an object located behind the terminal body through the area occupied by the display unit 151 of the terminal body.

There may be two or more display units 151 according to the embodiment of the mobile terminal 100. For example, in the mobile terminal 100, a plurality of display portions may be spaced apart from one another or may be disposed integrally with each other, or may be disposed on different surfaces.

(Hereinafter, referred to as a 'touch screen') in which a display unit 151 and a sensor for sensing a touch operation (hereinafter, referred to as 'touch sensor') form a mutual layer structure, It can also be used as an input device. The touch sensor may have the form of, for example, a touch film, a touch sheet, a touch pad, or the like.

The touch sensor may be configured to convert a change in a pressure applied to a specific portion of the display unit 151 or a capacitance generated in a specific portion of the display unit 151 into an electrical input signal. The touch sensor can be configured to detect not only the position and area to be touched but also the pressure at the time of touch.

If there is a touch input to the touch sensor, the corresponding signal (s) is sent to the touch controller (not shown). The touch controller processes the signal (s) and transmits the corresponding data to the controller 180. Thus, the control unit 180 can know which area of the display unit 151 is touched or the like.

The proximity sensor 141 may be disposed in an inner region of the mobile terminal or in the vicinity of the touch screen, which is enclosed by the touch screen. The proximity sensor refers to a sensor that detects the presence or absence of an object approaching a predetermined detection surface or a nearby object without mechanical contact using the force of an electromagnetic field or infrared rays. The proximity sensor has a longer life span than the contact sensor and its utilization is also high.

Examples of the proximity sensor include a transmission type photoelectric sensor, a direct reflection type photoelectric sensor, a mirror reflection type photoelectric sensor, a high frequency oscillation type proximity sensor, a capacitive proximity sensor, a magnetic proximity sensor, and an infrared proximity sensor. And to detect the proximity of the pointer by the change of the electric field along the proximity of the pointer when the touch screen is electrostatic. In this case, the touch screen (touch sensor) may be classified as a proximity sensor.

Hereinafter, for convenience of explanation, the act of recognizing that the pointer is positioned on the touch screen while the pointer is not in contact with the touch screen is referred to as "proximity touch" The act of actually contacting the pointer on the touch screen may be referred to as "contact touch. &Quot; The location where the pointer is proximately touched on the touch screen may refer to a position where the pointer corresponds vertically to the touch screen when the pointer is touched.

The proximity sensor detects a proximity touch and a proximity touch pattern (e.g., a proximity touch distance, a proximity touch direction, a proximity touch speed, a proximity touch time, a proximity touch position, a proximity touch movement state, and the like). Information corresponding to the detected proximity touch operation and the proximity touch pattern may be output on the touch screen.

The audio output module 152 may output audio data received from the wireless communication unit 110 or stored in the memory 160 in a call signal reception mode, a call mode or a recording mode, a voice recognition mode, a broadcast reception mode, The sound output module 152 also outputs sound signals related to functions (e.g., call signal reception sound, message reception sound, etc.) performed in the mobile terminal 100. [ The audio output module 152 may include a receiver, a speaker, a buzzer, and the like.

The alarm unit 153 outputs a signal for notifying the occurrence of an event of the mobile terminal 100. Examples of events that occur in the mobile terminal include call signal reception, message reception, key signal input, touch input, and the like. The alarm unit 153 may output a signal for notifying the occurrence of an event in a form other than the video signal or the audio signal, for example, vibration. In this case, the display unit 151 and the sound output module 152 may be a type of the alarm unit 153. The display unit 151 and the sound output module 152 may be connected to the display unit 151 or the sound output module 152, .

The haptic module 154 generates various tactile effects that the user can feel. A typical example of the haptic effect generated by the haptic module 154 is vibration. The intensity and pattern of the vibration generated by the haptic module 154 are controllable. For example, different vibrations may be synthesized and output or sequentially output.

In addition to the vibration, the haptic module 154 may include a pin arrangement vertically moving with respect to the contact skin surface, a spraying force or suction force of the air through the injection port or the suction port, a touch on the skin surface, contact of an electrode, And various tactile effects such as an effect of reproducing a cold sensation using an endothermic or exothermic element can be generated.

The haptic module 154 can be implemented not only to transmit the tactile effect through the direct contact but also to allow the user to feel the tactile effect through the muscular sensation of the finger or arm. At least two haptic modules 154 may be provided according to the configuration of the mobile terminal 100.

The memory 160 may store a program for processing and controlling the control unit 180 and temporarily store the input / output data (e.g., telephone directory, message, audio, still image, For example. The memory 160 may store the frequency of use of each of the data (for example, each telephone number, each message, and frequency of use for each multimedia).

In addition, the memory 160 may store data on vibration and sound of various patterns outputted when a touch is input on the touch screen.

The memory 160 may be a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, SD or XD memory), a RAM (Random Access Memory), SRAM (Static Random Access Memory), ROM (Read Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM A disk, and / or an optical disk. The mobile terminal 100 may operate in association with a web storage that performs a storage function of the memory 160 on the Internet.

The interface unit 170 serves as a path for communication with all external devices connected to the mobile terminal 100. The interface unit 170 receives data from an external device or supplies power to each component in the mobile terminal 100 or transmits data to the external device. For example, a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, a port for connecting a device having an identification module, an audio I / O port, A video input / output (I / O) port, an earphone port, and the like may be included in the interface unit 170.

The identification module is a chip for storing various information for authenticating the use right of the mobile terminal 100 and includes a user identification module (UIM), a subscriber identity module (SIM), a general user authentication module A Universal Subscriber Identity Module (USIM), and the like. Devices with identification modules (hereinafter referred to as "identification devices") can be manufactured in a smart card format. Therefore, the identification device can be connected to the mobile terminal 100 through the port.

When the mobile terminal 100 is connected to an external cradle, the interface unit may be a path through which power from the cradle is supplied to the mobile terminal 100, or various command signals input by the user to the cradle may be transmitted And may be a passage to be transmitted to the terminal 100. The various command signals or the power source inputted from the cradle may be operated as a signal for recognizing that the mobile terminal 100 is correctly mounted on the cradle.

The controller 180 typically controls the overall operation of the mobile terminal. For example, voice communication, data communication, video communication, and the like. The control unit 180 may include a multimedia module 181 for multimedia playback. The multimedia module 181 may be implemented in the control unit 180 or may be implemented separately from the control unit 180. [

The controller 180 may perform a pattern recognition process for recognizing handwriting input or drawing input performed on the touch screen as characters and images, respectively.

The power supply unit 190 receives external power and internal power under the control of the controller 180 and supplies power necessary for operation of the respective components.

The battery may be an internal battery configured to be chargeable, and may be detachably coupled to the terminal body for charging or the like. The connection port may be configured as an example of the interface unit 170 in which an external charger for supplying power for charging the battery is electrically connected.

The various embodiments described herein may be embodied in a recording medium readable by a computer or similar device using, for example, software, hardware, or a combination thereof.

According to a hardware implementation, the embodiments described herein may be implemented as application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays May be implemented using at least one of a processor, controllers, micro-controllers, microprocessors, and other electronic units for performing other functions. In some cases, The embodiments described may be implemented by the control unit 180 itself.

According to a software implementation, embodiments such as the procedures and functions described herein may be implemented with separate software modules. Each of the software modules may perform one or more of the functions and operations described herein. Software code can be implemented in a software application written in a suitable programming language. The software code is stored in the memory 160 and can be executed by the control unit 180. [

Hereinafter, the display unit 151 will be described on the assumption that the touch screen 151 is used to describe the embodiment of the present invention. However, the embodiments of the present invention are not limited to the touch screen 151.

In addition, in the embodiment of the present invention, the touch gesture refers to a gesture realized by touching or touching the touch screen 151 in a touch manner, and the touch input means an input received by the touch gesture.

The touch gesture may be a tap, a double tap, a drag, a flick, a press, a multi touch, a pinch in, a pinch-out pinch out).

The tab is an operation of lightly depressing and releasing the touch screen 151 once, which means a touch gesture such as a click of a mouse in a general personal computer.

In addition, the double tap is an operation of tapping the touch screen 151 twice rapidly, which means a touch gesture such as a double click of a mouse in a general personal computer.

In addition, the drag may be an operation of moving to a specific position while keeping the touch on the touch screen 151, and when the object is dragged, the object may be continuously moved according to the drag direction.

The flick refers to an operation of touching the touch screen 151 and then moving the contact in a specific direction (upward, downward, left, right or diagonal) and then releasing the contact. The mobile terminal 100 receives the touch input The processing of the specific operation is performed based on the flick direction, speed, and the like. For example, the page turning operation of the e-book can be performed based on the flick direction.

In addition, the press refers to an operation of continuously holding the touch for a predetermined time or longer after touching the touch screen 151. [

In addition, the multi-touch means an operation of touching a plurality of points of the touch screen 151 at the same time.

In addition, the pinch-in refers to an operation of dragging the touch screen 151 in a direction in which a plurality of pointers that are multi-touching approach each other. That is, a drag that starts from at least one point among a plurality of points that are multi-touched on the touch screen 151 and occurs in a direction in which a plurality of points that are multi-touched approach each other.

In addition, the pinch-out refers to an operation of dragging the touch screen 151 in a direction in which a plurality of pointers in a multitouch direction move away from each other. That is, dragging starts from at least one point among a plurality of points that are multi-touched in the touch screen 151 and occurs in a direction in which a plurality of points that are multi-touched move away from each other.

FIG. 2 is a perspective view of a mobile terminal or a portable terminal according to the present invention, as viewed from the front.

The disclosed mobile terminal 100 includes a bar-shaped terminal body. However, the present invention is not limited thereto, and can be applied to various structures such as a slide type, a folder type, a swing type, and a swivel type in which two or more bodies are relatively movably coupled.

The body of the mobile terminal 100 includes a casing 101, 102, In this embodiment, the case may be divided into a front case 101 and a rear case 102. [ A variety of electronic components are embedded in the space formed between the front case 101 and the rear case 102.

In some cases, electronic components may be mounted on the surface of the rear case 102 as well. Electronic components mounted on the surface of the rear case 102 include a battery, a USIM card, a memory card, and the like which are detachable by the user. At this time, the rear case 102 may further include a rear cover 103 for covering the surface thereof. When the rear cover 103 is removed, the surface of the rear case 102 is exposed.

As shown in FIG. 2, when the rear cover 103 is coupled, only a part of the side surface of the rear case 102 may be exposed. When the size of the rear cover 103 is reduced, Some may also be exposed. And an opening 103 'for exposing the camera 121' or the sound output unit 152 'to the outside when the rear cover 103 covers the entire rear surface of the rear case 102.

The cases 101, 102 and 103 may be formed by injection molding of synthetic resin or may be formed of a metal material such as stainless steel (STS) or titanium (Ti).

The display unit 151, the sound output module 152, the camera 121, the user input units 130/131 and 132, the microphone 122, the interface unit 170, and the like may be disposed in the terminal cases 101 and 102 .

The display unit 151 occupies most of the main surface of the front case 101. A sound output module 152 and a camera 121 are disposed in an area adjacent to one end of both ends of the display unit 151 and a user input unit 131 and a microphone 122 are disposed in an area adjacent to the other end. The user input unit 132 and the interface unit 170 may be disposed on the side surfaces of the front case 101 and the rear case 102. [

The user input unit 130 is operated to receive a command for controlling the operation of the mobile terminal 100 and may include a plurality of operation units 131 and 132. The operation units 131 and 132 may be collectively referred to as a manipulating portion and may be employed in any manner as long as the user operates in a tactile manner.

The contents inputted by the first or second operation unit 131 or 132 may be variously set. For example, the first operation unit 131 receives a command such as start, end, scroll, and the like, and the second operation unit 132 inputs a command such as the adjustment of the sound output from the sound output module 152 And the third operation unit 133 can receive a command such as switching to the touch recognition mode of the display unit 151 or the like.

3 is a rear perspective view of the portable terminal shown in FIG.

Referring to FIG. 3, a camera 121 'may be further mounted on the rear surface of the terminal body, that is, the rear case 102. The camera 121 'may have a photographing direction substantially opposite to the camera 121 (see FIG. 2), and may be a camera having different pixels from the camera 121.

For example, the camera 121 may have a low pixel so that the face of the user can be photographed and transmitted to the other party in case of a video call or the like, and the camera 121 ' It is preferable to have a large number of pixels. The cameras 121 and 121 'may be installed in the terminal body so as to be rotatable or pop-upable.

A flash 123 and a mirror 124 are additionally disposed adjacent to the camera 121 '. The flash 123 illuminates the subject when the subject is photographed by the camera 121 '. The mirror 124 allows the user to illuminate the user's own face or the like when the user intends to shoot himself / herself (self-photographing) using the camera 121 '.

An acoustic output 152 'may be additionally disposed on the rear surface of the terminal body. The sound output unit 152 'may implement the stereo function together with the sound output unit 152 (see FIG. 2A), and may be used for the implementation of the speakerphone mode during a call.

In addition to the antenna for talking and the like, a broadcast signal reception antenna 116 may be additionally disposed on the side of the terminal body. The antenna 116, which forms part of the broadcast receiving module 111 (see FIG. 1), can be installed to be able to be drawn out from the terminal body.

For convenience of explanation, it is assumed that the mobile terminal according to the present invention includes at least one of the components shown in FIG.

In particular, a terminal capable of performing mirroring with an external device according to the present invention includes a display unit for displaying a plurality of contents among components shown in FIG. 1, a wireless communication unit for receiving display information from the external device, A control unit for controlling the scale and display mode of the plurality of contents in consideration of the user input unit for receiving and the display information of the external apparatus and controlling the wireless communication unit so that the adjusted plurality of contents can be displayed through the external apparatus Wherein when the user input is received through the user input unit while the external device is displaying the adjusted plurality of contents, an operation mapped to the user input in consideration of the type of the content to which the user input is applied And when the external device performs the operation It is assumed that a control unit for controlling the wireless communication unit.

The display unit may display a plurality of contents in a thumbnail format on a list created based on a scrap date of a plurality of contents, and the control unit controls the scale and the size of the list in consideration of the display information of the external apparatus. The display mode can be adjusted to control the wireless communication unit to be displayed through the external device.

Also, the display information of the external apparatus may include a display size and an aspect ratio, and the controller may adjust the display mode of the content to a portrait mode or a landscape mode by comparing the aspect ratio of the display unit and the aspect ratio of the external apparatus have.

Here, the controller may determine an operation to be mapped to the user input according to the display mode of the adjusted content, and may control the wireless communication unit to allow the external device to perform the operation.

Hereinafter, embodiments related to a control method that can be implemented in a terminal configured as described above will be described with reference to the accompanying drawings.

4 is a diagram showing a concept of a mirroring system according to an embodiment of the present invention.

Referring to FIG. 4, the first terminal 100a has a mobile terminal type and the second terminal 100b has a television type. 2. Description of the Related Art [0002] In recent years, a so-called smart television having a wired / wireless communication unit capable of transmitting / receiving data with the outside and a separate processor has been supplied to a terminal of a television type. While the existing television is a unidirectional data movement that only displays the received broadcast data, the smart television may transmit data based on the user's input, so that the movement of data may be bi-directional.

In describing the embodiments of the present invention, a specific type of terminal is exemplified as shown in FIG. 4, but it is obvious that it is not limited to the terminal type.

In an exemplary embodiment of the present invention, a plurality of terminals may share mirroring content among a plurality of terminals by transmitting and receiving mirroring contents to and from a separate server 400 in order to perform a mirroring function. That is, when the mirroring contents to be shared are stored in the server 400, the server 400 can share the contents through the method of giving the access right to the mirroring contents stored in the terminal to be shared .

Meanwhile, the server 400 may be provided as a separate device, but a terminal acting as a host among a plurality of terminals may perform a role of a server instead. That is, the embodiments of the present invention do not limit the existence of the server 400, and may transmit and receive the mirroring data by terminal-based communication without having to store the mirroring contents in the server 400. [

Hereinafter, a control method for performing mirroring according to an embodiment of the present invention will be described with reference to FIG. 5 to FIG. Before describing the control method of the present invention, it is assumed that the first terminal 100a and the second terminal 100b are connected to each other by a mirroring function.

5 and 6 are views illustrating a method of performing mirroring by adjusting a scale and a display mode of contents according to an embodiment of the present invention. In this drawing, list data is taken as an embodiment of contents.

Referring to FIG. 5, the first terminal 100a may control the wireless communication unit so that the content 500 displayed on the first terminal 100a may be displayed and displayed on the second terminal 100b. Specifically, the first terminal 100a may adjust the scale and display mode of the content in consideration of the display information of the second terminal. Here, the display information may include the size and the aspect ratio of the display. The display mode is a mode for the direction in which the content is displayed, and may include a portrait mode and a landscape mode. 5, the display mode of the content 500 displayed on the first terminal 100a is the portrait mode, and the display mode of the content 501 displayed on the second terminal 100b is the landscape mode. That is, the second terminal 100b is preferably displayed in the landscape mode because the horizontal length of the display is longer than the vertical length. Conversely, since the vertical length of the display of the first terminal 100a is longer than the horizontal length, Mode. Therefore, when the first terminal 100a mirrors the content in the portrait mode to the second terminal 100b, the first terminal 100a can adjust the landscape mode suitable for the second terminal.

Also, the first terminal 100a can adjust the scale of the content in consideration of the display size of the mirrored second terminal 100b.

Meanwhile, the list data shown in the figure may include a plurality of scrap contents, and a plurality of contents may be displayed in a thumbnail format on a list created based on the scrap date. In addition to the scrap date, the list data may also generate a list based on the date the content was created, the date the content was modified, the date the content was last saved, and so on.

5, the content mirroring control method is exemplified by exemplifying the list data. However, the present invention is not limited thereto and may be applied to other types of contents (web content, application contents, etc.).

FIG. 6 illustrates a method of mirroring a lower level list according to an embodiment of the present invention.

6, a lower level list 610 displayed to be horizontally navigated to the first terminal 100a is displayed as a lower level list 611 converted to be vertically navigated to the second terminal 100b .

 Here, the lower level list means a list displayed when a user's input for selecting a list of specific dates is detected in the list data, and the contents corresponding to a specific date can be displayed in a thumbnail format.

7 to 9 are views illustrating a method of mirroring user input to content according to an embodiment of the present invention. According to an embodiment of the present invention, in addition to the display mirroring of the contents considering the display information of the mirrored external device, a method of mirroring the user input is also provided. Before the description of FIGS. 7 to 9, it is assumed that the first terminal and the second terminal are in a mirroring connection, and that the list data is converted from the landscape mode to the portrait mode according to the display information of the second terminal and mirrored do.

7, when a user input for dragging or flicking the list data 700 displayed in the first terminal 100a in the vertical direction is sensed, the first terminal 100a displays the list data 700 on the second terminal 100b, (For example, the press & move operation 702 of the remote controller) to move the list in the left-right direction and mirror the list data 701. That is, the first terminal 100a adjusts the user input in accordance with the list data mirrored in the portrait mode to the landscape mode. Specifically, when the first input / output terminal 701 receives a user input for dragging or flicking 701 in the vertical direction, the first terminal 100a can map the input and output to the press & move operation 702 in the left / have.

8, when a user input for tapping a list of the list data 800 displayed on the first terminal 100a is received, the first terminal 100a transmits the list data 800 to the second terminal 100b It is possible to convert any of the displayed list data 801 into an input for selecting (e.g., a press operation 802 of the remote controller) and mirror the input.

9, when a user input for dragging or flicking the lower-level list of the list data 900 displayed on the first terminal 100a in the left-right direction is received, the first terminal 100a transmits 2 list of the list data 901 displayed on the terminal 100b can be converted into an input (for example, a press & move operation 902 of the remote controller) moving up and down and mirrored.

Fig. 10 is a diagram for explaining mirroring of list data magnification and reduction operations.

10, when a user input for pinching down the lower level list of the list data 1000 displayed on the first terminal 100a is received, the first terminal 100a displays the list data 1000 displayed on the second terminal 100b, (For example, the wheel-down operation 1002 of the remote controller) for reducing the lower-level list of the list data 1001 and mirroring it. When the above operation is performed, the lower level lists 1010 and 1011 displayed in the first terminal 100a and the second terminal 100b may be reduced. In this case, the scale of the thumbnail of the contents included in the lower level list is reduced, and a thumbnail of more contents can be displayed.

The display mirroring method and the user input mirroring method of the second terminal with respect to the list data displayed on the first terminal have been described above.

Meanwhile, according to an embodiment of the present invention, various mirroring methods may be provided according to the type of contents. Hereinafter, an embodiment of a mirroring method according to various types of contents will be described with reference to FIGS. 11 to 19. FIG. Before the description of FIGS. 11 to 19, it is assumed that the first terminal and the second terminal are in a mirroring connection, and that the list data is converted from the landscape mode to the portrait mode and mirrored according to the display information of the second terminal do. The list data is extended to a lower level list to display contents in a thumbnail format. Here, the thumbnail may be displayed in a preview state in which the contents of the contents can be easily grasped.

11 is a diagram illustrating a method of mirroring text content according to an embodiment of the present invention.

11, when a user input 1110 for tapping a thumbnail 1100 of text content displayed on the first terminal 100a is received, the first terminal 100a displays the text content displayed on the second terminal 100b (For example, a press operation 1112 of a remote controller) for selecting a thumbnail 1101 of text content and mirroring the input. When the above operation is performed, the first terminal 100a and the second terminal 100b can display the texture contents on each full screen. In this case, the entire view screen can be configured differently according to the display information of each terminal. As shown in FIG. 11, the first terminal 100a having a small display size can constitute a full viewing screen 1120 of text contents in a notepad format, and conversely, the second terminal 100b having a large display size, The user can configure an entire view screen 1121 of text contents in an e-book format.

Meanwhile, when the user input 1130 for dragging or flicking the text content in the up / down direction is detected in the first terminal 100a while the entire contents of the text content are being displayed, (For example, the press & move operation 1132 of the remote controller) for turning over the bookcase of the text content displayed on the display unit 100b and mirroring the input.

In the case of a text content as described above, a full-view screen can be provided in an e-book format in the case of a terminal having a large display size, and a notepad format can be provided in a terminal having a small display size.

12 to 15 are diagrams illustrating a method of mirroring video content according to an embodiment of the present invention.

12, when the user input 1210 is tapped on the thumbnail 1200 of the video content displayed on the first terminal 100a, the first terminal 100a is displayed on the second terminal 100b, It can be converted into an input 1211 (e.g., a press operation 1212 of a remote controller) for selecting a thumbnail 1201 of video content and mirrored. When the above operation is performed, the first terminal 100a and the second terminal 100b can display the video content playback screens 1230 and 1231 on each full screen.

13 and 14 are diagrams for explaining a mirroring method for an operation of setting a playback position of video contents.

13, when a user input 1310 for dragging an indicator for controlling the reproduction of the video content 1300 is received at the first terminal 100a while the entire view of the video content is being executed, The controller 100a can convert the input video content 1301 into an input for changing the reproduction position of the video content 1301 being reproduced in the second terminal 100b (for example, the left / right key 1312 of the remote controller).

On the other hand, when the multi-touch user input 1410 that rotates clockwise to the first terminal 100a is received while the entire view of the video contents is being performed as shown in FIG. 14, the first terminal 100a The playback position of the video content 1401 being reproduced in the second terminal 100b can be converted into an input for forward conversion (for example, a right key 1412 of the remote controller) and mirrored. On the contrary, when the multi-touch user input 1420 rotating in the counterclockwise direction is received in the first terminal 100a, the first terminal 100a transmits the video content 1401 reproduced in the second terminal 100b (For example, the left key 1412 of the remote controller) to change the position backward and mirror it.

15 is a diagram illustrating a mirroring method for confirming additional information of video contents.

15, when a user input 1510 for tapping video content 1500 to the first terminal 100a is received while the entire view of the text content is being executed, (For example, a press operation 1511 of a remote controller) that displays additional information of the video contents 1501 displayed on the second terminal 100b and can mirror the input. Also in this case, the method of displaying the additional information can be different according to the display information of each terminal. 15, when the above operation is performed, the first terminal 100a having a small display size can display the additional information 1520 in the form of an overlay on the playback screen of the video content 1500 On the contrary, the second terminal 100b having a large display size can display the additional information 1521 on one side of the video content 1502 while dividing the screen.

16 to 19 are diagrams illustrating a method of mirroring image content according to an embodiment of the present invention.

16, when a user input 1610 for tapping a thumbnail 1600 of the image content displayed on the first terminal 100a is received, the first terminal 100a is displayed on the second terminal 100b, (For example, a press operation 1612 of a remote controller) for selecting a thumbnail 1601 of text content and mirroring the input. When the above operation is performed, the first terminal 100a and the second terminal 100b can display image contents on each full screen.

17 is a diagram illustrating a mirroring method for navigating an image content.

17, when the user input 1710 for dragging or flicking the image content 1700 displayed on the first terminal 100a is received while the entire view of the image content is being executed, the first terminal 100a Can be converted into an input 1711 (e.g., a left / right key 1712 of a remote controller) that moves to the previous and next images of the image content 1701 displayed on the second terminal 100b and mirrored .

18 is a diagram illustrating a mirroring method for a slide show operation of image contents. Here, the slide show operation means an operation of automatically moving to the next image every predetermined time.

18, when the user input 1810 for multi-touch dragging the image content 1800 displayed on the first terminal 100a is received while the entire view of the image content is being executed, the first terminal 100a Can be converted into an input 1811 (for example, a playback key 1812 of a remote controller) for executing a slide show of image contents displayed on the second terminal 100b, and mirrored.

19 is a diagram illustrating a method of mirroring image rotation operations of image content.

19, when the multi-touch user input 1900 that rotates clockwise to the first terminal 100a is received while the entire image content is being displayed, the first terminal 100a transmits the multi- (For example, a clockwise gesture 1902 or an up key of a remote controller) that rotates an image of the image content 1901 displayed on the display unit 100b clockwise by 90 degrees. On the contrary, when a multi-touch user input that rotates in a counterclockwise direction is received at the first terminal, an input for rotating the image of the displayed image content 1901 counterclockwise by 90 degrees (for example, (1902) or a down key) to be mirrored.

11 to 19, examples of providing various mirroring methods according to the types of contents have been described. As described above, the terminal can determine the mirroring operation that is mapped to the user input in consideration of the type of the content to which the user input is applied.

20 is a flowchart illustrating a method of controlling a terminal according to an embodiment of the present invention.

Referring to FIG. 20, a terminal capable of performing mirroring with an external device can display a plurality of contents on a display (S2010). Here, the plurality of contents may include text contents, video contents, image contents, audio contents, web contents, application contents, and the like.

Meanwhile, the terminal may display the plurality of contents in a thumbnail format on a list created based on a scrap date of a plurality of contents.

 The terminal can receive the display information from the external device to be mirrored (S2020), and adjust the scale and display mode of the plurality of contents in consideration of the display information of the external device (S2030). In this case, the display information may include a display size and an aspect ratio, and the terminal may adjust the display mode of the content to a portrait mode or a landscape mode by comparing the aspect ratio of the display and the aspect ratio of the external device.

In addition, the terminal may control the wireless communication unit so that a plurality of controlled contents can be displayed through the external apparatus (S2040).

Meanwhile, when displaying the plurality of contents in a thumbnail format on a list generated based on a scrap date of a plurality of contents, the terminal adjusts the scale and display mode of the list in consideration of the display information of the external apparatus, And may control the wireless communication unit so as to be displayed through the apparatus.

If the user input is received while the external device is displaying a plurality of controlled contents, the operation to be mapped to the user input is determined in consideration of the type of the content to which the user input is applied (S2050) The wireless communication unit may be controlled to perform the external device (S2060).

In this case, the terminal may determine an operation to be mapped to the user input according to the display mode of the adjusted content, and may remotely control the external device so that the external device performs the operation.

Here, the user input may be a tap, a double tap, a drag, a flick, a press, a multi touch, a pinch in, and a pinch out of the touch gesture.

20, a terminal according to an exemplary embodiment of the present invention can provide a display mirroring method and a user input mirroring method of content according to display information of a mirrored external device and a type of a mirrored content.

It will be apparent to those skilled in the art that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof.

The present invention described above can be implemented as computer readable codes on a medium on which a program is recorded. The computer readable medium includes all kinds of recording devices in which data that can be read by a computer system is stored. Examples of the computer-readable medium include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage, and the like, and also implemented in the form of a carrier wave (for example, transmission over the Internet) . Also, the computer may include a control unit 180 of the terminal.

Accordingly, the above description should not be construed in a limiting sense in all respects and should be considered illustrative. The scope of the present invention should be determined by rational interpretation of the appended claims, and all changes within the scope of equivalents of the present invention are included in the scope of the present invention.

100: mobile terminal 110: wireless communication unit
130: user input unit 151:
160: memory 180:

Claims (10)

1. A terminal capable of performing mirroring with an external device,
A display unit for displaying a plurality of contents;
A wireless communication unit for receiving display information from the external device;
A user input for receiving user input; And
Controls a scale and a display mode of the plurality of contents in consideration of display information of the external apparatus and controls the wireless communication unit so that the adjusted plurality of contents can be displayed through the external apparatus,
When the user input is received through the user input unit while the external device displays the adjusted plurality of contents, the operation to be mapped to the user input is determined in consideration of the type of the content to which the user input is applied And a controller for controlling the wireless communication unit such that the external device performs the operation. The method according to claim 1,
Wherein the display unit displays the plurality of contents in a thumbnail format on a list generated based on a scrap date of the plurality of contents,
Wherein the controller controls the wireless communication unit to display the external device by adjusting the scale and display mode of the list in consideration of the display information of the external device. The method according to claim 1,
Wherein the display information of the external device includes a display size and an aspect ratio,
Wherein the control unit compares the aspect ratio of the display unit and the aspect ratio of the external device to adjust the display mode of the content to a vertical mode or a horizontal mode. The method according to claim 1,
Wherein the control unit determines an operation to be mapped to the user input according to a display mode of the adjusted contents, and remotely controls the external apparatus so that the external apparatus performs the operation. The method according to claim 1,
Wherein the user input comprises:
At least one of a tap, a double tap, a drag, a flick, a press, a multi-touch, a pinch-in, and a pinch-out Is a touch gesture of the terminal. The method according to claim 1,
The type of the content is,
Wherein the terminal comprises at least one of text content, video content, image content, audio content, web content, and application content. A control method of a terminal capable of performing mirroring with an external device,
Displaying a plurality of contents;
Receiving display information from the external device;
Adjusting a scale and display mode of the plurality of contents in consideration of display information of the external device;
Controlling the wireless communication unit so that the adjusted plurality of contents can be displayed through the external device;
When the user input is received through the user input unit while the external apparatus displays the adjusted plurality of contents, the operation to be mapped to the user input is determined in consideration of the type of the content to which the user input is applied step; And
And controlling the wireless communication unit to perform the determined operation by the external device. 8. The method of claim 7,
Wherein the displaying the plurality of contents comprises:
And displaying the plurality of contents in a thumbnail format on a list generated based on a scrap date of the plurality of contents,
Controlling the wireless communication unit such that the plurality of contents can be displayed through the external device,
And controlling the wireless communication unit to display the image on the external device by adjusting a scale and a display mode of the list in consideration of the display information of the external device. 8. The method of claim 7,
Wherein the display information of the external device includes a display size and an aspect ratio,
Wherein adjusting the scale and display mode of the plurality of contents comprises:
Wherein the aspect ratio of the display unit is compared with the aspect ratio of the external device to adjust the display mode of the content to a vertical mode or a horizontal mode. 8. The method of claim 7,
Wherein determining an action mapped to the user input comprises:
And determining an operation to be mapped to the user input according to the display mode of the adjusted content.

Images