KR102304178B1 - User terminal device and method for displaying thereof - Google Patents

Abstract

A user terminal device and a display method thereof are provided. A display method of a user terminal device includes a display unit for displaying e-book contents, a bezel unit housing the display unit, a first touch sensing unit sensing a user touch input to the display unit, and detecting a user touch input to the bezel unit. When a touch interaction of touching at least two sides of the bezel is sensed through the second touch sensing unit and the second touch sensing unit, when a user interaction is detected at at least one side of the bezel through the second touch sensing unit, the user interaction and a control unit controlling the display of the e-book content in response to the type of , and the number of sides for which the user interaction is detected.

Description

User terminal device and its display method

The present invention relates to a user terminal device and a display method thereof, and more particularly, to a user terminal device capable of performing a user touch input not only on a display unit but also on a bezel housing the display unit, and a display method thereof.

With the development of electronic technology, various types of user terminal devices are being developed and distributed. In recent years, as the size of the user terminal device is minimized, its functions are more diversified, and the demand thereof is increasing.

The user terminal device may provide various contents such as multimedia contents or an application screen according to a user's request. A user may select a function that he/she wants to use by using a button or a touch screen provided in the user terminal device. The user terminal device may selectively execute a program according to an interaction with the user, and display the execution result.

On the other hand, as the functions that can be provided by the user terminal device are gradually diversified, various needs are also generated for a method of displaying content and a method of user interaction. That is, as the method of displaying content is changed and the types and functions of the content are greatly increased, the conventional interaction method of simply selecting a button or touching the touch screen is insufficient to perform various functions of the user terminal device. may occur.

Accordingly, there is a need for a user interaction technology that allows a user to use a user terminal device in a more convenient manner.

An object of the present invention is to provide a user terminal device capable of providing various functions according to a touch interaction sensed by at least one of a display unit and a bezel unit, and a display method thereof.

Another object of the present invention is to provide a user terminal device capable of providing various functions according to a touch interaction of touching at least two sides of a bezel part and a display method thereof.

According to an embodiment of the present invention for achieving the above object, a user terminal device includes: a display unit for displaying e-book contents; a bezel unit housing the display unit; a first touch sensing unit detecting a user interaction input to the display unit; a second touch sensing unit sensing a user interaction input to the bezel unit; and when a user interaction is detected on at least one side of the bezel unit through the second touch sensing unit, the e-book content is displayed in response to the type of the user interaction and the number of sides at which the user interaction is detected. It may include; a control unit for controlling the display.

When a single touch interaction is detected on the first side of the bezel while the e-book content is displayed, the controller displays a menu related to the e-book content, , when a multi-touch interaction is detected from the first side of the bezel, the display may be controlled to display bookmark information for the e-book content.

The control unit, when a drag interaction is detected through the first touch sensing unit while the e-book content is displayed, switches the screen of the e-book content page by page, , when a drag interaction is detected through the second touch sensing unit, to switch the screen of the e-book content chapter by chapter or switch the screen to other e-book content The display unit can be controlled.

When the drag interaction is detected at the horizontal side of the bezel unit, the control unit switches the screen of the e-book content chapter by chapter, and when the drag interaction is detected at the vertical side of the bezel unit, The display unit may be controlled to switch the screen to other e-book contents.

The controller controls the display unit to display a list of a plurality of e-book contents when a pinch-in interaction is detected in the bezel unit while the e-book contents are displayed. can do.

When a user interaction of touching and dragging a point on a vertical side of the bezel part is sensed while the e-book content is being displayed, the control unit is configured to include a point at which dragging ends from a row corresponding to the one point. The display unit may be controlled to highlight even a row corresponding to .

When a user interaction of simultaneously touching a first side of the bezel part and a second side adjacent to the first side is sensed, the control unit is configured to place the first side and the second side adjacent to the first side in a corner region between the touching points. The display unit may be controlled to display information related to the e-book content.

When a touch interaction of simultaneously touching a first side of the bezel part and a third side opposite to the first side is sensed, the controller is configured to simultaneously touch the bezel while the e-book content is being displayed The display screen is divided into two areas according to a virtual line connecting the selected points, the e-book content is displayed in a first area, and an execution screen of a second application is displayed in a second area The display unit may be controlled to do so.

The control unit is configured to include, while the display screen is divided so that the e-book content is displayed in a first area, and a second application is displayed in a second area, a first area in contact with the first area of the bezel unit When a touch interaction of touching the first side and dragging the second side in contact with the second area is sensed, the execution screen of the second application is removed from the second area and the execution screen of the third application is displayed. The display unit can be controlled.

The controller may be configured to, when detecting a drag interaction of continuously dragging from a first side of the bezel part to a second side adjacent to the first side while the e-book content is displayed, the e-book (e-book) You can change the font size of the content.

When the drag interaction proceeds in a clockwise direction, the control unit enlarges the text size of the e-book content, and when the drag interaction proceeds in a counterclockwise direction, the e-book (e) -book) You can reduce the text size of the content.

When a swipe interaction of simultaneously swiping a first side of the bezel part and a second side adjacent to the first side is detected while the page of the e-book content is displayed, the control unit is configured to: It is possible to control the page to be scraped according to the swipe interaction.

A display method of a user terminal device according to another embodiment of the present invention comprises: displaying e-book content on a display unit; and when a user interaction is detected on at least one side of the bezel part, controlling the display of the e-book content according to the type of the user interaction and the number of sides on which the user interaction is detected; may include

The controlling may include displaying a menu related to the e-book content when a single touch interaction is detected on the first side of the bezel while the e-book content is displayed. display, and when a multi-touch interaction is detected from the first side of the bezel part, bookmark information for the e-book content may be displayed.

In the controlling step, when a drag interaction is detected through the first touch sensing unit while the e-book content is displayed, the e-book content is displayed on a page-by-page basis. and when a drag interaction is detected through the second touch sensing unit, the screen of the e-book content is changed in chapter units or the screen is switched to other e-book content can do.

In the controlling step, when the drag interaction is detected at the horizontal side of the bezel unit, the screen is switched for the e-book contents chapter by chapter, and the drag interaction is detected at the vertical side of the bezel unit. In this case, the screen may be switched to other e-book contents.

The controlling may include displaying a list of a plurality of e-book contents when a pinch-in interaction is detected in the bezel unit while the e-book contents are displayed. .

In the controlling step, when a user interaction of touching and dragging a point on the vertical side of the bezel part is sensed while the e-book content is displayed, dragging is performed from a row corresponding to the point. You can make the line corresponding to the ending point be highlighted.

The controlling may include, when a user interaction of simultaneously touching the first side of the bezel part and the second side adjacent to the first side is sensed, a corner region between the first side and a point where the second side is touched. may display notification information of the user terminal device.

The controlling may include, when a touch interaction of simultaneously touching the first side of the bezel part and the third side opposite the first side is sensed while the e-book content is displayed, At the same time, the display unit is divided into two areas according to the side connecting the touched points, the e-book content is displayed in the first area, and the execution screen of the second application is displayed in the second area. can

In the controlling step, the display screen is divided so that the e-book content is displayed in a first area, and a second application is executed in a second area while abutting the first area of the bezel part. When a touch interaction of touching the first side and dragging the second side in contact with the second area is detected, the execution screen of the second application is removed from the second area and the execution screen of the third application is displayed. can

The controlling may include, when a drag interaction of continuously dragging from a first side of the bezel part to a second side adjacent to the first side is detected while the e-book content is displayed, the -You can change the font size of e-book contents.

In the controlling step, when the drag interaction proceeds in a clockwise direction, the text size of the e-book content is enlarged, and when the drag interaction proceeds in a counterclockwise direction, the e-book (e-book) You can reduce the text size of the content.

The controlling may include when a swipe interaction of simultaneously swiping the first side of the bezel part and a second side adjacent to the first side is detected while the page of the e-book content is displayed. , the page may be scraped according to the swipe interaction.

According to various embodiments of the present invention as described above, the user can perform various functions of the user terminal device by touching at least one of the display unit and the bezel unit.

1 is a block diagram schematically illustrating the configuration of a user terminal device according to an embodiment of the present invention;
2 is a block diagram showing the configuration of a user terminal device in detail according to an embodiment of the present invention;
3A to 3E are views for explaining a user terminal device including a bezel part capable of detecting a touch interaction, according to an embodiment of the present invention;
4 is a diagram for explaining the configuration of software stored in a storage unit according to an embodiment of the present invention;
5A to 14B are diagrams for explaining an embodiment of performing various functions of a user terminal according to a touch interaction of touching one side of a bezel, according to various embodiments of the present disclosure;
15A to 30 are diagrams for explaining an embodiment of performing various functions of a user terminal according to a touch interaction of touching at least two surfaces of a bezel, according to various embodiments of the present disclosure;
31A and 31B are diagrams for explaining an embodiment of performing a function of a user terminal according to a touch interaction of touching a bezel when the user terminal device is fully folded according to an embodiment of the present invention;
32 to 35 are views for explaining an embodiment in which a function mapped to a bezel is adjusted when the user terminal device rotates according to an embodiment of the present invention;
36 and 37 are flowcharts illustrating a display method of a user terminal device according to various embodiments of the present disclosure.
38 is a diagram illustrating a user interaction type and user interaction when a user interaction is detected on at least one side of a bezel part while an e-book content is displayed in the user terminal device according to an embodiment of the present invention; A diagram for explaining that the display of e-book contents is controlled according to the detected number of sides of the bezel part
FIG. 39 is a diagram illustrating that bookmark information related to the e-book content 3901 is displayed when a multi-touch interaction is detected on the first side of the bezel while the e-book content is displayed; FIG.
FIG. 40A is a view showing that the screen of the e-book content is changed chapter by chapter when a drag interaction is detected through the horizontal side of the bezel while the e-book content is displayed on the display unit of the user terminal device
FIG. 40B is a diagram illustrating that a screen is switched to another e-book content when a drag interaction is detected through the vertical side of the bezel while the e-book content is being displayed on the display unit; FIG.
41 is a diagram illustrating a plurality of e-book content lists being displayed by a pinch interaction on the bezel unit while the e-book contents are displayed on the display unit; FIG.
42 is a diagram illustrating a case in which a touch interaction is detected on the first vertical side of the bezel portion and a drag interaction is detected on the second vertical side while the e-book contents are displayed, the dragging ends from the row corresponding to the point where the touch is sensed; A diagram showing that the row corresponding to the point is highlighted
43 is a view illustrating that the display of e-book contents is controlled according to an interaction of touching two points of the bezel part;
FIG. 44 is a diagram illustrating that the font size is changed and displayed according to an interaction of continuously dragging at least two sides of the bezel while the e-book content is displayed; FIG.
45 is a diagram illustrating that a page is scraped according to a user interaction of simultaneously swiping or dragging a first side of a bezel part and a second side adjacent to the first side while a page of e-book content is displayed; FIG. drawing
46 is a diagram showing an e-book content being displayed on the first area of the display unit and a second application being displayed on the second area while touching the first side contacting the first area and dragging the second side contacting the second area When a touch interaction of
47 is a diagram illustrating a case in which a user interaction is detected on at least one side of a bezel in a state in which the e-book content is displayed in the user terminal device according to an embodiment of the present invention. ) is a flowchart for explaining the display method.

Since the present embodiments can apply various transformations and can have various embodiments, specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the scope of the specific embodiments, and it should be understood to include all transformations, equivalents and substitutions included in the spirit and scope of the disclosed technology. In describing the embodiments, if it is determined that a detailed description of a related known technology may obscure the subject matter, the detailed description thereof will be omitted.

Terms such as first, second, etc. may be used to describe various elements, but the elements should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another.

The terms used in the present application are used only to describe specific embodiments, and are not intended to limit the scope of rights. The singular expression includes the plural expression unless the context clearly dictates otherwise. In the present application, terms such as "comprises" or "consisting of" are intended to designate that the features, numbers, steps, operations, components, parts, or combinations thereof described in the specification exist, and are intended to indicate that one or more other It is to be understood that this does not preclude the possibility of addition or presence of features or numbers, steps, operations, components, parts, or combinations thereof.

In an embodiment, a 'module' or 'unit' performs at least one function or operation, and may be implemented as hardware or software, or a combination of hardware and software. In addition, a plurality of 'modules' or a plurality of 'units' are integrated into at least one module and implemented with at least one processor (not shown) except for 'modules' or 'units' that need to be implemented with specific hardware. can be

Hereinafter, an embodiment will be described in detail with reference to the accompanying drawings. 1 is a block diagram schematically illustrating a configuration of a user terminal device 100 according to an embodiment of the present invention. 1 , the user terminal device 100 includes a display unit 110 , a bezel unit 120 , a first touch sensing unit 130 , a second touch sensing unit 140 , and a control unit 150 . include In this case, the user terminal device 100 may be implemented as various types of devices such as a TV, a PC, a laptop PC, a mobile phone, a tablet PC, a PDA, an MP3 player, a kiosk, an electronic picture frame, a table display device, and the like. When implemented as a portable type device such as a mobile phone, tablet PC, PDA, MP3 player, or laptop PC, it may be referred to as a mobile device, but in this specification, it will be collectively referred to as a user terminal device.

The display unit 110 displays various image data and UI. In particular, the display unit 110 may be implemented as a touch screen in combination with the first touch sensing unit 130 . Also, the display unit 110 may be bent by bending lines corresponding to the plurality of hinge units 120 .

The bezel unit 120 is positioned on the edge of the display unit 110 to house the display unit 110 . In particular, the second touch sensing unit 140 may be included in the bezel unit 120 .

The first touch sensing unit 130 detects a user's touch interaction input on the display unit 110 . In addition, the second touch sensing unit 140 detects a user's touch interaction input on the bezel unit 120 .

The controller 150 controls the overall operation of the user terminal device 100 according to the touch interaction detected through the first touch sensing unit 130 and the second touch sensing unit 140 . In particular, when a first touch interaction of touching the display unit 110 is detected through the first touch sensing unit 130 , the control unit 150 performs a first function of the user terminal device 100 . And, when a second touch interaction of the same type as the first touch interaction for touching the bezel unit 120 is detected through the second touch sensing unit 140 , the controller 150 controls the second touch interaction of the user terminal device 100 . It can perform 2 functions. That is, even if the same type of touch interaction is detected, the controller 150 may perform different functions according to an area in which the touch interaction is sensed.

Specifically, when the drag interaction is detected through the second touch sensing unit 140 , the controller 150 can switch the screen in a higher unit than when the drag interaction is detected through the first touch sensing unit 130 . have.

In an embodiment of the present invention, when a drag interaction is detected through the first touch sensing unit 130 while the gallery application is running, the control unit 150 controls the display unit 110 to switch the display screen in units of files. can be controlled However, when a drag interaction is detected through the second touch sensing unit 140 , the controller 150 may control the display unit 110 to switch the display screen in units of folders.

In addition, when a drag interaction is detected through the first touch sensing unit 130 while the e-book application is running, the control unit 150 controls the display unit 110 to switch the display screen on a page-by-page basis. can control Also, when a drag interaction is detected through the second touch sensing unit 140 , the control unit 150 may control the display unit 110 to switch the screen in units of chapters.

Also, when the drag interaction is detected through the first touch sensing unit 130 , the controller 140 may switch the screen within the application, but when the drag interaction is detected through the second touch sensing unit 140 , , the execution screen can be switched among multiple applications.

For example, when a drag interaction is detected through the first touch sensing unit 130 while the first application is being executed, the controller 150 may control the display unit 110 to scroll the execution screen of the first application. can In addition, when a drag interaction is detected through the second touch sensing unit 140, the controller 150 removes at least a part of the execution screen of the first application from the display screen, and removes at least a part of the execution screen of the second application. The display unit 110 may be controlled to display.

Also, when a pinch-in interaction in which the distance between two touched points becomes close is detected through the first touch sensing unit 130 while the photo content is displayed, the controller 150 zooms out the photo content, and When the pinch-in interaction is detected through the second touch sensing unit 140 , the controller 150 may control the display unit 110 to display a folder list.

In addition, when a touch interaction touching at least two sides of the bezel unit 120 is detected through the second touch sensing unit 140 , the controller 150 controls the type of the touch interaction and the user corresponding to the at least two touched sides. A function of the terminal device 100 may be performed.

Specifically, when a touch interaction of simultaneously touching the first side of the bezel unit 120 and the second side adjacent to the first side is sensed while the image content is displayed, the controller 150 controls the first side and the second side The display unit 110 may be controlled to display information about the image content in a corner area between points touched.

Also, when a touch interaction of simultaneously touching the first side of the bezel unit 120 and the second side adjacent to the first side is sensed, the controller 150 controls a corner between the point where the first side and the second side are touched. The display unit 110 may be controlled to display notification information (eg, received message information, missed call information, update information, etc.) of the user terminal device in the area.

In addition, when a touch interaction of simultaneously touching the first side of the bezel unit 120 and the third side opposite the first side is sensed while the first application is running, the controller 150 controls the simultaneously touched point The display unit ( 110) can be controlled. In this case, the second application may be an application related to the first application. For example, when the first application is a phone application, the second application may be a memo application or a calendar application related to the phone application.

Also, when a drag interaction of continuously dragging from the first side of the bezel unit 120 to the second side adjacent to the first side is detected while the photo content is displayed, the controller 150 zooms in the photo content, When a drag interaction of continuously dragging from the first side of the unit 120 to the third side adjacent to the first side is detected, the controller 150 may control the display unit 110 to zoom out the photo content. In this case, the amount of zoom-in and zoom-out may be determined according to the number of sides of the bezel unit 120 at which the drag interaction is sensed or the length of the drag interaction.

In addition, a drag interaction of simultaneously touching the first side of the bezel part 120 and the second side opposite the first side while displaying the photo content is detected, and then dragging in opposite directions from the two touched points. In this case, the controller 150 may control the display 110 to rotate the photo content. In this case, the rotation direction of the photo content may be determined according to the direction of the drag interaction.

And, when a swipe interaction of simultaneously swiping the first side of the bezel unit 120 and the second side adjacent to the first side is detected while the first application is running, the controller 150 controls the swipe according to the swipe interaction. The display unit 110 may be controlled to display the execution screen of the second application in the first area of the execution screen of the first application.

In addition, while the display screen is divided so that the first application is executed in the first area and the second application is executed in the second area, the first side of the bezel unit 120 that comes into contact with the first area is touched, and the second area is touched. When a touch interaction of dragging the second side in contact with is sensed, the controller 150 removes the execution screen of the second application in the second area and controls the display 110 to display the execution screen of the third application. can do.

According to various embodiments of the present invention as described above, the user may be provided with various functions of the user terminal device 100 according to the touch interaction sensed by the bezel unit 120 .

Hereinafter, the present invention will be described in more detail with reference to FIGS. 2 to 32 . 2 is a block diagram illustrating the configuration of the user terminal device 200 in detail according to an embodiment of the present invention. As shown in FIG. 2 , the user terminal device 200 includes an image receiving unit 210 , an image processing unit 220 , a display unit 230 , a bezel unit 235 , a communication unit 240 , a storage unit 250 , It includes an audio processing unit 260 , a speaker unit 270 , a sensing unit 280 , and a control unit 290 .

On the other hand, FIG. 2 is a case in which the user terminal device 200 is a device having various functions, such as a content providing function and a display function, as an example, and various components are comprehensively shown. Accordingly, according to an embodiment, some of the components shown in FIG. 2 may be omitted or changed, and other components may be further added.

The image receiving unit 210 receives image data through various sources. For example, the image receiver 210 may receive broadcast data from an external broadcasting station, may receive VOD data from an external server in real time, and may receive image data from an external device.

The image processing unit 220 is a component that processes the image data received from the image receiving unit 210 . The image processing unit 220 may perform various image processing, such as decoding, scaling, noise filtering, frame rate conversion, and resolution conversion, on the image data.

The display unit 230 displays at least one of a video frame obtained by processing the image data received from the image receiving unit 210 by the image processing unit 220 and various screens generated by the graphic processing unit 293 . In an embodiment of the present invention, the display unit 230 may be implemented as a foldable flexible display, but this is only an embodiment and may be implemented as another display.

The bezel unit 235 is positioned at the edge of the display unit 230 to house the display unit 230 . In particular, as shown in FIG. 3A , the bezel part 235 may be positioned at the edges of four sides of the display part 230 . In addition, the display unit 230 and the bezel unit 235 may be folded by a folding line 310 as shown in FIG. 3 . In this case, the folding line 310 may be a line folded by the hinge part.

The communication unit 240 is configured to communicate with various types of external devices according to various types of communication methods. The communication unit 240 includes a Wi-Fi chip 241 , a Bluetooth chip 242 , a wireless communication chip 243 , and an NFC chip 244 . The control unit 290 performs communication with various external devices using the communication unit 240 .

In particular, the Wi-Fi chip 241 and the Bluetooth chip 242 perform communication using a WiFi method and a Bluetooth method, respectively. In the case of using the Wi-Fi chip 241 or the Bluetooth chip 242, various connection information such as an SSID and a session key are first transmitted and received, and then various types of information can be transmitted and received after communication connection using the same. The wireless communication chip 243 refers to a chip that performs communication according to various communication standards such as IEEE, ZigBee, 3rd Generation (3G), 3rd Generation Partnership Project (3GPP), and Long Term Evoloution (LTE). The NFC chip 244 refers to a chip operating in a near field communication (NFC) method using a 13.56 MHz band among various RF-ID frequency bands such as 135 kHz, 13.56 MHz, 433 MHz, 860 to 960 MHz, and 2.45 GHz.

The storage unit 250 may store various programs and data necessary for the operation of the user terminal device 200 . Specifically, the storage unit 250 includes a main area and

Programs and data for configuring various screens to be displayed in the sub area may be stored. Hereinafter, with reference to FIG. 4 , it is a diagram for explaining the structure of software stored in the user terminal device 200 . Referring to FIG. 4 , software including an OS 410 , a kernel 420 , middleware 430 , an application 440 , and the like may be stored in the storage unit 250 .

An operating system (OS) 410 controls and manages the overall operation of hardware. That is, the OS 410 is a layer in charge of basic functions such as hardware management, memory, and security.

The kernel 420 serves as a passage for transmitting various signals including a touch signal detected by the display unit 230 to the middleware 430 .

The middleware 430 includes various software modules that control the operation of the user terminal device 200 . According to FIG. 4 , the middleware 430 includes an X11 module 430-1, an APP manager 430-2, a connection manager 430-3, a security module 430-4, a system manager 430-5, It includes a multimedia framework 430-6, a UI framework 430-7, a window manager 430-8, and a handwriting recognition module 430-9.

The X11 module 430-1 is a module for receiving various event signals from various hardware provided in the user terminal device 200 . Here, the event may be variously set, such as an event in which a user gesture is detected, an event in which a system alarm occurs, an event in which a specific program is executed or terminated, and the like.

The APP manager 430 - 2 is a module for managing execution states of various applications 440 installed in the storage unit 250 . When an application execution event is detected from the X11 module 430-1, the APP manager 430-2 calls and executes an application corresponding to the event.

The connection manager 430 - 3 is a module for supporting a wired or wireless network connection. The connection manager 430 - 3 may include various detailed modules such as a DNET module and a UPnP module.

The security module 430 - 4 is a module supporting hardware certification, request permission, secure storage, and the like.

The system manager 430 - 5 monitors the status of each component in the user terminal device 200 , and provides the monitoring result to other modules. For example, if the remaining battery level is insufficient, an error occurs, or the communication connection state is cut off, the system manager 430-5 displays the monitoring result to the main UI framework 430-7 or the sub UI frame. It can be provided to the walk 430-9 to output a notification message or a notification sound.

The multimedia framework 430 - 6 is a module for playing multimedia content stored in the user terminal device 200 or provided from an external source. The multimedia framework 430 - 6 may include a player module, a camcorder module, a sound processing module, and the like. Accordingly, it is possible to perform an operation of reproducing various multimedia contents to generate a screen and sound.

The main UI framework 430-7 is a module for providing various UIs to be displayed in the main area of the display unit 230, and the sub UI framework 430-9 provides various UIs to be displayed in the sub area. module for The main UI framework 430-7 and the sub UI framework 430-9 include an image compositor module that composes various objects, a coordinate synthesizer that calculates coordinates at which objects are to be displayed, and coordinates that calculate the composed objects. It may include a 2D/3D UI toolkit that provides a rendering module for rendering to the UI, a tool for composing a 2D or 3D UI, and the like.

The window manager 430 - 8 may detect a touch event or other input event using the user's body or pen. When such an event is detected, the window manager 430-8 transmits an event signal to the main UI framework 430-7 or the sub UI framework 430-9 to perform an operation corresponding to the event.

In addition, when the user touches and drags the screen, the pitch angle, the roll angle, the yaw angle, etc. are calculated based on the sensor value detected by the writing module for drawing a line according to the drag trajectory or the motion detection unit 282 . Various program modules, such as an angle calculation module, may be stored.

The application module 440 includes applications 440 - 1 to 440 - n for supporting various functions. For example, it may include a program module for providing various services such as a navigation program module, a game module, an e-book module, a calendar module, an alarm management module, and the like. These applications may be installed by default, or the user may install and use them arbitrarily in the course of use. When an object is selected, the main CPU 294 may execute an application corresponding to the selected object using the application module 440 .

Since the software structure shown in FIG. 4 is only an example, it is not necessarily limited thereto. Accordingly, it goes without saying that some may be omitted, modified, or added as necessary. For example, the storage unit 250 includes a sensing module for analyzing signals sensed by various sensors, a messaging module such as a messenger program, a short message service (SMS) & multimedia message service (MMS) program, an email program, and phone information. Various programs such as a Call Info Aggregator program module, a VoIP module, and a web browser module may be additionally provided.

Again, referring to FIG. 2 , the audio processing unit 260 is a component that processes audio data of image content. The audio processing unit 260 may perform various processes such as decoding, amplification, and noise filtering on audio data. Audio data processed by the audio processing unit 260 may be output to the audio output unit 270 .

The audio output unit 270 is configured to output various types of audio data on which various processing operations such as decoding, amplification, and noise filtering have been performed by the audio processing unit 260 as well as various notification sounds or voice messages. In particular, the audio output unit 270 may be implemented as a speaker, but this is only an exemplary embodiment and may be implemented as an output terminal capable of outputting audio data.

The sensing unit 280 detects various user interactions. In particular, as shown in FIG. 2 , the sensing unit 280 may include a first touch sensing unit 281 , a second touch sensing unit 282 , a motion sensing unit 283 , and a bending sensing unit 284 . can

Specifically, the first touch sensing unit 281 may detect a user's touch interaction by using a touch panel attached to the rear surface of the display panel. The second touch sensing unit 282 may be positioned on the bezel unit 235 to detect a user's touch interaction. In this case, the first touch sensing unit 281 may be implemented as an electrostatic or pressure-sensitive touch sensor, and the second touch sensing unit 282 may be implemented as a proximity type touch sensor. However, this is only an exemplary embodiment, and the first touch sensing unit 281 and the second touch sensing unit 282 may be implemented with various touch sensors.

Meanwhile, the second touch sensing unit 282 may be disposed over the entire area of the bezel unit 235 , but this is only an exemplary embodiment and may be disposed only in a partial area (eg, a corner area).

In addition, as shown in FIG. 3B , the second touch sensing unit 282 may be disposed only on the bezel part 235 , but this is only an example, and as shown in FIG. 3C , the second touch sensing part 282 , the bezel part 235 . ) may be disposed on an edge region of the display unit 230 adjacent to each other, and may be disposed over the bezel unit 235 and the display unit 230 as shown in FIG. 3D . Also, when the display unit 230 is a flexible display, the second touch sensing unit 282 may be disposed in an area having a step with the display unit 230 as shown in FIG. 2 .

The motion detector 283 may detect a motion (eg, rotational motion, etc.) of the user terminal device 200 using at least one of an acceleration sensor, a geomagnetic sensor, and a gyro sensor. The bending detection unit 284 may detect at least one of whether the user terminal device 200 is folded and a folded angle based on the bending line using a bending sensor, an illuminance sensor, or the like. In this case, the bending detection unit 283 may be located on the folding line.

The control unit 290 controls the overall operation of the user terminal device 200 by using various programs stored in the storage unit 250 .

As shown in FIG. 2, the control unit 290 includes a RAM 291, a ROM 292, a graphic processing unit 293, a main CPU 294, first to n interfaces 295-1 to 295-n, bus 296 . In this case, the RAM 291 , the ROM 292 , the graphic processing unit 293 , the main CPU 294 , the first to n interfaces 295-1 to 295-n, etc. may be connected to each other through the bus 296 . .

The ROM 292 stores an instruction set for system booting and the like. When a turn-on command is input and power is supplied, the main CPU 294 copies the O/S stored in the storage unit 250 to the RAM 291 according to the command stored in the ROM 292 and executes the O/S. to boot the system. When booting is completed, the main CPU 294 copies various application programs stored in the storage unit 250 to the RAM 291 and executes the application programs copied to the RAM 291 to perform various operations.

The graphic processing unit 293 generates a screen including various objects such as items, images, and texts by using a calculation unit (not shown) and a rendering unit (not shown). The operation unit calculates attribute values such as coordinate values, shape, size, color, etc. of each object to be displayed according to the layout of the screen by using the control command received from the sensing unit 280 . The rendering unit generates screens of various layouts including objects based on the attribute values calculated by the calculation unit. The screen generated by the rendering unit is displayed in the display area of the display unit 230 .

The main CPU 294 accesses the storage unit 250 and performs booting using the O/S stored in the storage unit 250 . In addition, the main CPU 294 performs various operations using various programs, contents, data, etc. stored in the storage unit 250 .

The first to n-th interfaces 295-1 to 295-n are connected to the various components described above. One of the interfaces may be a network interface connected to an external device through a network.

In particular, the controller 290 controls the overall operation of the user terminal device 200 according to the user's touch interaction sensed through the second touch sensing unit 282 .

< of the display unit 230 . touch interaction and of the bezel part 235 touch interactive Distinguish>

When a touch interaction of touching the bezel unit 235 is detected through the second touch sensing unit 282, the control unit 290 may perform different functions depending on the area of the bezel unit 235 in which the touch interaction is detected. have.

Specifically, as shown in the left side of FIG. 5A , when a tap interaction 510 of tapping the bottom of the bezel part 235 is detected while a gallery application for displaying photo content is executed, the controller 290 controls the tap According to the interaction 510 , as shown on the right side of FIG. 5A , the display unit 230 may be controlled to display the home menu 520 in the lower right area of the display unit 230 . In this case, when the user terminal device 200 has a square shape, the bottom side of the bezel unit 235 may mean the bottom side based on the image content displayed on the display unit 230 . In addition, the home menu 520 displayed on the lower right side of the display unit 230 may include a home icon, a back icon, a view other window icon, and the like.

Also, as shown in the left side of FIG. 5B , when a tap interaction 530 of tapping the right side of the bezel part 235 is detected while the gallery application for displaying photo contents is executed, the controller 290 controls the tap According to the interaction 530 , as shown on the right side of FIG. 5B , the display unit 230 may be controlled to display an edit menu 340 for editing photo content on the right region of the display unit 230 .

That is, as shown in FIGS. 5A and 5B , the control unit 290 may control the display unit 230 to display a menu performing different functions according to an area touched on the bezel unit 235 .

In addition, the control unit 290 controls the bezel unit 235 through the second touch sensing unit 232 and when a first tap interaction of tapping the display unit 230 is detected through the first touch sensing unit 281 . When a second tap interaction of tapping is detected, different functions may be performed.

Specifically, as shown on the left side of FIG. 6A , when a tap interaction 610 of tapping a first item is detected while the screen 600 including a plurality of items is displayed, the controller 290 controls the tap interaction As shown on the right side of FIG. 6A according to 610 , the display unit 230 may be controlled to display the display screen 620 corresponding to the first item.

Also, as shown on the left side of FIG. 6B , when a tap interaction 630 of tapping the right side of the bezel part 235 is detected while the screen 600 including a plurality of items is displayed, the controller 290 ) according to the tap interaction 630, as shown on the right side of Figure 6b, the screen 600 may control the display unit 230 to display the screen 600' rotated 90 degrees counterclockwise. .

In addition, as shown in the left side of FIG. 6C , when a tap interaction 640 of tapping the right side of the bezel part 235 twice consecutively is detected while the screen 600 including a plurality of items is displayed, As shown on the right side of FIG. 6C according to the tap interaction 640 , the controller 290 controls a screen in which the execution screen 650 and the screen 600 of the application related to the currently executed application are rotated counterclockwise by 90 degrees. The display unit 230 may be controlled to simultaneously display 600 ′.

When a user interaction of touching a plurality of points of the bezel unit 235 is detected while a specific application is being executed, the controller 290 may perform a frequently used quick access function according to the application. For example, as shown on the left side of FIG. 7 , a user interaction of simultaneously tapping two points 710-1 and 710-2 on the left side of the bezel part 235 while the execution screen 700 of the gallery application is displayed When detected, the controller 290 displays a window 720 for performing the SNS sharing function, as shown on the right side of FIG. 7 in order to perform the SNS sharing function for a function frequently used by the user in the gallery application. The display unit 230 may be controlled.

Also, when a drag interaction is detected on the bezel unit 235 through the second touch sensing unit 282 , the controller 290 controls the display unit 230 in a higher depth unit (eg, , folders) can be browsed.

Specifically, as shown in the upper part of FIG. 8 , while the first photo content 810 stored in the first folder in the gallery application is displayed, the bottom edge of the bezel part 235 is displayed through the second touch sensing unit 282 . When a drag interaction 820 of touching a point and dragging in the left direction is detected, the controller 290 displays a UI 830 in which thumbnail images are displayed in units of folders, as shown in the center of FIG. 8 . The display unit 230 may be controlled to do so. Also, as shown in the lower part of FIG. 8 , the controller 290 may control the display 230 to display the second photo content 840 stored in a second folder different from the first folder.

As another example, as shown in FIG. 9A , while an e-book application is running, a point on the display unit 230 is touched through the first touch sensing unit 281 and then in a left direction. When the drag interaction 910 to drag is detected, the controller 290 may control the display 230 to display the next page according to the drag interaction 910 . That is, when the drag interaction 910 is detected on the display unit 230 , the controller 290 may switch the screen of the e-Book content in units of pages.

However, as shown in FIG. 9B , while the e-book application is running, a point of the bezel part 235 is touched through the second touch sensing unit 282 and then dragged in the left direction. When the drag interaction 920 is detected, the control unit 290 displays a UI 930 that guides the chapter of the e-Book content according to the drag interaction 910, and displays the first page of the next chapter ( 230) can be controlled. That is, when the drag interaction 920 is detected in the bezel unit 235 , the controller 290 may switch the screen of the (e-book) e-Book content in units of chapters that are higher in depth than in units of pages.

In addition, the control unit 290 detects a drag interaction on the bezel unit 235 through the second touch sensing unit 282 and detects a drag interaction at the display unit 230 through the first touch sensing unit 281 . It can perform different functions in different cases.

Specifically, as shown in FIG. 10A , a drag interaction 1010 of touching a point on the display unit 230 through the first touch sensing unit 281 and then dragging in an upward direction while the news content is displayed. When detected, the controller 290 may control the display 230 to scroll the news content in the same news content in a downward direction according to the drag interaction 1010 .

In addition, when a drag interaction 1020 of touching a point on the bottom of the bezel part 235 and dragging in the left direction is detected through the second touch sensing unit 282 while the news content is displayed, the controller 290 ) according to the drag interaction 1020 , as shown in FIG. 10B , may control the display unit 230 to display the history UI 1030 including the web page the user has recently visited. In this case, when one of a plurality of web pages included in the history UI 1030 is selected, the controller 290 may control the display 230 to display the selected web page in full screen.

Also, when a drag interaction 1040 of touching a point on the right side of the bezel part 235 and then dragging in an upward direction is detected through the second touch sensing unit 282 while the news content is displayed, the controller 290 ) may control the display unit 230 to display a browsing UI 1050 for browsing currently executed applications as shown in FIG. 10C according to the drag interaction 1040 . In this case, when one of a plurality of applications included in the browsing UI 1050 is selected, the controller 290 may control the display 230 to display the entire screen of the execution screen of the selected application.

In another embodiment, as shown on the right side of FIG. 11A , while an execution screen of a news application is displayed, a point of the display unit 230 is flicked upward through the first touch sensing unit 281 . ), when the flick interaction 1110 is detected, the controller 290 may control the display 230 to scroll the execution screen of the same news application in a downward direction according to the drag interaction 1110 .

However, as shown in the upper left of FIG. 11B , while the execution screen of the news application is displayed through the first application, a point on the right side of the bezel unit 235 is moved upward through the second touch sensing unit 282 . When the flicking flick interaction 1120 is detected, the controller 290 displays the execution screen of the music application, which is the third application, according to the flick interaction 1120 as shown in the upper right corner of FIG. 11B . ) can be controlled. That is, the controller 290 may switch the execution screen between the currently running applications through the flick interaction 1120 .

In addition, while the execution screen of the music application, which is the third application, is displayed, a drag interaction 1130 of touching a point on the right side of the bezel unit 235 through the second touch sensing unit 282 and then dragging in the downward direction is performed. When detected, the controller 290 moves the execution screen of the music application downwards according to the drag interaction 1130 as shown in the lower right of FIG. 11B to display it together with the execution screen of the news application. ) can be controlled. Then, while a part of the execution screen of the news application and a part of the execution screen of the music application are displayed together, a point on the bottom of the bezel part 235 is touched through the second touch sensing unit 282 and then dragged in the left direction. When the drag interaction 1140 is detected, the controller 290 controls a part of the execution screen of the music application and a part of the execution screen of the news application as shown in the lower left of FIG. 11B according to the drag interaction 1140 . By moving it to the left, the display unit 230 may be controlled to display a part of the execution screen of the SNS application that is the second application and a part of the execution screen of the memo application that is the fourth application. That is, the controller 290 moves the screen according to the drag amount of the drag interactions 1130 and 1140 through the drag interactions 1130 and 1140 to display execution screens for a plurality of applications, and performs multi-tasking for the plurality of applications. will be able to perform

In addition, when the pinch interaction is detected on the bezel part 235 through the second touch sensing unit 282 , the controller 290 detects the pinch interaction on the display unit 230 through the first touch sensing unit 281 . It performs a different function than in the case of

In general, after touching two points of the display unit 230 through the first touch sensing unit 281 while the photo content is being displayed, pinch-in interaction in which two touched points become closer to each other or a pinch in which two points move away from each other - When the out interaction is detected, the controller 290 may zoom-in or zoom-out the photo content according to the pinch-in interaction or the pinch-out interaction.

However, as shown in the upper part of FIG. 12 , after touching two points of the bezel part 235 through the second touch sensing unit 282 while the photo content 1210 is displayed, the two touched points are close to each other. When a losing pinch-in interaction is detected, the controller 290 may control the display 230 to display a folder screen 1220 including photo content as shown at the bottom of FIG. 12 . Also, as shown in the lower part of FIG. 12 , after touching two points of the bezel part 235 through the second touch sensing unit 282 while displaying the folder screen 1220 , the two touched points move away from each other. When the pinch-out interaction is detected, the controller 290 may control the display 230 to display the photo content 1210 as shown in the upper part of FIG. 12 .

Also, the control unit 290 may select or fix a partial area of the display unit 230 by touching at least one point of the bezel unit 235 .

Specifically, as shown in the upper part of FIG. 13A , while the first thumbnail images to the ninth thumbnail images 1310-1 to 1310-9 are displayed, the first point 1310 of the bezel part 235 is touched, and , when a drag interaction 1320 of touching an area of the display unit 230 and dragging in the downward direction is sensed, the control unit 290 moves to the first point 1310 as shown at the bottom of FIG. 13A . The first thumbnail image 1310-1, the fourth thumbnail image 1310-4, and the seventh thumbnail image 1310-7 included in the corresponding first column are fixed, and the plurality of displayed images in the second column and the third column are fixed. You can change the thumbnail images 1310-2, 1310-3, 1310-5, 1310-6, 1310-8, 1310-9 to other thumbnail images 1320-1 to 1320-6. That is, when a drag interaction is detected after touching a point 1310 of the bezel unit 235 corresponding to the first column, the controller 290 fixes the thumbnail image displayed in the first column and the thumbnail displayed in the other column. You can change the image to another thumbnail image.

In addition, while the editing screen for editing the photo content is displayed, as shown in the upper part of FIG. 13B , the two points 1330-1 and 1330-2 of the bezel part 235 are touched, and the lower region of the photo content is displayed. When a drag interaction 1340 of touching and dragging in the left direction is sensed, the controller 290 controls the first selected by two points 1330-1 and 1330-2 as shown in the lower part of FIG. 13B . The area 1350 maintains the existing setting values, while the display unit displays the second area 1360 except for the first area 1350 after processing the second area 1360 by applying other setting values. 230 can be controlled.

Also, as shown in FIG. 14A , while the first to fourth applications are simultaneously displayed, one point 1410 of the third application is touched, the right side of the bezel part 235 is touched, and then upwards. When the drag interaction 1420 is detected, the controller 290 may control the display 230 to fix the first application, the second application, and the fourth application, and to scroll the third application upward. have.

Also, while the web page is displayed, as shown in the upper part of FIG. 14B , two points 1430-1 and 1430-2 of the bezel part 235 are touched, and the bottom area of the web page is touched, and then the downward direction When the drag interaction 1440 of dragging is detected, the controller 290 controls the first region 1450 selected by the two points 1430-1 and 1430-2 as shown in the lower part of FIG. 14B as it is. On the other hand, the display unit 230 may be controlled to scroll the second areas 1460 - 1 and 1460 - 2 excluding the first area 1450 to another area according to the drag interaction 1440 .

< of the bezel part 235 Multi-touch input on multiple sides interaction >

When two points of the bezel unit 290 are touched, the controller 290 divides the display screen into a plurality of areas using the two touched points and controls the display unit 230 to display different images in the plurality of areas. can be controlled

Specifically, as shown in the upper part of FIG. 15A , while the photo content 1500 is displayed, the first point 1510 - 1 and the second point 1510 - 2 of the right side of the bezel part 235 are separated. In the case of a touch interaction for a preset time, the controller 290 guides detailed information of the photo content 1500 to a corner area defined by the first point 1510 - 1 and the second point 1510 - 2 . The display unit 230 may be controlled to display the UI 1520 . At this time, as shown in FIG. 15A , the controller 290 folds the photo content 1500 by a touch interaction of touching the first point 1510 - 1 and the second point 1510 - 2 , and the photo content ( 1500), the display unit 230 may be controlled to display a UI 1520 for guiding detailed information.

In addition, as shown in the upper part of FIG. 15B , while the photo content 1500 is displayed, the first point 1530-1 of the upper side and the second point 1530-2 of the lower side of the bezel part 235 are recorded. In the case of a touch interaction for a set time, the controller 290 divides the display screen into two by a first point 1530-1 and a second point 1530-2, as shown in the lower part of FIG. 15B. The display unit 230 may be controlled to display a part 1510 ′ of the photo content in the first area and display an execution screen of a memo application related to the photo content in the second area. That is, the control unit 290 may enter the multi-tasking mode through a touch interaction of touching a plurality of points of the bezel unit 235 .

In addition, as shown in the upper part of FIG. 16 , a touch interaction of touching two points 1610 and 1620 of the bezel part 235 through the second touch sensing unit 282 for a preset time in a state in which the display screen is turned off is performed. When detected, the controller 290 may control the display 230 to turn on the display screen as shown in the lower part of FIG. 16 . That is, power control of the display unit 230 can be performed through the multi-touch of the bezel unit 235 .

In addition, as shown in the upper part of FIG. 17 , while the map application 1730 is displayed, the first point 1710 of the left side and the second point 1720 of the right side of the bezel part 235 are touched for a preset time. In the case of touch interaction, the controller 290 divides the display screen into two by a first point 170 and a second point 1720 as shown at the bottom of FIG. 17 , and places a part of the map application in the first area. 1730 ′, and the display unit 230 may be controlled to display a web page 1740 in the second area. In this case, the web page 1740 may be an application executed before the map application 1730 is executed. That is, the control unit 290 may divide the screen through a touch interaction of touching a plurality of points of the bezel unit 235 .

Also, the controller 290 may enlarge or reduce the currently displayed screen through a drag interaction of dragging a plurality of sides of the bezel unit 235 .

Specifically, as shown in the upper part of FIG. 18 in a state in which the photo content 1810 is displayed, when a drag interaction of touching a point on the left side of the bezel part 235 and then dragging in a clockwise direction is detected, the control unit As shown in the middle and bottom of FIG. 18 , reference numeral 290 may control the display unit 230 to display photo contents 1810 ′ and 1810 ″ by enlarging according to a drag interaction.

In addition, when a drag interaction force that is dragged in a counterclockwise direction after touching a point of the bezel unit 235 is detected while the photo content 1810 is displayed, the controller 290 reduces the picture according to the drag interaction. The display unit 290 may be controlled to display content.

Also, the control unit 290 may adjust the number of images displayed on the display unit 230 through a drag interaction of dragging a plurality of sides of the bezel unit 235 .

Specifically, as shown in the upper part of FIG. 19 , after touching a point on the left side of the bezel unit 235 while the screen 1910 including nine images is running, clockwise to the upper side of the bezel unit 235 . When a drag interaction is detected, the controller 290 may control the display unit 230 to display a screen 1920 including four images among nine images as shown in the center of FIG. 19 . have. If the drag interaction continues to the right side of the bezel part 235, the controller 290 displays a screen 1930 including one image among four images, as shown in the lower part of FIG. 19 . The unit 230 may be controlled. Conversely, the control unit 290 may increase the number of images displayed on the display unit 230 according to a counterclockwise drag interaction with respect to the bezel unit 235 .

Also, the control unit 290 may turn on the display unit 230 according to a drag interaction input to two or more sides of the bezel unit 235 .

Specifically, as shown in the upper left of FIG. 20 , in a state in which the power of the display unit 230 is turned off (2010), after touching a point on the right side of the bezel unit 235, the bezel unit 235 is rotated in a clockwise direction. When a drag interaction of dragging up to the area below is detected, the controller 290 controls the display unit 290 to display time information in the first area 2020 of the display screen according to the drag interaction as shown in the upper right of FIG. 230) can be controlled. When the drag interaction continues to the left side of the bezel part 235, the control unit 290 displays time information and frequently used applications in the second area 2030 of the display screen as shown in the lower right corner of FIG. The display unit 230 may be controlled to display the shortcut icons together. If the drag interaction continues to the right side of the bezel part 235 , the controller 290 may control the display 230 to turn on the entire screen as shown in the lower left corner of FIG. 20 . That is, the control unit 290 may gradually turn on the display unit 230 through a drag interaction input to two or more sides of the bezel unit 235 .

Also, the controller 290 may search for a plurality of images through a drag interaction of simultaneously touching and dragging one point on each of the two sides of the bezel unit 235 .

Specifically, as shown in the upper part of FIG. 21 , while the first image 2110 is displayed, after touching one point on the right side and the bottom side of the bezel part 235 through the second touch sensing unit 282 . When the drag interaction 2120 - 1,2120 - 2 of dragging the two touched points to be closer to each other is sensed, the controller 290 controls the second image 2110 ′ in addition to the first image 2110 ′ as shown in the middle of FIG. 21 . The display unit 230 may be controlled to display the image 2130 . When a drag interaction of continuously dragging two touched points to become closer is detected, the controller 290 controls the first image 2110'' and the second image 2130', as shown at the bottom of FIG. 21 . And the display unit 230 may be controlled to simultaneously display the third image 2140 . In this case, the first image 2110 , the second image 2130 , and the third image 2140 may be pictures having similar colors, pictures of the same person, or pictures taken on the same date.

In addition, the controller 290 may rotate the screen using a drag interaction of simultaneously touching one point of opposite sides of the bezel part 235 and then dragging in the opposite direction.

Specifically, as shown in the upper part of FIG. 22 , while the photo content 2200 is displayed, a point on the left side of the bezel part 235 is touched through the second touch sensing part 282 and then dragged in an upward direction. When the first drag interaction 2210-1 and the second drag interaction 2210-2 for dragging in the downward direction after touching a point on the right side of the bezel part 235 are simultaneously sensed, the control unit 290 According to the first drag interaction 2210-1 and the second drag interaction 2210-2, the display unit 230 may be controlled to display the photo content 2200' rotated by 90 degrees in the clockwise direction.

Also, the controller 290 may perform various functions of the user terminal device through a swipe interaction of simultaneously swiping the first side of the bezel unit 235 and the second side adjacent to the first side.

Specifically, as shown in the upper part of FIG. 23 , while the gallery application 2310 is displayed, one point on the right side and the bottom side of the bezel unit 235 is touched through the second touch sensing unit 282 and then the touch is performed. When a swipe interaction (2310-1,2310-2) of swiping to bring one or two points closer is detected, the controller 290 controls an Internet application other than the gallery application 2310', as shown in the center of FIG. 21 . The display unit 230 may be controlled to display 2320 . When a swipe interaction of continuously swiping to bring the two touched points closer is detected, the controller 290 divides the display screen as shown at the bottom of FIG. The display unit 230 may be controlled to simultaneously display the applications 2320 ′. Accordingly, the controller 290 may enter a multi-tasking mode in which a plurality of applications can be used simultaneously.

In addition, as shown in the upper part of FIG. 24A , while the photo content 2410 is displayed, one point on the right side and the bottom side of the bezel part 235 is touched through the second touch sensing unit 282 and then touched. When a swipe interaction of swiping to bring two points closer is detected, the controller 290 divides the photo content 2410 into a plurality of areas 2410-1 to 2410-5 as shown in the lower part of FIG. 24A. The display unit 230 may be controlled to divide and apply different image setting values to each of the plurality of regions 2410 - 1 to 2410 - 5 .

Also, as shown on the left side of FIG. 24B , one point on the right side and the bottom side of the bezel unit 235 through the second touch sensing unit 282 while the UI 2420 in which the photo contents are sorted by day is displayed. When a swipe interaction of swiping to bring the two touched points closer after touching is detected, the controller 290 displays a UI 2430 in which photo contents are arranged by month, as shown in the center of FIG. 24B . The display unit 230 may be controlled to do so. While the UI 2430 in which the photo contents are arranged by month is displayed, a point on the right side and the bottom side of the bezel part 235 is touched through the second touch sensing unit 282, and then the two touched points are moved closer together. When a swiping swipe interaction is detected, the controller 290 may control the display 230 to display a UI 2440 in which photo contents are arranged by year, as shown on the right side of FIG. 24C .

In addition, as shown in the upper left of FIG. 24B , the right side of the bezel part 235 through the second touch sensing unit 282 while the screen 2450 for guiding the current weather is displayed while the weather application is running. and when a swipe interaction of swiping to bring the two touched points closer to each other after touching a point on the bottom side is sensed, the controller 290 divides the display screen into three regions as shown in the upper right of FIG. The display unit 230 may be controlled to display a screen 2460 that guides morning, lunch, and evening weather. Also, as shown in the upper right of FIG. 24B , one of the right side and the bottom side of the bezel unit 235 through the second touch sensing unit 282 while the screen 2460 for guiding breakfast, lunch, and dinner is displayed When a swipe interaction of swiping to bring two touched points closer to each other after touching a point is detected, the controller 290 divides the display screen into four areas as shown in the lower right of FIG. 24C for the next 4 days The display unit 230 may be controlled to display a screen 2470 for guiding the weather. In addition, while the screen 2470 for guiding the weather for 4 days as shown in the lower right of FIG. 24B is displayed, one point of the right side and the base of the bezel unit 235 through the second touch sensing unit 282 When a swipe interaction of swiping to bring two touched points closer after touching is detected, the controller 290 divides the display screen into seven regions as shown in the lower left of FIG. The display unit 230 may be controlled to display a guiding screen 2470 .

Meanwhile, in the above-described embodiment, a function performed when a swipe interaction of touching one point on the right side and the bottom side of the bezel part 235 and then swiping to bring the two touched points closer has been described. This is merely an embodiment, and when a swipe interaction of touching one point on the right side and the bottom side of the bezel part 235 and then swiping away from the two touched points is sensed, the technical idea of the present invention may be applied. have. At this time, when a swipe interaction of touching one point on the right side and the bottom side of the bezel part 235 and then swiping away from the two touched points is sensed, the controller 290 controls the right side of the bezel part 235 . and a function opposite to a function performed when a swipe interaction of swiping to bring two touched points closer to each other after touching a point on the base is detected.

In addition, when a swipe interaction of touching one point on the right side and the bottom side of the bezel part 235 and then swiping away from the two touched points is detected, the control unit 290 displays notification information in the lower right corner area The display unit 230 may be controlled to display (eg, a message reception notification, a missed call notification, an update notification, etc.).

Specifically, as shown in FIG. 25 , while the video application 2500 is displayed, a swipe interaction of touching one point on the right side and the bottom side of the bezel part 235 and then swiping the two touched points away from each other. When (2510-1,2510-2) is detected, the controller 290 displays text message notification information in the lower right corner area 2520 where the swipe interaction 2510-1,2510-2 is detected. The display unit 230 may be controlled. In this case, the size of the lower right corner area 2520 in which the text message notification information is displayed may be changed according to the swipe amount of the swipe interaction 2510 - 1,2510 - 2 .

Also, the controller 290 may perform a scroll function at different speeds using various drag interactions input to the bezel unit 235 .

As shown in FIG. 26A , a drag interaction 2610 of touching a point on the right side of the bezel part 235 through the second touch sensing unit 282 and then dragging in an upward direction is detected while the web page is displayed. In this case, the controller 290 may control the display unit 2330 to scroll the web page at the first speed.

Also, as shown in FIG. 26B , while the web page is displayed, a point on the bottom side of the bezel part 235 is touched through the second touch sensing unit 282 , and a point on the right side of the bezel part 235 is touched. When a drag interaction 2620 of touching and dragging in an upward direction is detected, the controller 290 may control the display unit 2330 to scroll the web page at a second speed twice faster than the first speed. .

Also, as shown in FIG. 26B , while a web page is displayed, one point on the right side and the bottom side of the bezel part 235 is touched through the second touch sensing unit 282 and then the two touched points are separated from each other. When the drag interaction 2630 is detected, the controller 290 may control the display unit 2330 to scroll the web page at a third speed four times faster than the first speed.

The controller 290 may provide a multi-tasking function through a case in which a swipe interaction of swiping after simultaneously touching two sides in contact is detected.

Specifically, as shown in the upper part of FIG. 27A , while the first application 2710 is displayed, one point of the bottom side and the right side of the bezel unit 235 is touched through the second touch sensing unit 282, respectively. When the swipe interaction 2720-1 and 2720-2 for swiping closer to each other is sensed, the controller 290 controls the swipe interaction 2720-1 and 2720-2 as shown at the bottom of FIG. 27A. Similarly, the display unit 230 may be controlled to reduce the size of the first application 2710 ′ and display the second application 2730 . The size of the reduced first application 2710 ′ may be changed to correspond to the swipe interaction.

Also, as shown in the upper part of FIG. 27B , while the third application 2740 is displayed, one point of the bottom side and the right side of the bezel unit 235 is touched through the second touch sensing unit 282 , respectively, and then each other When a swipe interaction 2750-1 and 2750-2 swiping away from each other is sensed, the controller 290 controls the swipe interaction 2720-1 and 2720-2 as shown at the bottom of FIG. 27B. , the display unit 230 may be controlled to enlarge and display the fourth application 2760 on the third application 2740'. The size of the enlarged fourth application 2760 may be changed to correspond to the swipe interaction.

The controller 290 may provide a multi-tasking function through a drag interaction of simultaneously touching opposite sides and then dragging in the same direction.

Specifically, as shown in the upper part of FIG. 28A , while the map application 2810 is displayed, the left side and the right side of the bezel unit 235 are touched through the second touch sensing unit 282, respectively, and then downward direction When a drag interaction (2820-1,2820-2) to drag is detected, the controller 290 controls the gallery application 2830 to come down from the top of the display screen according to the drag interaction (2820-1,2820-2). By controlling the display unit 230 , the display unit 230 may be controlled to simultaneously display the map application 2810 ′ and the gallery application 2830 as shown at the bottom of FIG. 28A .

In addition, as shown in the upper part of FIG. 28B , the display screen is divided into two regions so that the map application 2840 is displayed on the left region and the gallery application 2850 is displayed on the right region while the second touch sensing unit ( 282), when a drag interaction 2860-2 of touching a point on the left side of the bezel part 235 (2860-1), touching a point on the right side and then dragging in an upward direction is detected, the control unit ( 290) controls the display unit 230 so that the Internet application 2870 comes up from the bottom of the display screen according to the drag interaction 2860-2, and as shown at the bottom of FIG. 28B, the gallery application ( 2850') and the Internet application 2870 may be controlled to display the display unit 230 at the same time.

Also, as shown in the upper part of FIG. 29 , while the application screen 2900 is displayed, a first drag interaction 2910 of touching one point of each of the upper and left sides of the bezel part 235 and then dragging them to approach each other. -1) and when a second drag interaction 2910-2 of touching one point of each of the bottom and right sides of the bezel part 235 and dragging them to approach each other is simultaneously sensed, the controller 290 controls the As shown in the center, a pinching image effect is provided on the application screen 2900 according to the first drag interaction 2910-1 and the second drag interaction 2910-2, and again shown at the bottom of FIG. As described above, after returning to the application screen 2900, the application screen 2900 may be captured. In this case, the display unit 230 may be controlled to display a UI 2920 informing that the application screen 2900 is stored on the returned application screen 2900 .

In addition, as shown on the left side of FIG. 30 , the display screen is divided into two regions, the gallery application 3010 is displayed on the left region, and the memo application 3020 is displayed on the right region while the second touch sensing unit ( 282), when a drag interaction 3030-2 of touching a point on the left side of the bezel unit 235 (3030-1), touching a point on the right side and then dragging in an upward direction is detected, the control unit ( As shown in the right side of FIG. 30 , 290 may control the display unit 230 to fix the gallery application 3010 displayed in the right area and display the map application 3040 to the left area. In this case, the map application 3040 may be the most recently executed application before the gallery application 3010 and the memo application 3020 are executed. Accordingly, the controller 290 may switch the application displayed on one area of the display unit 230 to another application through the touch interaction 3030 - 1 and the drag interaction 3030 - 2 .

<Others bezel interaction >

In an embodiment of the present invention, when a call request is received from the outside while the square-shaped user terminal device 200 is being folded, the controller 290 may control the speaker unit 270 to output an incoming call sound. . At this time, as shown in the upper part of FIG. 31A , when a tap interaction of tapping the bezel part 235 is sensed, the controller 290 accepts a phone call and controls the speaker part 270 to output a phone call sound. can do. And, as shown in the middle of FIG. 31A while performing a phone call, when a drag interaction of touching a point on the long side and then dragging in an upward direction is detected, the controller 290 increases the volume of the phone call sound. It is possible to control the speaker unit 270 so as to

In another embodiment, when a text message is received from the outside while the square-shaped user terminal device 200 is being folded, the controller 290 controls the speaker unit 270 to output a notification guiding that the message is received. can be controlled At this time, as shown in the upper part of FIG. 31B , when a tap interaction of tapping an area of the bezel part 235 is detected, the controller 290 converts the received text message into TTS (TEXT-TO-SPEECH) conversion. The speaker unit 270 may be controlled to output audio through the . Then, as shown in the center of FIG. 31B while audio corresponding to the text message is output, when a drag interaction of touching a point on the long side and dragging in the upward direction is detected, the controller 290 responds to the text message. The speaker unit 270 may be controlled to increase the volume of the corresponding audio.

When the user terminal device 200 has a square shape, the controller 290 may maintain the main control area at the base, which is easy to access by the user, even when the user terminal device 200 is rotated.

As an embodiment of the present invention, the second touch sensing unit 282 of the user terminal device 200 includes a lower right area 3210 and an upper left area 3220 of the bezel unit 235 as shown in FIG. 32A . ) can be placed in In this case, when a touch interaction is detected in the lower right region 3210 , the controller 290 may perform a main control function (eg, a home menu call, an application control menu call, etc.). In addition, when a touch interaction is detected in the upper left region 3220 , the controller 290 may perform a sub-control function (eg, an additional function of a currently executed application, etc.).

In particular, as shown in the upper part of FIG. 32B , when the second touch sensing unit 282 is disposed in the lower right region 3210 and the upper left region 3220 , the user terminal device 200 rotates 90 clockwise in a clockwise direction. When a touch interaction is detected in the lower left region 3210 after being rotated, the controller 290 performs a main control function, and when a touch interaction is detected in the upper right region 3220, the controller 290 controls the sub control function can be performed.

As another embodiment, the second touch sensing unit 282 of the user terminal device 200 may be disposed over the entire area of the bezel unit 235 . In this case, when a touch interaction is detected in the lower right region 3230 , the controller 290 may perform a main control function (eg, a home menu call, an application control menu call, etc.). In addition, when a touch interaction is detected in the upper left region 3240 , the controller 290 may perform a sub-control function (eg, an additional function of a currently executed application, etc.).

And, as shown in FIG. 32C , when the user terminal device 200 is rotated 90 degrees clockwise and a touch interaction is continuously detected in the lower right region 3230 ′, the controller 290 controls the main control When a function is performed and a touch interaction is detected in the upper left region 3240 ′, the controller 290 may perform a sub control function.

According to the above-described embodiment, even when the user terminal device 200 rotates, the touch area for controlling the main control function can be continuously positioned at the bottom.

In addition, when a touch interaction of touching the bezel unit 235 is detected through the second touch sensing unit 282 and a shake interaction of shaking the user terminal device 200 through the motion sensing unit 283 is detected, The control unit 290 may perform a different function from a case in which only a touch interaction for touching the bezel unit 235 is sensed through the second touch sensing unit 282 .

Specifically, as shown in the left side of FIG. 33A , when a tap interaction 3310 of tapping the bottom of the bezel part 235 is detected while the gallery application is displayed, the controller 290 controls the As shown on the right, the display unit 230 may be controlled to switch the current screen to the home screen.

However, as shown in the upper part of FIG. 33B , a tap interaction 3320 of tapping the bottom of the bezel part 235 is sensed while the gallery application is running, and at the same time, the user terminal device 200 through the motion detector 283 . ), when a shake interaction is detected, the controller 290 displays a home menu 3330 including at least one icon in the corner area where the tap interaction 3320 is detected, as shown at the bottom of FIG. 33B . The display unit 230 may be controlled to do so. In this case, the home menu 3330 may include a home screen moving icon, a back icon, a different window view icon, a setting icon, and the like, as shown at the bottom of FIG. 33B .

As another embodiment, as shown in the upper side of FIG. 34A , when a tap interaction 3410 of tapping the top of the left side of the bezel part 235 is detected while the gallery application is displayed, the control unit 290 ) may control the display unit 230 to display a plurality of images 3420 to which different attribute values are applied to the photo content currently displayed at the bottom of the display unit 230 as shown at the bottom of FIG. 34A .

However, as shown in the upper part of FIG. 34B , a tap interaction 3430 for tapping the bottom of the bezel part 235 is sensed while the gallery application is running, and at the same time, the user terminal device 200 through the motion detector 283 . ), when a shake interaction is detected, the controller 290 controls the gallery application in the corner area where the tap interaction 3430 is detected, as shown at the bottom of FIG. 34B Context including at least one icon The display unit 230 may be controlled to display the menu 3440 . In this case, the home menu 3440 may include a previous photo content view icon, a next photo content view icon, an edit icon, and a delete icon, as shown at the bottom of FIG. 34B .

In addition, as shown in the first of FIG. 35 , while the first application 3500 is displayed, the user terminal device ( 200), when a shake interaction is detected, the controller 290 reduces the first application 3500 as shown in the second of FIG. 35 and displays a screen 3510 on which other applications are stacked like a card. The display unit 230 may be controlled to do so. 35 , while the screen 3510 on which the reduced first application and other applications are stacked like a card is displayed, the left and right sides of the bezel part 235 are continuously touched and motion is sensed at the same time When a shake interaction that harms the user terminal device 200 is detected through the unit 283 , the controller 290 controls a screen 3520 in which the second application is located at the top as shown in the third of FIG. 35 . The display unit 230 may be controlled to display When the touch on the right side is released, the controller 290 may control the display 230 to display the screen 3530 enlarged to the full screen of the second application as shown in the fourth of FIG. 35 .

Hereinafter, a display method of the user terminal device 200 according to an embodiment of the present invention will be described with reference to FIGS. 36 and 37 . 36 is a flowchart for explaining an embodiment of performing different functions when a touch interaction is detected on the display unit 230 and the bezel unit 235, respectively, according to an embodiment of the present invention.

First, the user terminal device 200 displays an image (S3610).

Then, the user terminal device 200 detects a touch interaction on the display unit 230 or the bezel unit 235 (S3620). In this case, the touch interaction may include at least one of a tap interaction, a drag interaction, a swipe interaction, and a pinch interaction.

When the touch interaction is detected on the display unit 230, the user terminal device 200 performs the first function (S3630), and when the touch interaction is detected on the bezel unit 235, the user terminal device 200 A second function is performed (S3640). That is, even if the same type of touch interaction is input, the user terminal device 200 may perform different functions depending on whether the touch interaction is input to the display unit 230 or the bezel unit 235 .

37 is a flowchart for explaining an embodiment of performing a function of the user terminal device 200 when a touch interaction is detected on a plurality of sides of the bezel part 235 according to an embodiment of the present invention.

First, the user terminal device 200 displays an image (S3710).

Then, the user terminal device 200 determines whether a touch interaction for touching at least two sides of the bezel part 235 is detected (S3720).

When a touch interaction touching at least two sides of the bezel part 235 is detected (S3720-Y), the user terminal device 200 performs a function corresponding to the type of the touch interaction and the two touched sides (S3730) .

According to various embodiments of the present invention as described above, the user can perform various functions of the user terminal device 200 by touching at least one of the display unit 230 and the bezel unit 235 .

<e-book book ) specialized users interaction >

38 is a diagram illustrating a case in which user interaction is detected on at least one side of the bezel unit 235 in a state where e-book content is displayed in the user terminal device 200 according to an embodiment of the present invention; It is a diagram explaining that the display of e-book contents is controlled according to the type of user interaction and the number of sides of the bezel part where the user interaction is detected.

47 is a diagram illustrating a case in which user interaction is detected on at least one side of the bezel part 235 in a state where e-book content is displayed in the user terminal device 200 according to an embodiment of the present invention. - It is a flowchart for explaining a method of displaying book contents (e-book).

Referring to FIG. 38 , an e-book content 3801 , a menu 3805 , and a bezel unit 3808 are shown on the display unit 230 . The e-book content 3801 may have a hierarchical structure. The menu 3805 may include a plurality of menu items capable of performing a function related to the e-book content 3801 . The bezel part 3803 may house the display part 230 .

The user terminal device 200 includes a first touch sensing unit 281 that detects a user interaction input to the display unit 230 , and a second touch sensing unit 282 that detects a user interaction input into the bezel unit 235 . and when a user interaction is detected on at least one side of the bezel unit 235 through the second touch sensing unit 282, an e-book corresponds to the type of user interaction and the number of sides on which the user interaction is detected. ) may include a control unit 290 for controlling the display of the content. That is, the control unit 290 variously transforms e-book contents or relates to e-book contents according to user interaction input to the display unit 230 and the bezel unit 235 . The display unit 230 may be controlled to display information.

The display method of the user terminal device 200 includes the step of displaying the e-book content on the display unit 230 ( S4701 ) and when the user interaction is detected from at least one side of the bezel unit 235 , the type of user interaction and The method may include controlling the display of the e-book content in response to the number of sides detected by the user interaction ( S4703 ).

The bezel part 235 houses the display part 230 and may include four sides. However, this is only an exemplary embodiment, and the bezel part 235 may include three sides, four sides, and five or more sides depending on the shape of the display unit 230 . Each side of the bezel unit 235 may detect a user interaction through the second touch sensing unit 282 . The user interaction type may include a single touch interaction, a multi-touch interaction, a drag interaction, a flip interaction, and a combination of two or more user interactions listed above. For example, a touch and drag interaction may be a combination of two or more user interactions.

Referring to FIG. 38 , when the user touches the bezel unit 3803 while the e-book content 3801 is displayed on the display unit 230 , the control unit 290 controls the bezel through the second touch sensing unit 282 . A user interaction 3807 input to the unit 3803 may be detected. The controller 290 may determine the type of user interaction. The user interaction may be a single touch interaction. Also, the control unit 290 may determine the number of sides of the bezel unit 3803 in which the user interaction is detected. The controller 290 may determine whether a user interaction is detected from one side or a plurality of sides of the bezel part, and control the display of the e-book content accordingly. Controlling the display of the e-book content may mean displaying various information related to the e-book content.

While the e-book content is being displayed, if a single touch interaction 3807 is detected on the first side of the bezel part 3803, the controller 290 displays a menu screen 3805 related to the e-book content. The unit 230 may be controlled. The menu may include a plurality of items. For example, the menu may include 'Scrap', 'Bookmark', 'Highlight' and 'Memo' as items. The controller 290 may control the display 230 to display the menu 3805 overlapping the displayed e-book content 3801 . That is, if the user touches 3807 one side of the bezel 3803 with one finger while the e-book content 3801 is displayed on the display 230, the controller 290 displays the e-book content. The display unit 230 may be controlled to display a menu 3905 related to the e-book content by overlapping. When a single touch interaction is detected on any of the four sides of the bezel unit 3803 while the e-book content is displayed, the controller 290 controls the display unit 230 to display a menu related to the e-book content. can be controlled

When a multi-touch interaction is detected on the first side of the bezel unit 235 , the controller 290 may control the display unit 230 to display bookmark information for the e-book content.

Referring to FIG. 39 , e-book contents 3901, bookmarks and highlight information 3909 are shown. The bezel part 3903 houses the display part 230 . While the e-book content 3901 is being displayed, if multi-touch interactions 3905 and 3907 are detected on the first side of the bezel part 3903 , the controller 290 controls a bookmark related to the e-book content 3901 . The display unit 230 may be controlled to display the information 3909 .

Also, when the multi-touch interactions 3905 and 3907 are detected from the first side of the bezel part 3903 while the e-book content 3901 is being displayed, the controller 290 controls the highlight information on the e-book content. The display unit 230 may be controlled to display .

The bookmark information 3909 may be location information on a specific page of the e-book content 3901 . The highlight information may be a display of a specific area of the e-book content 3901 to be visually distinguished.

Referring to FIG. 40A , when a drag interaction is detected through the horizontal edge of the bezel unit 4003 while the e-book content 4001 is displayed on the display unit 230 of the user terminal device 200, the e-book It is shown that the screen is switched in units of chapters.

When a drag interaction is detected through the first touch sensing unit 281 while the e-book contents are being displayed, the control unit 290 switches the screen of the e-book contents page by page, and the second touch sensing unit 282 When a drag interaction is detected through , the controller 290 may control the display 230 to switch the screen of the e-book content chapter by chapter or switch the screen to other e-book content.

For example, if the user drags the display unit 230 while the e-book content is displayed on the display unit 230 , the currently displayed page may be switched to the next page or the previous page. The first touch sensing unit 281 may be electrically connected to the display unit 230 . When the user drags the display unit 230 , the first touch sensing unit 281 may detect a user interaction.

When the drag interaction is detected through the second touch sensing unit 282 , the control unit 290 determines which side of the horizontal side or the vertical side of the bezel unit 4003 is the drag interaction, and determines which side of the bezel unit 4003 the drag interaction is detected. It is possible to control the display of the e-book content based on the location.

For example, if the user drags 4005 a horizontal side of the bezel unit 4003 while the e-book content 4001 is displayed on the display unit 230 , the control unit 290 currently displays the content on the display unit 230 . The display unit 230 may be controlled to display the chapter of the e-book content 4001 being converted into the e-book content 4007 of the next chapter. In addition, the controller 290 may control the display unit 230 to display the previous chapter or the next chapter according to the left and right direction of the drag.

Also, if the user drags (4015) the vertical side of the bezel part 4013 while the e-book content 4001 is displayed on the display unit 230, the control unit 290 controls the The display unit 230 may be controlled so that the e-book content is converted to other e-book content and displayed.

Referring to FIG. 40B , on the display unit 230 , an e-book content 4011 , a reduced image 4017 of the e-book content, a reduced image 4019 of other e-book content, and other e-book content 4021 are displayed on the display unit 230 . ) is displayed.

If the user drags the vertical side 405 of the bezel part 4013 while the e-book content 4011 is being displayed on the display unit 230 , the controller 290 may display the e-book content on the display unit 230 . The display unit 230 may be controlled to display a reduced image 4017 of , and a reduced image 4019 of other e-book contents.

As the user drags the vertical side 4015 of the bezel unit 4013 , the reduced images of the plurality of e-book contents may be scrolled and displayed on the display unit 230 . The size of the reduced image of the e-book content located in the center among the plurality of reduced images of the e-book content may be displayed relatively large. When the user releases the drag operation, a reduced image 4019 of the centrally located e-book content may be selected, and other e-book content may be displayed on the display unit 230 .

That is, when a drag interaction is detected on the horizontal side of the bezel unit 4013 while the e-book contents are being displayed on the display unit 230, the control unit 290 switches the screen of the e-book contents chapter by chapter, When a drag interaction is detected on the vertical side of the bezel unit 4013 , the controller 290 may control the display unit 230 to switch the screen in units of e-book contents.

When a pinch-in interaction is detected in the bezel unit 235 while the e-book contents are displayed on the display unit 230 of the user terminal device 200 , the control unit 290 displays a list of a plurality of e-book contents. The display unit 230 may be controlled to do so.

Referring to FIG. 41 , an e-book content 4101, a plurality of e-book content lists 4107, and a category list 4111 of the e-book content are shown on the display unit 230 of the user terminal device 200. has been The plurality of e-book contents list 4107 may be an arrangement of reduced images of the plurality of e-book contents. The category list 4111 of the e-book contents may be an image in the form of a bookshelf, and a plurality of e-book contents may be classified and displayed according to categories.

When the user pinches in the bezel unit 4101 while the e-book contents are displayed on the display unit 230 , a plurality of e-book contents lists 4107 may be displayed on the display unit 230 . The pinch-in is an operation of bringing two fingers into contact with the bezel part and moving the two fingers so that the distance between the two fingers becomes closer while maintaining the contact.

When the user pinches in the bezel while the plurality of e-book content lists 4107 are displayed on the display unit 230 , the controller 290 displays the plurality of e-book contents classified by category on the display unit 230 . The display unit 23 may be controlled to display the book contents 4111 .

When the user pinches out the bezel while a plurality of e-book contents 4111 classified by category are displayed on the display unit 230 , the control unit 290 displays a list of a plurality of e-book contents 4107 . The display unit 230 may be controlled to do so. The pinch-out is an operation of bringing two fingers into contact with the bezel part and moving the two fingers so that the distance between the two fingers increases while maintaining the contact.

If the user pinches out the bezel while a plurality of e-book contents lists 4107 are displayed on the display unit, the controller 290 controls the display unit 230 to display the e-book contents 4101. can

When detecting a touch interaction on the first vertical side of the bezel part 235 and detecting a drag interaction on the second vertical side of the bezel part 235 while the e-book content is being displayed, the controller 290 controls a point corresponding to the point where the touch is sensed. The display unit 230 may be controlled to highlight and display a row corresponding to a point where the dragging ends from the row. The highlighted display may be visually distinct display.

Referring to FIG. 42 , an e-book content 4201 and a highlighted area 4207 are shown on the display unit 230 . When the user touches and drags the vertical side 4205 of the bezel unit 4203 while the e-book contents 4201 are displayed on the display unit 230, a row of e-book contents corresponding to the touched point This could be the highlight. While the user drags, the highlight area may expand according to the drag distance. That is, the highlight area may be determined according to the drag length.

When the user interaction of simultaneously touching the first side of the bezel unit 235 and the second side adjacent to the first side of the bezel unit 235 is detected while the e-book content is displayed on the display unit 230, the control unit 290 controls the first side and the display unit 230 may be controlled to display information related to the e-book content in a corner area between the points where the second side is touched.

Referring to the upper drawing of FIG. 43 , the e-book content 4301, two points 4303 and 4305 touched by the user, and information 4307 related to the e-book content are shown.

While the e-book content 4301 is displayed on the display unit 230 , the first point 4303 of the first side of the bezel unit 235 and the second point 4305 of the second side of the bezel unit 235 are displayed. When a touch interaction is detected for a set time, the controller 290 displays information related to the e-book content 4301 in a corner area defined by the first point 4303 and the second point 4305 . The display unit 230 may be controlled. At this time, as shown in the upper part of FIG. 43 , the control unit 290 folds the corner of the e-book content 4301 by a touch interaction of touching the first point 4303 and the second point 4305 . The display unit 230 may be controlled to display a UI guiding information related to the e-book content 4301, for example, detailed information of the e-book content.

Referring to the lower drawing of FIG. 43 , an e-book content 4311 , a second application execution screen 4317 , and two points 4313 and 4315 touched by the user are shown.

When a touch interaction of simultaneously touching the first side of the bezel part and the third side opposite the first side is sensed while the e-book content 4311 is being displayed, the controller 290 controls the point at which they are simultaneously touched The display screen is divided into two areas along a virtual line connecting 4313 and 4315, the e-book content 4311 is displayed in the first area, and the second application is displayed in the second area. The display unit 230 may be controlled to display the execution screen 4317 .

The first area and the second area may have the same size, but this is only an example, and the first area and the second area may have different sizes. A portion 4311 of the entire page of the e-book content may be displayed in the first area. In addition, the e-book content may be reduced (4311) and displayed in the first area. In the second area, the most recently executed second application 4317 among a plurality of applications currently being multitasked may be displayed.

When a drag interaction of continuously dragging from the first side of the bezel unit 235 to the second side adjacent to the first side is detected while the e-book content is displayed on the display unit 230, the control unit 290 controls the -You can enlarge or reduce the font size of the book contents. That is, the font size of the e-book content can be changed.

Referring to FIG. 44 , e-book contents 4401 , 4405 , 4409 and drag interaction directions 4403 , 4407 and 4411 having different font sizes are shown.

While the e-book content is displayed on the display unit 230, if the user starts dragging from the first side 4403 of the bezel unit and continuously drags to the second side 4407 adjacent to the first side 4403, The control unit 290 may control the display unit 230 to display the size of the letters gradually enlarged whenever the drag direction is changed.

For example, while the e-book content 4401 having the first font size is displayed on the display unit 230 , the user starts dragging from the first side 4403 of the bezel unit 235 to the second side ( In the case of continuously dragging up to 4407 , the e-book content 4405 having the second font size is displayed, and in the case of continuously dragging up to the third side 4411 , the font size is the third size. In e-book content 4409 may be displayed.

While the e-book content 4409 having the third font size is displayed on the display unit 230 , the user starts dragging from the third side 4403 of the bezel unit 235 to the second side 4407 . In the case of continuous dragging, the e-book content 4405 having the second font size is displayed, and in the case of continuous dragging to the first side 4403, the e-book content 4405 having the first font size is displayed. Book contents 4401 may be displayed.

That is, when the drag interaction proceeds in the clockwise direction, the control unit 290 gradually enlarges the text size of the e-book content according to the drag direction, and when the drag interaction proceeds in the counterclockwise direction, the control unit 290 controls the The font size of the e-book content can be reduced in stages according to the drag direction.

Also, when the drag interaction proceeds in the clockwise direction, the controller 290 reduces the text size of the e-book content in stages according to the drag direction, and when the drag interaction proceeds in the counterclockwise direction, the controller 290 controls the The font size of the e-book content can be enlarged step by step according to the drag direction.

In addition, when the drag interaction proceeds in the clockwise direction, the controller 290 continuously enlarges the text size of the e-book content according to the drag movement distance, and when the drag interaction proceeds in the counterclockwise direction, the controller 290 can continuously reduce the font size of the e-book content according to the drag movement distance.

Also, when the drag interaction proceeds in the clockwise direction, the controller 290 continuously reduces the text size of the e-book content according to the drag movement distance, and when the drag interaction proceeds in the counterclockwise direction, the controller 290 can continuously enlarge the font size of the e-book contents according to the drag movement distance.

Referring to Fig. 45, an e-book content 4501, a reduced image 4507 of the e-book content, and a scrap icon 4509 are shown.

When a swipe interaction of simultaneously swiping the first side of the bezel and the second side adjacent to the first side is detected while a page of e-book content is displayed on the display unit, the control unit 290 may scrape the page according to the swipe interaction.

When scraping a page, the controller 290 may control the display 230 to generate a reduced image of the page while an animation in which the page is gradually reduced is displayed. Also, the controller 290 may control the display 230 to display the reduced image of the page as if sucked into the scrap icon. When the user touches the scrap icon, the controller 290 may control the display 230 to display reduced images corresponding to the plurality of scraped pages. The thumbnail image may include address information for navigating to a corresponding page.

For example, if the user simultaneously swipe the first side 4503 and the second side adjacent to the first side while the e-book content 4501 is displayed on the display unit 230, the control unit 290 may The two-touch sensing unit 282 may simultaneously detect a swipe interaction on both sides and control the display unit 230 to generate a reduced image of a page being displayed according to the swipe interaction. The controller 290 may control the display 230 to display the process of reducing the page as an animation. As the page is gradually reduced, a reduced image 4507 of the page is generated, and as the reduced image moves to the scrap icon 4509 , the reduced image 4507 may disappear from the screen.

Also, when a user interaction of simultaneously dragging the first side of the bezel unit 235 and the second side adjacent to the first side is detected while the page of the e-book content is displayed on the display unit 230 , the control unit 290 ) can scrap the page according to the drag interaction.

Referring to FIG. 46 , an e-book content 4601 , a second application 4607 , and a third application 4609 are shown in the divided area of the display unit 230 .

While the e-book content 4601 is displayed on the first area of the display unit 230 and the second application 4605 is displayed on the second area of the display unit 230 , the first side of the bezel unit 235 is in contact with the first area. When a touch interaction of touching and dragging the second side in contact with the second area is sensed, the controller 290 converts the execution screen of the second application displayed in the second area to the execution screen of the third application to be displayed. The display unit 230 may be controlled.

For example, the display unit 230 is divided into two areas to display the e-book content 4601 in the first area and the second application that the user touches in the first area while the second application is displayed in the second area. When the first side 4603 is touched and the second side 4605 in contact with the second region is dragged, the controller 290 converts the second application 4607 to the third application 4609 according to the drag operation and displays the display. The display unit 23 may be controlled to do so.

Conversely, if the user touches the second side in contact with the second area and drags the first side in contact with the first area, the e-book content may be switched to another application and displayed according to the drag operation.

That is, the controller 290 controls the display unit 230 to display the application displayed near the touch point by fixing the screen of the application displayed near the touch point while the display screen is divided, and switching the application displayed near the drag point to another application. can

According to various embodiments of the present invention as described above, the user can perform various functions of the e-book contents by touching at least one of the display unit 230 and the bezel unit 235 .

In addition, the display method of the user terminal device according to the above-described various embodiments may be implemented as a program and provided to the user terminal device. Specifically, a non-transitory computer readable medium in which a program including a control method of a user terminal device is stored may be provided.

The non-transitory readable medium refers to a medium that stores data semi-permanently, rather than a medium that stores data for a short moment, such as a register, cache, memory, etc., and can be read by a device. Specifically, the various applications or programs described above may be provided by being stored in a non-transitory readable medium such as a CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM, and the like.

In addition, although preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and the technical field to which the present invention belongs without departing from the gist of the present invention as claimed in the claims In addition, various modifications may be made by those of ordinary skill in the art, and these modifications should not be individually understood from the technical spirit or perspective of the present invention.

110, 230: display unit 120, 235: bezel unit
130: first touch sensing unit 140: second touch sensing unit
150, 290: control unit 210: image receiving unit
220: image processing unit 240: communication
250: storage unit 260: audio processing unit
270: audio output unit 280: sensing unit

Claims (24)

In the user terminal device,
a display unit for displaying content;
a bezel unit housing the display unit;
a first touch sensing unit detecting a first user interaction input to the display unit;
a second touch sensing unit for sensing a second user interaction input to the bezel unit; and
When a user interaction is detected on at least one side of the bezel unit through the second touch sensing unit, a control unit configured to control the display of the content in response to the type of the second user interaction and the number of sides on which the second user interaction is detected including;
The control unit is
When a touch interaction of simultaneously touching a first side of the bezel unit and a second side opposite the first side is sensed while the content is displayed, the display unit may be displayed according to a virtual line connecting the touched points. dividing the screen into two areas, and controlling the display unit to display the content in a first area among the divided areas and display an execution screen of an application in a second area among the divided areas,
While the content is displayed in the first area and the execution screen of the application is displayed in the second area, a first side of the bezel part that comes in contact with the first area is touched, and controlling the display unit to remove an execution screen of the application in the second area and display an execution screen of another application when a touch interaction of dragging two sides is detected. According to claim 1,
The content is e-book content,
The control unit is
When a single touch interaction is detected on the first side of the bezel while the e-book content is displayed, a menu related to the e-book content is displayed, and the bezel part When a multi-touch interaction is detected on the first side, the user terminal device according to claim 1, wherein the display unit is controlled to display bookmark information for the e-book content. According to claim 1,
The content is e-book content,
The control unit is
When a drag interaction is detected through the first touch sensing unit while the e-book content is being displayed, the screen is switched for the e-book content page by page, and the second When a drag interaction is detected through the touch sensing unit, controlling the display unit to switch the screen of the e-book content chapter by chapter or switch the screen to other e-book content A user terminal device, characterized in that. According to claim 1,
The content is e-book content,
The control unit is
When a drag interaction is detected at the horizontal side of the bezel, the screen is switched for each chapter of the e-book content, and when the drag interaction is detected at the vertical side of the bezel, another e-book ( e-book) A user terminal device, characterized in that for controlling the display unit to switch the screen to the content. According to claim 1,
The control unit is
When a pinch-in interaction is detected in the bezel unit while the contents are being displayed, the user terminal device according to claim 1, wherein the display unit is controlled to display a list of a plurality of contents. According to claim 1,
The control unit is
When an interaction of touching and dragging a point on the vertical side of the bezel part is detected while the content is displayed, controlling the display part to highlight from a row corresponding to the one point to a row corresponding to a point where the dragging ends a user terminal device. According to claim 1,
The control unit is
When a user interaction of simultaneously touching the first side of the bezel part and the second side adjacent to the first side is sensed, the first side and the second side adjacent to the first side are touched in a corner region between the touching points. and controlling the display unit to display information related to the content. delete delete According to claim 1,
The control unit is
The user terminal device according to claim 1, wherein the font size of the content is changed when a drag interaction of continuously dragging from a first side of the bezel part to a second side adjacent to the first side is detected while the content is displayed. 11. The method of claim 10,
The control unit is
When the drag interaction proceeds in a clockwise direction, the text size of the content is enlarged, and when the drag interaction proceeds in a counterclockwise direction, the text size of the content is reduced. According to claim 1,
The control unit is
When a swipe interaction of simultaneously swiping the first side of the bezel part and the second side adjacent to the first side is detected while the page of the content is displayed, the page is scraped according to the swipe interaction a user terminal device. A display method of a user terminal device capable of inputting a user touch into a display unit and a bezel unit housing the display unit, the display method comprising:
displaying e-book contents on the display unit; and
When a user interaction is detected on at least one side of the bezel part, controlling the display of the e-book content according to the type of the user interaction and the number of sides on which the user interaction is detected; includes; do,
The controlling step is
When a touch interaction of simultaneously touching the first side of the bezel unit and the second side opposite the first side is sensed while the content is being displayed, the display unit is displayed according to an imaginary line connecting the touched points. dividing a screen into two areas, displaying the content in a first area of the divided areas, and displaying an execution screen of an application in a second area of the divided areas; and
While the content is displayed in the first area and the execution screen of the application is displayed in the second area, a first side of the bezel part that comes into contact with the first area is touched, and a second side that comes into contact with the second area is touched. When a touch interaction of dragging a side is detected, removing the execution screen of the application in the second area and displaying the execution screen of another application; 14. The method of claim 13,
The content is e-book content,
The controlling step is
When a single touch interaction is detected on the first side of the bezel while the e-book content is displayed, a menu related to the e-book content is displayed, and the bezel part When a multi-touch interaction is detected on the first side, the display method of claim 1, characterized in that the bookmark information for the content of the e-book is displayed. 14. The method of claim 13,
The content is e-book content,
The controlling step is
When a drag interaction is detected through the first touch sensing unit of the user terminal device while the e-book content is displayed, the screen is switched between the e-book content page by page, , when a drag interaction is detected through the second touch sensing unit of the user terminal device, the screen of the e-book content is switched chapter by chapter or the screen is converted into another e-book content A display method, characterized in that for switching. 16. The method of claim 15,
The controlling step is
When the drag interaction is detected at the horizontal side of the bezel, the screen is switched for each chapter of the e-book content, and when the drag interaction is detected at the vertical side of the bezel, another e-book (e-book) Display method characterized in that the screen is converted to content 14. The method of claim 13,
The controlling step is
When a pinch-in interaction is detected in the bezel unit while the content is being displayed, a list of a plurality of content is displayed. 14. The method of claim 13,
The controlling step is
Display, characterized in that, when a user interaction of touching and dragging a point on the vertical side of the bezel part is detected while the content is being displayed, from a row corresponding to the one point to a row corresponding to a point where the dragging ends Way. 14. The method of claim 13,
The controlling step is
When a user interaction of simultaneously touching the first side of the bezel part and the second side adjacent to the first side is sensed, the first side and the second side adjacent to the first side are touched in a corner region between the touching points. A display method characterized in that the notification information of the user terminal device is displayed. delete delete 14. The method of claim 13,
The controlling step is
When a drag interaction of continuously dragging from a first side of the bezel part to a second side adjacent to the first side is detected while the contents are displayed, the text size of the contents is changed. 14. The method of claim 13,
The controlling step is
When the drag interaction proceeds in the clockwise direction, the text size of the content is enlarged, and when the drag interaction proceeds in the counterclockwise direction, the text size of the content is reduced. 14. The method of claim 13,
The controlling step is
When a swipe interaction of simultaneously swiping the first side of the bezel part and the second side adjacent to the first side is detected while the page of the content is displayed, the page is scraped according to the swipe interaction display method.

Images