WO2017014475A1

WO2017014475A1 - Electronic device and control method therefor

Info

Publication number: WO2017014475A1
Application number: PCT/KR2016/007536
Authority: WO
Inventors: 박정록; 조시연; 김대현; 노완호; 박진형
Original assignee: 삼성전자 주식회사
Priority date: 2015-07-17
Filing date: 2016-07-12
Publication date: 2017-01-26

Abstract

The present invention provides an electronic device and a control method therefor. The electronic device of the present invention comprises: a touch display including a main display area and a plurality of auxiliary display areas positioned at a side surface of the main display area; and a processor for controlling the touch display such that a UI for text input is provided in the plurality of auxiliary display areas of the touch display, and for controlling the touch display such that text inputted according to a touch interaction is provided in the main display area of the touch display, when the touch interaction with the UI is inputted.

Description

Electronic device and control method thereof

The present disclosure relates to an electronic device and a control method thereof, and more particularly, to an electronic device having a touch display including a main display area and a plurality of auxiliary display areas positioned on a side of the main display area, and a control method thereof. It is about.

Recently, various electronic devices have been provided. In particular, a wearable electronic device worn and operated by a user, such as a smart watch, has been provided. The wearable electronic device has a problem in that the user always has a high portability but a small touch display.

There are some problems in inputting characters through a wearable electronic device having such a small touch display. In particular, since the touch display is small, the keyboard UI for inputting characters is small. Therefore, due to the small size of the keyboard UI, not only the user's input speed of text is slow but also the possibility of touching the wrong letter key during the text input increases, so that the user cannot input the text smoothly. In addition, when the keyboard UI is displayed on a small touch screen, there is a problem that covers most of the screen.

SUMMARY The present disclosure has been made to solve the above-described problem, and an object of the present disclosure is to provide a UI for inputting a character to a plurality of auxiliary display areas of a touch display, thereby allowing a user to perform text input more conveniently and intuitively. An electronic device and a control method thereof are provided.

According to an exemplary embodiment of the present disclosure, an electronic device includes a touch display including a main display area and a plurality of auxiliary display areas positioned on side surfaces of the main display area; And controlling the touch display to provide a UI for inputting text to the plurality of sub display areas of the touch display, and when a touch interaction for the UI is input, inputs the input characters according to the touch interaction. And a processor for controlling the touch display to provide the main display area.

The touch display may be integrally extended from the main display area and bent toward the first side of the main display area and integrally extended from the main display area and the first curved auxiliary display area smaller than the main display area. The display device may include a second curved auxiliary display area that is bent to a second side of the display area and smaller than the main display area.

In addition, the UI may include a plurality of UI elements, and each of the plurality of UI elements may correspond to at least one character.

The processor may provide a UI element corresponding to a collection among the plurality of UI elements to the first curved auxiliary display area, and provide a UI element corresponding to a consonant among the plurality of UI elements in the second curved auxiliary display area. The touch display can be controlled to provide to.

The processor may provide a character corresponding to the touched UI element to the main display area when a user touch of one of the plurality of UI elements is detected and swipes toward the main display area. To control the touch display.

The processor may delete a character provided in the main display area when a user touch is swiped in a predetermined direction in the main display area.

The processor may execute a specific application according to the user touch when a user touch for simultaneously swiping the first curved auxiliary display area and the second curved auxiliary display area is detected.

The processor may execute a camera application according to the user touch when a user touch for simultaneously swiping the first curved auxiliary display area and the second curved auxiliary display area in a left and right direction is detected.

The processor may detect a direction of the electronic device while the camera application is running and display a live view image on at least a portion of the main display area and the plurality of auxiliary display areas based on the detected direction of the electronic device. The touch display may be controlled to display the touch screen.

On the other hand, according to an embodiment of the present invention for achieving the above object, a control method of an electronic device having a touch display including a main display area and a plurality of auxiliary display areas located on the side of the main display area, Providing a UI for text input in the plurality of auxiliary display regions of the touch display; And when a touch interaction with respect to the UI is input, providing a character input according to the touch interaction to the main display area of the touch display.

The touch display may be integrally extended from the main display area and bent toward the first side of the main display area and integrally extended from the main display area and the first curved auxiliary display area smaller than the main display area. The display device may include a second curved auxiliary display area bent to a second side of the display area and smaller than the main display area.

The providing of the UI may include providing a UI element corresponding to a collection among the plurality of UI elements to the first curved auxiliary display area, and displaying a UI element corresponding to a consonant among the plurality of UI elements. It may be provided to the curved auxiliary display area.

In the providing of the main display area, when a user touch of one of the plurality of UI elements is swiped and then swipe toward the main display area is detected, a character corresponding to the touched UI element is detected. The main display area may be provided.

The method may further include deleting a character provided in the main display area when a user touch is swiped in a predetermined direction in the main display area.

The method may further include executing a specific application according to the user touch when a user touch for simultaneously swiping the first curved auxiliary display area and the second curved auxiliary display area is detected.

The executing may include executing a camera application according to the user touch when a user touch for simultaneously swiping the first curved auxiliary display area and the second curved auxiliary display area in a left and right direction is detected.

The method may further include detecting a direction of the electronic device while the camera application is running; The method may further include displaying a live view image on at least a portion of the main display area and the plurality of auxiliary display areas based on the detected direction of the electronic device.

According to various embodiments of the present disclosure as described above, a user may more conveniently input text and execute various functions by using the auxiliary display area of the electronic device.

1 is a block diagram schematically illustrating a configuration of an electronic device according to an embodiment of the present disclosure;

2A-2C illustrate types of touch displays, according to various embodiments of the present disclosure;

3 is a block diagram illustrating a configuration of an electronic device in detail according to an embodiment of the present disclosure;

4 is a diagram illustrating an example of a software configuration of an electronic device according to an embodiment of the present disclosure;

5A to 10 are diagrams for describing an embodiment of performing various functions using an auxiliary display area of an electronic device according to various embodiments of the present disclosure;

11 is a flowchart illustrating a control method of an electronic device according to an embodiment of the present disclosure.

The embodiments may be variously modified and have various embodiments, and specific embodiments will be illustrated in the drawings and described in detail in the written description. However, this is not intended to limit the scope to the specific embodiments, it should be understood to include all transformations, equivalents, and substitutes included in the scope of the disclosed spirit and technology. In describing the embodiments, when it is determined that the detailed description of the related known technology may obscure the gist, the detailed description thereof will be omitted.

Terms such as first and second may be used to describe various components, but the components should not be limited by the terms. The terms are only used to distinguish one component from another.

The terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting of scope. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, the terms "comprise" or "consist" are intended to indicate that there is a feature, number, step, action, component, part, or combination thereof described in the specification, and one or more other It is to be understood that the present invention does not exclude the possibility of the presence or the addition of features, numbers, steps, operations, components, parts, or a combination thereof.

In an embodiment, the module or unit performs at least one function or operation, and may be implemented by hardware or software, or a combination of hardware and software. In addition, the plurality of modules or the plurality of units may be integrated into at least one module except for the modules or units that need to be implemented with specific hardware, and are implemented as at least one processor (not shown). Can be.

In addition, in an embodiment, the UI may include at least one of a configuration for receiving a user interaction and a configuration indicating notification information. In this case, the UI may include a UI element, and the UI element may include elements for providing notification information as well as elements capable of interacting with the user and allowing feedback such as visual, audio, and olfactory according to a user input. . In this case, the UI element may be expressed in the form of at least one of an image, a text, and a video, or if there is an area in which the above information is not displayed but feedback is possible according to a user input, the area may be referred to as a UI element.

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram illustrating an example of a basic configuration of an electronic device 100 for describing various embodiments of the present disclosure. As shown in FIG. 1, the electronic device 100 includes a touch display 110 and a processor 120. Meanwhile, the electronic device 100 of FIG. 1 may be a wearable device such as a smart watch, but this is only an example, and may be a TV, a PC, a laptop PC, a mobile phone, a tablet PC, a PDA, an MP3 player, a kiosk, an electronic photo frame, and the like. It may be implemented in various types of devices, such as a table display device. When implemented as a portable type of device, such as a mobile phone, a tablet PC, a PDA, an MP3 player, a laptop PC, etc., the mobile terminal may be referred to as a portable terminal.

The touch display 110 outputs image data and receives a user touch. In particular, the touch display 110 may include a main display area and a plurality of auxiliary display areas. In this case, the main display area may mean an area disposed on the front surface of the electronic device 100, and the auxiliary display area may be disposed on the side of the main display area. Here, the main display area and the sub display area may be interpreted in different meanings. For example, a relatively large area of the two areas may be referred to as a main display area, and a small area may be defined as an auxiliary display area.

In particular, the touch display 110 may be implemented as a flexible display, and includes a main display area provided on the front surface of the electronic device 100 and a first curved surface extending from the main display area and bent to the first side of the electronic device 100. The display device may be configured as a second curved auxiliary display area extending from the auxiliary display area and the main display area and bent to the second side surface of the electronic device 100. Meanwhile, the first auxiliary display area and the second auxiliary display area may include only a portion which is bent and fixed in the side direction of the electronic device 100, but this is only an example and may correspond to the side direction of the electronic device 100. It may include not only a bent and fixed portion but also a flat portion located in front of the electronic device 100.

In this case, an area of each of the sub display areas may be smaller than that of the main display area. In addition, the plurality of curved display areas may form different surfaces from the main display area. For example, when the main display area is disposed on the front surface of the electronic device 100, the plurality of auxiliary display areas may include a right side surface, among the surfaces forming the appearance of the electronic device 100. It may be disposed on different surfaces such as a left side surface, an upper side surface, a lower side surface, and the like. The surface including the main display area and the surface including the at least one auxiliary display area may be fixed to form an obtuse angle. The shape, location, and number of auxiliary display areas may be variously implemented according to embodiments. This will be described in detail with reference to the accompanying drawings. On the other hand, when the auxiliary display area is located on the side surface of the surfaces forming the appearance of the electronic device 100, the auxiliary display area may be referred to as an edge area. Also, the auxiliary display area may be modified in various terms, such as a sub area and a side area, according to an exemplary embodiment.

2A to 2C are diagrams illustrating examples of external appearance of an electronic device 100 including a touch display 110 divided into a main display area 111 and two sub display areas 113-1 and 113-2. to be.

According to FIG. 2A, the main display area 111 is disposed on the front surface, and the auxiliary display areas 113-1 and 113-2 are disposed on the upper side and the lower side, respectively. In this case, the main display area 111 and the sub display areas 113-1 and 113-2 are divided by boundary lines 115-1 and 115-2. In addition, according to FIG. 2A, each of the auxiliary display regions 113-1 and 113-2 may be disposed at an obtuse angle with the main display region 111 so as to be viewed from the front direction. However, the curved auxiliary display area is only an example, and the auxiliary display area may be implemented in other forms. For example, according to FIG. 2B, the main display area 111 and the plurality of auxiliary display areas 113-1 and 113-2 may be disposed on one surface. That is, one display may be divided into a main display area 111 and two sub display areas 113-1 and 113-2 through the boundary surfaces 115-1 and 115-2. As another example, according to FIG. 2C, the first auxiliary display area 113-1 of the plurality of auxiliary display areas 113-1 and 113-2 may be disposed on the same plane as the main display area 111, and the second The auxiliary display area 113-2 may be bent to the lower side of the main display area 111.

In particular, the touch display 110 displays a UI for inputting characters in the auxiliary display areas 113-1 and 113-2. In this case, the UI may include a plurality of UI elements, and each of the plurality of UI elements may correspond to at least one character.

The processor 120 controls the overall operation of the electronic device 100 using the main display area and the plurality of auxiliary display areas of the touch display 100.

In detail, when a situation for text input is detected, the processor 120 may provide the touch display 110 to provide a UI for text input to the plurality of auxiliary display areas 113-1 and 113-2 of the touch display 110. Can be controlled. In this case, the processor 120 may provide various types of UIs according to a type of language or a text input method. This will be described later.

In particular, the processor 120 provides a UI element corresponding to a collection among a plurality of UI elements to the first curved auxiliary display area 113-1, and provides a UI element corresponding to a consonant among the plurality of UI elements to a second curved auxiliary surface. The touch display 110 may be controlled to provide the display area 113-2.

In particular, when a user's touch of swiping toward the main display area 111 is detected after touching one of the plurality of UI elements, the processor 120 touches the character corresponding to the touched UI element 110. The touch display 110 may be controlled to provide the main display area 111 of FIG. This will be described later in detail with reference to the drawings.

In addition, when a user's touch of swiping in a predetermined direction is detected in the main display area 111, the processor 120 may delete the input character. In addition, when a user touch is swiped in a direction opposite to a preset direction on the main display area 111, the processor 120 may perform a space bar function.

When a user touch that simultaneously swipes the first curved auxiliary display area 113-1 and the second curved auxiliary display area 113-2 is detected, the processor 120 may execute a specific application according to the user touch. For example, when a user touch for simultaneously swiping the first curved auxiliary display area 113-1 and the second curved auxiliary display area 113-2 in the right direction is detected, the processor 120 according to the user touch. You can run the camera application.

In addition, the processor 120 detects a direction of the electronic device 100 while the camera application is executed and based on the detected direction of the electronic device 100, the main display area 111 and the plurality of auxiliary display areas 113. The touch display 110 may be controlled to display a live view image on at least some of the -1,113-2.

3 is a block diagram illustrating an example of a detailed configuration of an electronic device 200 implemented as a smart watch. According to FIG. 3, the electronic device 200 may include a memory 210, a GPS chip 220, a communication unit 230, a video processor 240, an audio processor 250, a button 260, a microphone unit 270, The camera 280 includes a speaker 290, a touch display 300, a sensor 310, and a processor 320.

The memory 210 may store various programs and data necessary for the operation of the electronic device 200. Specifically, the memory 210 may store programs, data, and the like for configuring various screens to be displayed in the main display area and the sub display area. The processor 320 displays contents in each of the main display area and the sub display area of the touch display 300 using the programs and data stored in the memory 210. In other words, the processor 320 may control the touch display 300 to display content. In particular, the processor 320 may display a UI for text input on the auxiliary display area of the touch display 300. In addition, when a user touch is made to a boundary line corresponding to a main display area, an auxiliary display area, and a boundary therebetween, the processor 320 performs a control operation corresponding to the touch.

The GPS chip 220 is a component for receiving a GPS signal from a global positioning system (GPS) satellite and calculating a current position of the electronic device 200. The processor 320 may calculate the user location using the GPS chip 220 when using a navigation program or when the current location of the user is required.

The communication unit 230 is a component that performs communication with various types of external devices according to various types of communication methods. The communication unit 230 includes a Wi-Fi chip 231, a Bluetooth chip 232, a wireless communication chip 233, an NFC chip 234, and the like. The processor 320 communicates with various external devices using the communication unit 230.

The Wi-Fi chip 231 and the Bluetooth chip 232 communicate with each other by WiFi and Bluetooth. In the case of using the Wi-Fi chip 231 or the Bluetooth chip 232, various connection information such as SSID and session key may be transmitted and received first, and then various communication information may be transmitted and received using the same. The wireless communication chip 233 refers to a chip that performs communication according to various communication standards such as IEEE, zigbee, 3rd Generation (3G), 3rd Generation Partnership Project (3GPP), Long Term Evoloution (LTE), and the like. The NFC chip 234 refers to a chip operating in a near field communication (NFC) scheme using a 13.56 MHz band among various RF-ID frequency bands such as 135 kHz, 13.56 MHz, 433 MHz, 860-960 MHz, 2.45 GHz, and the like.

The video processor 240 is a component for processing video data included in content received through the communication unit 230 or content stored in the memory 210. The video processor 240 may perform various image processing such as decoding, scaling, noise filtering, frame rate conversion, resolution conversion, and the like on the video data.

The audio processor 250 is a component for processing audio data included in content received through the communication unit 230 or content stored in the memory 210. The audio processor 250 may perform various processing such as decoding, amplification, noise filtering, and the like on the audio data.

When the playback program for the multimedia content is executed, the processor 320 may drive the video processor 240 and the audio processor 250 to play the content. The touch display 300 may display an image frame generated by the video processor 340 in at least one of the main display area and the sub display area. In addition, the speaker 290 outputs audio data generated by the audio processor 250.

The button 260 may be various types of buttons such as a mechanical button, a touch pad, a wheel, and the like formed in an arbitrary area such as a front part, a side part, a back part, etc. of the main body of the electronic device 200.

The microphone 270 is a component for receiving a user voice or other sound and converting the same into audio data. The processor 320 may use a user's voice input through the microphone unit 270 in a call process or convert the user's voice into audio data and store it in the memory 210.

The camera 280 is a component for capturing a still image or a moving image under the control of a user. The camera 280 may be implemented in plurality, such as a front camera, a rear camera, and a side camera. As described above, the camera 280 may be used as a means for acquiring an image of the user in an embodiment for tracking the gaze of the user.

When the camera 280 and the microphone 270 are provided, the processor 320 may perform a control operation according to a user voice input through the microphone 270 or a user motion recognized by the camera 280. That is, the electronic device 200 may operate in a motion control mode or a voice control mode. When operating in the motion control mode, the processor 320 activates the camera 280 to photograph the user, tracks the user's motion change, and performs a control operation corresponding thereto. When operating in the voice control mode, the processor 320 may operate in a voice recognition mode that analyzes a user voice input through the microphone 270 and performs a control operation according to the analyzed user voice.

In the electronic device 200 supporting the motion control mode or the voice control mode, a voice recognition technology or a motion recognition technology may be used in the above-described various embodiments. For example, when a user takes a motion as if selecting an object displayed on a home screen or pronounces a voice command corresponding to the object, it may be determined that the object is selected and a control operation matched to the object may be performed. .

As described above, the touch display 300 may be divided into a main area and a plurality of sub display areas. The touch display 300 may be implemented as various types of displays such as a liquid crystal display (LCD), an organic light emitting diodes (OLED) display, a plasma display panel (PDP), and the like. The touch display 300 may also include a driving circuit, a backlight unit, and the like, which may be implemented in the form of an a-si TFT, a low temperature poly silicon (LTPS) TFT, an organic TFT (OTFT), or the like. Meanwhile, the touch display 300 may be implemented as a touch screen in combination with a touch sensor included in the sensor 310.

In this case, the touch sensor may include at least one of a touch panel and a pen recognition panel. The touch panel may detect a user's finger gesture input and output a touch event value corresponding to the detected touch signal. The touch panel may be mounted under both the main sub region and the sub display area of the touch display 300 or may be mounted only under the sub display area of the touch display 300. The touch panel detects a user's finger gesture input may be a capacitive type or a pressure sensitive type. The electrostatic type is a method of calculating touch coordinates by sensing fine electricity generated by a user's human body. The pressure-sensitive type includes two electrode plates embedded in the touch panel and calculates touch coordinates by detecting that the upper and lower plates of the touched point are in contact with each other so that current flows.

The pen recognition panel detects a user's pen gesture input according to the operation of the user's touch pen (eg, a stylus pen or a digitizer pen), and outputs a pen proximity event value or a pen touch event value. can do. The pen recognition panel may be mounted under at least one of the main area and the plurality of auxiliary display areas of the touch display 300. The pen recognition panel may be implemented by, for example, an EMR method, and detect a touch or a proximity input according to a change in the intensity of the electromagnetic field due to the proximity or touch of the pen. In detail, the pen recognition panel includes an electromagnetic induction coil sensor (not shown) having a grid structure and an electronic signal processor (not shown) that sequentially provides an AC signal having a predetermined frequency to each loop coil of the electromagnetic induction coil sensor. Can be configured. If there is a pen incorporating a resonant circuit in the vicinity of the loop coil of the pen recognition panel, the magnetic field transmitted from the loop coil generates a current based on mutual electromagnetic induction in the resonant circuit in the pen. Based on this current, an induction magnetic field is generated from the coil constituting the resonant circuit in the pen, and the pen recognition panel detects the induction magnetic field in the loop coil in a signal receiving state, so that the pen's approach position or touch position is changed. Can be detected.

Meanwhile, the electronic device 200 according to an embodiment of the present disclosure has been described as being implemented as a flexible display including a main display area and a plurality of auxiliary display areas. However, this is only an example and may include a plurality of general displays. The main display area and the plurality of sub display areas can be configured using the?. When implemented as a general display, a plurality of displays may be connected to each other to form a curved display.

The sensor 310 detects various states and user interactions of the electronic device 200. In particular, the sensor 310 may detect a holding state in which the user is holding the electronic device 200. In detail, the electronic device 200 may be rotated or tilted in various directions. In this case, the sensor 310 may detect a tilt of the electronic device 100 held by the user based on the rotational movement or the gravity direction using at least one of various sensors such as a geomagnetic sensor, a gyro sensor, an acceleration sensor, and the like. Can be. In addition, the sensor 310 may detect the touch area and the grip pressure distribution of the auxiliary display area.

The processor 320 includes a RAM 321, a ROM 322, a CPU 323, a graphics processing unit (GPU) 324, and a bus 325. The RAM 321, the ROM 322, the CPU 323, and the Graphic Processing Unit (GPU) 324 may be connected to each other through the bus 325.

The CPU 323 accesses the memory 210 and performs booting using an operating system stored in the memory 210. In addition, various operations are performed using various programs, contents, data, etc. stored in the memory 210.

The ROM 322 stores a command set for system booting. When the turn-on command is input and the power is supplied, the CPU 323 copies the O / S stored in the memory 210 to the RAM 321 according to the command stored in the ROM 322, and executes the O / S to execute the system. Boot up When the booting is completed, the CPU 323 copies various programs stored in the memory 210 to the RAM 321 and executes the programs copied to the RAM 321 to perform various operations. When booting of the electronic device 200 is completed, the GPU 324 displays a UI screen in an activated area of the main area and the auxiliary display area. In detail, the GPU 324 may generate a screen including various objects such as an icon, an image, and a text by using a calculator (not shown) and a renderer (not shown). The calculator calculates attribute values such as coordinates, shapes, sizes, and colors for displaying the objects according to the layout of the screen. The renderer generates screens of various layouts including objects based on the attribute values calculated by the calculator. The screen generated by the renderer is provided to the touch display 300 and displayed on the main area and the sub display area, respectively.

In addition, although not shown in FIG. 3, according to an embodiment, various external input ports for connecting to a USB port through which a USB connector may be connected in the electronic device 200, or various external terminals such as a headset, a mouse, and a LAN may be used. Of course, it may further include a DMB chip, various sensors, etc. to receive and process a Digital Multimedia Broadcasting (DMB) signal.

Meanwhile, as described above, various programs may be stored in the memory 210. 4 is a diagram for describing a structure of software stored in the electronic device 200. According to FIG. 4, the memory 210 may store software including an OS 410, a kernel 420, middleware 430, an application 440, and the like.

An operating system (OS) 410 performs a function of controlling and managing overall operations of hardware. In other words, the OS 410 is a layer that performs basic functions such as hardware management, memory, and security.

The kernel 420 serves as a path for transmitting various signals including the touch signal sensed by the sensor 310 to the middleware 430.

The middleware 430 includes various software modules for controlling the operation of the electronic device 200. According to FIG. 4, the middleware 430 includes an X11 module 430-1, an APP manager 430-2, a connection manager 430-3, a security module 430-4, a system manager 430-5, The multimedia framework 430-6, the UI framework 430-7, the window manager 430-8, and the handwriting recognition module 430-9 are included.

The X11 module 430-1 is a module that receives various event signals from various hardware included in the electronic device 200. Here, the event may be variously set, such as an event in which a user gesture is detected, an event in which a system alarm occurs, an event in which a specific program is executed or terminated, and the like.

The APP manager 430-2 is a module that manages execution states of various applications 440 installed in the memory 210. When the application execution event is detected from the X11 module 430-1, the APP manager 430-2 calls and executes an application corresponding to the event.

The connection manager 430-3 is a module for supporting a wired or wireless network connection. The connection manager 430-3 may include various detailed modules such as a DNET module and a UPnP module.

The security module 430-4 is a module supporting certification, request permission, secure storage, and the like for hardware.

The system manager 430-5 monitors the state of each component in the electronic device 200, and provides the monitoring result to other modules. For example, when the battery level is low, or when an error occurs, when the communication connection state is lost, the system manager 430-5 sends the monitoring result to the main UI framework 430-7 or the subUI frame. Provided to the walk (430-9) can output a notification message or notification sound.

The multimedia framework 430-6 is a module for playing multimedia content stored in the electronic device 200 or provided from an external source. The multimedia framework 430-6 may include a player module, a camcorder module, a sound processing module, and the like. Accordingly, an operation of reproducing and reproducing a screen and sound by reproducing various multimedia contents may be performed.

The main UI framework 430-7 is a module for providing various UIs to be displayed on the main display area of the touch display 300, and the sub UI framework 430-9 is used to display various UIs to be displayed on the auxiliary display area. Module to provide. The main UI framework 430-7 and the sub UI framework 430-9 include an image compositor module constituting various objects, a coordinate synthesizer for calculating coordinates at which the object is to be displayed, and coordinates for calculating the configured object. And a 2D / 3D UI toolkit that provides a tool for constructing a UI in 2D or 3D form, and a rendering module for rendering.

The window manager 430-8 may detect a touch event or other input event using a user's body or a pen. When such an event is detected, the window manager 430-8 transmits an event signal to the main UI framework 430-7 or the sub UI framework 430-9 to perform an operation corresponding to the event.

In addition, when the user touches and drags the screen, a pitch module, a roll angle, a yaw angle, or the like is calculated based on a writing module for drawing a line according to the drag trajectory or a sensor value detected by the sensor 310. Various program modules may be stored, such as an angle calculation module.

The application module 440 includes applications 440-1 through 440-n to support various functions. For example, it may include a program module for providing various services such as a navigation program module, a game module, an e-book module, a calendar module, an alarm management module, and the like. These applications may be installed by default, or may be arbitrarily installed and used by a user during use. When an object is selected, the CPU 423 may execute an application corresponding to the selected object by using the application module 440.

The software structure shown in FIG. 4 is merely an example, and is not necessarily limited thereto. Therefore, some may be omitted, modified, or added as necessary. For example, the memory 210 includes a sensing module for analyzing signals sensed by various sensors, a messaging module such as a messenger program, a short message service (SMS) & multimedia message service (MMS) program, an e-mail program, and a phone information collector. (Call Info Aggregator) Various programs may be additionally provided, such as a program module, a VoIP module, a web browser module, and the like.

Meanwhile, as described above, the electronic device 200 may be implemented as various types of devices such as a mobile phone, a tablet PC, a laptop PC, a PDA, an MP3 player, an electronic picture frame device, a TV, a PC, a kiosk, and the like. Therefore, the configuration described with reference to FIGS. 3 and 4 may be variously modified according to the type of the electronic device 200.

Hereinafter, various embodiments of the present invention will be described with reference to the accompanying drawings.

When a situation in which text input is required is detected, the processor 320 may control the touch display 300 to provide a UI for text input in the plurality of auxiliary display areas. In detail, as illustrated in FIG. 5A, when a user touch touching the text input window 505 is input while the text input window 505 is provided in the main display area 510, the processor 320 may display FIG. 5B. As illustrated in FIG. 2, the touch display 300 may be controlled to provide a UI for text input in the plurality of auxiliary display regions 520-1 and 520-2.

In the above-described embodiment, the situation in which the character input is required is described as the situation in which the character input window 505 is displayed. However, this is only an example and may be determined as a situation in which the character input is required in other situations. For example, various situations, such as a situation for entering a phone number and a situation for entering a text message, may be determined as a situation in which text input is required.

Meanwhile, the UI for text input includes a plurality of UI elements as shown in FIG. 5B, and each UI element may correspond to at least one letter, symbol, or function. For example, the first UI element may correspond to ".", ",", "?", "!", And the second UI element may correspond to "a", "b", and "c". The third UI element may correspond to “d”, “e” and “f”, and the fourth UI element may correspond to “g”, “h” and “i”, and the fifth UI element. May correspond to “j”, “k” and “l”, and the sixth UI element may correspond to “m”, “n” and “o”, and the seventh UI element may correspond to “p”, ” q "," r ", and" s ". The eighth UI element may correspond to" t "," u ", and" v. "The ninth UI element may correspond to" w "and" x. " , "y", "z", and the tenth UI element may correspond to the "enter" function. On the other hand, the character array shown in Figure 5b is only one embodiment, other character array can be applied to the present invention.

In addition, the processor 320 may input a character according to a user touch input through the UI displayed on the plurality of auxiliary display areas 520-1 and 520-2.

In particular, when one character corresponds to one UI element, when a user touch is detected that swipes toward the main display area 510 after touching one of the plurality of UI elements, the processor 320 may detect the user's touch. A character corresponding to the touched UI element may be input.

However, when a plurality of characters correspond to one UI element, the processor 320 may input characters through touch input and swipe input.

In detail, as illustrated in FIG. 5B, while the UI for text input is displayed on the plurality of auxiliary display regions 520-1 and 520-2, the processor 320 may display the fourth UI through the sensor 310 as illustrated in FIG. 5C. A user's touch on an element can be detected.

After the user touches the fourth UI element once, as illustrated in FIG. 5D, when a user touch is detected in which the user touches the fourth UI element and then swipes toward the main display area 510, the processor 320 may display FIG. 5E. As shown in FIG. 2, "g" may be input to the text input window 505. In this case, when the user touches the fourth UI element once, the processor 320 may control the touch display 300 to provide an indicator indicating that "g" is selected for the fourth UI element.

However, after the user touches the fourth UI element twice, as illustrated in FIG. 5D, when a user touch is detected in which the user touches the fourth UI element and then swipes toward the main display area 510, the processor 320 may detect the user's touch. "h" may be input to the character input window 505. In this case, when the user touches the fourth UI element once, the processor 320 may control the touch display 300 to provide an indicator indicating that "h" is selected for the fourth UI element.

In addition, after the user touches the fourth UI element three times, as illustrated in FIG. 5D, when a user touch is detected in which the user touches the fourth UI element and then swipes toward the main display area 510, the processor 320 may detect a user touch. "i" may be input to the text input window 505. In this case, when the user touches the fourth UI element once, the processor 320 may control the touch display 300 to provide an indicator indicating that "i" is selected for the fourth UI element.

As described above, the user may input text with one hand through the touch input and the swipe input, and the text input may be performed without covering the main display area 510. It can also have an entertaining element that bounces the abacus through the swipe input.

In addition, the processor 320 may perform various character input functions according to a user touch input to the main display area while the UI for text input is displayed on the plurality of auxiliary display areas.

In an embodiment of the present invention, as shown in FIG. 6A, when a character is input in the main display area 610, a swipe input in a left direction after touching an area of the main display area 610 is performed. If 630 is detected, the processor 320 may delete the most recently input character. At this time, if a swipe input is detected more than a predetermined number of times, the processor 320 may delete all the input characters.

In another exemplary embodiment, as shown in FIG. 6B, when a character is input in the main display area 610, a swipe input 640 in a right direction after touching an area of the main display area 610 is performed. If detected, the processor 320 may perform a space function corresponding to a space bar.

Meanwhile, in the above-described embodiment, the delete function and the space bar function have been described, but this is only an example, and various functions such as a copy function and a cut function may be performed according to a user input input to the main display area. .

Meanwhile, in the above-described embodiment, the UI for text input is described as being English. However, this is only an example, and the UI for text input may be another language. For example, as illustrated in FIGS. 7A and 7B, a UI for inputting Korean may be provided.

In particular, FIG. 7A displays a UI for inputting Korean in a narrative manner. In this case, a UI element for inputting a consonant in Korean is provided with a UI element for inputting a consonant in the first auxiliary display area 720-1 and a UI for inputting a vowel in the second auxiliary display area 720-2. The element can be placed. In addition, in FIG. 7B, a UI for Korean input in a celestial manner is displayed.

In addition, a plurality of auxiliary display areas may display a UI for inserting various texts such as numbers and special symbols instead of characters.

In addition, when a user touch is sensed after touching one point of each of the plurality of auxiliary display regions and swiping in the same direction, the processor 320 may execute a preset application according to the user touch. In detail, as shown in FIG. 8A, while touching a point of each of the plurality of auxiliary display areas 820-1 and 820-2 while the clock screen is displayed on the main display area 810, the user swipes in the right direction. When the swipe input 830 is detected, the processor 320 may execute the camera application and control the touch display 810 to display a live view image as shown in FIG. 8B.

In this case, the processor 320 may execute another application according to the position of the sub display area where the swipe input is detected or the direction of the swipe input.

For example, when a user's touch to touch a point of each of the plurality of sub display areas 820-1 and 820-2 and then swipe in the left direction is detected while the clock screen is displayed on the main display area 810, the processor displays the processor. 320 may execute the telephone application.

As another example, while a clock screen is displayed on the main display area 810, one point of the first sub display area 820-1 among the sub display areas 820-1 and 820-2 is touched and then moved to the right direction. When a user's swipe is detected, the processor 320 may execute a schedule application.

As another example, while the watch screen is displayed on the main display area 810, one touch point of the second sub display area 820-2 among the sub display areas 820-1 and 820-2 is moved to the left direction. When a user's swipe is detected, the processor 320 may execute a memo application.

As described above, by executing a specific application through a swipe input sensed in the plurality of auxiliary display areas, the user can execute the application more quickly, as with a shortcut key.

Meanwhile, in the above-described embodiment, the type of the user touch for quick application execution is described as a swipe input. However, this is only an example and may be applied to another touch such as a long press input for touching the auxiliary display area for a predetermined time. Can be implemented with tangible user input.

In addition, the application executed according to the user touch may be set at the time of manufacture, but this is only an embodiment and may be set by the user.

In addition, the processor 320 detects the direction of the electronic device 200 through the sensor 310 while the camera application is running, and at least one of the main display area and the plurality of auxiliary display areas based on the detected direction of the electronic device. The touch display 300 may be controlled to display a live view image on a portion. In an embodiment of the present disclosure, as shown in FIG. 9A, the touch display 300 divides the main display area into two, and thus, the first main display area 910-1 and the second main display area 910-2. ), A first auxiliary display area 920-1 and a second auxiliary display area 920-2.

While the camera application is running, the processor 320 may detect the direction of the electronic device 200. In this case, the processor 320 may detect the direction of the electronic device 200 through the motion sensor, but this is only an example. The processor 320 captures a user's eye by using the camera 280 to capture the user's gaze. By detecting, the direction of the electronic device 200 may be detected.

The processor 320 may control the touch display 300 to display a live view image on at least a portion of the touch display 300 according to the orientation of the electronic device 200.

In detail, when the camera application is first executed, the processor 320 displays a live view image on the first main display area 910-1 and the second main display area 910-2, as shown in FIG. 9B. The touch display 300 may be controlled.

When the user who wears the electronic device 200 with his left hand rotates the electronic device 200 in a counterclockwise direction, as illustrated in FIG. 9C, the processor 320 may display the first main display area 910-1. ) And the touch display 300 to display the live view image on the first auxiliary display area 920-1. As another example, when a user wearing the electronic device 200 with his left hand rotates the electronic device 200 in a clockwise direction, the processor 320 may include the second main display area 910-2 and the second auxiliary display area. The touch display 300 may be controlled to display a live view image at 920-2. That is, the processor 320 may control the touch display 300 to detect a change in posture of the electronic device 200 and to display the live view image by moving another area of the touch display 300.

Meanwhile, in the above-described embodiment, the live view image is moved according to the direction of the electronic device 200. However, this is only an example, and various types of images (for example, a web page image and a message window screen) are described. Etc.) may move along the direction of the electronic device 200.

As described above, by moving and displaying the live view image according to the direction of the electronic device, the user can more conveniently check the live view image and take a picture.

In addition, when a plurality of cameras 280 are provided in the electronic device 200, the processor 320 may display a plurality of live view images captured by the plurality of cameras in the plurality of main display areas. Can be controlled. In detail, as illustrated in FIG. 10, the processor 320 displays a live view image captured by the first camera in the first main display area 1010-1, and the second main display area 1010-2. ), The touch display 300 may be controlled to display a live view image captured by the second camera. In this case, the first auxiliary display area 1020-1 may display a UI for capturing a live view image displayed on the first camera, and the second auxiliary display area 1020-2 may be displayed on the second camera. A UI for capturing a live view image may be displayed.

As described above, by dividing the main display area 1010 of the electronic device into a plurality of areas and capturing a plurality of live view images, it is possible to take a picture in more various environments.

11 is a flowchart illustrating a control method of the electronic device 100 according to an embodiment of the present disclosure. First, the electronic device 100 according to an embodiment of the present disclosure may include a touch display including a main display area and a plurality of auxiliary display areas positioned on side surfaces of the main display area.

First, the electronic device 100 provides a UI for text input on a plurality of auxiliary display areas of the touch display in operation S1110. In this case, the UI for text input includes a plurality of UI elements, and each of the plurality of UI elements may correspond to at least one letter, number, symbol, or special character.

In operation S1120, the electronic device 100 determines whether a touch interaction is input through the UI. In this case, the touch interaction may be a swipe interaction that touches one of the plurality of UI elements and swipes toward the main display area.

When a touch interaction is input through the UI (S1120-Y), the electronic device 100 inputs a character according to the touch interaction (S1130). That is, the electronic device 100 may provide the input text according to the touch interaction to the main display area 111 of the touch display 110.

According to various embodiments of the present disclosure as described above, a user may more conveniently input text using the auxiliary display area of the electronic device 100 without disturbing viewing of the main display area.

Meanwhile, the control method of the electronic device according to various embodiments of the present disclosure may be implemented as a program and provided to a display device or an input device. In particular, the program including the control method of the display apparatus may be stored and provided in a non-transitory computer readable medium.

The non-transitory readable medium refers to a medium that stores data semi-permanently and is readable by a device, not a medium storing data for a short time such as a register, a cache, a memory, and the like. Specifically, the various applications or programs described above may be stored and provided in a non-transitory readable medium such as a CD, a DVD, a hard disk, a Blu-ray disk, a USB, a memory card, a ROM, or the like.

In addition, although the preferred embodiment of the present invention has been shown and described above, the present invention is not limited to the specific embodiments described above, but the technical field to which the invention belongs without departing from the spirit of the invention claimed in the claims. Of course, various modifications can be made by those skilled in the art, and these modifications should not be individually understood from the technical spirit or the prospect of the present invention.

Claims

In an electronic device,

A touch display including a main display area and a plurality of auxiliary display areas positioned on side surfaces of the main display area; And

The touch display is controlled to provide a UI for text input to the plurality of auxiliary display areas of the touch display, and when a touch interaction with respect to the UI is input, the text input according to the touch interaction is displayed in the touch display. And a processor for controlling the touch display to provide a main display area.
The method of claim 1,

The touch display,

A first curved auxiliary display area which is integrally extended from the main display area and bent to the first side of the main display area and which is integrally extended from the main display area and bent to the second side of the main display area And a second curved auxiliary display area smaller than the main display area.
The method of claim 2,

The UI includes a plurality of UI elements,

Each of the plurality of UI elements corresponds to at least one character.
The method of claim 3,

The processor,

Providing a UI element corresponding to a collection among the plurality of UI elements to the first curved auxiliary display area;

And controlling the touch display to provide a UI element corresponding to a consonant of the plurality of UI elements to the second curved auxiliary display area.
The method of claim 3,

The processor,

If a user touch is detected after swiping one of the plurality of UI elements toward the main display area, the touch display is controlled to provide a character corresponding to the touched UI element to the main display area. An electronic device, characterized in that.
The method of claim 1,

The processor,

And when a user touch is swiped in a predetermined direction in the main display area, a character provided in the main display area is deleted.
The method of claim 2,

The processor,

And when a user touch for simultaneously swiping the first curved auxiliary display area and the second curved auxiliary display area is detected, executing a specific application according to the user touch.
The method of claim 7, wherein

The processor,

And when a user touch for simultaneously swiping the first curved auxiliary display area and the second curved auxiliary display area in a left and right direction is detected, executing the camera application according to the user touch.
The method of claim 8,

The processor,

The touch display is configured to detect a direction of the electronic device while the camera application is running and to display a live view image on at least a portion of the main display area and the plurality of auxiliary display areas based on the detected direction of the electronic device. An electronic device, characterized in that for controlling.
A control method of an electronic device having a touch display including a main display area and a plurality of auxiliary display areas positioned on a side of the main display area.

Providing a UI for text input in the plurality of auxiliary display regions of the touch display; And

And when a touch interaction with respect to the UI is input, providing a character input according to the touch interaction to a main display area of the touch display.
The method of claim 10,

The touch display,

A first curved auxiliary display area which is integrally extended from the main display area and bent to the first side of the main display area and which is integrally extended from the main display area and bent to the second side of the main display area And a second curved auxiliary display area smaller than the main display area.
The method of claim 11,

The UI includes a plurality of UI elements,

And each of the plurality of UI elements corresponds to at least one character.
The method of claim 12,

Providing the UI,

Providing a UI element corresponding to a collection among the plurality of UI elements to the first curved auxiliary display area;

And a UI element corresponding to a consonant among the plurality of UI elements to the second curved auxiliary display area.
The method of claim 12,

The providing to the main display area may include

And after a user touches one of the plurality of UI elements and detects a user touch that swipes toward the main display area, providing a character corresponding to the touched UI element to the main display area. Way.
The method of claim 10,

And deleting a character provided in the main display area when a user touch is swiped in a predetermined direction in the main display area.