KR20100041150A - A method for controlling user interface using multitouch - Google Patents

Abstract

PURPOSE: A user interface control method using multi touch is provided to enable a user to easily input or manipulate data using multi touch, thereby solving inconvenience problem of data input. CONSTITUTION: A multi touch sensing unit(14) senses the multi touch on a display unit(18). A memory(16) stores data included in a selection area which is set by the multi touch and file information. The display unit displays a graphic object related to the stored information. A control unit(12) sets the selection area by the sensed multi touch. In order that components perform an operation corresponding to the user input of the graphic object, the control unit performs control operation.

Description

A method for controlling user interface using multitouch}

The present invention relates to a user interface control method and a display device.

More specifically, the present invention relates to a method in which a user can easily input or manipulate data using a multi-touch in a device equipped with a display, and an apparatus in which the method is implemented.

Today, electronic systems such as TVs, cell phones, MP3 players, PMPs, notebooks and PDAs are equipped with various input / output devices.

Various input / output devices are provided for the user to conveniently control the above systems.

Among electronic systems, portable devices such as mobile phones, MP3 players, PMPs, notebook computers, PDAs, and the like are small in size, and thus, there are limitations in mounting input / output devices.

Recently, such portable devices have been equipped with touch screens, touch pads, and navigation pads as part of efforts to improve user interfaces. By adopting a touch screen, it replaces mouse and keyboard input in a general PC, and allows a user to input data and input commands in various ways even in a small space.

Despite these various user interfaces, the input and output of devices are subject to a lot of limitations, and the user also feels uncomfortable.

For example, in the case of a mobile phone, various additional functions are added in addition to a call function. For example, an MP3 player, a camera function, and an internet connection function are added.

Many of these functions have been added, but due to the limited input means, the user may not be able to easily control these functions particularly easily in a mobile terminal such as a mobile phone. There is a need for an improved user interface control method.

An object of the present invention is to provide a method that can eliminate the inconvenience of data input by multi-touch.

An object of the present invention is to provide a method for easily manipulating data such as texts and files by multi-touch.

An object of the present invention is to provide a user interface that is convenient for a user in a mobile terminal such as a mobile phone, in which the user's input means is limited.

A user interface control method according to an embodiment of the present invention comprises the steps of: setting a selection area by detecting a multi-touch on a display; And displaying a graphic object associated with the selection area.

According to one or more exemplary embodiments, a method of controlling a user interface includes: setting a selection area by detecting a multi-touch; Displaying a graphic object associated with the selection area; Detecting user input for the graphical object; And performing an operation corresponding to the user input.

A display device according to an embodiment of the present invention, a multi-touch sensing unit for detecting a multi-touch on the display; A memory for storing information on data or files included in the selection area set by the multi-touch; A display for displaying a graphical object associated with the stored information; And a controller configured to set the selection area by the detected multi-touch, to cause the display to display the graphic object, and to control the components to perform an operation corresponding to a user input to the graphic object. Include.

According to the present invention, it is possible to provide a convenient and improved data input method.

According to the present invention, it is possible to provide a method for easily manipulating data such as texts and files by multi-touch.

The present invention can provide a convenient and improved user interface especially in a mobile terminal such as a mobile phone in which a user's input means is limited.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 illustrates a configuration of a display apparatus according to an embodiment of the present invention.

The display device according to the present invention may include any electronic device equipped with a mobile terminal such as a mobile phone, an MP3 player, a PMP, a notebook computer, a PDA, and a non-mobile terminal such as a monitor or a TV.

As shown in FIG. 1, the display apparatus 10 according to the present invention includes a multi-touch sensing unit 14 that detects a multi-touch from a user, and information on data or files included in an area set by the multi-touch. A memory 16 for storing the information, a display 18 for displaying a graphic object related to the stored information, and an area set by the detected multi-touch, and causing the display 18 to display the graphic object, It may be configured to include a control unit 12 for controlling the components to perform an operation corresponding to the user input to the graphic object.

In the present invention, the multi-touch means that the user touches the touch screen with a plurality of touch means, unlike the user's one touch means, for example, a finger or a stylus pen, as is widely known.

The present invention uses a multi-touch technique to detect the touch of at least two fingers or stylus pens. Many techniques for detecting multi-touch are known, and for example, a technique for detecting multi-touch on a capacitive touch screen as described in US Patent Publication No. 20070257890 can be used. In addition, various techniques for detecting multi-touch on the display may be applied to the present invention, such as a method using an ultrasonic sensor, a method using an infrared sensor, or a method using an image sensor.

As the multi-touch sensing unit 14, any means capable of detecting the multi-touch as described above may be used. The multi-touch sensing unit 14 detects a plurality of touch means for touching the touch screen, that is, whether a finger or a stylus pen is touched and its touch position. As will be described later, when two points are touched, the inside of the rectangle formed by the two points can be set as the selection area. In addition, the multi-touch sensing unit 14 may detect movement of a plurality of input means, and may also detect drag and drop gestures.

The memory 16 stores information on data or files included in the area when the selection area is set by multi-touch. In addition, the memory 16 may store various data or software necessary for the operation of the display device.

As the display 18, various display devices such as an LCD may be used. In addition to outputting various information to the user, the display 18 displays any graphic object related to the information stored in the memory or related to the selection area set by multi-touch. The graphic object may include an icon, characters that may represent a characteristic of stored information, a file name, and the like.

The controller 12 controls the overall operation of the display device. In addition, the control unit 12 stores the information related to the selection area set by the two touches in the memory 16, causes the display 18 to display the graphic object, and the user's predetermined reference to the graphic object. When an input is received, an operation according to a user input is performed. User input and the operations performed accordingly are described in more detail below.

2 illustrates a method of selecting a region using multi-touch according to an embodiment of the present invention.

2 is a mobile phone 20 called a full touch screen phone as an embodiment of the display device of the present invention. The full touch screen phone does not have a key button for inputting a phone number. Instead, the touch screen 21 is mounted on a display having a relatively large area of the screen among mobile terminals, and the phone number is input to the touch screen when the power is turned on. It will provide a button.

In the mobile phone 20 of FIG. 2, the touch screen 21 may be configured by combining a display and a touch screen panel. The display corresponds to the display 18 of FIG. 1, and the touch screen panel includes a multi-touch sensor unit ( 14).

As shown in FIG. 2, when a touch is detected at any two points on the touch screen 21, as shown in FIG. 2, a rectangle that diagonally crosses the line connecting the two points 22a and 22b is shown. The area can be set as the selection area 24.

In another embodiment, a circle whose diameter is a line connecting two points can be set as the selection area 24.

In addition, various types of figures that may be defined by the multi-touch may be set as the selection area 24.

In another embodiment, it may be determined that the multi-touch is performed when the touch means is removed from the touch screen 21, not when the touch means touches the touch screen 21. In this case, when the user touches the touch means on the touch screen 21, there is no change, and it is determined that the multi-touch is made at the moment the user releases the touch means.

Therefore, while the user touches the finger on the touch screen 21, the user moves the finger, that is, drags to adjust the size or position of the selection area 24. After setting the desired area 24 correctly, and then releasing the finger, it can be determined that there was a multi-touch at the time of releasing.

3A and 3B illustrate a method of controlling a user interface using area setting according to an embodiment of the present invention. According to an embodiment of the present invention, if a multi-touch is detected, a selection area may be set, and a user may easily access data, files, etc. included in the set area.

3A is a screen of a text message reading mode state.

As shown in FIG. 3A, it is assumed that a text message includes a sentence "Hello ... That's OK". The text message may be a message currently being written, may be a received message, or may be a stored message.

When a touch is detected at two points 25a and 25b, the rectangle is thereby set as the selection area. In order to allow the user to recognize the set rectangle, the boundary line of the set area may be displayed or the inside of the set area may be inverted and highlighted. The set rectangle contains the text "That's OK".

When the user releases his or her finger, information related to the text included in the selection area, for example, the text itself, is stored in a memory (not shown) of the mobile phone 20, and the graphic object 27 is displayed on the screen as shown in FIG. 3B. The graphic object 27 may include an icon, a start word of the text, a stored time, and the like so that the user can easily recognize what text is displayed by the graphic object 27. 3b shows the start word of the text.

The graphic object 27 is floating, ie movable, on the touch screen 21. The user may drag the graphic object 27 to another position on the screen. In addition, the graphic object 27 is displayed in front of the screen in preference to other displayed information, and is displayed as it is even if the operation mode of the mobile phone 20 is changed.

According to an embodiment, the graphic object 27 may be displayed as translucent. The size of the graphic object 27 can be set by the user, and the size of the graphic object 27 can be changed by touching and dragging the outline of the graphic object 27, such as changing the size of a window of a general PC by dragging and dropping a mouse. Can be.

The controller (not shown) of the mobile phone 20 may separately store information for linking the graphic object 27 and the text stored in the memory in the memory to access the stored text in response to a user input.

4A and 4B illustrate a method of accessing and using copied data, according to one embodiment of the invention.

4A is a text message creation screen of the mobile phone 20. As the operation mode is changed in the state of FIG. 3B, the graphic object 27 is displayed as it is despite the operation mode change. At the top of the screen, there is a message window 29 displaying the input characters, and at the bottom, a key button 28 for inputting numbers and characters may be displayed.

When the user touches the graphic object 27 in the state of FIG. 4A, information corresponding to the graphic object 27 is input at the position where the current cursor is located or the position at which the current character is to be input as shown in FIG. 4B.

As shown in FIG. 4B, the graphic object 27 that has been used once, i.e. the user has touched once, may be made to disappear from the display or may remain displayed. Also, in another embodiment, once the graphic object 27 is used once, the user touches a null input area of the display, for example, an edge portion of the screen, so that the graphic object 27 disappears. In another embodiment, the graphic object 27 touched once may be displayed on the bottom or side of the screen as a work display window of a window of a PC.

According to an exemplary embodiment, when the user drags and drops the graphic object 27 to a position to input text, information related to the graphic object 27 may be input to the dropped position.

5A and 5B illustrate a method of deleting data copied by area setting, according to an embodiment of the present invention.

As shown in FIG. 5A, when the graphic object 27 is displayed, when the user drags the graphic object 27 to the outside of the touch screen 21, the graphic object 27 may be deleted.

As shown in FIG. 5B, a message may be displayed asking whether to delete the graphic object 27 before deleting. If the user touches 'Yes', information related to the graphic object 27 stored in a memory (not shown) or the information thereof may be displayed. You can delete the information to access the.

According to the exemplary embodiment, the trash icon is displayed on the screen, not outside the touch screen 21, and the graphic object 27 is deleted when the user drags and drops the graphic object 27 onto the trash icon. Can be configured.

As described above, by using the drag and drop gesture of the user, the user interface can be provided more intuitive and easy to understand.

6A to 6C illustrate a method of controlling a user interface using multi-touch according to an embodiment of the present invention.

6A shows a screen of a schedule management mode of the mobile phone 20. The calendar 30 is displayed in the middle of the screen, and the schedule 31 of the date is displayed at the bottom.

When the user sets the region to include the text "Birthday Party" of the schedule 31, the graphic object 32 related to the schedule information is displayed on the display as shown in FIG. 6B. As in the above-described embodiment, the graphic object 32 is a floating state on the screen and can be moved in position, and is displayed as it is even if the operation mode is changed. It is also displayed in front of the screen than other displayed objects. The description of the other graphic object 32 is also the same as in the above-described embodiment.

6C illustrates a state in which the operation mode is changed to the text message creation mode. The graphic object 32 is displayed as it is. When the user touches the graphic object 32, information corresponding to the graphic object 32 may be input at a position of a current cursor or a position at which a character is to be input.

5A and 5B, when the user drags and drops the graphic object 32 at a position where text is to be input, information corresponding to the graphic object 32 is input at the dropped position. It may be.

In the above-described examples, an example of copying a part in a text message or schedule management mode and using it when composing another text message has been described. However, in the present invention, a user inputs text into a display device such as a schedule, a memo, or a file name edit. Applicable in all cases where

7A and 7B illustrate a method of selecting a plurality of files by setting a selection area according to an embodiment of the present invention.

7A shows a preview mode of image files such as photographs. Previews of multiple image files are displayed. When the user touches two points 33a and 33b, the rectangular area is thereby set as the selection area. At this time, the image files included in the selection area, access information of img4, img5, img6, img7, that is, location information where the file is stored, or address information are stored in the memory. The controller (not shown) may access the file corresponding to the graphic object 32 through the location information where the file is stored. According to an embodiment, the selected file itself may be configured to be stored in a memory.

As described above, in addition to the rectangle, various areas that can be defined by the multi-touch can be set as the selection area.

As illustrated in FIG. 7B, a graphic object 34 representing information related to a file included in a rectangular area may be displayed on the screen. The graphic object 34 may include the file name of some of the files included in the selection area, or may include any information that allows other users to recognize which file the graphic object 34 represents.

In the state of FIG. 7B, even when the user moves to another folder, the graphic object 34 may be displayed as it is, and when the user touches or drags and drops the graphic object 34, the graphic object 34 may be copied to the corresponding folder. By the above method, a user interface for more convenient file management can be provided to the user.

In one embodiment, the same method can be applied to moving a file attachment or other file. For example, when attaching a file to a text message or an email for wireless Internet, the file can be conveniently inserted by touching or dragging and dropping the graphic object 34 displayed on the screen.

In the above embodiment, only the case of moving a file to another folder or inserting a file has been described. However, the above embodiment may be applied to all cases where an operation of selecting other files is required.

8A and 8B illustrate a method of selecting a file from a file list by area setting according to an embodiment of the present invention.

8A shows a screen on which the MP3 file play list is displayed on the mobile phone 20. When the user touches two points 35a and 35b, a rectangular area is thereby set. The location or memory address information where files (xxx.mp3, yyy.mp3) included in the set area are stored is stored in the memory. The controller may access the files through location information of the files included in the selected area.

As shown in FIG. 8B, a graphic object 37 related to a file included in a selected area is displayed on the screen. The graphic object 37 may include the file name of some of the files included in the selection area, or may include any information that allows other users to recognize which file the graphic object 37 represents.

As in the above-described embodiment, the user may touch or drag and drop the graphic object 37 to copy the files displayed by the graphic object 37 to another location or attach the file.

As in the above-described embodiment, when attaching a file to a text message or an email for wireless Internet, the file can be conveniently inserted by touching or dragging and dropping the graphic object 37 displayed on the screen.

In the above embodiment, only the case of moving a file to another folder or inserting a file has been described. However, the above embodiment may be applied to all cases where an operation of selecting other files is required.

1 illustrates a configuration of a display apparatus according to an embodiment of the present invention.

2 illustrates a method of selecting a region using multi-touch according to an embodiment of the present invention.

3A and 3B illustrate a method of controlling a user interface using area setting according to an embodiment of the present invention.

4A and 4B illustrate a method of accessing and using copied data, according to one embodiment of the invention.

5A and 5B illustrate a method of deleting data copied by area setting, according to an embodiment of the present invention.

6A to 6C illustrate a method of controlling a user interface using multi-touch according to an embodiment of the present invention.

7A and 7B illustrate a method of selecting a plurality of files by setting an area according to an embodiment of the present invention.

8A and 8B illustrate a method of selecting a file from a file list by area setting according to an embodiment of the present invention.

Description of the main parts of the drawing

10: display device 12: control unit

14: multi-touch sensing unit 16: memory

18: display

Claims (24)

Detecting a multi-touch on the display and setting a selection area; And And displaying a graphic object associated with the selection area. The method of claim 1, And the graphic object represents information about data or a file included in the selection area. The method of claim 1, And the graphical object is floating on the display. The method of claim 1, And the graphical object is semi-transparently displayed on the display. The method of claim 1, Detecting a gesture of dragging the graphic object out of the display; And And deleting the graphical object from the display. The method of claim 1, The multi-touch is by two touch means, and the selection area is formed by a rectangle with a straight line connecting the touch points of the two touch means diagonally. The method of claim 1, And the multi-touch is by two touch means, and the selection area is formed by a circle having a diameter of a straight line connecting the touch points of the two touch means. The method of claim 1, And the multi-touch is detected by a position when the two touch means are separated from the display. Detecting a multi-touch and setting a selection area; Displaying a graphic object associated with the selection area; Detecting user input for the graphical object; And Performing an operation corresponding to the user input; User interface control method comprising a. 10. The method of claim 9, Performing an operation corresponding to the user input, And inserting data or a file corresponding to information stored in the graphic object at a current cursor position. 10. The method of claim 9, And the user input is a gesture of dragging and dropping the graphic object. 10. The method of claim 9, Performing an operation corresponding to the user input, And inserting data or a file corresponding to the information stored in the graphic object in the drop position. 10. The method of claim 9, And the graphic object represents information about data or a file included in the selection area. 10. The method of claim 9, And the graphical object is floating on the display. 10. The method of claim 9, And the graphical object is semi-transparently displayed on the display. 10. The method of claim 9, Detecting a gesture of dragging the graphical object out of the display; And And deleting the graphical object from the display. 10. The method of claim 9, And the multi-touch is detected by a position when the two touch means are separated from the display. 10. The method of claim 9, The multi-touch is by two touch means, and the selection area is formed by a rectangle with a straight line connecting the touch points of the two touch means diagonally. 10. The method of claim 9, And the multi-touch is by two touch means, and the selection area is formed by a circle having a diameter of a straight line connecting the touch points of the two touch means. A multi-touch sensing unit detecting a multi-touch on the display; A memory for storing information on data or files included in the selection area set by the multi-touch; A display for displaying a graphical object associated with the stored information; And A controller configured to set the selection area by the detected multi-touch, cause the display to display the graphic object, and control the components to perform an operation corresponding to a user input to the graphic object; Display device comprising a. The method of claim 20, And the user input is a touch on the graphic object. The method of claim 20, The operation corresponding to the user input is an operation of inserting data or a file corresponding to information stored in the graphic object at a position of a current cursor. The method of claim 20, And the user input is a gesture for dragging and dropping the graphic object. The method of claim 20, The operation corresponding to the user input is an operation of inserting data or a file corresponding to information stored in the graphic object in the drop position.

Images