KR20080011973A - Portable digital device having graphic user interface system which uses two cursors and two pointing devices with switch function - Google Patents

Abstract

A portable digital device having an interface system, which includes two cursors and two optical pointing devices having a switching function, is provided to realize convenience like a computer in a small electronic device by maximizing character processing ability through two optical pointing devices and two cursors. Two optical pointing devices(3a,3b) enable a user to estimate movement of the cursor displayed in a screen without viewing a screen by checking a moving distance of a finger on the pointing device based on a reference point recognizable to a finger. The reference point is a small projection or groove. Each optical pointing device operates the pointer without depending on each other by forming two pointers in a virtual keyboard window for inputting characters. A virtual keyboard is divided into two sections and two pointers respectively manage one divided section without passing a boundary between two sections.

Description

Portable Digital Device having Graphic User Interface System which uses two cursors and two pointing devices with switch function}

Figure 1 shows two types of Querty Phone, Figure 1a is a model LG9200 Qwerty Phone produced and sold by LG, the keyboard is provided in a slide form, Figure 1b is a Blackberry 8700 model keyboard is provided on the front of the phone.

2 shows a mobile phone using two optical pointing devices 201L and 201R provided by the present invention. Each optical pointing device has four function buttons 202 to 205, which perform different functions when the mobile phone is used, and the arrangement of these function buttons fully performs the function of the pointing device with both hands or with one hand. Do it.

3 shows a schematic structure of a portable digital electronic device incorporating an optical pointing device having a switch function. 3a and 3b are cross-sectional views of the optical pointing device described in Korean Patent No. 10-0495686, and 3c is a 3d The cross-sectional view cut along AB is shown schematically. The principle of the optical pointing device is to flip the optical mouse and move the hand on the objective surface 320 to move the pointer. Light passes through the condenser lens 370 from a light source such as a photodiode 330 and irradiates the objective surface 320. The reflected and reflected light is transmitted to the sensor 340 to convert the movement of the hand into an electrical signal. And when the objective switch 320 is pressed so that the click switch 350 is mounted on the substrate 360 to serve as a switch function of the pointing device, the objective surface is included in the fixed state of the optical pointing device module 310. The upper module is moved downward to operate the click switch, which is designed to facilitate the execution command by allowing the pointing device to simultaneously perform the function of the function button. Nevertheless, since the function of the function button is additionally required in the UI (user interface), the function buttons may be additionally arranged on the top, bottom, left, and right sides of the optical pointing device, and 380 and 390 indicate this in 3d. In FIG. 3C, 380 shows an off button and 390 shows an on button.

Figure 4 shows an example applied to the actual mobile phone Figure 4c is the current model of Samsung's SPH-B3100 using a double hinge to enable the horizontal and vertical view of the screen. Similarly, the pointing device of the present invention (Fig. 2) can be applied to both the case of the landscape view is advantageous, such as the e-mail or multimedia player function and the portrait view case for the phone function only, and in a small space compared to the conventional QWERTY phone shown in Fig. Not only does it have a built-in keyboard, but it can also provide a graphical user interface (GUI).

Figure 5 is a state of using the mobile phone provided in the vertical view of the present invention can be used with one hand can be easily dialed even when using the phone and further the structure of the bar-type to enable the handset function to provide.

Figure 6 is a state of the mobile phone provided by the present invention when using the landscape view in this case can use both hands to enable smooth operation of the character input and GUI system.

FIG. 7 shows that a phone call can be made identical to a conventional mobile phone by using a GUI system in a portrait view mode. FIG. 7A is an initial screen displayed when the mobile phone is turned on, and the pointer (cursor) is positioned on the telephone call program icon and then double-clicked (double-tap the optical pointing device) to open the virtual keypad shown in FIG. 7B, and then 011-813. To dial -9715, move the cursor to the desired number from '0' to '5' (Fig. 7c) and press it in turn (press the optical pointing device) to enter the number on the screen. Move the cursor to the 'call' button on the keypad (press the optical pointing device) to make a call. To end the phone call, move the pointer over the 'end' button on the virtual keypad and press the pointing device. Unlike the existing mobile phone, you do not need to delete the wrong number even if you enter the wrong number, just select the wrong number and press the 'Cancel' button to delete and then enter a new number. This input method is the same as the method of modifying a character using a mouse in a computer. It is an advantage of the mobile phone provided by the present invention that the implementation of a function of the existing mobile phone is also possible through a program. Therefore, even if you are familiar with the call method using the keypad of the existing mobile phone may provide an interface that can be used as it is. For example, even if you enter only the last number 9715 (by moving the pointer over '5' on the virtual keypad to enter the final number '5', press and hold the pointing device). 9715) appears on the screen, press the 'call' button. Also, if you press and hold '1' for a long time like the shortcut function of the existing mobile phone (when you hold the pointer over '1' on the virtual keypad and press and hold the optical pointing device for a long time), you can connect to the phone number already entered.

8 illustrates the use of an e-mail program as in the GUI system of an existing computer in a landscape view mode in a mobile phone having a pointing device provided by the present invention. 8a shows the main screen in landscape view when the mobile phone is turned on, and when the pointer (cursor) is placed on the email program icon and double-clicked (the optical pointing device is pressed twice quickly), the email program as shown in FIG. 8b is displayed. Open and place the pointer again in the transmission inbox and double-click to display the list of transmission letters as shown in Figure 8c. Clicking on one of the e-mail lists opens the contents of the e-mail so that a new entry or modification can be made as shown in FIG. 8D. In this case, place the pointer where you want to enter the text to activate the text input mode, and then double-click to display the virtual keyboard at the bottom of the screen and place the part you want to enter text directly on the virtual keyboard. (FIG. 8E). And if you want to enter the body after the title input, as shown in Figure 8f If you place the pointer on the body and double-click the virtual keyboard automatically appears at the bottom of the screen and the text is located directly above the virtual keyboard, easy to check the characters when typing Done. When the text input is completed, move the pointer to 'Quit' and click (Fig. 8h) to exit the text input mode. At this time, if you press the upper button (903R) of the right pointing device, the extended menu will appear on the screen. Move to the 'Send' menu and click on it to send an email. Finally, click on the 'Exit' menu shown in FIG. 8i to return to the main screen.

FIG. 9 illustrates the virtual keyboard. FIG. 9A shows the virtual keyboard displayed on the screen when the text input mode is started. The left and right pointers are automatically positioned at 'f' and 'j'. At this time, two pointers (cursors) appear on the left and right sides of the virtual keyboard, and each pointer does not cross the center line of the keyboard and operates only in the left region 901 and the right region 902 of the keyboard. It is designed to double the efficiency of text input by the left thumb and the right thumb in the same way as the keyboard. In other words, the two pointers do not intersect with each other, and no matter how the optical pointing device is operated, the pointer is fixed to the left and right sides so that the movement of the two fingers (thumbs) is more freely (especially for the fingers and the right hand who are not mainly used). The resulting text input will result in the same input efficiency as the QWERTY keyboard. In addition, the functions of 'Shift', 'Space bar' and 'Enter' frequently used in the character input mode as shown in FIG. 8g are handled by the function buttons around the optical pointing device so that the characters can be easily operated in a fixed position. It shows an increase in the efficiency of the input. In fact, pressing the 'shift' function button causes the virtual keyboard's keys to change to uppercase mode (Fig. 9b). Further, when using these function buttons, the finger is released from the optical pointing device, and then the function button is pressed again, and then the optical pointing device is pressed again. When you come back, you won't be able to find the right place for your thumb. To solve this problem, if you make a reference point (903) by digging a protrusion or groove so that it does not interfere with the movement of the thumb on the optical pointing device, press the function button and the thumb will be located again at this reference point. The device (reference points-903L, 903R) and the virtual keyboards ('f', 'j') have reference points in absolute coordinates. Therefore, in the character input process, the position of the thumb is always detected from the reference point 903 in the optical pointing device so that it is easy to grasp the movement of the thumb to find a key to be input next. Therefore, the character input method using the touch paddle according to the present invention also enables mechanical character input by finger position grasping like a keyboard.

10 shows an example of actually inputting a character using reference points 903L and 903R on the optical pointing device. In the text input process, "... I am fine." To "... I am" are entered, followed by a space and then 'fine'. Fig. 10A is a state of inputting '... am' and Fig. 10B is a situation in which the space function button is pressed to enter a space. Then, the virtual keyboard automatically moves the pointer to 'j' (Fig. 10C). Since the thumb on the left side has not moved, the left pointer remains at its original position 'a', and if only the right hand thumb is pressed back to the reference point 903R after pressing the space function button, the position of the pointer and left and right thumb is always moved in association. Each key of the keyboard corresponds to a certain coordinate on the pointing device, and has the same effect as using an actual keyboard.

FIG. 11 shows that a mobile phone having two optical pointing devices and a plurality of function buttons provided by the present invention can be easily operated on a GUI system such as a mouse. FIG. 11A illustrates a document selection using a right pointing device and a left command execution button (a function button located at the bottom of the left pointing device) during the process of deleting a document in the trash. FIG. 11B is a right pointing while pressing a command execution button. The process of moving a document to the trash using a device. In the case of one pointer and two pointing devices, since the pointing devices operate independently, there is an advantage that the hand can be easily used as it is, thus ensuring convenience as a mouse.

FIG. 12 illustrates a mobile phone having a double sliding structure in which two optical pointing devices enable both horizontal and vertical viewing modes. Comparing the cell phone of this structure with the cell phone shown in FIG. 1A, the cell phone provided in the present invention provides a space that can expand the screen as much as the keypad length of the body of the cell phone of FIG. If the size of the screen is the same, the size of the mobile phone can be as small as the keypad length of the body of the mobile phone of Figure 1a. Therefore, it is much more efficient in space than a QWERTY phone using a conventional keypad and more various functions are possible in terms of GUI implementation.

FIG. 13 lists electronic dictionaries (FIGS. 13B and 13C) having two optical pointing devices and electronic dictionaries (FIG. 13A) having existing keypad input devices. In the case of an electronic dictionary having an optical pointing device provided by the present invention, FIG. Since it is operated on the basis of the system, each built-in dictionary can be used in the same way as the application program of the computer. When holding and using the electronic dictionary with both hands, as shown in FIG. 5 and the optical pointing device with only one hand as shown in FIG. The electronic dictionary provided by the present invention enables the use of multiple dictionaries at the same time as shown in FIG. In other words, if you click the radio button at the bottom of the screen that says "Work ...", you will be able to use the Japan-Korean dictionary in the same way as the English-Korean dictionary shown on the screen.

14 shows an arrangement of functional buttons having a rectangular structure unlike those shown in FIGS. 2 to 13 in a mobile phone and an electronic dictionary equipped with two optical pointing devices provided by the present invention. As such, depending on the function and structure of the device, the position and arrangement of the function buttons can be changed, and the number of function buttons can be changed in various ways.

FIG. 15 shows a structure for making text input using the virtual keyboard easier by making two or more reference points in the optical pointing device and associating these reference points with two or more keys of the virtual keyboard. The left and right optical pointing devices each have four reference points (2101-L1, L2, L3, L4, R1, R2, R3, R4) so that these reference points can be defined as s, e, f, c, j, i, l, When it is associated with m, it is a principle that allows the user to know the characters inputted from the relative position from the reference point felt by the finger without looking at the screen. For example, when the virtual keyboard starts, the pointer is automatically positioned at 'f' and 'j' of the virtual keyboard, and the finger on the optical pointing device is positioned at the reference points 1501-L1 and R1 as shown in FIG. 15B and the pointer is moved. When you start to move, the finger on the left optical pointing device moves from 1501-L1 to L3, so the pointer moves from 'j' to 's' on the screen. If you press the optical pointing device in that state, 's' is input. In other words, perception of relative position using the reference point makes it possible to determine the direction of the pointer by constantly looking at the screen because it can already determine in which direction and how much to move the finger, even when using the keyboard, even with the optical pointing device. You don't have to. As a result, even in the input method using the pointer, the absolute position can be set like the keyboard, and the virtual keyboard has the same convenience as the actual keyboard. The only difference is that the actual keyboard uses all five fingers, while the virtual keyboard using the optical pointing device uses only one finger.

FIG. 16 illustrates a method of using a reference point of the optical pointing device when inputting letters or numbers in the portrait view mode. FIG. 16A illustrates an arrangement of reference points when the mobile phone is opened in the portrait view mode in the mobile phone structure using the double hinge shown in FIG. 4. That is, the reference point 1501-L1 is located above and 1501-R1 is located below. In portrait view mode, instead of the virtual keyboard, the keypad of the existing mobile phone is placed on the screen (virtual keypad) to input letters or numbers in the same way as a mobile phone using a normal keypad. In this case, FIG. 16B illustrates the relationship between the virtual keypad and the reference point. If the virtual keypad appears on the screen at the start of the portrait mode and is automatically programmed to position the pointer over '5' as shown in Figure 16A, the finger will span four reference points as shown in the right circle of Figure 16B. Starting movement from the position provides the same ease as using the keypad of an existing mobile phone as shown in FIG. 15. 16C and 16D show the movement of a finger to input '8' and '0'. That is, from the position where the finger contains four reference points (pointer is above '5') to the position containing three reference points (pointer is above '8') and finally the reference point corresponding to 1501-R3 (the pointer is It shows moving to '0'). Therefore, it is shown that the input of letters and numbers can be easily developed in both the horizontal mode and the vertical mode using a plurality of reference points.

The computer was converted into a graphical user interface (GUI) system since the advent of the mouse as a device that allows the computer monitor to move the pointer that specifies the content and location of the executable command. Since then, as computers have become smaller and more portable, touch pads and pointing sticks have been developed and used as built-in pointing devices that can replace the mouse for added convenience. However, the difference between these devices and the mouse is that, unlike the mouse moving the mouse itself to set its position, these devices have a fixed position and instead operate the buttons that execute commands in the process of setting the pointer position by moving a finger. Additional movement is required, which makes it less convenient to use than a mouse. For this reason, even though these pointer input devices exist in the main body of the notebook, a separate mouse is connected and used. In particular, compared to these built-in pointing devices, the convenience of the mouse is that the operation required for positioning the pointer and the action of the command button for the execution command is performed by pressing the button while the finger is stopped after the movement of the mouse. It's about to end. On the other hand, the touchpad or pointing stick needs to additionally move the position of the finger in order to execute the execution command after setting the position of the point, thereby causing inconvenience in the continuity and speed of the input operation compared to the mouse. For example, a mouse can perform various execution commands by using the thumb, index finger, or middle finger without additional movement of the hand or finger at the same time as selecting the position coordinates, whereas a touchpad or pointing stick device may be unnatural to operate a command button. Additional wrist and finger movements are required, and in some cases, both hands must be used to complete execution command input. In order to improve such inconvenience in use and to facilitate the use of a mouse, a structure in which a function button is arranged above and below the pointing device has been devised. And a mobile phone with a pointing device according to the invention of the present invention is shown in FIG. On the other hand, in the case of a mobile phone using a pointing device up to now, the text input using the pointing device is relatively slower than that of a keyboard, but a keyboard system has not been replaced even though a GUI system can be implemented in a small digital electronic device such as a mobile phone. Moreover, it cannot be applied to devices that require text input directly, and for this reason, the keyboard increases the size of the mobile phone for additional text input to the mobile phone in addition to the numeric keypad. In order to bring about the inconvenience of using a reduced key, it was difficult to develop a small electronic device such as a mobile phone with a GUI system that is easy to input text.

Thanks to advances in semiconductor technology, vacuum tube-based computers in the mid-20th century are now miniaturized and integrated so that they can fit in your hand. (laptop) Routines have already been made through computers and PDAs. Furthermore, devices such as mobile phones, MP3s, and PDPs, which were previously unrelated to computers, are in charge of additional functions such as computers, or are in the age of convergence where these functions are included in portable Palmtop computers. With this trend, it is time to introduce computer graphical user interface (GUI) systems to these small electronic devices. However, the pointing device, which is essential for a graphical user interface (GUI) system of a computer, has not been applied due to the space limitations and usage problems in which these small electronic devices can be mounted. In order for the pointing device to be applied to use a graphical user interface (GUI) system, the size of the touchpad or pointing stick already used in a laptop computer and the like and the structure and size of the function button used with the pointing device need to be changed. will be. Designed to overcome this point, touch screens and input pens are currently applied to palmtop computers or PDAs, but such touch screens or input pens cannot be applied to a size that needs to be operated with one hand such as a mobile phone. Accordingly, it is a technical problem of the present invention to provide a mobile phone using a pointing device that has the convenience of a mouse and may be operated by one finger in some cases.

The touchpad, pointing stick, and trackball, which are used as pointing devices, are devices for changing the position of the pointer. The principles for moving the pointer differ from each other, but they require additional operation of the function buttons to execute commands in the GUI system. . In fact, even when a group of sentences are selected and moved in word processing in a GUI system or when a file is selected and moved at the same time, the function button arrangement of the present invention is arranged as shown in FIG. 2B. It can be easily operated to provide a mobile phone with a true GUI system without an array of buttons. So far, mobile phones using pointing devices have the advantage of introducing a GUI system, but using only one pointing device still has limitations in the text input method. It was because. In particular, the input method using the pointing device and the virtual keyboard becomes slower than the QWERTY phone, which cannot be followed in terms of character input efficiency. In order to overcome this point, it is a configuration of the present invention to use the left and right fingers (thumbs) at the same time by using two optical pointing devices to bring the same character input efficiency as a QWERTY phone. In addition to the increase, furthermore, the operation of the pointer in the GUI system is much easier than when using an optical pointing device. As shown in FIG. 11, even in operation such as drag & drop, convenience of use is brought like a mouse. Furthermore, the present invention provides two optical pointing devices and two pointers associated with the optical pointing device to achieve fast character input, and makes reference points on the optical pointing device to provide convenience in use such as a keyboard to realize convenience of the keyboard. do.

The present invention by applying two optical pointing devices, which are built-in pointing devices that can replace the mouse of a computer, to a phage-type small electronic device such as a mobile phone and a PDA, maximizes the character processing capability such as a computer, This brings the convenience of the function. Especially when it is held and operated with one hand, such as a mobile phone, it will be easier to convert an operating system that stays at DOS level to a Windows system when compared to a computer. And the advantage of the pointing device having the function of the function button to be provided by the present invention is that the input of the character brings the same input method as the existing keypad-type mobile phone when the user uses the mobile phone provided by the present invention It also brings another advantage that it is easy to use. Ultimately, the problem of keyboard processing, which is the most difficult in the process of miniaturizing computers and retaining computer functions of small digital electronic devices including mobile phones, will be solved, resulting in the rise of mobile phones or mobile phone-sized computers with computer functions. . In the case of conventional mobile phones up to now, all functions were operated by buttons, so when the optical pointing device is embedded in the mobile phone as in the present invention, the key button on the front of the mobile phone is replaced with the pointing device, and instead, the key button arrangement is displayed on the screen. Will be replaced by a virtual button. Therefore, the number and size of the key buttons, which had to be limited due to the size of the mobile phone, could no longer be a limiting factor. For example, as shown in FIG. 1, a comparison is made between a Qwerty mobile phone that facilitates text input for the expansion of a text input function and a mobile phone having the pointing device of FIG. 2 designed according to the present invention. You can cut your phone size in half. Moreover, if you use the pointing device to input text on the screen, you do not have to worry about the stress of the finger that comes with operating the small keyboard key provided on the Qwerty mobile phone. Furthermore, as shown in FIG. 17, the importance of the optical pointing device can be easily used like a mouse, which shows how ergonomically efficient the space is used, and furthermore, it is possible to apply a true GUI system. It is more prominent. Accordingly, the present invention enables the development of a Palmtop computer into a Fingertop computer. In this case, the mobile phone will no longer be a simple phone but will serve as a terminal of the information age, and the Internet, the basis of the information age, will face another era.

Claims (13)

In the hand-held portable digital devices such as mobile phones, electronic dictionaries and PDAs with GUI system, a composite input device composed of two pointing devices having a switch function to facilitate the input of characters and symbols through a virtual keyboard and the same GUI system Apparatus according to claim 1, wherein the pointing device is an optical pointing device In claim 1, at least one small protrusion or groove that a finger can recognize is made and used as a reference point to determine the distance direction of the finger movement on the pointing device, and to predict the movement of the pointer on the screen in advance without looking at the screen. A virtual keyboard program that has a pointing device and pointer location associated with it A virtual keyboard program having a function of generating two pointers in a virtual keyboard window for character input in a GUI system of claim 1 so that each pointing device independently operates a different pointer. In claim 4, the virtual keyboard is divided into two virtual keyboard program having a function of having two pointers to each of the divided virtual keyboard The virtual keyboard program having a function of allowing the left and right pointers to cover only the divided areas without crossing the boundary surface of the divided virtual keyboard of claim 5. A virtual keyboard program having a function of placing a pointer on a key of a virtual keyboard to be matched with a reference point on an optical pointing device when the virtual keyboard program of claim 4 is opened and the virtual keyboard is opened on the screen. When the finger moves out of the pointing device and operates a function button while the virtual keyboard of the present invention is already open on the screen, the pointer is returned to the key position of the virtual keyboard to be matched with a reference point on the optical pointing device. Even if you do not leave the optical pointing device, you can operate the special keys such as Tab, Caps, and Korean / English toggle keys on the virtual keyboard, if your finger is accompanied by a deviation from the reference point. Virtual keyboard program having a function of reducing the signal to a key position of a virtual keyboard to be matched with a reference point on the optical pointing device The virtual keyboard program having a function of corresponding reference points on the left and right optical pointing devices to the 'f' and 'j' keys of the virtual keyboard when the virtual keyboard is operated in the QWERTY mode. In claim 3, when the keys on the virtual keyboard associated with the reference point are 'f' and 'j', the left optical pointing device is located at the point immediately off the object surface from the center, and the right pointing device is located from the center to the left. Optical pointing device with at least one protrusion or groove just off the object surface The mobile phone which allows the operation only by the hand holding the hand held in one hand in claim 1 When the small electronic device of claim 1 is a mobile phone, the GUI system having a function of automatically releasing a virtual keypad on the screen to operate a telephone call function when the mobile phone is operated in the portrait view mode. A virtual keypad program having a function of positioning a pointer over a key button corresponding to a reference point on an optical pointing device when the virtual keypad of claim 12 is started.

Images