1 Title [0001] Method of implementing a touch-based universal TV remote. Technical Field [0002] This invention relates to a TV remote. Particularly, this invention relates to a touch screen or touch pad based physical TV remote and a touch screen based software TV remote (mobile App). Background Art [0003] ATV remote is a quite common device in almost every home. It is used to operate a TV set, like turn on/off, switch channels, adjust volumes, play videos and so on. A traditional TV remote is full of buttons due to the demands for the number of operation commands over the traditional TV set. [0004] Nowadays, digital contents are coming into people's daily life through all sorts of Internet-based devices like smart TVs, TV boxes and TV dongles and so on, with a trend of replacing the traditional content delivery method. Accordingly, the remote has been changed greatly to a small and smart device with only a few buttons. For example, an Apple TV's remote has seven buttons including "Up", "Down", "Left", "Right", "Enter", "Menu" and "Play/Pause" buttons; and a Google Nexus Player's Remote has nine buttons including "Up", "Down", "Left", "Right", "Enter", "Return", "Home", "Play/Pause" and "Voice" buttons. These remotes can be viewed as modern remotes. In addition, TV remote Apps for iOS and Android become available now, by which the user can just touch on a smart phone or tablet to control the TV content. These software remotes normally simulate the physical remotes by placing the buttons on the touch screen. [0005] Along with the rapid development of technology, personal computers are becoming smaller and smaller. Now a kind of Mini PCs start appearing in the market which uses a TV 2 screen as the monitor and by which, the users are able to not only watch TV or movies, but also play games, shop online, access cloud services and many other things. At the same time, TV boxes and smart TVs are being added more powerful features like what a computer can do. Accordingly, a new type of remote or control - mini keyboard was brought into the market, which can be used to control the variety of computer contents over the TV screen. Atypical mini keyboard may consist of TV remote buttons, mouse touch pad and essential key buttons from the traditional keyboard. Problems [0006] The main problems existing over the current available remotes can be summarized as the following. [0007] 1. For the traditional physical remote, due to the number of buttons, every time when it is being used, the user has to lower head to locate the button and press it, and then raise head to see the effect on the screen. It is a painful procedure to modern people! [0008] 2. The modern remote can greatly reduce the number of head up and down, but the problem is that, it is unable or very hard to control pointer-based and touch-based contents, such as most of computer applications, mobile Apps and website contents. A touch pad may be placed on the modern remote to fix the problem - for example, Samsung's Smart Remote integrates a touch pad, but the added space will bring great inconvenience that the user may have to move the remote in hand to reach the pad by the thumb finger. Even though a well-designed layout is applied, it is still hard to achieve the convenience that one finger can do everything without moving the remote in hand, because the direction keys and the touch pad require sufficient space respectively. [0009] 3. As for the software remote, it does bring great convenience and economic benefits to the users because it can easily integrate a touch mouse and keyboard in one App so that the user is able to control nearly all the contents; but a fatal problem is that, the user has no feeling of the buttons over a touch screen so that the pain of having head up and down frequently comes back although there are only a few buttons! Therefore, simply 3 simulating the buttons on a physical remote cannot achieve the expected effects. [0010] 4. The mini keyboard so far is a powerful and handy control which integrates the modern remote, the mousepad and the keyboard. But due to having too many of key buttons, it lets users experience the same pain with using the traditional remote and moreover, it is impossible to be made as small and smart as a modern remote. [0011] 5. In addition, no matter it is the physical remote or the software remote, a slight problem is that, there are only four direction operations including "Up", "Down", "Left" and "Right". The resources on a traditional TV are normally organised in a list, therefore these four direction operations are competent enough. However, the modern resources, most organised in a grid, for example Apple TV Box organises the resources in a 4x5 grid. Therefore, if the user wants to navigate to the resource at a corner, the user has to press to buttons; for example, to the top left recourse, the user needs to press "left" then "top" or "top" first and then "left". In a word, eight direction commands may be needed to meet the new requirements. SUMMARY OF THE INVENTION [0012] In a broad aspect of the invention there is provided a method of remotely controlling a TV using a touch motion sensing TV remote control having a touch sensitive surface capable of sensing touch motions of a finger on the touch sensitive surface, the TV being responsive to remote control signals representing key events of direction up, down, left and right keys, the method comprising the step of configuring the touch motion sensing TV remote control to respond to sensing of 3A a touch move of the finger towards each of up, down, left and right by emitting remote control signals representing a number of pairs of down and up events of the corresponding direction key, wherein the touch move of the finger is defined as the finger moving on a touch sensitive surface so as to trigger continuous motion move events. In one embodiment, the method further comprises the step of configuring the touch motion sensing TV remote control to response to sensing of a touch flick of the finger towards each of up, down, left and right by emitting remote control signals representing a single pair of down and up events of the corresponding direction key, wherein the touch flick of the finger is defined as the finger moving quickly over the touch sensitive surface so as to trigger one or a few pairs of motion move events. In one embodiment, the pairs of down and up events of the direction keys are merged into single touch or press events in a system which does not distinguish down and up events of the direction keys. In one embodiment, wherein the converting steps are implemented in software located in the touch most sensing TV remote control. In one embodiment, the touch motion sensing TV remote control is provided in a programmed smart phone. [0013] The new operation methods of the invention in an embodiment are simply summarised as the following but not limited thereto: touch down and quick up: "enter" key double touch down and quick up: "return" key triple touch down and quick up: "home" key Move straight: "direction" keys Move in a curve or circle or straight and one of the edge areas: "volume up" and "volume 4 Down" keys [0014] The invention will bring the following advantageous effects. [0015] 1. To a physical remote, a touch pad with the new TV operation method can completely replace the old direction keys and some of others. In this way, the remote is able to integrate the mouse and even keyboard functions over the pad without having to increase the remote size. As a result, it is possible to perform all the operations by only one thumb finger without moving the remote in hand. [0016] 2. To a software remote, the new method enables users to perform the operations over a touch screen without having to lower and raise head frequently. [0017] 3. The new method can bring more operation possibilities like the navigation to the extra four directions. [0018] 4. The new method brings great extensibility for future requirements; for example, using more different finger motions to implement more operations such as page up, page down, and contents scrolling and so on. [0019] 5. Touch based keys would bring more comfortable feelings to the user compared with physical buttons. [0020] The difference between a touch screen and a touch pad in this document is that, the touch screen means the screen which can display contents like a smart phone's screen; while the touch pad means the pad on a control device which normally do not display anything such as the touchpad on a laptop. Drawings [0021] Figure 1 shows how a remote with the invention works.
5 [0022] Figure 2 shows the finger motions over the touch screen or touch pad which can trigger the "Direction" key events. [0023] Figure 3 shows the finger motions over the touch screen or touch pad which can trigger the "Volume Up" key events. [0024] Figure 4 shows the finger motions over the touch screen or touch pad which can trigger the "Volume Down" key events. [0025] Figure 5 shows another finger motion option to implement the "Volume Up" key and "Volume Down" key events. [0026] Figure 6 shows a motion track in a coordinate, used to demonstrate an algorithm to calculate the triggered events. [0027] Figure 7 shows an example of a remote design based on the invention, which integrates the mouse feature. [0028] Figure 8 shows an example of a remote design based on the invention, which integrates the mouse and keyboard features. [0029] Figure 9 shows an example of a remote design based on the invention, which integrates the mouse, keyboard, and voice control features. Invention Embodiments [0030] Figure 1 illustrates how a remote with the invention works. Over the touch screen or touch pad, the finger motion events are captured; then the motion events are converted to the key events of a remote; after that the key events are transmitted to the controlled device; and finally the effects are displayed on the screen. The procedure 1 in Figure 1 is performed by the touch screen or touch pad and the corresponding software. Procedure 3 is performed by existing technologies like Infrared, Bluetooth, WiFi, FR and the 6 corresponding equipment and so on. Procedure 4 is performed by the controlled device. Procedure 2 is the invention part, comprising a number of conversion methods. [0031] The touch motions and the corresponding events performed by one finger are defined as the following. 1. Touch Down The finger touches the screen or pad before leaves, triggering a Motion Down event. 2. Touch Up The finger leaves the screen or pad after a Touch Down, triggering a Motion Up event. 3. Double Touch The finger touches and leaves the screen or pad twice in a short interval, triggering two pairs of Motion Down and Motion Up events. 4. Triple Touch The finger touches and leaves the screen or pad three times in a short interval, triggering three pairs of Motion Down and Motion Up events. 5. Touch Move The finger moves on the screen or pad, triggering continuous Motion Move events. It is normally accompanied a Motion Down event at the beginning and a Motion Up event at the end. 6. Touch Flick The finger flicks quickly over the screen or pad, triggering one or a few pairs of Motion Move events. It is normally accompanied a Motion Down event at the beginning and a Motion Up event at the end. [0032] The keys which can be substituted comprise but not limited to the following, with Down and Up events respectively. 1. Direction Up Key 2. Direction Down Key 3. Direction Left Key 4. Direction Right Key 5. Enter Key 6. Return Key 7 7. Home Key 8. Volume Up Key 9. Volume Down Key [0033] The conversion methods from touch motions to key events are defined as the following. 1. Touch Down is converted to Enter Key Down event. 2. Touch Up is converted to Enter Key Up event. 3. Double Touch is converted to a pair of Return Key Down event and Return Key Up event. 4. Triple Touch is converted to a pair of Home Key Down event and Home Key Up event. 5. Touch Move towards up is converted to a number of pairs of Down and Up events of the Direction Up Key. The item 5 in Figure 2 shows some motion track examples for Touch Move towards up. 6. Touch Move towards down is converted to a number of pairs of Down and Up events of the Direction Down Key. The item 6 in Figure 2 shows some motion track examples for Touch Move towards down. 7. Touch Move towards left is converted to a number of pairs of Down and Up events of the Direction Left Key. The item 7 in Figure 2 shows some motion track examples for Touch Move towards left. 8. Touch Move towards right is converted to a number of pairs of Down and Up events of the Direction Right Key. The item 8 in Figure 2 shows some motion track examples for Touch Move towards right. 9. Touch Flick towards up is converted to a pair of Down and Up events of the Direction Up Key. 10. Touch Flick towards down is converted to a pair of Down and Up events of the Direction Down Key. 11. Touch Flick towards left is converted to a pair of Down and Up events of the Direction Left key. 12. Touch Flick towards right is converted to a pair of Down and Up events of the Direction Right Key. 13. Touch Move in a towards-up curve, or in a clockwise circle is converted to a number of 8 pairs of Down and Up events of the Volume Up Key. Figure 3 shows some motion track examples for the two motions performed by right hand and left hand respectively. 14. Touch Move in a towards-down curve or in an anticlockwise circle is converted to a number of pairs of Down and Up events of the Volume Down Key. Figure 4 shows some motion track examples for the two motions performed by right hand and left hand respectively. [0034] The motions described above can be performed anywhere over the touch screen or touch pad, rather than the location where they are drawn in Figure 2, Figure 3 and Figure 4. [0035] For Volume Up Key and Volume Down Key, they can be alternatively substituted by Touch Move towards up and down at one of the edge areas of the screen or pad. For example, as shown in Figure 5, the whole screen or pad are divided into two parts: the centre area (17) and the four edge bars (where 18, 19, 20, 21, 22, 23, 24, 25 are); in the centre area, the conversion methods described above can be applied; in one of the edge bars, Touch Move towards up or towards right are converted to a number of pairs of Down and Up events of the Volume Up Key (e.g. 18, 19, 20, 21 in Figure 5), and Touch Move towards down or towards left are converted to a number of pairs of Down and Up events of the Volume Down Key (e.g. 22, 23, 24, 25 in Figure 5). In this way it is much simpler to implement compared with the "Touch Move in a curve, or in a circle". [0036] To implement the conversion to Enter Key, Return Key and Home Key events, it can be simply catching the touch motion events, and dispatch the corresponding key events instead. [0037] To implement the conversion to Direction Keys by Touch Move, a certain number of moved pixels (or dots) on X axis and Y axis in a coordinate respectively can be used to trigger one pair of the Direction Key events. Figure 6 shows an example. Suppose the left bottom corner of the screen or pad is the centre of the coordinate; a finger moves from A to B; on X axis, it is from 100 to 600 and on Y axis, it is from 100 to 800; if 20 pixels is set as the trigger number; during the moving procedure, on each of the Motion Move event, the 9 number of pixels on X axis and Y axis is accumulated respectively; when it is equal to or greater than 20 on X axis, trigger one pair of Down and Up events of the Direction Right Key, and then subtract 20 on the accumulated number; when it is it is equal to or greater than 20 on Y axis, trigger one pair of Down and Up events of the Direction Up Key, and then subtract 20 on the accumulated number; therefore, theoretically the moving from A to B can trigger 25 pairs of the Direction Right Key events and 35 pairs of the Direction Up Key events; but be aware that the Motion Move event may not be captured over every pixel, especially when moving fast, one Motion Move event may be captured once over an uncertain number of pixels. [0038] To implement the conversion to Direction Keys by Touch Flick, a certain number of moved pixels (or dots) in the direction can be used to trigger one pair of the Direction Key events. It is similar with that of the conversion by Touch Move described above, but the difference is that, Touch Flick is one motion which triggers only one pair of the Direction Key events while Touch Move is a series of motions which trigger a number of pairs of the Direction Key events. Touch Flick is normally implemented by Motion Move events with a check logic that, when the Motion Up event is captured, if the total number of moved pixels is smaller than a certain number like 20, it is supposed to be a Touch Flick motion and trigger only one pair of the Direction Key events. [0039] To implement the conversion to Volume Up Key and Volume Down Key events, the same methods described above can be used, but for the move in a curve or in a circle, it needs to judge whether the previous points on the track fall into a curve or circle to decide to trigger the Direction Key events or the Volume Key events; for the move at the edge area, it can simply apply the calculation on a single X axis or Y axis to trigger the corresponding Volume Up Key or Volume Down Key events. [0040] For some systems, they do not recognize Down and Up events, and instead recognize only one event called "Pressed" (or "Clicked") on a key and "Touch" on a touch screen or touch pad. In this case, Motion Down and Motion Up events are merged into one Motion Touch event which can be triggered by either of the two event motions; for the remote keys, the pair of Down and Up events can be represented by one "Pressed" or 10 "Clicked" event. [0041] Moreover, a Touch Move motion at one of the vertical edges can also be converted to a Mouse Wheel event (the scrolling event of the middle button on a mouse) which is useful to scroll a web page or a document vertically. For example, use the right edge area to continuously trigger Mouse Wheel events by moving the finger up and down. Examples of Invention [0042] Figure 7 to 9 show three example remote designs which are based on the invention. These designs can be applied on the touch pad of a physical remote or the touch screen of a smart phone. [0043] Figure 7 shows a simple TV remote. On this remote, the area above the dashed line 27 can be a structured touch region where all the keys are implemented. 26 is the Touch and Touch Move area for either "Direction Mode" or "Mouse Mode" (in the former mode, the area is used for Direction Keys, Enter Key, Return Key and Home Key while in the latter mode, the area is used as a mouse pad); 28 is the Touch Move area for Volume Up Key and Volume Down Key; 29 is the Touch Move area for the Mouse Wheel event or the Page Up and Page Down events (the "PgUp" and "PgDn" buttons on a keyboard); 30 is the Power on/off key; 31 is a key to switch between "Direction Mode" and "Mouse Mode"; 32 is the Home Key (go back to the home screen); 33 is the Menu Key (show menu list); 34 is the Return Key (return to the previous screen). [0044] Figure 8 shows the same layout with that in Figure 7, but on this remote, it is added one more key 35 - Keyboard, which is used to add one more mode ("Keyboard Mode") on the touch area. When this key is pressed, the touch area can be a keyboard with showing the keys, by which the user is able to input characters to the device. When the key is pressed again, the mode is switched back to the "Direction Mode" or "Mouse Mode", with hiding the keyboard. In this way, the touch screen or touch pad can be used for three purposes which are switched by the two keys.
11 [0045] Figure 9 shows a remote design with "Voice" function. It has the same layout with that in Figure 8, but the "Keyboard" key 36 in this design is used for one more purpose that, when it is being held, it is in "Voice Control" mode; and in this mode, the user can talk to control the TV contents. [0046] The three designs are in three cost levels and they are just general guidelines to implement a the-invention-based remote. In practical applications, the key layout and the key purpose need to be designed according to the actual demands and the user experience.