AU2016100780A4

AU2016100780A4 - Smart text input for remote screens

Info

Publication number: AU2016100780A4
Application number: AU2016100780A
Authority: AU
Inventors: Wenzhi Wang
Original assignee: Hubi Tech Pty Ltd
Current assignee: HUBI TECHNOLOGY Pty Ltd
Priority date: 2016-05-29
Filing date: 2016-05-29
Publication date: 2016-06-30
Anticipated expiration: 2024-05-29

Abstract

The invention provides an interactive viewing system, comprising: a target viewing screen associated with a target processor adapted to display a target screen text input field on the target viewing screen to a viewer and adapted to receive text input received from a TV remote control and insert the text input received from the TV remote control into the target screen text input field; a text input injector application running on or in association with the target processor; a hand-held mobile device associated with the viewer, the hand-held mobile device having text input capability and running a viewer interaction application and being connected via a wireless link with the text input injector application, wherein: the viewer interaction application is configured to receive text input from the viewer and to send the text input to the text input injector application over the wireless link; and the text input injector application is further configured to receive the text input from the client application and to inject the text input from the client application into the target screen text input field. YNS-Fn rd -5 t U v w x ~~~~J POW$____ Fiur 1

Description

TITLE

[0001] SMART TEXT INPUT FOR REMOTE SCREENS

TECHNICAL FIELD

[0002] This invention relates to remote text input. In particular, this invention relates to a software input terminal on a smart device such as a smart phone, a tablet and a smart watch, connected to a target device with remote screen such as a smart TV.

BACKGROUND ART

[0003] Along with the increasing popularity of Internet connected smart TVs and Internet TV boxes, more and more on-demand streaming services are available on big screens at home. Typical representatives include YouTube, Netflix, Apple TV, and Google Movies. When watching a program on a TV screen, viewers normally use a standard TV remote control to navigate and find the target contents. Standard TV remote controls do not include keyboard functionality, either in the form of a hard keyboard or a programmed soft keyboard on a touchscreen, and yet an increasing amount of interaction with a smart TV requires text input. It is difficult using a standard TV remote control to find what you want among a huge number of contents. "Voice Search" was invented to find the desired content easily and efficiently, but the reality is that the accuracy was not as people expected and actually not many people prefer to speak in front of a TV set, especially in a quiet environment. "Text Search" is still the most popular way to find desired content and it is a standard function on nearly all of the streaming apps. A drawback of text search is the requirement to navigate between characters over a virtual keyboard on the target television screen with the arrow keys of the TV remote, to input keywords .

[0004] Aside from searching for media content inside a streaming service, the problem of text input into a TV screen also presents itself in connecting to a Wi-Fi network on the required provision of a password, logging into accounts to input a username and password, opening a browser to access a website requiring specification of the URL, filling in a form on a website. Of course, it is possible to attach a standard keyboard to many smart TVs typically via a USB port to solve this problem, but most people are reluctant to place unsightly keyboard in front of a TV set or a coffee table. Consequently, remote control devices without keyboarding input persist despite the inconvenience.

[0005] Further, because televisions may be owned by viewers for many years, viewers are often stuck with input technology originating from many years ago.

[0006] It is an object of the current invention to overcome one or more of the above-mentioned disadvantages by providing an adaptation to existing systems.

SUMMARY OF INVENTION

[0007] In accordance with a first broad aspect of the invention there is provided an interactive viewing system, comprising: a target viewing screen associated with a target processor adapted to display a target screen text input field on the target viewing screen to a viewer and adapted to receive text input received from a TV remote control and insert the text input received from the TV remote control into the target screen text input field; a text input injector application running on or in association with the target processor; a hand-held mobile device associated with the viewer, the handheld mobile device having text input capability and running a viewer interaction application and being connected via a wireless link with the text input injector application, wherein: the viewer interaction application is configured to receive text input from the viewer and to send the text input to the text input injector application over the wireless link; and the text input injector application is further configured to receive the text input from the client application and to inject the text input from the client application into the target screen text input field.

[0008] In one embodiment, the system further comprises: a capture focus event application running on or in association with the target processor adapted to detect text input field focus events; and a send event application running on or in association with the target processor adapted to send a notification of the focus event to the hand-held mobile device over the wireless link; and a receive event application running on the hand-held mobile device adapted to receive the notification of the focus event; and wherein: the viewer interaction application is configured to be activated after receiving the notification of the text input field focus event.

[0009] In another embodiment, the viewer interaction application is configured to be activated by the viewer.

[0010] In one embodiment, the viewer interaction application is further configured to send the text input after the viewer activates a send command after completion of the text input.

[0011] In another embodiment, the viewer interaction application is further configured to send the text input after each keystroke is input by the viewer.

DRAWINGS

[0012] Figure 1 depicts a hand-held mobile device, target viewing screen and TV remote control in accordance with an embodiment of the invention.

[0013] Figure 2 shows the processes involved in the invention.

[0014] Figure 3 shows the processes involved in the invention in the context of the inputting device and the target device.

[0015] Figure 4 shows the typical components of the input view on the inputting device based on the invention.

[0016] Figure 5 shows the processes involved in the invention in the context of multiple inputting devices and the target device.

DETAILED DESCRIPTION OF EMBODIMENTS

[0017] Embodiments of the invention will now be described with reference to the above mentioned drawings.

[0018] The invention aims to provide an easy, accurate and efficiently way to input texts onto a remote screen, utilizing the ubiquitous smart mobile devices like smart phones, tablets and smart watches. The invention is a technology or method of using the soft keyboard and other text input methods of a smart mobile device to input text into a focusable text component, such as text input field, on a remote screen.

[0019] In this embodiment the invention involves two software applications that are installed on an inputting device and the target device respectively. The application installed on the inputting device is a client application and the application installed on the target device is a server application (or service). The implementation method comprises six processes, including Capture Focus Event, Transfer Focus Event, Show Input View, Capture Inputs, Transfer Inputs, Inject Inputs.

[0020] Referring first to Figure 1, the system of the invention includes a target viewing screen 1, which may be a smart TV screen, associated with a target processor 2. Target processor 2 may be incorporated within target viewing screen 1, or may be incorporated in a standalone TV box such as an Apple TV and using target viewing screen 1 as a display output. Target processor 2 is adapted to display a target screen text input field 3 on target viewing screen 1 to a viewer 4, and is adapted to insert text input received from viewer 4 into target screen text input field 3 if viewer 4 is operating TV remote control 5.

[0021] TV remote control 5 is typically a standard TV remote control which permits text input into target screen text input field 3 via known means, such as navigation using direction keys over a soft keyboard 6 displayed on target viewing screen 1, mouse or other pointing device or voice or similar, but may be any TV remote control. The invention is typically applied to augment a system originally incorporating a standard TV remote control.

[0022] The system according to the invention also incorporates a hand-held mobile device 7 which the invention adapts to be an alternative text input device to target screen text input field 3. Hand-held mobile device 7 is typically a general purpose smart phone or tablet such as an iPhone, iPad running iOS or a similar smart phone or tablet running Android.

[0023] This embodiment of the invention the viewer interaction application is incorporated in a client software application installed on hand-held mobile device 7 and the text input injector application is incorporated in a server application which is installed on or associated with target processor 2. The system may be adapted to allow multiple hand-held mobile devices to input texts into the target device at the same time. The hand-held mobile device 7 is the inputting device where the texts are typed and captured; and the target processor 2 is the device where the input texts are input and displayed on the target viewing screen 1.

[0024] The method of this embodiment comprises six applications or processes, as shown in Figure 2, including Capture Focus Event 101, Transfer Focus Event 102, Show Input View 103, Capture Inputs 104, Transfer Inputs 105, and Inject Inputs 106.

[0025] Figure 3 shows the processes and further details in the context of both devices.

[0026] The first process is Capture Focus Event 101. This is a process hosted by the server application on target processor 2.

When a text field is the focus of target processor 2 in readiness for viewer input, for example during text search or request for use in are or similar, conventionally by default the virtual keyboard will be called and displayed on target viewing screen 1, then the user can operate on the virtual keyboard with TV remote control 5, typically by using direction keys on TV remote control 5 to highlight the appropriate virtual keys on soft keyboard 6, to perform text input. In this embodiment of the invention, the server application incorporates a focus monitoring process continuously monitoring for the focus event, and once it is triggered, the server application captures the event and may stop the virtual keyboard being displayed on the target viewing screen 1. Persons skilled in the art will appreciate various methods of monitoring for text input focus offered by most operating systems.

[0027] The focus (normally visually indicated by a cursor) is kept in the target text field, waiting for key input. There are different ways available for target processor 2 to stop showing the virtual keyboard on target viewing screen 1, depending on the local operating system. For Android, one practical way is to replace the default system Input Method with a "NULL" Input Method. Here "NULL" Input Method can be an Android App which shows an invisible virtual keyboard. In some cases, it may not be necessary or desirable to stop showing the virtual keyboard, for example in implementations which may allow simultaneous input from either TV remote control 5 or hand-held mobile device 7.

[0028] The second process is Transfer Focus Event 102, comprising Send Focus Event 102a implemented in the server application on target processor 2 and Receive Focus Event 102b implemented in the client application on hand-held mobile device 7. When the server application captures the focus event, it immediately sends the event to the client application. This process involves communication over a wireless link between the two applications. Typically, the server and client applications establish the wireless link at the initiation, and then keep the communication channel live. Alternatively, the server can initiate the wireless link on receipt of a first focus event. The focus event can be sent to the client application in any form as long as the client application can recognize it, and it is not necessarily in the same form or protocol captured by the server application on target processor 2. For example, the server application and client application can communicate the focus event simply use a single byte of integer value 1 representing the focus event.

[0029] The third process is Show Input View 103. When the client application receives the focus event from the server application, Show Input View 103 shows an Input View on a touchscreen of handheld mobile device 7 in readiness for viewer 4 to input texts on the touchscreen. A typical Input View comprises four components as shown in Figure 4, including a text field 301, a "Send" button 302, a "Cancel" button 303, and a system keyboard 304. The text field 301 is used to display the input text. When the "Send" button 302 is pressed, Send Input process 105a is activated causing the input text in text field 301 to be sent over the wireless link to the server application in or associated with the target processor 2.

The "Cancel" button 303 is used to click and hide the Input View. System keyboard 304 is provided by the operating system of handheld mobile device 7 and used to input the text on the touchscreen.

[0030] The fourth process Capture Inputs 104. When the user inputs texts locally using the system keyboard 304 on hand-held mobile device 7, the input text is captured and displayed in the text field 301. The input texts may be generated by typing on the system keyboard, or by hand writing, or by voice recognition or by other text input method, depending on what input methods the local handheld mobile device operating system provides and which method the user chooses .

[0031] The fifth process is Transfer Inputs 105, comprising Send Inputs 105a implemented in the client application on hand-held mobile device 7 and Receive Inputs 105b implemented in the server application on target processor 2. The captured input contents will be displayed on the text field 301. Once the user clicks the "Send" button 302, the contents in the text field 301 will be sent to the server application as a whole and received by the Receive Inputs 105b process. The sent inputs may also include invisible characters such as "Enter" key, "Tab" key and so on.

[0032] The sixth process is Inject Inputs 106 running on the server application on the target processor 2 which injects the text input received from the hand-held mobile device. After Receive Inputs 105b the server application receives the inputs, the server application uses Inject Inputs 106 to inject the inputs into the focused text field or area. Different operating systems have different ways of implementing the text injection, but persons skilled in the art are aware that most operating systems provide the corresponding APIs for programs to access and write to text input fields.

[0033] In the context of multiple hand-held mobile devices, such as hand-held mobile device 10 and hand-held mobile device 11 as shown in Figure 5, the server application may operate Send Event 102a to send the focus event to the multiple client applications running on each hand-held mobile device 10, 11 at the same time, and the clients may send the inputs back at different times. In this case, the server application can overwrite or append the previous inputs, or just simply accept the first received input and abandon the later inputs. The implementation depends on the practical requirements. For example, multiple occupants of the house may be running the client application on their respective hand-held mobile devices or a single occupant may have multiple hand-held mobile devices break the client application. Typically, the first received input should take precedence.

[0034] Persons skilled in the art will appreciate that modifications and variations can be made to the invention within the broadest scopes of the invention described above.

[0035] For example, the viewer interaction application is not necessarily activated by receiving a focus event from the target processor 2. An alternative is when a target screen text input field 3 appears on the target viewing screen 1 and is noticed by viewer 4, viewer 4 may activate the viewer interaction application (or access it if it is already running) and enter text on hand-held mobile device, which is either transferred to the server application irrespective of whether a focus event has occurred or on the condition that a focus event has occurred. Clearly, the server application can only inject the text input in the event that they focus event has occurred, and it will be appreciated that appropriate software on the server application and client application can be arranged to provide appropriate circumstances for text input transfer and/or injection.

[0036] Further, the viewer interaction application may not accumulate text input until viewer 4 activates a send command after completion of the text input by pressing a send button to activate send inputs 105a, and instead may activate send inputs 105a after each keypress. In such an embodiment, it would be appropriate to keep the target screen text input field 3 visible so that the viewer 4 can view the progress of their text entry on the hand-held mobile device 7.

[0037] Further still, while it is anticipated that the invention is normally implemented by augmenting an existing viewing system with a TV remote control by providing the server application to run on the target processor in parallel with an existing program running on the target processor which interacts with the remote control, the invention extends to implementations in which the server application is incorporated within an existing viewing system.

[0038] The words "application" or "process" is used interchangeably unless the context requires otherwise and extends to any programmed capabilities implemented in software running or available on a processor satisfying the described features, and include sub-programs or lines of code contained within larger programs .

[0039] The words "text input field" refers to an input field for receiving text input and does not refer to any particular visual appearance or area on the screen. As such, while a text input field is usually visible, it may be invisible or may become invisible and still be an active text input field. The term "focus event" in relation to a text input field refers to the state of a program focusing on or waiting for input into the text input field, typically but not necessarily displayed by the visual appearance of a cursor in a text input field.

[0040] The broadest aspect encompasses embodiments where the functionality of the hand-held mobile device is added to a system having any type of primary, dedicated or existing TV remote control, which may be supplied with the standard television or television box such as an Apple TV, or may have a keyboard input capability, and the broadest aspect is not restricted to embodiments where the hand-held mobile device augments a standard TV remote control without any keyboard input capability of its own.

Claims

1. An interactive viewing system, comprising: a target viewing screen associated with a target processor adapted to display a target screen text input field on the target viewing screen to a viewer and adapted to receive text input received from a TV remote control and insert the text input received from the TV remote control into the target screen text input field; a text input injector application running on or in association with the target processor; a hand-held mobile device associated with the viewer, the handheld mobile device having text input capability and running a viewer interaction application and being connected via a wireless link with the text input injector application, wherein: the viewer interaction application is configured to receive text input from the viewer and to send the text input to the text input injector application over the wireless link; and the text input injector application is further configured to receive the text input from the client application and to inject the text input from the client application into the target screen text input field.

2. The interactive viewing system of claim 1, further comprising: a capture focus event application running on or in association with the target processor adapted to detect text input field focus events; and a send event application running on or in association with the target processor adapted to send a notification of the focus event to the hand-held mobile device over the wireless link; and a receive event application running on the hand-held mobile device adapted to receive the notification of the focus event; and wherein: the viewer interaction application is configured to be activated after receiving the notification of the text input field focus event.

3. The interactive viewing system of claim 1, wherein the viewer interaction application is configured to be activated by the viewer.

4. The interactive viewing system of any one of claims claim 1 to 3, wherein the viewer interaction application is further configured to send the text input after the viewer activates a send command after completion of the text input.

5. The interactive viewing system of any one of claims 1 to 3, wherein the viewer interaction application is further configured to send the text input after each keystroke is input by the viewer.