WO2013126507A1

WO2013126507A1 - Mobile embedded servers systems and methods related thereto

Info

Publication number: WO2013126507A1
Application number: PCT/US2013/027030
Authority: WO
Inventors: Francois LABERGE; Christopher Allen
Original assignee: Brass Monkey, Inc.
Priority date: 2012-02-21
Filing date: 2013-02-21
Publication date: 2013-08-29

Abstract

Featured are systems and methods for forming a network connection between a host device and an endpoint device and more particularly, embedding a server or server like functionality in a mobile device for communication across a network. Also featured is a communication method for communicating over a network that includes establishing a local network connection between an application executing on an endpoint device, and a browser application executing within a web browser on a host device; downloading, by the endpoint device, from a server device, data for use by the browser application; and providing, by the endpoint device, the data for downloading over the local network, so that the browser application can retrieve the data from the endpoint device over the local network connection for subsequent use by the browser application.

Description

MOBILE EMBEDDED SERVERS SYSTEMS AND

METHODS RELATED THERETO

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to and benefit of Provisional Patent Application Serial No. 61/601,213, filed on February 21, 2012, the teachings of which are incorporated herein by reference in their entirety.

TECHNICAL FIELD

The present invention relates generally to the field of electronic communication over a network as well as systems and methods implementing or embodying such communications. More particularly, the present invention relates to embedding a server or server-like functionality in a mobile device for communication across a network to a host device. In more specific aspects, the present invention relates to communication between a game application executing within a browser and a remote application within, for example, a mobile device.

BACKGROUND

With the rise of mobile computing, more and more devices are able to connect to computer networks. Network communication is no longer limited to PCs and laptops. Mobile phones and smart phones can typically connect not only to mobile telephone networks, but also Wi-fi networks, Bluetooth® networks, USB networks, etc. Other devices, which historically were not able to connect to computer networks, are entering the market already configured to access the internet. Televisions, DVD players, and even home appliances have network and internet capability.

Mobile applications have also become increasingly popular. Smart phones, such as the iPhone® device, Android® phones, or Blackberry® phones can download and run a broad array of different applications, from games and movie players to productivity and work tools. Browser applications are attractive to developers because they can be relatively easy to develop and deploy. Browser applications also provide dynamic and customizable content for display within a web browser. A browser application developer could create, for example, a video game or media presentation application that can run within a web browser.

Browser applications, such as FLASH® applications, Unity™ applications, JavaScript® applications, and the like, can be downloaded and run by a web browser without the need to install the application directly onto a local computer. Rather, a user can run a browser application simply by directing his or her browser to a URL that provides a browser application. The web browser will then load and execute the browser application directly, downloading it just as if the user were viewing a static web page. Many browser applications are game applications that users can connect to, control, and play for entertainment.

However, browser applications are typically subject to security restrictions. Web browsers often employ a security scheme, sometimes called sandbox security that restricts browser applications from accessing resources on the local machine. For example, web browsers will often prevent browser applications from accessing files on the local hard drive, operating system resources, etc. Web browsers may also prevent browser applications from communicating over a network, such as a local or wide area network connected to the local machine.

Unfortunately, because of such a security mechanism of the web browser, it may be difficult for a mobile device to establish a network connection with a game application executing within the web browser.

Technology has advanced so as to provide devices, systems and methods that can establish a network connection between an application (e.g., game application) executing within the web browser in a host device and a remote endpoint device. Such devices, systems and methods can be found in U.S. Patent Nos. 8, 019,867; 8,019,878; 8,024,469; 8,166, 181 ; and

8, 171 ,145, the teaching of all of which are incorporated herein by reference. It also is noted that these patents are owned by the assignee of the subject

application.

[00010] While there has been some advances in technology, there continues to a be a need to improve the capabilities of such devices, systems and methods and there continues to be a need to establish such network connections as well as using such connections to minimize time and requirements for downloading of files and data. In this way, the time and system requirements involved with downloading files and data using conventional techniques (e.g., via the internet) can be reduced and in the case of a gaming application, reducing the time taken before the gaming experience can begin.

[0001 1] It thus would be desirable to provide devices, systems and methods that allow the formation of such a network connection between a host device and an endpoint device. It would be particularly desirable to provide such devices, systems and methods in which the endpoint device is provided with additional functionality in comparison to prior art devices. More specifically, such an endpoint device can have some server like functionality so as to, for example, cache files and data for communication between the endpoint device and the host device using the network connection.

BRIEF SUMMARY OF THE INVENTION

[00012] The present invention features systems, device and methods for the formation of a network connection between a host device and an endpoint device and relates to electronic communication and methods for implementing the same. More particularly, the present invention also features embedding a server or server-like functionality in a mobile device for communication across a network. The server technology may be http / web server technology. However, other protocols and servers may be possible, such as WebSockets, Web RTC, Adobe Flash's RTMP, RTMPT, RTMFP streaming protocols, and Unity Web Player's UDP/TCP networking capabilities, etc.

[00013] In a particular aspect of the present invention, there is featured a

method for communicating over a network. Such a method includes establishing a local network connection between an application executing on an endpoint device, and a browser application executing within a web browser on a host device and downloading, by the endpoint device, from a server device, data for use by the browser application. Such a method also includes providing, by the endpoint device, the data for downloading over the local network, so that the browser application can retrieve the data from the endpoint device over the local network connection for subsequent use by the browser application.

[00014] In embodiments of the present invention, in such a method, the data is an executable file that can be launched by the browser application and/or the data is application data that can be used by the browser application during execution. Also, in further embodiments the browser application is a game and the application executing on the endpoint device is a controller application for playing the game.

[00015] In yet a further embodiment, the step of storing includes rendering the data by the endpoint device. In addition, the data can be used to coordinate and control graphical output from multiple devices in communication with the local network.

[00016] According to another aspect or embodiment of the present invention, there is featured a method of controlling a game. Such a method includes establishing a local network connection between a controller application executing on an endpoint device, and a browser application executing within a web browser on a host device and storing in the endpoint device data for use by the browser application. Such a method also includes retrieving the stored data by the browser application from the endpoint device over the local network, so that the browser application can subsequently use the retrieved data; and executing, by a host device in communication with a computer network, a game application within a web browser. In yet further embodiments, such a method further includes sending data, by the controller application and the game application, to one another over the communication channel for controlling and playing the game application within the web

browser.

[00017] In accordance with yet another aspect/embodiment of the present invention there is featured a system for controlling an application within a web browser. Such a system includes a host device configured to execute a first application within a web browser and an endpoint device configured to communicate with the host device over a computer network and to execute a second application. Such a system further includes a communication channel established on a local network between the second application and the first application. In such a system, the endpoint device is configured and arranged to download data, from a server device, the data for use by the first application and the endpoint device also is configured and arranged to transmit the downloaded data to the browser application for subsequent use by the browser application.

[00018] Such systems and methods of the present invention have a number of beneficial effects as compared to conventional systems and methods. In particular, the caching of files and data along with communicating such data and files over the local network can reducing latency or time for downloading data and files, as compared to conventional systems and methods. In such conventional systems and methods, data and files are communicated from a server to a host device over a combination of networks (e.g. , LAN or WAN) including the Internet where time or latency delays can be introduced anywhere along the communication pathway and/or by devices involved with such communications.

[00019] Other aspects and embodiments of the invention are discussed below.

BRIEF DESCRIPTION OF THE DRAWINGS

[00020] For a fuller understanding of the nature and desired objects of the present invention, reference is made to the following detailed description taken in conjunction with the accompanying drawing figures wherein like reference character denote corresponding parts throughout the several views and wherein:

Figs. 1 A - E are various illustrative views illustrating systems and related methods for connecting host devices and endpoint devices over a network.

Fig. 2 is an illustrative view of a system and related method in which a local network connection is provided between a host device and endpoint device(s) as well interconnecting both to a server device.

Fig. 3 is a block diagram illustrating a computer network architecture suitable for operation with the systems and methods of the present invention.

Fig. 4 is a block diagram of an exemplary computer processing device embodied in the host device and/or the endpoint device.

Fig. 5 is a flowchart depicting a method for use in connection with establishing a network connection between the host device and the endpoint device.

Fig. 6 is a block diagram illustrating some of the operations of the system of Fig. 2;

Fig. 7 is an illustrative block diagram view of a simplified system with certain aspects or features removed for clarity (e.g., the server and related interconnects thereto for establishing the network connection is not shown) that shows caching of data and files.

Fig. 8 is another illustrative block diagram view of a system according to the present invention showing establishing a network connection and caching of data and files (e.g., application files).

Fig. 9 includes illustrative views of multiple screen scenarios that can be controlled and/or coordinated by one or more endpoint devices of the present invention acting as servers.

Fig. 10A is an illustrative block diagram view illustrating video/audio streaming using the systems and methods of the present invention.

Figs. 10B - C are illustrative views of controller generated (Fig. 10B) and game generated (Fig. IOC) video and audio buffering using systems and methods of the present invention. Fig. 11 is an illustrative block diagram view illustrating a method and system that embodies such embodiments as methods and systems related to sharing data and files, rendering and streaming data and files, and controlling graphical output among multiple devices.

DEFINTIONS/TERMINOLOGY

The instant invention is most clearly understood with reference to the following definitions/terminology:

In some instances, the term Game is used herein instead of App, Application, or Game/ App for brevity. However, all methods and system outlined here may apply to general applications as well.

SDK (software development kit) shall be understood to be referring to applications and the apps/games they control. The SDK can be used to create applications that turn PC, mobile devices, and even custom hardware into both controllers and games that can communicate bidirectionally with each other.

Full SDK shall be understood to allow developers to make custom controllers using any technology for the User Interface that they desire.

SFC (seriously fun controller) SDK shall be understood to include controllers of the present invention, and in one embodiment can include for example, a Brass Monkey SDK controller (http://playbrassmonkey.com/). Such SFC SDK controllers are made using a layout tool that generates a schema containing the layout information. The schemas can be registered with a central gaming system so that they can be immediately downloaded within a controller application of the present invention.

A controller application of the present invention, in one embodiment is a mobile app that can be used in connection with a central gaming website, and provides an interface for users to control and play games. Additionally the App provides interfaces for purchasing Virtual Currency as well as pay as you go payments.

Games made using both the Full SDK controller or the controller of the present invention can be hosted on a gaming portal website. The website can provide links to applications for the wireless endpoint device; games company info, developer info, online game info (Profiles, High Scores, Commenting), etc. The website can be displayed on the wireless endpoint device, or on other computing devices.

The term controller shall be understood to be referring to either a dedicated controller or a controller using the controller application of the present invention, as well as to a wireless endpoint device or endpoint device according to the present invention.

Host shall be understood to be referring to a computing system that is used to host a game or application and where the game or application can be executed on such a computing system.

The term LAN shall be understood to be referring to a Local Area Network, typically a wired and/or wireless Ethernet environment.

The phrase Self-Hosting shall be understood to be a term for storing data on a wireless endpoint device (User Info, Game Save Data, and

Game/Apps) and loading data from games, user profile, and game, etc. from the endpoint device to another endpoint device, to a central server, or to the host, so that the content is available on those other devices.

P2P hosting shall be understood to be a term for hosting a game by a wireless endpoint device, so that the game can be played by other wireless endpoint devices. The game can be executed wholly by a single endpoint device, or portions of the game can be executed and served by multiple endpoint devices.

A computer readable medium shall be understood to mean any article of manufacture that contains data that can be read by a computer or a carrier wave signal carrying data that can be read by a computer. Such non- transitory computer readable media includes but is not limited to magnetic media, such as a floppy disk, a flexible disk, a hard disk, reel-to-reel tape, cartridge tape, cassette tape or cards; optical media such as for example a DVD or CD-ROM and writeable compact disc; magneto -optical media in disc, tape or card form; or paper media, such as punched cards and paper tape. Such transitory computer readable media includes a carrier wave signal received through a network, wireless network or modem, including radio- frequency signals and infrared signals.

DETAILED DESCRIPTION

[00046] The present invention is directed to systems, device and methods for the formation of a network connection between a host device and an endpoint device and relates to electronic communication and methods for implementing the same. In addition, such systems and devices further feature embedding a server or server-like functionality in a mobile device or the endpoint device for communication across a network. As described further herein, providing the endpoint device or the mobile device with such server or server-like functionality provides a mechanism by which information, data and/or files can be cached or stored in the endpoint device or the mobile device and then later uploaded or communicated as needed over a network (e.g., a local network) to a host device and/or other endpoint devices. The server technology may be http / web server technology. However, other protocols and servers may be possible, such as WebSockets, Web RTC, Adobe Flash's RTMP, RTMPT, RTMFP streaming protocols, and Unity Web Player's UDP/TCP networking capabilities, etc.

[00047] Also, the figures and discussion herein concerning the systems and methods of the present invention illustrate the structure of the logic of the present invention as embodied in computer program software for execution on a computer, digital processor or microprocessor. Those skilled in the art will appreciate that the figures and discussion illustrate the structures of the computer program code elements, including logic circuits on an integrated circuit that function according to the present invention. As such, the present invention, in whole or part, is practiced in its essential embodiment(s) by a machine component that renders the program code elements in a form that instructs a digital processing apparatus (e.g., computer) to perform a sequence of function step(s) corresponding to those shown in the flow diagrams and/or description herein. [00048] Figs. 1 A - E and 2 are various illustrative views that illustrate systems and related methods for connecting host devices and endpoint devices over a network. Such systems and methods may be used to allow for communications between an application being executed on a mobile endpoint device (i.e., a controller device) and an application being executed within a web browser on a host device.

[00049] More particularly, Fig. 1 A illustrates a desktop install application

where the application is installed onto a desktop computer and a controller is communicatively coupled to the desktop so that user's actions create input events that are sent to the application. For example, if the user pressed a jump button on the controller, the game application would react to the command and have a character jump on the screen.

[00050] Fig. IB illustrates an input relaying environment in which inputs by the user using the controller are relayed via the internet to a server which is running the application on a web browser. This approach involves an internet round trip and performance also depends upon the performance of the internet.

[00051] Fig. 1C illustrates an Apple AirPlay type of approach in the controller that is coupled to a television via an AirPlay enabled device that inputs using an HDMI cable, to the television. In this approach, the game is calculated and rendered graphically and audio/visually locally on the controller which allows different graphics to be to be shown on the screen. The video/audio is then streamed to the AirPlay enabled device.

[00052] Fig. ID illustrates a Mobile to Mobile/Tablet environment in which a user can create input using a mobile device such as a phone for example and this input is sent to the application being executed on the Mobile/Tablet device. In this way, the mobile device controls execution of the application. In this environment, the Mobile/Tablet device acts as the host device and can send messages back to the mobile device or controller. In this environment the devices are running native code so they have access to more network protocols unlike browsers. [00053] Fig. IE illustrates a cloud gaming type of environment where the

games or applications are run in the cloud. This approach, like the input relaying environment approach has inputs by the user using the controller being relayed via the internet to the cloud and then back to a local computer which depicts the actions resulting from the inputs. Additionally the game play graphics and audio are rendered on the server and streamed as video to the local computer. As with the example in Fig. IB this approach involves an internet round trip and performance also depends upon the performance of the internet. See also the discussion herein regarding conventional systems and methods, where data and files are communicated from a server to a host device over a combination of networks (e.g., LAN or WAN) including the Internet and where time or latency delays can be introduced anywhere along the communication pathway and/or by devices involved with such communications .

[00054] In an embodiment of the present invention, the host device may

execute a game, or other application, within a web browser (i.e., a browser application). An endpoint device, in communication with the host device over a LAN, may execute another application, such as a controller application for playing the game. Other aspects or embodiments of the present invention are described hereinafter. While reference may be made herein to a game or game application, this shall not be considered as limiting as it is within the scope of the present invention for the application to be any application known by those skilled in the art or hereinafter developed, that can be executed on or using a web browser.

ESTABLISHING LOCAL NETWORK CONNECTION

[00055] Typically, devices on a network can discover other devices on the network by broadcasting their presence on the network, and monitoring broadcasts from other devices. However, if a browser application, for example a FLASH application, is running within a web browser on a machine (e.g., a computing device) connected to a network, the security mechanisms of the web browser may prevent the browser application from discovering or communicating with other devices connected to the network, even if those other devices are running applications that could otherwise communicate or interact with the browser application.

[00056] Also, web browsers often employ security schemes, sometimes

referred to as sandbox security, to ensure that browser applications cannot cause harm to a local machine. These security schemes may restrict a browser application from using local machine resources, such as the hard drive of the computer, I/O, or networking capabilities.

[00057] As described hereinafter; the system, device and method of the present invention embody a system and method for connecting such devices over a network. More particularly, such a system and method can be used to connect and facilitate communication between a browser application running within a web browser and an application executing on an endpoint device. For example, as shown in Fig. 2, a server device in communication with both the web browser and the controller application can send the private network address of the endpoint device to the browser application. Thereafter, a network connection can be established between the browser application and another application so that the browser application can communicate directly with the controller or another application to send and receive data, and thus create an exception to the web browser's security.

[00058] First described herein is a system and method that may be used to connect and facilitate communication between a browser application running within a web browser and an application executing on an endpoint device. As indicated herein, reference also shall be made to U.S. Patent Nos. 8, 019,867; 8,019,878; 8,024,469; 8,166,181; and 8,171,145, the teaching of all of which are incorporated herein by reference, for further details concerning such devices, systems and methods including details not described further herein.

[00059] Referring now to Fig. 3, there is provided a system and architecture 10 for controlling content within a web browser. The system may include a host device 12, which may execute browser application 14 within the web browser 16. [00060] Fig. 3 shows host device 12 as a desktop computer, however the host device 12 is not limited to desktop computers. For example, the host device may be a laptop computer, server computer, game console, mobile device, smart phone, internet television, home appliance, network- and processor- enabled remote control toy, internet enabled billboard, medical display device, iPad®, or any other device capable of running a browser application 14 within a web browser 16.

[00061] Fig. 3 also shows endpoint devices 18, 20. Endpoint device 18 may execute application 22, which can be an application configured to communicate with the browser application 14. Such an endpoint device may typically be a network-enabled wireless handheld device, such as a PDA or smart phone, and the application 22 is typically a mobile application.

However, the endpoint device also can be any type of device capable of running application 22 and connecting to private network 24, such as a laptop or desktop computer, a gaming console, a game controller, etc.

[00062] Endpoint device 20 may be another endpoint device within system 10.

Endpoint device 20 is depicted here in Fig. 3 merely to show that other devices may be included within the system 10 and connected to a private network 24. Although not shown, this endpoint device 20 may execute an application such as application 22, may execute other applications capable of communicating with browser application 14, or may execute applications capable of communicating with other browser applications, for example.

[00063] The host device 12 and endpoint devices 18, 20 may be connected to a private network 24. The private network 24 can typically be a local area network (LAN), but may also be other types of networks, including but not limited to, a wired LAN, one or more LAN segments, a wireless LAN, a wide area network, a Bluetooth® network, a USB network, etc. Accordingly, the connections 26, 28 may be wired or wireless connections and may employ any appropriate communication technologies and protocols to enable communication between the devices. [00064] Private network 24 also may comprise multiple networks coupled

together, multiple LAN segments, etc. For the sake of simplicity, this disclosure will describe the private network 24 as a wired/wireless Ethernet LAN, typical of internet enabled households and businesses. Wired/wireless routers, such as those provided by Linksys® and other companies, can provide such a network. However, one skilled in the art will recognize that the inventions herein may operate within various different types of network topologies including those hereinafter developed.

[00065] The private network 24 is shown as being separated from the public network 30 by a network interface device 32. The public network 30 may be any type of LAN or WAN. In this example, the public network 30 can represent a public network such as the internet.

[00066] The network interface device 32 can provide a bridge between a

private network 24 and the public network 30, and also can provide a security barrier between the private network 24 and the internet, such as for example, a firewall. Typical examples of a network interface device 32 are cable modems, DSL modems, Tl connections, and the like. Devices connected to the public network 30 (i.e., the internet) typically cannot penetrate the security barrier of a network interface device 32 to discover or communicate with devices on a private network 32. However, in an embodiment, the network interface device 32 may allow devices on the private network 24 to access the public network 30 so that they can browse the internet, download applications, and communicate with devices on the internet. Network interface device 32 may be a single network interface device, or may be multiple network interface devices that provide multiple access points between the private network 24 and the internet.

[00067] Devices connected to the private network 24 may receive a private network address, while devices connected to the public network 30 may receive public network addresses. A private network address is an address specific to the private network 24. Devices that receive a private network address from the private network 24 can typically communicate freely with other devices on the private network 24. However, devices on the public network 30 typically cannot discover or communicate with devices on the private network 24 having private network addresses because of the security mechanisms in the network interface device 32. A typical example is a home or business LAN, where devices on the LAN can communicate with each other, and can access the internet, but devices on the internet cannot discover or initiate communications with the devices on the LAN.

[00068] Because the network interface device 32 is connected to the public network 30, the network interface device may receive a public network address. A public network address is an address that can be accessed by devices on the public network 30 and/or the private network 24. Because, in this example, the public network 30 represents the internet, the public network address received by the network interface device is a public internet address, which can be accessed by other devices on the internet. The opposite may also be true.

[00069] In other words, devices on the public network 30 (i.e., internet devices) can "see" the network interface device 32 and its public network address, but they cannot "see" devices on the private network 24. Devices on the private network 24 can not only "see" the other devices on the private network 24 and access them using their private network addresses; they can also "see" devices on the public network 30 and can access them using their public network addresses.

[00070] Fig. 3 also shows a server 34 (e.g., registration server), which is

executing an address registration process 36. The server 34 can be any type of computing device or group of devices capable of executing such an address registration process 36. Typically, server 34 is a server computer, an internet server, a group of servers, a cluster of servers, or any configuration of server computers.

[00071] The system 10 shows a typical configuration for the inventions

described herein. However, such a system 10 is not the only architecture within which the inventions described herein can operation. One skilled in the art will recognize that various system architectures can support the systems, methods, and apparatuses described herein. Also, although not shown, any of the applications and processes can be run by any computing device within the system 10, or by any computing device capable or running the applications and processes. For example, the host device 12 may execute the address registration process 36 and/or application 22; the endpoint devices 18, 20 may execute web browser 16, browser application 14, and or the address registration process 36; and server 34 may execute the web browser 16, browser application 14, and/or application 22.

[00072] The systems and methods that can be used to connect and facilitate communication between a browser application running within a web browser and an application executing on an endpoint device may be implemented as hardware, software, or a combination of hardware and software. There is shown in Fig. 4, a block diagram of a typical processing architecture, which may execute software applications and processes. The computer processing device 200 is coupled to a display 202 for graphical output. The processor 204 is a computer processor capable of executing software. Typical examples are computer processors (such as Intel® or AMD® processors), ASICs, microprocessors, digital signal processors and the like. The processor 204 is coupled to memory 206, which is typically a volatile RAM memory for storing instructions and data while processor 204 executes.

[00073] The processor 204 also is coupled to storage device 208, which is a non-volatile storage medium, such as a hard drive, NVRAM, FLASH drive, tape drive, DVDROM, optical drive, other computer readable storage medium or similar device. The program 210 is one or more computer programs containing instructions, criterion and/or data, and is stored on the storage device 208. In a typical scenario, the processor 204 may load some or all of the instructions and/or data of one or more programs 210 into memory 206 for execution. The program 210 can be any computer program or process including, but not limited to web browser 16, browser application 14, address registration process 36, application 22, or any other computer application or process (e.g., operating system, email, word processing, etc.). The program 210 may include various instructions, criterion, code segments and subroutines, which, when loaded into the memory 206 and executed by the processor 204 cause the processor to perform various operations, some or all of which may effectuate the methods associated with the present inventions.

Although not shown, as is known to those skilled in the art, such a computer processing device 200 typically includes various forms of input and output (I/O). Such I/O may include network adapters, USB adapters,

Bluetooth radios, mice, keyboards, touchpads, displays, touch screens, LEDs, vibration devices, speakers, microphones, sensors, or any other input or output device for use with such a computer processing device 200.

There is shown in Fig. 5, a flowchart diagram that depicts a process for controlling content and/or establishing two - way communication with a web browser. Fig. 6 also shows an example of the communications between endpoint device 18, host device 12, and server 34. The process may be implemented as software executing on one or more devices. For example, the process may run on host device 12, endpoint devices 18, 20, server 34, or any combination thereof. The process also may be implemented on any other capable device or in any appropriate architecture.

Such a content/communication control process may operate to control content within a web browser and/or engage in two-way communication with an application running within a web browser. More specifically, such a process may facilitate connectivity and two-way communication between the browser application 14 and application 22. However, because a video game runs as a browser application (e.g. , a FLASH application) within the web browser 16, and is subject to the security restrictions of web browser, the controller application and the video game may not be able to discover and communicate with each other.

In order for the applications to communicate, a server 34 (and/or address registration process 36) may receive (Step 302) a communication 402 from application 22, which is running on endpoint device 18. In one embodiment, application 22 may send the communication 402 in order to register its addresses with the server 34. Accordingly, this communication 402 may include the private network address of endpoint device 18 (i.e., the private network address within private network 24), and also may include the public address of endpoint device 18. In this example, since network interface device 32 is the bridge between the private network 24 and public network 30, the public network address of endpoint device 18 may match, at least in part, the public network address of network interface device 32.

[00079] In some cases, the communication 402 also may contain an application

ID. For example, the application 22 may have an application ID that it sends to server 34 along with the public and private network addresses.

[00080] Browser application 14 (running within browser 16 on the host device

12) also may send a communication 404 to the server 34. Similarly, this communication 404 from browser application 14 may serve to register browser application 14 with the address registration process 36. The communication 404 may contain the public network address of the host device 12, and also may contain an application ID that uniquely identifies the browser application 14. In this discussed above, the public network address of host device 12 may match, in whole or in part, the public network address of the network interface device 32, because the network interface 32 provides the interface between the host device 12 and the public network 30. These communications 402, 404 from the browser application 14 and/or from application 22 also contains other information, such as the private network address of host device 12, other information about the host device 12, application data, user data, or any other type of information.

[00081] Also, the server 34 may receive communications from other endpoint devices and host devices (not shown). In one embodiment, the server 34 is an internet server that receives communications (including network addresses and application IDs) from devices connected to various private networks throughout cyberspace. The server 34 may record and catalog the information it receives from these various devices in, for example, a database or other data store.

[00082] Upon receipt of the communications 402,404, server 34 may determine

(Step 304) whether endpoint device 18 and host device 12 are connected to the same local network based upon the public network addresses within the communications. As described above, the public network address is typically associated with a public-facing device connected to the local network (e.g. , the network interface device 32). Therefore, in this example, when the server 34 receives communication 402 from endpoint device 18, the public network address may indicate that endpoint device 18 is connected to a private network 24 because the communication is being routed through the network interface device 32. Similarly, the public network address in communication 404 from the host device 12 may indicate that the host device 12 is connected to the private network 24 and that the communication 404 is being routed through the network interface device 32.

[00083] By comparing the public network address of various devices in this way, the server 34 can determine which devices are connected to the same private networks. The public network addresses may match, in whole or in part, or may otherwise be related. For example, the server 34 may determine that the two devices are connected to the same private network if the public network addresses of the two devices match. In another example, the server 34 may determine that the two devices are connected to the same private network if the public network addresses of the two devices have the same subnet address. The server 34 may also contain a lookup table or other device that provides information about which public network addresses are associated with various private networks. By utilizing the lookup table or other mechanism, the server 34 can determine whether the received public network addresses indicate that the devices are connected to the same private network.

[00084] The system 10 also may facilitate (Step 306) establishing a network connection between browser application 14 and the application 22 by providing the private network address of endpoint device 18 to the browser application 14 in a communication (Step 308). As discussed herein, the

browser application 14 and application 22 may not be able to connect or communicate with each other if the security mechanisms of the web browser 16 prevent such communication. However, the browser application 14 may be able to communicate directly with application 22 because the browser application 14 knows the private network address of endpoint device 18 through such communication. By providing the private network address of endpoint device 18 to browser application 14, the system may be able to modify the security policy (Step 312) of the web browser 16 to allow communication 408 between the browser application 14 and application 22.

[00085] In further embodiments, the address registration process 36 and/or server 34 provides the private network address of endpoint device 18 to browser application 14 only if the application 22 and browser application 14 are compatible applications. For example, if the address registration process determines (Step 310) that the browser application 14 is a game application and application 22 is a controller application for that game, then the address registration process 36 may determine that the application 22 and browser application 14 are compatible applications and send the private network address of endpoint device 18 to browser application 14, so that browser application 14 and application 22 can communicate with each other.

[00086] However, if browser application 14 and application 22 are not

compatible, then the address registration process may not send the private network address to browser application 14. For example, if the browser application 14 is a video game, but application 22 is an application for drawing pictures, then they are likely not compatible. In this case, there may be no need to send the private network address of endpoint device 18 to browser application 14 in order to initiate communication between browser application 14 and application 22.

[00087] In further embodiments, the address registration process 36 may

determine (Step 310) whether applications are compatible in various ways. For example, the address registration process 36 may compare the application IDs received from application 22 and browser application 14. If the application IDs indicate the applications are compatible, the address registration process may provide (Step 308) the private network address of endpoint device 18 to the browser application 14. For example, the application IDs may indicate that the applications are compatible if the application IDs match. Also, the application IDs may indicate compatibility if they are associated in some other way. For example, the address registration process 36 may have a lookup table or hash table of application IDs that links compatible application IDs.

[00088] The application IDs may be any unique identifier and may include information such as a code representing the application and/or its version number, the name of the application, etc. The opposite may also be true.

[00089] In further embodiments, the address registration process 36 may

provide (Step 308) the private network address to browser application 14 in various ways. In one embodiment, the browser application 14 may periodically poll the address registration process 36 and ask for the address. Upon receipt of the request, the address registration process 36 may send browser application 14 a list of the private network addresses of all endpoint devices that are running compatible applications and connected to the private network 24. In another embodiment, the address registration process 36 may send the private network address of an endpoint device to the browser application 14 whenever an endpoint device running a compatible application and connected to the private network 24, registers with the address registration process 36.

[00090] In yet another embodiment, the address registration process 36 may periodically broadcast lists of private network addresses, which can be retrieved by browser application 14 whenever appropriate. Other schemes of providing (Step 308) the private network addresses to browser application 14 may also be employed.

[00091] If the security of the web browser 16 prevents communication between the browser application 14 and application 22, then the system may modify (Step 312) the security policy of the web browser 16 to allow for such

communication there between. For example, endpoint device 18 may send (Step 314) a policy file 406 to the host device 12. Referring to both Figs. 5 and 6, in further embodiments, endpoint device 18 may send (Step 314) the policy file 406 to the host device 12 by serving the policy file through a web server.

[00092] Upon receipt of the private network address of endpoint device 18, the browser application 14 may access the web server executing on endpoint device 18 and download the policy file 406. Policy files are typically known in the art. In this example, the policy file 406 may contain a new security policy for the web browser 16 or changes to the security policy of the web browser that allow for communication between the browser application and application 22. When the browser application 14 receives policy file 406, it may apply the policy file to the security policy of the web browser 16 to allow for such communication.

[00093] In another embodiment, the security policy of web browser 16 may be modified (Step 312) through an HTML5 handshake. HTML5 handshakes are also typically known in the art. In this case, application 22 and the browser application 14 may perform the HTML5 handshake in order to modify the security profile of the web browser 16 and allow communication between browser application and application 22.

[00094] In some embodiments, the methods and systems may facilitate

communication between the browser application 14 and application 22. As discussed, the web browser 16 may include security mechanisms that prevent the browser application 14 from discovering or communicating with endpoint devices 18, 20 over the private network 24. To facilitate communications, application 22 may send the private network address of endpoint device 18 to one or more network-enabled devices.

[00095] In the example above, application 22 may send the public and/or private network address of endpoint device 18, and/or an application ID to the server 34. However, one skilled in the art will recognize that application 22 may send such information to any device, or may broadcast such information to any device, in order to facilitate communications between application 22 and the browser application 14. Similarly, browser application 14 may send the public network address of the host device 12, the private network address of host device 12, and/or an application ID to the server 34 in order to facilitate communication with application 22. However, one skilled in the art will recognize that the browser application 14 may send such information to any device, or may broadcast such information to any device, in order to facilitate communications between application 22 and the browser application.

[00096] In further embodiments, application 22 and/or the browser application

14 may establish socket connections in order to facilitate communications. As used herein, the term socket refers to any unidirectional or bidirectional inter- or intra-process data or communication flow over a network. The socket connections may be initiated by application 22, browser application 14, or both. The socket connections may be any continuous or intermittent communication connections between application 22 and the browser application 14 through the private network 24. Reference also shall be made to the issued US patents identified herein for further details of such socket connections.

[00097] In particular embodiments, a socket connection may be a TCP

connection. As known in the art, TCP is a reliable communication protocol. The TCP protocol provides features that limit packet and data loss between endpoints, as well as features that ensure packets are received and/or processed in a correct order. Accordingly, the browser application 14 and application 22 may send loss-sensitive data over a TCP socket connection. As an example, if the browser application 14 is a game and application 22 is a game controller application, loss-sensitive data may include control data such as button presses, command acknowledgements, control data, setup and initiation data, etc. In general, the data sent over the TCP socket connection may be any data that the application deems to be loss-sensitive. In other words, any data where the absence of receipt of the data on the receiving end of the socket could cause problems, inconsistencies, or bugs in the application may be deemed loss-sensitive data.

[00098] The socket connection also may be a UDP connection. As known in the art, UDP is an unreliable communication protocol and does not include mechanisms to ensure receipt of data. Accordingly, browser application 14 and application 22 may send loss-tolerant data over such a UDP socket connection. Loss tolerant data may include any type of data where the loss of at least some of the data will not cause problems, inconsistencies, or bugs in the application. Typical examples of loss tolerant data include streaming video, streaming audio, etc. These types of data are considered loss tolerant because, if some of the data are not received, the audio or video stream may not be significantly interrupted. In some embodiments, other types of data, such as accelerometer measurements from a smart phone, vibration commands for enabling a vibration device in a phone, etc., are considered loss-tolerant data. The opposite may also be true.

[00099] Application 22 and/or browser application 14 also may generate

network packets for transmission over the socket connection(s) by using pre- constructed packet headers and/or pre-constructed packet templates. The packet may include a body, which may contain a data payload, and a header, which may contain routing and other information concerning the transmission of the packet over the network. Such a header may include an indicator, which may identify whether the packet contains loss-sensitive traffic and should be routed over a particular socket, or contains loss-tolerant traffic and should be routed over another particular socket. The indicator may be any data type known in the art capable of indicating how the packet should be routed.

[000100] During development, an application developer may decide what types of application data are loss-tolerant and what types are loss-sensitive. In this case, the developer may determine which pre-constructed header to use for different types of data. Additionally/alternatively, the browser application 14 and application 22, or a development environment used to create browser application 14 and/or application 22, may provide defaults for particular data types. For example, browser application 14 and application 22 may, by

default, use a pre-constructed header for loss-tolerant data whenever they send streaming audio or streaming video data over the network to each other. One skilled in the art will recognize that various defaults may be employed so that particular types of data are routed over the appropriate type of socket connection.

[000101] Also, the appropriate routing of loss-tolerant data and loss-sensitive data over different socket connections may result in reduced latency and more efficient network communication between application 22 and the browser application 14.

[000102] Although the descriptions above provide examples with a single

endpoint device, a single host device, and a single server, one skilled in the art will recognize that any combination employing single or multiple devices are possible. Thus, any combination of single or multiple endpoint devices, single or multiple host devices, and/or single or multiple address registration server devices may fall within the scope of the invention. For example, multiple endpoint devices may connect to a single host device, a single endpoint device may connect to a single host device, single or multiple host and/or endpoint devices may communicate with single or multiple address registration server devices, etc. Other combinations are also within the scope of the invention.

[000103] Furthermore, the browser application 14 and application 22 may be compatible applications designed to communicate with each other. For example, the browser application 14 may be a game and application 22 may be a game controller application designed to be compatible with one another. Similarly, the browser application 14 and application 22 may be the same application, e.g. , they may be the same game running on different platforms and communicating, for example, in a head-to-head match.

Additionally/alternatively, browser application 14 and application 22 may be different applications that can communicate with each other over the network. DATA/APPLICATION CACHING

[000104] As indicated herein, the present invention is directed to systems, device and methods for the formation of a network connection between a host device and an endpoint device. Such systems and methods include configuring the host device and endpoint device so that a network connection can be established there between so as to also allow communications there between. In this way, the endpoint device can provide at least instructions, data and information to an application being executed on the host device, for example, in a web browser of the host device.

[000105] In addition, such systems and devices are further configured and

arranged so as to embed a server or server-like functionality in a mobile device or the endpoint device (e.g. , wireless endpoint device) for communication across a network. As described further herein, providing the end point device or the mobile device with such a server or server-like functionality provides a mechanism by which information, data and/or files can be cached or stored in the endpoint device or the mobile device and then later uploaded or communicated as needed over a network (e.g., a local network) to a host device and/or other endpoint devices. Such an endpoint device also can be configured and arranged, so that when it is in such communication with the host device over a network (e.g., LAN), to execute another application, it is a controller application for controlling the application (e.g., playing the game). The server technology may be http / web server technology. However, other protocols and servers may be possible, such as WebSockets, Web RTC, Adobe Flash's RTMP, RTMPT, RTMFP streaming protocols, and Unity Web Player's UDP/TCP networking capabilities, etc.

[000106] As described herein, when the browser application is being executed in a web browser of the host device, the security in the web browser can prevent the applications from communicating with each other. Thus and as described herein, in order to allow such communication, a server device can send the private network address of the endpoint device to the browser application. The browser application can thereafter communicate with the controller application to send and receive data, in order to create an exception to the web browser's security.

[000107] Once the exception is created, the browser application and the

controller application can establish one or more sockets over the LAN as described herein. The sockets can include a TCP socket and/or a UDP socket, and can carry loss-sensitive and/or loss tolerant data between the applications. In an embodiment, the data can be used to control content displayed by a screen coupled to the host device, play a game hosted by the host device, and the like.

[000108] In further aspects/embodiments of the present invention, in order to provide additional functionality, the endpoint device is configured and arranged so as to include server capabilities or the endpoint device may be a server. In further aspects/embodiments, the endpoint device can have at least some server-like functionality and may act as a server. As described in more detail below, such an endpoint device that includes server capabilities can be used to cache files and data for communication between the endpoint device and the host device, render and/or stream data such as audio and video for use and display by the browser application or by other endpoint devices, or control and coordinate graphical output of devices across the network that are in communication with the endpoint device and/or involved in the execution and control of the game or the browser application. An endpoint device with server functions can provide other features and advantages as well.

[000109] In more particular embodiments, the host device may be a laptop

computer, server computer, game console, mobile device, smart phone, internet television, home appliance, network- and processor- enabled remote control toy, internet enabled billboard, medical display device, iPad®, or any other device capable of running a browser application within a web browser. Such a host device may be coupled to the network or LAN by means of a wired connection or a wireless connection (e.g. , wired or wireless Ethernet connection). [0001 10] While reference is made here to an endpoint device, it should be

recognized that a plurality of endpoint devices may be coupled to the network or LAN (see e.g., Fig. 9). Such an endpoint device can be coupled to the network using wired or wireless techniques. In a more specific embodiment, the endpoint device may be a network-enabled wireless handheld device, such as a PDA or smart phone. However, the endpoint device also can be any type of device capable of running an application and connecting to a private network or LAN, such as a tablet, laptop or desktop computer, a gaming console, a game controller, etc.

[0001 1 1] The following describes the embodiments directed to file and data caching and sharing as contemplated for the present invention. Reference shall be made to Fig. 7 which is an illustrative block diagram view of a simplified system with certain aspects or features removed for clarity (e.g. , the server and related interconnects thereto for establishing a network connection are not shown). Reference also should be made to Fig. 8 which is another illustrative block diagram view of a system 100 according to the present invention also showing caching of data and files (e.g., application files).

[0001 12] In further embodiments, the endpoint device may act as a server to provide files and data to the host device, or to other devices connected to the LAN. For example, if the host device is executing a game application, the endpoint device can store and serve files and data related to the game application, so that the host device can retrieve the files from the mobile endpoint device rather than from the internet. Because the endpoint device and the host device may be connected over a LAN, downloading these files from the endpoint device may be faster than downloading the files over the internet from another server, the cloud, etc. Examples of files that can be downloaded from the endpoint device to the host device include, but are not limited to: images, video, audio, game data, and executable files, scripts, application files, etc. Application files can include .SWFs for Flash, .unity3d for Unity Web Player, and html/css/js for HTML5, etc. In some embodiments, the caching technique can be applied over many types of network transports techniques such as Smart Sockets that use a TCP and/or UDP connection to carry loss-sensitive and loss-tolerant data across the network. However, caching also can take place over an HTTP connection.

Configuring the endpoint device or mobile endpoint device so as to embody a server or server functionality has a number of advantageous effects. Applications and other files and data can be downloaded directly from the mobile endpoint device to the host device over the local network. It is expected that such downloading over the local network would result in less time and resources as compared to downloading over the internet from a server each time the user wishes to use the application or play the game. The devices also can become http proxies that cache all resources automatically, with minimal change to code. In some instances, media or other files also can be streamed from the mobile endpoint device to the host device without the need for a plugin application.

Referring now to Fig. 7, there is shown a part of a system 100 for the caching of data, files, images, video, audio, game data, and/or executable files, scripts, application files, user Profile data and other game related data such as Avatars, User Write-ups, Friend Lists, and the like. According to the caching technique of the present invention, at least some or all of the data, files and the like associated with a given application are stored on the endpoint device 1 10. In particular embodiments, the endpoint device is a mobile device such as a smart phone or a tablet. Also, in more specific embodiments a HTTP server is built into the mobile/tablet application running on the endpoint device 1 10.

Once the exception is created in the host device 120, the host device knows the local IP address of the endpoint device 1 10. Thus, the host device 120 can be configured and arranged so as to request the data (e.g., profile settings things like Avatar images/names or the like) from the endpoint device 1 10. For example, the host device 120 can forward an http GET request(s) to the endpoint device for the files/data. Such a communication is sent over the local network. As indicated above, the endpoint device 1 10 (e.g., mobile

device/Tablet) is configured and arranged so as to embody a server or server like functionality. In particular embodiments, an HTTP server is built into the application running on the endpoint device. Upon receipt of the request (GET request(s)), the endpoint device 1 10 determines if the requested files/data are being cached on the endpoint device. In particular embodiments, the files/data being requested are preferably stored in the endpoint device's storage medium (e.g., hard drive, solid state drive, etc.).

In additional embodiments, a dedicated controller application that would be loaded into the endpoint device 1 10 can be provided to a central app store, or app depository, with Flash, Unity, or Native apps fully preloaded. Also, in some embodiments, only the heavy resources or large files may be preloaded, then the app logic itself can be decoupled from the larger files so that it can be updated much faster. The app logic file thus can be uploaded at the time the application or game application is to be executed on the host device.

In the case such files/data are not stored on the endpoint device 1 10 when the request(s) is/are made, then the endpoint device undertakes a process of communicating with another endpoint device on the network (such as shown in Fig. 3) to obtain such information (files/data) or communicating with another source 140 located on a public network such as the internet on which such information is being stored. The requested information is then transmitted (e.g., pushed or pulled) to the endpoint device 1 10 from the other endpoint device or the public network source (e.g., Cloud). This may occur when the user first attempts to access the application or game. Alternatively, such downloading can occur at a different time such as when, for example, the user registers with the server 34 (Fig. 6)

In further embodiments, the endpoint device 1 10 can undertake a process of checking with the other endpoint devices or the public network source to determine if the files and/or data have been updated since such files and data were last downloaded to the endpoint device 1 10. If it is determined that the files/data have been updated, the process can include syncing the files/data stored on the endpoint device 110 with the updated files/data.

If the files/data being requested is already stored on the endpoint device 110 or after going through the process of obtaining or updating such files/data, the endpoint device sends a HTTP response or other appropriate response to the host device 120 and also sends the requested files/data.

Thereafter, the host device 120 can cause the application to begin execution of the application or game application (e.g., execution in the web browser) and the application or controller application being executed on the endpoint device can thereafter control the functioning of the application or game application being executed on the host device.

Referring now to Fig. 8 there is shown a system 100 according to the present invention and showing establishing a network connection between the host device and the endpoint device and caching of data and files (e.g., application files) in the endpoint device. In the illustrated embodiment, the host device 120 and endpoint device 110 undergo a process so that a network connection can be established between an application being executed in a web browser of the host device and an application being executed on the endpoint device.

In regards to the host device 120, a small size "bootstrap" application for JavaScript, Flash, or Unity is provided and used to register the host device with the registration server 34 on startup. Because of the security restrictions (i.e., sandboxing) the host device provides their public IP address to the registration server 34 at start-up. Similarly, the endpoint device 110 provides their public-IP address and their private IP address to the registration server 34. As described above, the registration server using the provided information as well as any other information it may have, determines if the host device 120 and the endpoint device 110 can be communicatively coupled and if yes, forwards the private network IP address of the endpoint device to the host device, more particularly to the bootstrap application on the host device. A list 112 of host devices may be provided to the user of the endpoint device and the user may select a particular host device and send a command to the registration server. The registration server may in turn relay this command to the host device telling the host device to open up a network connection to the controller application of the endpoint device.

After receiving the private network IP address for the endpoint device, the host device bootstrap application can now communicate with the endpoint device. In addition and as described herein, such communication can occur using any of a number of transports, such as sockets, web sockets polling, etc. In particular embodiments, the bootstrap application can ask for the URL for the files of the full application including the application itself in the case of Flash/Unity applications.

As indicated above in the discussion regarding Fig. 7, upon receiving the request from the host device, the endpoint device 110 determines if the requested files/data are being cached on the endpoint device such as, for example, by having the Flash, Unity, or Native applications fully preloaded. If not, the process continues with obtaining the requested information (files and/or data) from another endpoint device on the network or by

communicating with another source 140 located on a public network such as the internet on which such information is being stored. After obtaining the requested information, it is then transmitted to the host device.

Also, the process can proceed to determine if the files/data to be uploaded needs to be updated and if so, updating the files/data. If the files/data being requested is already stored on the endpoint device 110 or after going through the process of obtaining or updating such files/data, the endpoint device sends a HTTP response or other appropriate response to the host device 120 and also sends the requested files/data to the host device. More particularly, the URL(s) of the full application files/data is sent to the bootstrap application so that it can load them from the http server built into the controller application of the endpoint device. Reference shall be made to the discussion regarding Fig. 7 for additional details regarding this process not otherwise provided here.

As also described more fully in connection with Fig. 7; the host device 120 can forward an http GET request(s) to the endpoint device for the files/data and the endpoint device can provide a HTTP response to this request also sending files/data from the cache. See the discussion regarding Fig. 7, for further details regarding this particular process.

The full application is now appropriately loaded into the web browser running on the host device 120. In addition, the bootstrap application passes the local IP address of the controller application to this full application and preferably thereafter shuts itself down. Now that the full application is in operation or being executed, the process thereafter continues with the controller application providing appropriate inputs to the full application on the host device 120 so that the endpoint device 1 10 can thereafter control the functioning of the application or game application being executed on the host device.

In addition, after the full application is loaded, then the cache or storage medium of the endpoint device may be used to store data such as scores, profiles, etc. to the medium or to fetch such data from the storage medium that the application or game application decides to save or retrieve.

Thus, using the caching technique of the present invention, it is possible to store application files and data on the endpoint device 1 10 (e.g., mobile endpoint device). For Flash/Unity, the application is typically a single file (.flv or .unity3d) while for HTML5, for example, the application may consist of javascript, ess, json/data, and html pages, etc. In either case, these files can be stored on and served by the wireless endpoint device, so that the host device 120 or other devices (e.g., other endpoint devices) on the LAN can retrieve these files from the endpoint device without having to download them from the internet.

While caching a Flash/Unity application on the endpoint device 1 10, the .flv/.unity3d file is downloaded from a server 140 and cached to the endpoint device the first time a user attempts to execute the application. After the application is cached to the endpoint device 110, when the application is subsequently executed by the user, the application can be loaded directly from the endpoint device to the host device 120 over the LAN.

In an alternative embodiment, Flash/Unity apps are split up so that code/data is not loaded until it is needed. Getting even the initial application load sizes to be low can prove difficult. With an http server running on the endpoint device, large executable files can be more easily downloaded.

HTML5 games typically include multiple files. If an HTML5 game is designed so that all of its logic is run on the client, then its containing html page could be cached/streamed from the endpoint device into an iframe on a central server's website. The accompanying javascript, ess, images, and data files can also easily be cached and then loaded via http.

In this case the files needed to render the application can be compressed into one file before being transported to the endpoint device, and then decompressed on the endpoint device so that they are available for the web browser on the host computer to display it.

In further embodiments, the system is configured so as to include a plurality of endpoint devices 110. As discussed further herein these plurality of endpoint devices can share application and media files with each other over the LAN.

One of the issues with attempting to make Rich User Interfaces in Web Applications is that load times may become dramatically worse as more images or other heavy/larger files and resources are used, especially if the files are being downloaded over the internet. In particular, by caching the files on the endpoint device and by loading the website's ess, images, audio, video, etc. from the endpoint device's cache, load times may decrease dramatically. This performance advantage can allow for a richer graphics/audio than is typical of web games which usually trend towards using less

images/audio/video to reduce load times. In some instances, users can come to a website before having installed the mobile app, or they can have the app but not have it running when first opening the website. This means that wireless endpoint devices may not be available for LAN based loading. One approach to alleviate this may be to use a light weight interface when users are browsing the website without a wireless endpoint device. Once the wireless endpoint device is used, a richer User Interface can be provided.

User Profile data and other game related data can be cached to the mobile endpoint device. This data can include Avatars, User Write-ups, Friend Lists, etc. Because most Mobile Devices have a lot of memory, large files such as video recordings of game play can be cached, then

served/streamed to the host device and, for example, integrated back into the game. As an example, users may record their own victory or taunt videos, or take a picture of themselves that can be dynamically loaded into the game during play.

Saving and caching game state will also be possible using this technique. In many cases, it may not even be necessary to fully render the output on the host web browser. The data object being stored on the device can instead be delivered via the Brass Monkey local (LAN based) socket connection as a binary or serialized file, allowing the host application to utilize it.

APIs (application programming interfaces) may be used to facilitate all these caching techniques. Game code/source would be bundled and registered with central game servers just as they will be with Controller Schema/Code itself. User/Game data read/write APIs for in game and in Controller integration will have to be implemented as well. Also, most data will not just be transient and will need to be stored to the cloud and synchronized when users log into their account from other Devices. Example: A User can change their User Profile from a Gaming Portal Website. The changes can then be synchronized those changes to any Controller Devices they have. It also is expected that caching game or application content on an endpoint device(s) will reduce the number of downloads from a server. This will reduce strain on the server so that the server can be used for other purposes, such as hosting other files and data.

Also, if the game application is cached on a mobile endpoint device, the host device can provide the game without the need to download from the internet. This also can allow for game play even if no internet connection is present. An example of this would be a racing game that has local shared screen multiplayer or multiplayer that can be played without an internet connection. When a user attempts to play over the internet, they may receive a message telling them that they need an internet connection for certain features.

Also, it is within the scope of the present invention for recently played game screens, applications and other data such as web pages can be cached to the mobile endpoint device. Certain social features can be disabled such as Facebook integration and friend chat etc. though local chat can still be supported. Applications that do not require internet connectivity can work in this way.

A cached website, for example, can be served from a mobile endpoint device. In an embodiment, the page and connect code and the

JS/HTML/CSS/Images etc. could be cached and provided by the endpoint device.

PLURALITY OF ENDPOINT DEVICES

In further aspects/embodiments of the present invention, a plurality of endpoint devices 110 can share application and media files with each other over the LAN. The endpoint devices may copy the application and media files from each other in order to propagate the cache of application and media files onto additional endpoint devices. Propagating the cache of files will allow copies of the files and data to be retained by multiple endpoint devices. Then, when the additional endpoint devices connect to other host devices, the files can be shared directly over the LAN from the additional endpoint devices. This can also reduce the number of times that the files are downloaded from the internet, which can reduce the load on the internet server, and reduce the time needed to transfer the game files to the host device.

MULTIPLE DISPLAYS

[000148] According to another aspect or embodiment of the present invention, the endpoint device(s) 110 can be configured and arranged so as to be used as central servers for coordinating the display of games on multiple displays. In such an embodiment, server code for controlling and/or coordinating graphics displayed by multiple screens can be downloaded onto and executed by one or more endpoint devices. In addition, audio output from multiple devices can also be controlled by an endpoint device executing server code. This can allow for complex audio environments, such as interactive experiences in stadiums or highly technically implemented small venue shows.

[000149] By using an authoritative server approach a multitude of screens can be used allowing players to have their own dedicated screens to play against other players who may be sharing a screen or using their own as well. For example, an endpoint device, acting as a server, can control and/or dictate, by communicating over the LAN, the content that is displayed on the multiple screens. The endpoint device can, for example, control the content displayed on other endpoint devices, on a central screen, on multiple shared screens, etc. In some instances, all or part of the application graphics can be displayed on any or all of the screens.

[000150] Complex setups with video output broken up across a large grid of screens or in more unique configurations also are possible with an endpoint device acting as a central server. Many unique experiences can be created as well such as using some screens for ambient lighting. Screens can be coordinated using two different techniques. So long as each screen has a website loaded, the endpoint device(s) can initiate loading games into all coordinates screens at once. For example, the following techniques can also be used. In one technique, the game can be simulated/rendered on the controller, and then portions of the video output can be sent to respective screens to create a cohesive large video output. In the second technique, one or more controllers each can keep the clients in sync. In such architecture, the controllers can send each other commands to control screen output.

Alternatively, lock-step type server/client architectures can be used. The authoritative server approach might have discernible visual differences if the personal computer (PC) of some of the screens has different latency. The lock-step approach may require the game's front-end and simulation to be deterministic. Effectively, in the lock-step approach, screens are coordinated by issuing the same input messages to each and having the games only update their states once all of them have signaled that they've received the message.

Some examples of multiple screen scenarios that can be controlled and/or coordinated by one or more endpoint devices of the present invention acting as servers are illustrated in Fig. 9.

As indicated herein, the controllers or endpoint devices of the present invention can be used as servers to coordinate typical Client/Server

Multiplayer Game Architectures.

In bars and other public events there could be dedicated screens that are designed more for spectating than playing. An example would be in playing games where players do not want to be interrupted by replays, or where players wish to watch the game from a bird's eye view.

In further embodiments, alternate interfaces can be applied to other content as well, and is not limited only to games. An example is an interactive music experience Game/App. One screen could be for navigating available songs and queuing/requesting them, and playing the song or a portion thereof as a preview. Then, the other host's viewpoints could be used for showing music visualizations and outputting the actual audio, while yet another could be specialized for the disk jockey (DJ) to use the controller technology described herein for a Virtual DJ Interface.

VIDEO/AUDIO STREAMING Referring now to Figs. lOA-C, there is shown how video and audio can also be hosted/served/streamed from the endpoint device 110 to other devices. This can allow the host device 120, running the game, to retrieve the audio and video from the endpoint device rather than from the internet. This can also allow the endpoint device to render the video and audio, i.e., to create graphics and audio, that the game application can use during execution. For example, browsers often have weak support for 3D Graphic Rendering. WebGL is not fully supported in all Browsers and Adobe Flash's 3D support hasn't even exited beta. The endpoint device 110 can be used to render and/or stream video that may not otherwise be available through the browser.

Most mobile devices have the ability to render graphics in an offscreen buffer. Graphics rendered by this buffer can be served/streamed by the wireless endpoint device, to other devices over the LAN. For example, a game can be run on the endpoint device and drawn in the background, while the on-screen buffer displays other graphics on the screen. The other graphics can be, for example, graphics simulating a game controller.

The graphics rendered in the off-screen buffer can be streamed to a host device 120 which then displays the video in real time on its screen. In the present invention, one can stream/play video in a browser without any desktop installation. The technique of the present invention just requires an endpoint device 110 powerful enough to capture and/or render graphical data. In some instances, the game application can also be run and hosted by the endpoint device that is performing the graphics rendering.

The endpoint device 110 also can render and serve/stream audio data, or any other type of data that requires processing and/or streaming.

Browsers sometimes do not offer robust support for complex audio rendering. When complex audio rendering is needed, the endpoint device of the present invention can perform the rendering, and serve/stream the resulting audio to the host device and/or other wireless endpoint devices over the LAN.

Phone Peripherals such as the Mophi Pulse may add force feedback functionality. These peripherals, and like devices, can generate controller shaking based on certain ranges of sound coming out of the iOS Device's speakers. APIs for games according to the present invention can be provided to call back to the controller to rumble/shake based on actions in the game. For example: crashing an in-game car may make the endpoint device, which is being used to control the car, shake in response to the crash.

Various embodiments have been described herein and these embodiments include methods and systems related to sharing data and files, rendering and streaming data and files, and controlling graphical output among multiple devices. One skilled in the art will recognize that the various embodiments can each function and be implemented individually.

Alternatively, some or all of the various embodiments can function and be implemented in combination with each other, as shown in Fig. 11.

The present disclosure makes reference to various block diagrams and flowcharts. One skilled in the art will recognize that the order and

configuration of components in the block diagrams and steps in the flowcharts are not limitations and are provided for illustration only. Various appropriate configurations of components and devices, and various orders and sequences of operation may fall within the scope of the claims.

While the invention has been described in connection with the specific embodiments thereof, it will be understood that it is capable of further modification. Furthermore, this application is intended to cover any variations, uses, or adaptations of the invention, including such departures from the present disclosure as come within known or customary practice in the art to which the invention pertains.

Incorporation by Reference

All patents, published applications and other references disclosed or referred to herein are hereby expressly incorporated by reference in their entireties by reference,

Equivalents Those skilled in the art will recognize, or be able to ascertain using no more than routine experimentation, many equivalents of the specific embodiments of the invention described herein. Such equivalents are intended to be encompassed by the following claims.

Claims

What is claimed is:

1. A method for communicating over a network comprising: establishing a local network connection between an application executing on an endpoint device, and a browser application executing within a web browser on a host device;

downloading, by the endpoint device, from a server device, data for use by the browser application;

providing, by the endpoint device, the data for downloading over the local network, so that the browser application can retrieve the data from the endpoint device over the local network connection for subsequent use by the browser application.

2. A method as set forth in claim 1, wherein the browser application is a game and the application executing on the endpoint device is a controller application for playing the game.

3. A method as set forth in claim 1, wherein the data is an executable file that can be launched by the browser application.

4. A method as set forth in claim 1, wherein the data is application data that can be used by the browser application during execution.

5. A method as set forth in claim 1, wherein the step of storing includes rendering the data by the endpoint device.

6. A method as set forth in claim 1, wherein the data is used to coordinate and control graphical output from multiple devices in communication with the local network.

7. A method as set forth in claim 6, wherein the multiple devices comprise endpoint devices.

8. A method as set forth in claim 6, further comprising outputting the graphical output to one or more display devices.

9. A method as set forth in claim 8, wherein said outputting the graphical output is to a single display having one of multiple windows or a split screen type of display.

10. A method as set forth in claim 1, wherein the data includes one of video, audio or a combination thereof and the method further comprises the endpoint device streaming the respective one of the video, audio or the combination thereof to the host device and the host device causing the audio to be played or the video to be displayed.

11. A method as set forth in claim 10, further comprising the endpoint device rendering one of an audio signal or a video signal for serving or streaming to the host device.

12. A method of controlling a game, the method comprising:

establishing a local network connection between a controller application executing on an endpoint device, and a browser application executing within a web browser on a host device;

storing, in the endpoint device, data for use by the browser application;

retrieving the stored data, by the browser application, from the endpoint device over the local network, so that the browser application can subsequent use the retrieved data; and

executing, by a host device in communication with a computer network, a game application within the web browser.

13. The method of controlling the game of claim 12, further comprising the steps of:

sending data, by the controller application and the game application, to one another over the local network for controlling and playing the game application within the web browser.

14. The method of controlling the game of claim 12, wherein said establishing includes coupling a plurality of endpoint devices to the local network; wherein said storing includes storing, in one of the plurality of endpoint devices, data for use by the browser application; and wherein said method further includes downloading from said one endpoint device to the other of the plurality on endpoint devices the data for use by the browser application.

15. A method as set forth in claim 12, further comprising the step of coordinating and controlling graphical output and outputting the graphical output to one or more display devices.

16. A method as set forth in claim 15, wherein said outputting the graphical output is to a single display having one of multiple windows or a split screen type of display.

17. A method as set forth in claim 12, wherein the data includes one of video, audio or a combination thereof and the method further comprises the endpoint device streaming the respective one of the video, audio or the combination thereof to the host device and the host device causing the audio to be played or the video to be displayed.

18. A method as set forth in claim 17, further comprising the end point device rendering one of an audio signal or a video signal for serving or streaming to the host device.

19. A system for controlling an application within a web browser, the system comprising:

a host device configured to execute a first application within a web browser; an endpoint device configured to communicate with the host device over a local network and to execute a second application;

a communication channel established on a local network between the second application and the first application; wherein the endpoint device is configured and arranged to download data, from a server device, so that the data can be used by the first application; and

wherein the endpoint device is configured and arranged to transmit the downloaded the data to the browser application for subsequent use by the browser application.

20. The system as set forth in claim 19, further comprising:

a plurality of endpoint devices each being coupled to the local network; and wherein at least one of the plurality of endpoint devices is configured and arranged so as communicate the downloaded data to each of the other of the plurality of endpoint devices.

21. The system of as set forth in claim 19, further comprising one or more display devices; and wherein said second application is configured so as coordinate and control graphical output and outputting the graphical output to said one or more display devices.

22. A system as set forth in claim 21, wherein such outputting the graphical output is to a single display having one of multiple windows or a split screen type of display.

23. A system as set forth in claim 19, wherein the downloaded data includes one of video, audio or a combination thereof; and the second application is configured so the endpoint device streams the respective one of the video, audio or the combination thereof to the host device so that the host device plays back the audio or cause the video to be displayed.

24. The system as set forth in claim 23, wherein the second application is configured so as to render the one of an audio signal or a video signal for serving or streaming to the host device.