CN101449265A - Mobile global virtual browser with head mounted display for browsing and interacting with world wide web - Google Patents

Abstract

The present disclosure describes a new method for browsing and interacting with the World Wide Web and other users while on the move, comprising the following components: 1) a mobile communication device including but not limited to a BlackBerry, a personal digital assistant, or a smart phone; 2) specialized glasses or viewing surfaces including but not limited to LCD (liquid crystal display) glasses, 3D goggles, organic LED (light emitting diode) panels; 3) a data input device including but not limited to a data glove, a finger sensor, a data input pen, or a gesture device; 4) a mobile communication device headset; and 5) a third generation or more powerful wireless carrier, a voice over IP provider, or a wireless LAN technology such as WiFi or Wi-Max, or Bluetooth.

Description

Mobile global virtual browser with head mounted display for browsing and interacting with world wide web

technical field

The invention satisfies the requirement of providing fully interactive web browsing experience to mobile users. Specifically, this disclosure describes an interactive system and method that allows mobile network (web) users to browse, participate in online games, and communicate with the World Wide Web and other users, typically via new user interfaces for mobile devices utilizing large virtual color display screens. interact.

Background technique

Currently, mobile users must gaze at small LCD screens on mobile cellular devices to view web content. The systems and methods of the present disclosure expand the perception screen and enable users to experience rich auditory/visual network impact. The disclosed system is a mobile, global service using next-generation smartphones for playing online games and interacting with the World Wide Web (WWW). The system can be seen as an always-on, always-available WWW service that provides smartphone users with the ability to overcome the visual limitations of current smartphones by using liquid crystal display (LCD) glasses. Currently, cellular phone users must gaze at a small LCD screen to access and view pages that have been greatly reduced compared to the rich content of the World Wide Web. On these preliminary pages, there is only basic text. The small size of smartphone LCD screens is the main reason why current technology is limited by this weakening. That won't change for the foreseeable future, as manufacturers are reducing rather than increasing their hardware footprint. The disclosed system will allow users to see a virtual screen in front of them that is the same size as a large size television screen. Greatly increasing the perceived real estate of the screen allows for many possibilities, including: amazing gaming realism, deeply immersive web interaction, streaming movies, sports, and video, video conferencing, visual virtual digital technology , and many other possibilities. The system requires next-generation smartphones, LCD glasses with stereo speakers, finger pointing devices, data input pens, or data gloves, cellular headsets, and third-generation (3G) (or 4G) wireless carrier (wireless carrier) ), a Voice over IP (VoIP) provider, or if a cellular signal is not available or required, then online gaming requires a wireless LAN such as WiFi (within the same building) or Wi-Max (within the same city) ( LAN) technology.

Related prior art includes: World Intellectual Property Organization Patent WO2 005/054782 Al: Mobile Geographic Information Display System; U.S. Publication 20020015008: Computer System and Headset-Mounted Display Device; U.S. Publication 20030092384: Piconetwork Radiotelephone Interfaces for Dice Vehicles Pointless Vehicles Including and Wire ；美国公开20030160736：Portable Heads-up Display for Vehicle；美国公开20050096096：WirelessCommunications Systems for Masks or Helmets；美国专利4,636,866：Personal Liquid Crystal Image Display；美国专利5,509,048-RadioTransceiver with Interface Apparatus Which Visually Displays Informationand Method Therefor；美国专利5,556,224：Radio FrequencyCommunication Device Including Mirrored Surface；美国专利5,969,698：Manually Controllable Cursor and Control Panel in a VirtualImage；美国专利5,977,950：Manually Controllable Cursor in a VirtualImage；美国专利6,073,033：Portable Telephone with Integrated Heads-up Display and DataTerminal Functions ; US Patent 6,091,376: Mobile Telephone Equipment with Head-up Display; US Patent 6,091,546: Eyeglass Interface System; US Patent 6,697,721: Safety Devices for Use in M otor Vehicles; US Patent 6,747,611: Compact Optical System and Packaging for Head Mounted Display; and, US Patent 6,941,248: System for Operating and Observing Making Use of Mobile Equipment. None of these prior art discloses a unique combination of this system that allows mobile users to enjoy a full web experience.

Contents of the invention

The present invention is a novel system and method for browsing and interacting with the World Wide Web and other users while on the move, comprising the following components: 1) a mobile communication device including but not limited to a Blackberry, Personal Digital Assistant, tablet PC, or smart phone; 2 ) including but not limited to LCD (Liquid Crystal Display) glasses, three-dimensional eyepieces, organic LED (Light Emitting Diode) panels specially made glasses and/or viewing surfaces; 3) including but not limited to data gloves, data 4) mobile communication device headsets; and 5) third-generation or greater wireless carriers or voice-over-IP providers, and/or if a cellular signal is unavailable or not If required, a Wireless Local Area Network (LAN) technology such as WiFi (within the same building) or Wi-Max (within the same city) is required to play online games.

Currently, mobile users must gaze at small LCD screens on mobile cellular devices to view web content. The system and method of the present invention expand the perceived screen and enable users to experience rich auditory/visual network impact. In addition, currently users must fumble with uncomfortable small keypads to dial numbers, send text messages, navigate menus, and enter data. The system and method of the present invention provide a brand new interface to the user.

It is an object of the present invention to provide a system and method for accessing the World Wide Web while on the move, viewing and interacting with the Web or other users via a large virtual color screen and stereo sound.

Yet another object of the present invention is to allow users to enjoy rich audio-visuals of the World Wide Web without being limited by the small LCD screen of a mobile device.

Yet another object of the present invention is to provide a sign language command such as American Sign Language (American Sign Language) or other conventionally designed gesture signal language for interacting with and controlling the system.

Another object of the present invention is to provide a voice-controlled command language for interacting with and controlling the system.

Another object of the present invention is to provide a method for inputting, sending and receiving text messages or SMS messages bypassing the small keypad of the current handset, which is achieved by: allowing the use of gesture commands and language commands for phone dialing, and Accept call, decline call, change line, start meeting, alert when voicemail or text or SMS message is received and who sent it, including scheduled messages easily via gesture command or voice.

Yet another object of the present invention is to provide a screen overlay technology activated by voice or gestures that label each possible virtual screen option (hyperlink, cursor movement, selection, input, etc.) and enable the user to instantly select the option via voice, gesture, or eye focus.

It is yet another object of the present invention to provide a system and method for viewing streaming movies, sports, and video on a large virtual color screen with stereo sound while moving.

Yet another object of the present invention is to provide a system and method for playing online, real-time multiplayer games on a large virtual color screen with stereo sound while moving.

Yet another object of the present invention is to provide a system and method for videoconferencing on a large virtual color screen with stereo sound while moving, which can also provide mobile users with the ability to create and edit The ability to document and spreadsheet without a computer boosts mobile worker productivity.

Another object of the present invention is to provide a system and method for simultaneously experiencing audio-visual virtual holidays with geographically separated people on a large virtual color screen with stereo sound (three-dimensional graphics as much as possible) while moving, The system and method book a virtual vacation by viewing an on-screen map, pointing at the map to select, and ordering a package.

Yet another object of the present invention is to provide a system and method for viewing and listening to museum digital artwork through a large virtual color screen with stereo sound while moving.

Yet another object of the present invention is to provide a system and method for building and interacting with a stereoscopic virtual world and a social world via a large virtual color screen and stereo sound while moving.

It is yet another object of the present invention to provide a system and method that allows users to assemble and organize their virtual digital lives while on the move via a large virtual color screen and stereo sound, wherein human contact can be seen on the virtual screen Pull up and activate via voice, eye focus or gesture command.

Yet another object of the present invention is to provide a system and method for viewing GPS data via a large virtual color screen and stereo, which allows the user to view and listen to restaurants, traffic reports, current location, navigation directions for a desired destination and other accurate and detailed data.

Yet another object of the present invention is to provide a system and method for participating in a social group network that is not geographically constrained via a large virtual color screen with stereo artifact) to interact with virtual netizens in real time.

For a full understanding of the objects and nature of the present invention, reference should be made to the following detailed description taken in conjunction with the accompanying drawings.

Description of drawings

For a full understanding of the objects and nature of the present invention, reference should be made to the accompanying drawings, in which:

Figure 1 shows a general diagram of a conventional system with components and their inputs and outputs;

Figure 2 shows a typical user under a first generation system that uses a controller that is being worn;

Figure 3 shows a typical user in a preferred second generation system that replaces the controller with software and runs directly on the smartphone or mobile phone.

The figure reference numbers shown in the figures are as follows:

10-Smartphone or mobile device

12-Controller (microprocessor based)

14-Phone/controller interface

16 - Head Mounted Display with Stereo Speakers

18-Head Mounted Display/Controller Interface

20 - Headphones

22-Headphone/controller interface

24 - Data gloves or pointing devices

26-Data glove/controller interface

Detailed ways

An important aspect of the system is that it is mobile and does not require a computer to work (eg, no portable computer is required). In addition, the mobile device interface is a new user interface compared to the small LCD screen and outdated keypad of current mobile device interfaces, and is similar to Microsoft Corporation's Disk Operating System (DOS), which was developed in conjunction with another software called Windows. compared to the graphical user interface. Also, while the system is built around existing components that currently exist, it is integrated by hardware and software interfaces that can be implemented by those skilled in the art. Mobile users require relatively minimal additional equipment - the equipment required for visualization are sleek LCD glasses. To interact with the system, a data glove, a virtual mouse, or a data input pen can be used to form characters, numbers, etc., by using the input device in the air, to complete any form of text, which the system analyzes and translates to the screen/file text on to allow web browsing. Small headphones, such as headphones with earbuds (earbuds), are used for stereo sound. Online gaming by geographically separated players can be achieved with real-time feedback without the need for a huge gaming console such as PS3 or Xbox360. The best mobile devices (especially smartphones) are equipped with onboard RAM, Bluetooth, 8GB HDD, USB ports, and integrated GPS capability. All components need to have their own power supply, or be powered by the USB bus (or otherwise), in order to maintain mobility. Wireless LAN capability for retrieving system updates Internet etc. WiFi will allow players to interact within buildings or in the same room if an internet connection is not available. Wi-Max capability will allow players to play online games within the same city without a cellular connection.

As shown in the preferred embodiment of Fig. 3, in order to use the system, the user uses the following components: 1) a mobile communication device, including but not limited to a Blackberry, a personal data assistant, a tablet PC, or a smart phone (10), which is connected to the first Three generations or more powerful wireless carrier or voice over IP provider connected for wireless network access. Specifically, in this case, a Samsung SGH-i310 smartphone is attached to the individual to allow "hands-free" operation via one or more data gloves (24), however as is well known to those skilled in the art 2) a display device (16) incorporating special glasses or various viewing surfaces known to those skilled in the art, including but not limited to LCD (liquid crystal display) glasses, three-dimensional eyepieces, Or organic LED (light emitting diode) panels. In this case, i-glasses (i-glasses) (16) are mounted on the head, with LCD screens placed in front of the user's eyes; 3) data input devices (24), including but not limited to those known by those skilled in the art Known data gloves, finger sensors, data input pens, or other gesture devices. In the preferred embodiment shown, 5DT data glove(s) (24) are donned; and 4) a mobile communication device headset (20) compatible with the SGH-i310 smartphone (10), Put it on your ear.

In operation, the i-glasses video display (16) interfaces with the Samsung SGH-i310 smartphone (10) through the TV-OUT jack of the smartphone (10). The smartphone (10) device supports NTSC and PAL output signals through cables terminated with output cables. These cables connect to the i-glasses (16) and may require an adapter to go from composite video to the S-video input on the video glasses (16), as known to those skilled in the art.

The headset (20) is wirelessly connected via bluetooth network technology (22). Similarly, the data glove(s) (24) connect to the smartphone (10) via the data glove "hub" via Bluetooth technology.

In operation as shown in the alternative embodiment of Figure 1, the SGH-i310 smartphone (10) is used to connect to the Internet via a service provider, as usual. However, the image viewed by the user is a virtual color panel that "floats" in front of the person and appears as a large screen of considerable size. For example, at 13 feet the user perceives a 70" screen. This image is reproduced on the LCD screen of the display (16) through the action of parallax and transformation software known to those skilled in the art running on the controller or mobile device , the image appears as a large color screen. It is also possible to use a special mirror (half-silvered, for example) that can provide eye-tracking capabilities that will allow the user looking at a specific point on the virtual screen to make a selection. A cursor or The arrow appears on the screen, just like a normal computer screen looks. The TV OUT port of the SGH-i310 smart phone (10) drives the display device (16) and audio components. As known to those skilled in the art, the current video standard is used to drive the display device (16), while current audio format standards are used to drive the audio components.

Movement and positioning (gesture tracking) of a finger or hand with a data input device (24) facilitates movement in the browser. Motion and positioning are translated by the system to move a virtual cursor or pointer on the virtual screen (16). A software driver is used to move the cursor much like a computer mouse for pointing a personal computer. Spatial positioning is relative and may employ body sensors. First, the data input device (24) uses relative positioning so that the hand can be positioned in stereoscopic space and the system will translate the virtual screen cursor position to be within the real area of the virtual screen. Once the position stabilized for a short period, subsequent hand movements were tracked directly on the virtual screen. This gives the user the flexibility to place the data entry device (24) anywhere in three-dimensional space and start navigating. Some examples of gesture tracking hardware are: a magnetic loop attached to the finger (which is tracked by its magnetic field); a transceiver attached to the finger, hand or arm whose transmissions are captured by the controller (12) or mobile device (10) or a data glove device (24) that transmits its position and gesturing data to the controller (12) or mobile device (10). Additionally, body sensors can be strategically located and used for gesturing. Finger joints can also be made more convenient by using magnetic rings or data gloves (24).

The screen overlay help feature is activated by sign language commands or voice commands. First, the sign language commands use a subset of American Sign Language, but other sign language commands can be constructed from the ground-up as known to those skilled in the art. The system superimposes an intuitive marking scheme with visual cues to the user of all available options or selected alphanumeric, color, or symbolic data. The user visually inspects the screen (16) to determine which option they wish to activate or select. The user can point to the symbol to perform the action, or can speak their choice, which will work through the voice recognition subsystem. By using eye-tracking technology, such as using mirrors or lasers, it is also possible to make selections based on the focus of the user's vision.

As is known to those skilled in the art, a very large bandwidth is required to carry all the video and data components of the wireless signal. The system is designed to work with any bandwidth and will migrate in power as third generation wireless networks increase their bandwidth capacity or move to fourth generation networks. Voice, video, and data need to be carried wirelessly.

Mobile device computing power will be primarily determined by central processing unit (CPU) and storage constraints. Next generation ARM or mobile Pentium processors with 2 gigabytes of RAM can be used to drive computations in mobile devices (10). Specialized operations (graphics, speech recognition, spatial positioning, video encoding/decoding, gesture recognition) can be offloaded from the mobile device by encoding and burning into EPROM (Electrically Programmable Read-Only Memory). This hardware solution is faster than its software equivalent.

Alternative embodiments include the following: 1) A hardware controller device (12) as shown in Figures 1 and 2 is used as a central control point for interfacing via hardwired cables rather than running dedicated software on the smartphone (10) itself All equipment; 2) the head-mounted display (16) can be a three-dimensional eyepiece instead of LCD glasses; 3) the display (16) can be an organic light-emitting diode (OLED) surface; 4) the display (16) can be an advanced optical invisible glasses; 5) the display (16) can be a holographic or interference-based image; 6) the display (16) can be a human retina, which uses nanotechnology or other advanced biotechnology to transmit digital video signals to the optic nerve; 7) data The input device (24) can be a virtual mouse, pointing device, or body sensor instead of the data glove (24); 8) the system interconnection can be wired instead of wireless; 9) the sign language command input can be replaced by or Add voice recognition technology; 10) smart phone (10) devices can be replaced by tablet personal computers; 11) mobile cellular devices (10) can be replaced by satellite-based communication devices; and 12) gesture tracking can use lasers to detect eye movements and location.

In operation of the alternative embodiment shown in Figures 1 and 2, an interface cable (14) connects the mobile communication device (10) and the controller device (12). The controller (12) performs all operations to control the interface between the mobile communication device (10), display device (16), device headset (20), and data input device (24). A second cable (audio/video) (18) connects the display device (16) and the controller (12). A third cable (data) (26) connects the data input device (24) to the controller (12). In this alternative embodiment of the system, wireless technology (22) is used to connect the headset (20) with the controller device (12). All other connections will use wired connections. The preferred embodiment of the present invention employs a wireless local area network protocol, such as Bluetooth, WiFi, or Wi-Max as described above and shown in FIG. 3 .

The system components (SC) of this preferred embodiment include: 1) i-glasses video head-mounted display (16) (LCD glasses); 2) Samsung SGH-i310 smart phone (10) or similar device (Nokia N95, Samsung SCH-A990); 3) standard Bluetooth headset compatible with SGH-i310 (20); 4) 5DT Data Glove Super Series (two are required if virtual keypad will be used); and 5) 5DT Wireless for Data Glove Kit, but one skilled in the art can readily envision similar components for implementing the systems and methods disclosed herein.

The following are the specifications of the preferred embodiment system components described above, but other specifications for implementing the systems and methods disclosed herein can be readily envisioned by those skilled in the art:

SC1-i-Glasses Video Head Mounted Display(16)(LCD Glasses)

I-O Display Systems, Menlo Park, CA

a) 1.44 million pixels per display

b) The virtual image size is 70" at 13 feet

c) 24-bit color depth

d) PAL/NTSC combination or S-video (S-video) input: scaling to 800×600

e) Audio: Full Stereo

f) Weight: 7 oz

g) Power supply: battery/charger packaged with lithium polymer battery - 4 hours

SC2 - Samsung SGH-i310 smartphone (10) or similar

a) Windows Mobile 5 operating system

b) Intel Xscate PXA272 processor, running at 416Mhz

c) User accessible 128MB ROM

d) 64MB RAM

e) 8GB hard disk capacity

f) Interface:

Expansion: SDIO, microSD, TransFlash

Serial: RS-232, 115200 bit/s

USB: USB 2.0

Bluetooth: Bluetooth 2.0

g) TV OUT video output jack

h) Batteries - Li-ion

SC4-5DT Data Gloves (24) Super Series (Two if using the virtual keypad)

5DT Irvine, CA

a) 8-bit deflection resolution

b)USB1.1 interface

c) Onboard processor

d) Bluetooth wireless connectivity up to 20 meters

e) Left and right hand version

f) Based on optical dimension

g) Power supply through USB interface

h) Sampling rate minimum 75 Hz

5DT Wireless Kit for SC5-Data Gloves

5DT Irvine, CA

a) Using 2.4Ghz Bluetooth spread spectrum technology

b) belt-worn kit with battery pack

c) One unit sends data from two data gloves

d) Small form factor, 4.33 x 3.27 x 1.65 inches

e) light, 10.6 oz

The system can be configured as described in the following scenarios. Ralph came to Tokyo on business. Cindy is one final weekend away from surgery and wants to relax and see or talk to Ralph. At the appointed time, Cindy puts on her LCD glasses, puts on the sign language command gloves, and connects to the smartphone. She logs into her personal virtual universe. A virtual menu floats in front of her on the 36" virtual screen. One choice is "The Magic of Italy". She selects the option with finger movements interpreted by the glove, and a picture of Ralph appears. Since he has Logs in so he says hello - he's so excited about the virtual vacation that he can't wait. When Cindy logs in, he sees her picture. They talk about their day-to-day for a few minutes, then decide to go on their virtual vacation. Remember, Ralph is physically in Tokyo and Cindy is in the US.

A floating menu shows their possible destinations—Milan, Venice, Florence, Rome, Assini, Sorrento, Pompeii, or Capri. They choose Venice and travel by gondola at dusk. As they tour the canal, both see the same authentic and professional video footage of the journey. Thanks to the cellular connection, of course, they can talk to each other in real time, and they can discuss with each other what they've both seen. In essence, they felt together even though they were physically thousands of miles apart.

Later, Cindy wanted to go to the Vatican and walked through the Sistine Chapel. Ralph agrees. With a few sign language commands, a virtual menu of options appears on the virtual screen, and Cindy chooses a tour of the Vatican. Now the two are virtually wandering the Vatican and entering the Sistine Chapel. They saw the mural "Creation of Adam" on the ceiling. Ralph pauses the picture so they can discuss it. After praising it for five minutes, they changed the angle via multiple sign language commands. They even increased the magnification. The next stop is Rome and the Coliseum!

Although physically separated, the two enjoyed a trip to Italy, felt the joy of playmates, and conversed with each other through the techniques disclosed in this application, which created a wide bridge between them and allowed them to be together and share unique experiences.

As certain changes could be made in the interactive web browsing and online gaming system and method without departing from the scope of the inventions involved herein, all matter contained in the specification or shown in the drawings is to be regarded as examples explanatory and not limiting.

Claims (12)

1. system, be used for that the user browses when mobile, game on line and generally speaking with WWW and other user interactions, this system comprises:

Mobile communication equipment;

The head mounted display that communicates with described mobile communication equipment;

Data input device, using gesture produces the input that communicates with described mobile communication equipment;

The headphone that communicates with described mobile communication equipment;

Cellular carrier, ip voice provider or the wireless lan technology that described mobile communication equipment is connected with other users in WWW or the LAN such as WiFi, Wi-Max or bluetooth.

2. the system of claim 1, wherein, described mobile communication equipment is blackberry, blueberry, personal digital assistant, dull and stereotyped PC or smart phone.

3. the system of claim 1, wherein, described head mounted display is cover LCD (LCD) glasses, three-dimensional safety goggles, or organic LED (light emitting diode) panel.

4. the system of claim 1, wherein, described data input device is one or more data glove, finger sensor, data input pen or the equipment that postures.

5. the system of claim 1, wherein, described data input device uses eye movement to follow gesture or uses eye movement to replace gesture.

6. the system of claim 1, wherein, described data input device uses voice command to follow gesture or uses voice command to replace gesture.

7. system, be used for that the user browses when mobile, game on line and generally speaking with WWW and other user interactions, this system comprises:

Hardware control equipment;

The mobile communication equipment that communicates with described hardware control equipment;

The head mounted display that communicates with described hardware control equipment;

Data input device, using gesture produces the input that communicates with described hardware control equipment;

The headphone that communicates with described hardware control equipment; And

Cellular carrier, ip voice provider or the wireless lan technology that described mobile communication equipment is connected with other users in WWW or the LAN such as WiFi, Wi-Max or bluetooth.

8. the system of claim 7, wherein, described mobile communication equipment is blackberry, blueberry, personal digital assistant, dull and stereotyped PC or smart phone.

9. the system of claim 7, wherein, described head mounted display is cover LCD (LCD) glasses, three-dimensional safety goggles, or organic LED (light emitting diode) panel.

10. the system of claim 7, wherein, described data input device is one or more data glove, finger sensor, data input pen or the equipment that postures.

11. the system of claim 7, wherein, described data input device uses eye movement to follow gesture or uses eye movement to replace gesture.

12. the system of claim 7, wherein, described data input device uses voice command to follow gesture or uses voice command to replace gesture.

Images