CN103890836B - The bluetooth with power management or other wave points for head mounted display - Google Patents
The bluetooth with power management or other wave points for head mounted display Download PDFInfo
- Publication number
- CN103890836B CN103890836B CN201180050791.3A CN201180050791A CN103890836B CN 103890836 B CN103890836 B CN 103890836B CN 201180050791 A CN201180050791 A CN 201180050791A CN 103890836 B CN103890836 B CN 103890836B
- Authority
- CN
- China
- Prior art keywords
- user input
- processor
- state
- user
- peripheral hardwares
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000001755 vocal effect Effects 0.000 claims abstract description 12
- 230000002093 peripheral effect Effects 0.000 claims description 23
- 230000009471 action Effects 0.000 claims description 11
- 238000000034 method Methods 0.000 claims description 11
- 238000012545 processing Methods 0.000 claims description 10
- 230000000007 visual effect Effects 0.000 claims description 5
- 230000004886 head movement Effects 0.000 claims description 4
- 230000005236 sound signal Effects 0.000 claims description 4
- 238000004891 communication Methods 0.000 claims description 3
- 238000001514 detection method Methods 0.000 claims description 3
- 230000008569 process Effects 0.000 claims description 3
- 239000003607 modifier Substances 0.000 claims 1
- 230000006870 function Effects 0.000 description 21
- 230000000712 assembly Effects 0.000 description 4
- 238000000429 assembly Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000000284 resting effect Effects 0.000 description 3
- 230000004913 activation Effects 0.000 description 2
- 239000011469 building brick Substances 0.000 description 2
- 230000001413 cellular effect Effects 0.000 description 2
- 238000013519 translation Methods 0.000 description 2
- 230000014616 translation Effects 0.000 description 2
- 240000003604 Dillenia indica Species 0.000 description 1
- PEDCQBHIVMGVHV-UHFFFAOYSA-N Glycerine Chemical compound OCC(O)CO PEDCQBHIVMGVHV-UHFFFAOYSA-N 0.000 description 1
- 230000005355 Hall effect Effects 0.000 description 1
- HBBGRARXTFLTSG-UHFFFAOYSA-N Lithium ion Chemical compound [Li+] HBBGRARXTFLTSG-UHFFFAOYSA-N 0.000 description 1
- 244000131316 Panax pseudoginseng Species 0.000 description 1
- 235000005035 Panax pseudoginseng ssp. pseudoginseng Nutrition 0.000 description 1
- 235000003140 Panax quinquefolius Nutrition 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 235000008434 ginseng Nutrition 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 229910001416 lithium ion Inorganic materials 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012827 research and development Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/41—Structure of client; Structure of client peripherals
- H04N21/422—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]
- H04N21/42203—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS] sound input device, e.g. microphone
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
- G06F1/26—Power supply means, e.g. regulation thereof
- G06F1/32—Means for saving power
- G06F1/3203—Power management, i.e. event-based initiation of a power-saving mode
- G06F1/3206—Monitoring of events, devices or parameters that trigger a change in power modality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
- G06F1/26—Power supply means, e.g. regulation thereof
- G06F1/32—Means for saving power
- G06F1/3203—Power management, i.e. event-based initiation of a power-saving mode
- G06F1/3234—Power saving characterised by the action undertaken
- G06F1/325—Power saving in peripheral device
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
- G06F1/26—Power supply means, e.g. regulation thereof
- G06F1/32—Means for saving power
- G06F1/3203—Power management, i.e. event-based initiation of a power-saving mode
- G06F1/3234—Power saving characterised by the action undertaken
- G06F1/3287—Power saving characterised by the action undertaken by switching off individual functional units in the computer system
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/012—Head tracking input arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/0304—Detection arrangements using opto-electronic means
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/41—Structure of client; Structure of client peripherals
- H04N21/4104—Peripherals receiving signals from specially adapted client devices
- H04N21/4122—Peripherals receiving signals from specially adapted client devices additional display device, e.g. video projector
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/41—Structure of client; Structure of client peripherals
- H04N21/422—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]
- H04N21/4223—Cameras
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/439—Processing of audio elementary streams
- H04N21/4394—Processing of audio elementary streams involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/44008—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/443—OS processes, e.g. booting an STB, implementing a Java virtual machine in an STB or power management in an STB
- H04N21/4436—Power management, e.g. shutting down unused components of the receiver
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
- G02B2027/014—Head-up displays characterised by optical features comprising information/image processing systems
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0179—Display position adjusting means not related to the information to be displayed
- G02B2027/0187—Display position adjusting means not related to the information to be displayed slaved to motion of at least a part of the body of the user, e.g. head, eye
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Computer Hardware Design (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Optics & Photonics (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
A kind of head mounted display (HMD) system, the system include wireless front end, and the wireless front end explains verbal order and/or hand motion and/or body gesture, with optionally according only to needing to activate subsystem components to execute specific order.
Description
Cross-Reference to Related Applications
This application claims in the U.S. Provisional Application No. 61/384,419 of the submission on the 20th of September in 2010 and in 2010 9
The rights and interests of the U.S. Provisional Application No. 61/384,543 that the moon 20 was submitted to.By quoting the whole of above one or more application
Teaching combines here.
Technical field
Present disclosure is related to a kind of head mounted display (HMD) system or other video eyeglasses devices, and which has control
The wireless front end of the activation of other systems component.
Background technology
The miniscope of research and development can provide large format, high-resolution colour in very little shape factor in the recent period
Picture and video flowing.One purposes of this class display is head mounted display (HMD) device for being worn over user face or head,
Similar a pair of glasses or earphone.The electronic component being attached in such device has become extremely complex, and can wrap now
Include integrated data processor, wave point and other input equipments, such as head tracking accelerometer, video camera, speech recognition circuit
With software and other assemblies.
Content of the invention
In preferred environment, head mounted display (HMD) system includes high resolution micro display, (multiple) Mike
Wind and (multiple) speaker, multiple satellite interfaces (such as wave point), multiple radio network interfaces, multiple video cameras (visible ray,
Infrared ray etc.), multiple audio frequency apparatuses, multiple keyboards, multiple game controller devices and other peripheral hardwares.
Wave point (such as bluetooth, WiFi, honeycomb or other wave points) becomes the front end of the system and controls other
When system component is activated.When the HMD systems first switch on power supply, it can be for example similar to bluetooth earphone with least work
Can work, so as to provide only the function of speaker and mike.Paired device is (such as the compatible mobile phone of bluetooth (i.e.
Apple iPhone or Google Android smart phones)) can be used in the handsfree mode feeding audio session and quilt
Feeding audio session.In such a mode, the other functions of the HMD systems are maintained at park mode, until earphone identifies wake-up
Other systems component and the verbal order of function.
For example, when the blue tooth front end detects incoming data flow with specific audio-frequency assembly, the audio-frequency assembly is passed
To the speech identifying function for being presented as circuit or software.If the speech identifying function determines that the audio-frequency assembly is to needing video
The verbal order of the system of output, the front end will cause the embedded data processor to open any other peripheral equipment, such as
Video display unitss.When the video display unitss are no longer needed, the processor and video driver circuit are put back into sleep pattern.
In another example, the user can say the order of the more complicated function of needing the onboard processing device.For example,
The user can ask " pending " task list for reading today.The inventory can be by as run on a host computer
The application program of Microsoft Outlook is generated.Executing this order not only needs to wake up the flush bonding processor to access this
Ground mass-memory unit is so as to obtaining the To Do List information, in addition it is also necessary to generate the audio signal corresponding with the spoken language
Word to sound circuit.
In another example, the user can ask to check that being attached to web mail as Microsoft Word files disappears
Agenda file in breath.This may need to wake up the flush bonding processor, activate the local micro display, and and extranets
Connection set up by network, for example, pass through bluetooth, WLAN (WLAN) circuit and/or cellular modem circuit.Then, these
Circuit only will be energized at that time.Then, the flush bonding processor is contacted with main frame (such as personal computer), so as to retrieving and showing
Show the network information.
In order to support this function, the wireless front end has wherein contained form or other logics, it is executed
The function of voice to word, or posture is converted into word, then can also explain these voices or posture order and seek advice from one
Row need the component being powered to execute these oral or posture orders.
Description of the drawings
According to the explanation in greater detail below of the example embodiment of the present invention, the above will be apparent, such as at these
Show in accompanying drawing, wherein run through these different views, identical reference refers to identical part.These figures are not necessarily
It is to scale relative to, but shows emphatically embodiments of the invention.
Fig. 1 is head mounted display (HMD) system for including miniscope and other peripheral interfaces.
Fig. 2 is another perspective view, shows the HMD systems for being mounted with video camera peripheral hardware for being worn over user's head.
Fig. 3 is high layer function figure, shows the controller of the HMD and its host-processor as such as personal computer
Purposes.
Fig. 4 is the high-level block diagram of the component of the HMD component internals.
Fig. 5 is more detailed figure, shows the electronic building brick for using in certain embodiments.
Fig. 6 is high-level flowchart, shows that how the blue tooth front end works so as to controlling which as system controller
When his various HMD system components become an effective example.
Fig. 7 is that those of verbal order, the list of posture order or headwork order and the subsystem are activated
Part.
Specific embodiment
The explanation to example embodiment is presented herein below.
Fig. 1 shows that head mounted display (HMD) system 100, the system generally include framework 1000, band 1002, the back of the body
Portion 1004, speaker 1006, cantilever or arm 1008 and miniscope sub-component 1010.
HMD 100 can be that the Kopin Corporation based on the Taunton from Massachusetts are available
Golden-iTMHands-free communication and control device.Side (particularly relative with miniscope sub-component 1010 one of the device
Side) " hot shoe " peripheral port 1020 is may include, what which was as explained further below can provide corresponding connection to accessory, so as to
User can delete the various components for being attached to HMD 100.
Fig. 2 is the view of the HMD 100 for being worn over user's head, and wherein shooting machine part 1060 has been placed in port 1020
In.Video camera 1060 can include that Voice & Video is sensed and two abilities of record, and can be that " bullet is convex for example similar
In the little profile factor of wheel ".As will be explained further below, the user (can be headwork there is provided user command
Or voice command) controlling display 1010, the setting of video camera 1020 and execute other functions.
The HMD systems can mainly as or standalone personal computers or " smart phone " device and/or as main
The remotely control of machine personal computer or other devices plays a role.For this purpose it is proposed, it is various electronics to be included in the housing
Circuit, including such as will understand quickly microcomputer, one or more wave points, associated memorizer or other deposit
Storage device, position, action and/or attitude sensor.These camera positions, action and/or attitude sensor are used to track
The position of the head or hand of the user, action and/or posture, and the mike receives audio input command.
The wave point enables HMD systems 100 to be used as smart phone itself or be used as 200 (ginseng of Host computing device
See Fig. 3) remotely control.For example, main frame 200 can be laptop computer, mobile phone, RIM Blackberry, Apple
IPhone, Google Android or other smart phone devices, with similar with remotely control HMD system 100 itself or more
High computational complexity and similar or bigger network connectivty.Main frame 200 can be connected further to other networks, such as pass through
Wide area network (WAN) connection 210 is connected to the Internet.HMD systems 100 and main frame 200 pass through such as by Bluetooth link or wireless local
The appropriate wireless connection provided by net (WLAN) 150 is connected to each other.
As shown in figure 3, HMD systems 100 receive the input from the user, such as via the phonetic entry of mike, via
The hand exercise and/or the head for tracking circuit via accelerometer or other headworks of video camera or other optical pickocffs is moved
It is input into.By in system 100 software (and/or with main frame 200 in software cooperation) these content translations into device 100
And/or the order that 200 system of main frame is capable of identify that.If these contents are the orders for HMD systems 100, it can carry
They, and if they are the orders for main frame 200, they are sent to main frame by bluetooth and/or WiFi interfaces 150
200.Then, according to the operating system/application software of their own, main frame 200 explains that these translated orders are various so as to execute
Function.
There may be the order for obtaining and showing file in these orders.
These orders are probably to receive phone calls.
Other orders are probably retrieval from the information of the Internet and are shown, such as Email or webpage.
Other orders are probably in virtual monitor to select the visual field, then return selected screen to the remote equipment
Curtain.It is to be further understood that the very big breadth of virtual viewing area may with the operating system that runs on main frame 200 on
Or the application software of interior operation is associated, because the various voice commands and/or head of the display 1010 being supplied on HMD 100
Portion's action only have selected a part for the large-scale virtual viewing area in the visual field.
More letters of the function being able to carry out in response to detecting phonetic entry, action and/or posture with regard to HMD 100
Breath, is that on May 5th, 2010 submits to entitled " using action and the remotely control of the host application program of voice command
(Remote Control of Host Application Using Motion and Voice Commands) " (serial numbers
12/774,179, attorney 0717.2098-001) and in the entitled " by action, body of submission on 2 1st, 2011
The wireless hand-free formula with detachable accessory of body posture or voice command control calculates earphone (Wireless Hands-Free
Computing Headset with Detachable Accessories Controlled by Motion, Body
Gesture, or Vocal Commands) " (serial number 13/018,999, attorney 0717.2102-001) common
It is made that in pending U.S. Patent application that explanation, the full content of wherein each file are incorporated herein by reference.
HMD systems 100 can provide other functions to other accessories beyond video camera, such as extra speaker, extra
Video camera, light source and hot shoe 1020 or any other electronic building brick in self-powered battery can be put into.
Fig. 4 is the high-level block diagram of the simplification of the non-limiting example embodiment of HMD systems 100.These components are collected
Into in the various places to HMD shells as shown in Figure 1.HMD systems 100 can include eye container (eye pod) component
4000, the component includes above-mentioned miniscope 4010 and one or more mikes 4020.One or more speakers 4030
(item 1006 in referring to Fig. 1) is positioned adjacent in the housing receiver of user's ear.System electronics can be placed on the frame
Appropriate location (such as back 1004) on frame or in the framework, and including open media application platform (OMAP) processor
4110th, power/audio companion chip 4102, display driver 4104, head-tracker 4105, circuit board 4106 and wireless
LAN/ blue tooth interfaces 4108.Power supply, such as lithium ion battery 4200 is additionally provided with the housing.
Head-tracker circuit 4106 can be included using hall effect sensor, MIM diodes, accelerometer, gyroscope
And/or the circuit of converter detection headwork and posture, such as along or transverse movement and rotation around X-axis, Y-axis and Z axis
Posture.
HMD systems 100 can be so that from external input device, (such as wireless mouse, trace ball can pass through blue tooth interface
The keyboard of 4108 wireless connections) receives input.Software in WLAN/BT front ends 4108, OMAP 4100 and/or main frame can be used
In the gesture for explaining that the video camera is detected.Video camera plate 4060 can be with optionally provided video input.
OMAP processors may include CPU, and potentially include nonvolatile memory and/or read-only storage
The on-chip memory of device (ROM), such as random access memory (RAM).The OMAP can be by Texas Instruments,
Inc. sell and used the processor of Texas Instruments models OMAP 3530 of multimedia processor or more new edition
This.The OMAP is typically more powerful than WLAN/BT interface 4108, more power consumption processor.
In this example, and from the available 65950 power/audio companion chips of TPS of Texas Instruments to
This system provides audio frequency, USB, Keyboard Control and battery charging function.
WLAN/BT interfaces 4108 can be the circuit of model LBEE 1W8NEC interfaces or with similar or higher ability
The wireless module of renewal.
The display driver can be that the Kopin Corporation of the Westborough from Massachusetts can
The display driver of model KCD-A 910 for obtaining.
And from the available miniscopes 4010 of Kopin can include model 113LV, 152LV, 230LV, WQVGA or
The acceptable miniscope of other manufacturers.
The received microphone signal of process of NCS modules 4400, to provide speech identifying function and to audio frequency companion chip
4102 produce audio signal.
Fig. 5 is more detailed block diagram, shows the connectedness between the various assemblies of Fig. 4.
Can be more easily seen from this figure, WLAN/BT front ends 4108 can serve as UART interface.It is programmed to control
OMAP processors 4100 processed and the correspondingly energization of audio frequency and power supply companion chip 4102 or off-position, and it is applied to other
The electric power of peripheral function (such as video camera, display, WLAN modems etc.).
The flow chart that Fig. 6 be a diagram that several illustrative functions, wherein WLAN/ blue tooth front ends 4108 can be controlled to other
The energization of system component.Fig. 7 is sample table, and the table can be kept determining when unlatching/various devices of closing by front end 4108
Power supply, and should be referred in the discussion of Fig. 6 below.
In the first example, under state 6000, system component is probably initially in park mode, only enables bluetooth ear
The function (such as mike 4020 and speaker 4030) of machine.This shape can be entered when HMD systems 100 first switch on power supply
State 6000.Thus, for example in state 6100, calling in the event of incoming call, these audio inputs and output function are in state
6110 keep active (for example make mike 4020 in opening and enable speaker 4030).But in state 6120, OMAP
4100th, display 4010, video camera 4060 and other assemblies are maintained at park mode.During end of calling, process and return to state
6000.
In another example, from resting state 6000, the voicefrequency circuit can detect that specific user voice command, such as
" reading my To Do List ".From this state 6200, the OMAP is activated under state 6210.Then, the OMAP is in state
6220 can access local storage region (such as SDRAM memory) to access the text being associated with the local task list of user
Part, so that obtain the same day pending detailed task list.The To Do List data are in (the now To Do List word of state 6240
It is converted to voice) audio interface is returned from OMAP 4100, and and then heard in the speaker from earphone 100.Once
Complete this function and just enter state 6500, then now the system returns resting state 6000, now the system is only used as again
Bluetooth earphone plays a role, and waits another user command.
In another example, in state 6300, user says order and indicates that she wishes to check on www.NYTimes.com
New York Times homepages.When this order is received, the OMAP is activated in state 6310.WLAN interface is activated,
The OMAP is enable to send order (as combined described in Fig. 3) so as to obtaining New York Times masters to the primary processor
The HTML content of page.Miniscope 1010 is also enabled at a certain moment of entrance state 6330, it is contemplated that the user wishes aobvious
Show webpage.The html web page of New York Times.com is obtained in other ways can, if for example the earphone has in itself
Ability accesses the Internet by onboard bluetooth, WLAN circuitry (WiFi) or by aiding in cellular modem.Anyway, one
Denier obtains the content, then just which is formatted and display on the miniscope in state 6350.At certain for the moment
Carve, the user can send another order to disable the display (or the order may time-out), this moment again
Enter sleep mode 6000.
Therefore, when the bluetooth and/or WiFi front ends user voice command detect incoming audio stream, the bluetooth
And/or WiFi circuitry automatically wakens up the OMAP and other ancillary equipment, but only it is carried out the order actually those equipment necessary.
Therefore, the bluetooth and/or WiFi front ends retain information such as in the form of form or data base entries, indicate which ancillary equipment
And/or OMAP (if any) needs to be powered to executing each in many different user orders.
In another example (referring to Fig. 7), the video camera can detect the gesture of user, and provide input to right
The energization of the specific components of system 100.In this case, these audio-frequency assemblies are enabled first, but detect verbal order with
Read reconciliation to loosen one's grip gesture by activation at least video camera 4060 and OMAP 4100.
In another example, the user can be used as input head movement.In such a mode, for example when the user just
When using head movement translation, scaling or otherwise navigation large format virtuality viewing area, may there was only headwork
Tracking 4106 and display are the associated components being activated.
So, the function of the HMD systems is the order control by appropriate verbal order, posture order or come host computer
System, and explained by the bluetooth and/or WiFi interfaces, according only to the part for needing wakeup subsystem electronic component.
After the activity is completed, the HMD systems can be returned to sleep pattern by another verbal order or time-out.Now, the bluetooth
And/or WiFi circuitry return to resting state wait external contact signal, such as call, another voice command or pass through WLAN
Computer interface order.
The teaching of all patents cited herein, published application and list of references all passes through to quote is tied with entire contents
Close here.
Although the present invention, ordinary skill people is specifically had shown and described by reference to its example embodiment
Member can be it should be appreciated that in the case of without departing from the scope of the present invention included by claims, wherein
Various changes are carried out in terms of form and details.
Claims (17)
1. one kind is used for the method for controlling head mounted display (HMD) system, and the system includes miniscope, user input
Equipment, first processor and two or more other peripheral hardwares, methods described include:
First state is entered by only enabling the first processor and the user input device;
Detection user input;
The user input is construed to verbal order or posture order using the first processor;And
Based on the verbal order or the posture order, sent by the first processor enable described two or more
Selected peripheral hardware in other peripheral hardwares order and the first processor send disable other peripheral hardwares order and enter second
State;
Wherein described two or more other peripheral hardwares include that second processing device, the second processing device are configured to relative to institute
State first processor more power consumption.
2. method according to claim 1, wherein described two or more other peripheral hardwares are selected from the group, and described group by wide
Domain network interface, the second processing device, display, video camera and headwork tracker are constituted.
3. method according to claim 1, wherein described user input is the verbal order for accessing file, and
And apply electric power to the second processing device and the miniscope in second state, then show the file.
4. method according to claim 3, wherein described user input is the verbal order for accessing webpage, and
And second state further makes wave point be connectable to network to obtain the webpage.
5. method according to claim 1, wherein described two or more other peripheral hardwares are included in the first state
The video camera being activated and the second processing device, the user input are gestures, and the second processing device is entering institute
The gesture is explained before stating the second state.
6. method according to claim 1, wherein described two or more other peripheral hardwares include wave point, and institute
State the second state and the user input is sent to by main frame by the wave point.
7. method according to claim 1, wherein described user input is the one side for manipulating the display
Request, and information on display described in second status modifier is presented.
8. a kind of Portable earphone device, including:
Miniscope, for showing visual information;
User input device, there is provided user input;
Two or more other peripheral hardwares, there is provided one or more peripheral hardware input signals, one or more of peripheral hardware input letters
The input of number indication sensor or the user action from user and/or in the sound input of the user at least one
Individual, wherein described two or more other outer to be located at first state disabled;
First processor, is configured to for the user input to translate into one or more orders for detecting, only described first
Processor and the user input device are activated in the first state;And
Processor controller, for being enabled and/or being disabled described based on one or more of command selections for detecting
One or more peripheral hardwares in miniscope and described two or more other peripheral hardwares;
Wherein described two or more other peripheral hardwares include that second processing device, the second processing device are configured to relative to institute
State first processor more power consumption.
9. device according to claim 8, wherein one or more of peripheral hardwares include:
One or more mikes, for receiving audio signal from the user, and
Wherein described first processor is further included:
Speech recognition device, for processing audio signal to detect voice command.
10. device according to claim 8, wherein one or more of peripheral hardwares are motion detectors, and described dynamic
Two or more action inputs are provided as detector, described two or more action inputs are indicated along two or more axles
The user action.
11. devices according to claim 10, wherein described motion detector be for detect the hand of the user and/
Or the video camera of body gesture motion.
12. devices according to claim 10, wherein described motion detection are the head movements for detecting the user
Head movement tracking equipment.
13. devices according to claim 8, wherein described two or more other peripheral hardwares further include that communication connects
Mouthful, the communication interface provides one or more wireless between the Portable earphone device and another data handling equipment
Link.
14. devices according to claim 8, wherein described user input are described to control by first processor process
The many aspects of the presentation of visual information on miniscope.
15. devices according to claim 14, wherein described user input control the visual field.
16. devices according to claim 15, wherein user input control zoom factor, shift factor or ratio because
Son.
17. devices according to claim 14, wherein described user input select hyperlink item in web displaying.
Applications Claiming Priority (5)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US38454310P | 2010-09-20 | 2010-09-20 | |
US38441910P | 2010-09-20 | 2010-09-20 | |
US61/384,543 | 2010-09-20 | ||
US61/384,419 | 2010-09-20 | ||
PCT/US2011/051760 WO2012040030A2 (en) | 2010-09-20 | 2011-09-15 | Bluetooth or other wireless interface with power management for head mounted display |
Publications (2)
Publication Number | Publication Date |
---|---|
CN103890836A CN103890836A (en) | 2014-06-25 |
CN103890836B true CN103890836B (en) | 2017-03-15 |
Family
ID=45874290
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201180050791.3A Active CN103890836B (en) | 2010-09-20 | 2011-09-15 | The bluetooth with power management or other wave points for head mounted display |
Country Status (4)
Country | Link |
---|---|
EP (1) | EP2617202A4 (en) |
JP (1) | JP5974008B2 (en) |
CN (1) | CN103890836B (en) |
WO (1) | WO2012040030A2 (en) |
Families Citing this family (64)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9952664B2 (en) | 2014-01-21 | 2018-04-24 | Osterhout Group, Inc. | Eye imaging in head worn computing |
US9965681B2 (en) | 2008-12-16 | 2018-05-08 | Osterhout Group, Inc. | Eye imaging in head worn computing |
US9400390B2 (en) | 2014-01-24 | 2016-07-26 | Osterhout Group, Inc. | Peripheral lighting for head worn computing |
US9298007B2 (en) | 2014-01-21 | 2016-03-29 | Osterhout Group, Inc. | Eye imaging in head worn computing |
US9229233B2 (en) | 2014-02-11 | 2016-01-05 | Osterhout Group, Inc. | Micro Doppler presentations in head worn computing |
US20150205111A1 (en) | 2014-01-21 | 2015-07-23 | Osterhout Group, Inc. | Optical configurations for head worn computing |
US9715112B2 (en) | 2014-01-21 | 2017-07-25 | Osterhout Group, Inc. | Suppression of stray light in head worn computing |
US9851787B2 (en) | 2012-11-29 | 2017-12-26 | Microsoft Technology Licensing, Llc | Display resource management |
US20140282273A1 (en) * | 2013-03-15 | 2014-09-18 | Glen J. Anderson | System and method for assigning voice and gesture command areas |
US9810906B2 (en) | 2014-06-17 | 2017-11-07 | Osterhout Group, Inc. | External user interface for head worn computing |
US9671613B2 (en) | 2014-09-26 | 2017-06-06 | Osterhout Group, Inc. | See-through computer display systems |
US9575321B2 (en) | 2014-06-09 | 2017-02-21 | Osterhout Group, Inc. | Content presentation in head worn computing |
US10254856B2 (en) | 2014-01-17 | 2019-04-09 | Osterhout Group, Inc. | External user interface for head worn computing |
US9829707B2 (en) | 2014-08-12 | 2017-11-28 | Osterhout Group, Inc. | Measuring content brightness in head worn computing |
US11103122B2 (en) | 2014-07-15 | 2021-08-31 | Mentor Acquisition One, Llc | Content presentation in head worn computing |
US20160019715A1 (en) | 2014-07-15 | 2016-01-21 | Osterhout Group, Inc. | Content presentation in head worn computing |
US9529195B2 (en) | 2014-01-21 | 2016-12-27 | Osterhout Group, Inc. | See-through computer display systems |
US10649220B2 (en) | 2014-06-09 | 2020-05-12 | Mentor Acquisition One, Llc | Content presentation in head worn computing |
US10684687B2 (en) | 2014-12-03 | 2020-06-16 | Mentor Acquisition One, Llc | See-through computer display systems |
US9448409B2 (en) | 2014-11-26 | 2016-09-20 | Osterhout Group, Inc. | See-through computer display systems |
US9299194B2 (en) | 2014-02-14 | 2016-03-29 | Osterhout Group, Inc. | Secure sharing in head worn computing |
US9746686B2 (en) | 2014-05-19 | 2017-08-29 | Osterhout Group, Inc. | Content position calibration in head worn computing |
US20150277118A1 (en) | 2014-03-28 | 2015-10-01 | Osterhout Group, Inc. | Sensor dependent content position in head worn computing |
US10191279B2 (en) | 2014-03-17 | 2019-01-29 | Osterhout Group, Inc. | Eye imaging in head worn computing |
US11227294B2 (en) | 2014-04-03 | 2022-01-18 | Mentor Acquisition One, Llc | Sight information collection in head worn computing |
US9841599B2 (en) | 2014-06-05 | 2017-12-12 | Osterhout Group, Inc. | Optical configurations for head-worn see-through displays |
US9939934B2 (en) | 2014-01-17 | 2018-04-10 | Osterhout Group, Inc. | External user interface for head worn computing |
US9594246B2 (en) | 2014-01-21 | 2017-03-14 | Osterhout Group, Inc. | See-through computer display systems |
US20150205135A1 (en) | 2014-01-21 | 2015-07-23 | Osterhout Group, Inc. | See-through computer display systems |
US9529199B2 (en) | 2014-01-21 | 2016-12-27 | Osterhout Group, Inc. | See-through computer display systems |
US11737666B2 (en) | 2014-01-21 | 2023-08-29 | Mentor Acquisition One, Llc | Eye imaging in head worn computing |
US11892644B2 (en) | 2014-01-21 | 2024-02-06 | Mentor Acquisition One, Llc | See-through computer display systems |
US9532714B2 (en) | 2014-01-21 | 2017-01-03 | Osterhout Group, Inc. | Eye imaging in head worn computing |
US9753288B2 (en) | 2014-01-21 | 2017-09-05 | Osterhout Group, Inc. | See-through computer display systems |
US9494800B2 (en) | 2014-01-21 | 2016-11-15 | Osterhout Group, Inc. | See-through computer display systems |
US11487110B2 (en) | 2014-01-21 | 2022-11-01 | Mentor Acquisition One, Llc | Eye imaging in head worn computing |
US9651784B2 (en) | 2014-01-21 | 2017-05-16 | Osterhout Group, Inc. | See-through computer display systems |
US9740280B2 (en) | 2014-01-21 | 2017-08-22 | Osterhout Group, Inc. | Eye imaging in head worn computing |
US11669163B2 (en) | 2014-01-21 | 2023-06-06 | Mentor Acquisition One, Llc | Eye glint imaging in see-through computer display systems |
US9836122B2 (en) | 2014-01-21 | 2017-12-05 | Osterhout Group, Inc. | Eye glint imaging in see-through computer display systems |
US9766463B2 (en) | 2014-01-21 | 2017-09-19 | Osterhout Group, Inc. | See-through computer display systems |
US9846308B2 (en) | 2014-01-24 | 2017-12-19 | Osterhout Group, Inc. | Haptic systems for head-worn computers |
US9401540B2 (en) | 2014-02-11 | 2016-07-26 | Osterhout Group, Inc. | Spatial location presentation in head worn computing |
US20150241964A1 (en) | 2014-02-11 | 2015-08-27 | Osterhout Group, Inc. | Eye imaging in head worn computing |
US20160187651A1 (en) | 2014-03-28 | 2016-06-30 | Osterhout Group, Inc. | Safety for a vehicle operator with an hmd |
US9672210B2 (en) | 2014-04-25 | 2017-06-06 | Osterhout Group, Inc. | Language translation with head-worn computing |
US10853589B2 (en) | 2014-04-25 | 2020-12-01 | Mentor Acquisition One, Llc | Language translation with head-worn computing |
US9651787B2 (en) | 2014-04-25 | 2017-05-16 | Osterhout Group, Inc. | Speaker assembly for headworn computer |
US9423842B2 (en) | 2014-09-18 | 2016-08-23 | Osterhout Group, Inc. | Thermal management for head-worn computer |
WO2015179877A2 (en) * | 2014-05-19 | 2015-11-26 | Osterhout Group, Inc. | External user interface for head worn computing |
US10663740B2 (en) | 2014-06-09 | 2020-05-26 | Mentor Acquisition One, Llc | Content presentation in head worn computing |
US9684172B2 (en) | 2014-12-03 | 2017-06-20 | Osterhout Group, Inc. | Head worn computer display systems |
JP6516464B2 (en) * | 2014-12-15 | 2019-05-22 | Kddi株式会社 | Wearable search system |
USD751552S1 (en) | 2014-12-31 | 2016-03-15 | Osterhout Group, Inc. | Computer glasses |
USD753114S1 (en) | 2015-01-05 | 2016-04-05 | Osterhout Group, Inc. | Air mouse |
CN104615243A (en) * | 2015-01-15 | 2015-05-13 | 深圳市掌网立体时代视讯技术有限公司 | Head-wearable type multi-channel interaction system and multi-channel interaction method |
US9933995B2 (en) * | 2015-01-25 | 2018-04-03 | Harman International Industries, Incorporated | Headphones with integral image display |
US20160239985A1 (en) | 2015-02-17 | 2016-08-18 | Osterhout Group, Inc. | See-through computer display systems |
ES2835877T3 (en) * | 2015-06-09 | 2021-06-23 | Nokia Technologies Oy | Transition of a virtual reality headset from operation in a first power mode to operation in a second power mode |
US9298283B1 (en) | 2015-09-10 | 2016-03-29 | Connectivity Labs Inc. | Sedentary virtual reality method and systems |
CN105797366A (en) * | 2016-03-25 | 2016-07-27 | 中国传媒大学 | Head-wearing type interactive audio game terminal based on sound source location |
DE102016207009A1 (en) * | 2016-04-26 | 2017-10-26 | Krones Aktiengesellschaft | Operating system for a machine of the food industry, in particular the beverage industry |
TWI599966B (en) * | 2016-05-10 | 2017-09-21 | H P B Optoelectronic Co Ltd | Gesture control modular system |
USD864959S1 (en) | 2017-01-04 | 2019-10-29 | Mentor Acquisition One, Llc | Computer glasses |
Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101506868A (en) * | 2006-09-08 | 2009-08-12 | 索尼株式会社 | Display device and display method |
Family Cites Families (18)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH10312631A (en) * | 1997-05-14 | 1998-11-24 | Nec Niigata Ltd | Sound reproducing device and information processing device |
US6408257B1 (en) * | 1999-08-31 | 2002-06-18 | Xerox Corporation | Augmented-reality display method and system |
WO2001035390A1 (en) * | 1999-11-09 | 2001-05-17 | Koninklijke Philips Electronics N.V. | Speech recognition method for activating a hyperlink of an internet page |
US7899915B2 (en) * | 2002-05-10 | 2011-03-01 | Richard Reisman | Method and apparatus for browsing using multiple coordinated device sets |
JP3988632B2 (en) | 2002-11-28 | 2007-10-10 | 日本電気株式会社 | Control method of glasses-type display |
JP2004349915A (en) * | 2003-05-21 | 2004-12-09 | Matsushita Electric Ind Co Ltd | Remote control unit |
KR100594117B1 (en) * | 2004-09-20 | 2006-06-28 | 삼성전자주식회사 | Apparatus and method for inputting key using biosignal in HMD information terminal |
JP4364861B2 (en) * | 2005-11-01 | 2009-11-18 | 株式会社ソニー・コンピュータエンタテインメント | Information display device |
JP2008054085A (en) * | 2006-08-25 | 2008-03-06 | Hitachi Ltd | Broadcast receiving apparatus and starting method thereof |
US8386259B2 (en) * | 2006-12-28 | 2013-02-26 | Intel Corporation | Voice interface to NFC applications |
KR101441873B1 (en) * | 2007-01-12 | 2014-11-04 | 코핀 코포레이션 | Head mounted monocular display device |
US9116340B2 (en) * | 2007-05-14 | 2015-08-25 | Kopin Corporation | Mobile wireless display for accessing data from a host and method for controlling |
US8150699B2 (en) * | 2007-05-17 | 2012-04-03 | Redstart Systems, Inc. | Systems and methods of a structured grammar for a speech recognition command system |
IL183637A (en) * | 2007-06-04 | 2013-06-27 | Zvi Lapidot | Distributed head-mounted display |
WO2009054619A2 (en) * | 2007-10-22 | 2009-04-30 | Moon Key Lee | Augmented reality computer device |
JP5217386B2 (en) * | 2007-11-21 | 2013-06-19 | 株式会社ニコン | Control device, head mounted display device, program, and control method |
US8416152B2 (en) | 2008-06-11 | 2013-04-09 | Honeywell International Inc. | Method and system for operating a near-to-eye display |
US8957835B2 (en) * | 2008-09-30 | 2015-02-17 | Apple Inc. | Head-mounted display apparatus for retaining a portable electronic device with display |
-
2011
- 2011-09-15 WO PCT/US2011/051760 patent/WO2012040030A2/en active Application Filing
- 2011-09-15 EP EP11827249.1A patent/EP2617202A4/en not_active Withdrawn
- 2011-09-15 JP JP2013529323A patent/JP5974008B2/en active Active
- 2011-09-15 CN CN201180050791.3A patent/CN103890836B/en active Active
Patent Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101506868A (en) * | 2006-09-08 | 2009-08-12 | 索尼株式会社 | Display device and display method |
Also Published As
Publication number | Publication date |
---|---|
WO2012040030A2 (en) | 2012-03-29 |
CN103890836A (en) | 2014-06-25 |
EP2617202A4 (en) | 2015-01-21 |
WO2012040030A3 (en) | 2014-03-27 |
JP2014503085A (en) | 2014-02-06 |
JP5974008B2 (en) | 2016-08-23 |
EP2617202A2 (en) | 2013-07-24 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN103890836B (en) | The bluetooth with power management or other wave points for head mounted display | |
US8736516B2 (en) | Bluetooth or other wireless interface with power management for head mounted display | |
KR102411124B1 (en) | Electronic device and method for performing task using external electronic device in electronic device | |
US10229565B2 (en) | Method for producing haptic signal and electronic device supporting the same | |
US9122307B2 (en) | Advanced remote control of host application using motion and voice commands | |
US8862186B2 (en) | Lapel microphone micro-display system incorporating mobile information access system | |
KR102537543B1 (en) | Intelligent electronic device and operating method thereof | |
CN107643921A (en) | For activating the equipment, method and computer-readable recording medium of voice assistant | |
CN110058777A (en) | The method and electronic equipment of shortcut function starting | |
CN111524501B (en) | Voice playing method, device, computer equipment and computer readable storage medium | |
US20150052375A1 (en) | Information processing method and electronic device | |
CN110827820B (en) | Voice awakening method, device, equipment, computer storage medium and vehicle | |
CN105940371A (en) | User configurable speech commands | |
KR102391298B1 (en) | electronic device providing speech recognition service and method thereof | |
KR102546510B1 (en) | Method for providing information mapped between plurality inputs and electronic device supporting the same | |
KR20190032026A (en) | Method for providing natural language expression and electronic device supporting the same | |
US20200075008A1 (en) | Voice data processing method and electronic device for supporting same | |
CN108694032A (en) | Use the picture outputting method of external equipment and the electronic equipment for supporting it | |
CN111341317B (en) | Method, device, electronic equipment and medium for evaluating wake-up audio data | |
CN108898000A (en) | A kind of method and terminal solving lock screen | |
CN112100528A (en) | Method, device, equipment and medium for training search result scoring model | |
KR101727900B1 (en) | Mobile terminal and operation control method thereof | |
KR20110133295A (en) | Mobile terminal and operating method thereof | |
CN109478131A (en) | Audio input output method and terminal device | |
CN114360538A (en) | Voice data acquisition method, device, equipment and computer readable storage medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
C06 | Publication | ||
PB01 | Publication | ||
C10 | Entry into substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
C14 | Grant of patent or utility model | ||
GR01 | Patent grant |