CN106293101A - A kind of man-machine interactive system for head-mounted display and method - Google Patents
A kind of man-machine interactive system for head-mounted display and method Download PDFInfo
- Publication number
- CN106293101A CN106293101A CN201610869952.6A CN201610869952A CN106293101A CN 106293101 A CN106293101 A CN 106293101A CN 201610869952 A CN201610869952 A CN 201610869952A CN 106293101 A CN106293101 A CN 106293101A
- Authority
- CN
- China
- Prior art keywords
- head
- mounted display
- ultrasonic
- hand held
- held controller
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/014—Hand-worn input/output arrangements, e.g. data gloves
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Position Input By Displaying (AREA)
Abstract
The present invention relates to a kind of man-machine interactive system for head-mounted display and method.This man-machine interactive system includes the first interactive unit being installed on hand held controller and at least one second interactive unit being installed on head-mounted display body, and wherein, the first interactive unit includes the first RF transceiver and at least one ultrasonic transmitter；Second interactive unit includes the ultrasonic sensor that the second RF transceiver and at least three are generally aligned in the same plane；First interactive unit sends data signal and ultrasonic signal to the second interactive unit, second interactive unit receives and processes data signal and ultrasonic signal, obtains control instruction that hand held controller sends and the hand held controller locus relative to head-mounted display body.By implementing the present invention, interactive system need not photographic head, is not afraid of the impact of ambient light, it is not required that takies processor and carries out the image procossing of complexity；Reduce power consumption simultaneously, reduce cost.
Description
Technical field
The present invention relates to head-mounted display field, more particularly, it relates to a kind of man-machine interaction for head-mounted display
System and method.
Background technology
The technology such as virtual reality (VR), augmented reality (AR) are formed newly because each side technology is gradually ripe in recent years later
The hot fields of product.Because these technology can bring the sensory experience of more various dimensions, the therefore people in these fields to people
Machine is the most also the focus of current industry research and development.
In terms of market situation, the VR equipment of current main flow is mainly by being divided into two big classes:
1. low cost, uses mobile phone to wear the VR helmet as display, the most most typical representative be google relatively
" paper box " the VR display (as shown in Figure 1) issued time early, it is storm wind witch mirror series that domestic typical products represents.This kind of VR
The handle that the man-machine interaction of equipment generally bluetooth connects carries out man-machine interaction by button, for the object in Virtual Space then
Measured the head pose of beholder by the gyroscope system of embedded in mobile phone, used by the quasi-heart alignment UI arranged in picture and press
Key interacts；Or the inertial navigation system of multiaxis MEMS sensor composition is set on handle to measure attitude and the motion of handle
Trend is to interact.
2. high cost, such as HTC VIVE and the comprehensive VR equipment of SONY PLAYSTATION VR mono-class, this kind of set
Standby feature is to arrange multiple locating base station in the environment using VR equipment to enter the locus of player's helmet or handle
Row is measured.As shown in Fig. 2 (including Fig. 2 a and Fig. 2 b), the Lighthouse technology that such as HTC VIVE uses make use of machinery rotation
The X turned, Y-axis line is swept iraser and is provided with, to scan the activity space of player, the helmet and controller, the light that quantity is very many
Sensing device, by measure iraser scanning arrive light-sensitive device time difference rebuild the helmet and controller locus and
Attitude.
HoloLens mono-class additionally also having such as Microsoft uses the AR equipment that image procossing interacts.
For first kind low cost VR equipment, their interactive means is single, lacks spatial depth sense；Simultaneously with MEMS structure
The inertial navigation system become needs the attitude of the output of multiple MEMES inertial navigation systems to merge, and owing to current industry is the biggest
Part Magnetic Sensor inter device consistency is all not so good, causes the angle of drift deviation calculated bigger, it is difficult to preferably to measure
Relativeness between controller and display.
Being required to arrange locating base station in VR scene for Equations of The Second Kind VR equipment prior art, installation and debugging are complicated and numerous
Trivial, hardware cost is high and must be limited in certain region by activity space.
Using the interactive means of image recognition class for other, data processing amount is big, and power consumption is high, and time delay and response are not
Very well.
Summary of the invention
The present invention is directed to above-mentioned defect of the prior art, it is provided that a kind of man-machine interactive system for head-mounted display and
Method.
The technical solution adopted for the present invention to solve the technical problems is: construct a kind of man-machine friendship for head-mounted display
System mutually, described head-mounted display includes head-mounted display body and at least one hand held controller, described man-machine interactive system
It is installed on described head-mounted display body with at least one including the first interactive unit being installed on described hand held controller
The second interactive unit, wherein,
Described first interactive unit includes the first RF transceiver and at least one ultrasonic transmitter；
Described second interactive unit includes the supersonic sensing that the second RF transceiver and at least three are generally aligned in the same plane
Device；
Described first interactive unit sends data signal and ultrasonic signal to described second interactive unit, described second friendship
Unit receives and processes described data signal and ultrasonic signal mutually, obtains control instruction and institute that described hand held controller sends
State the hand held controller locus relative to described head-mounted display body.
Further, of the present invention in the man-machine interactive system of head-mounted display, described first interactive unit
Also include: be connected with described ultrasonic transmitter, for driving described ultrasonic transmitter to launch the driving electricity of ultrasonic signal
Road；
It is connected with described drive circuit and described first RF transceiver, is used for controlling described drive circuit and described first
First microcontroller of RF transceiver.
Further, of the present invention in the man-machine interactive system of head-mounted display, described first interactive unit
Also include being connected with described first microcontroller, for sensing the control instruction induction apparatus of user operation instruction；
Described control instruction induction apparatus includes that attitude detection module, mechanical key, capacitance touch button, capacitance touch are sliding
One or more in bar, capacitive touch panel, digital flexion sensor, finger navigation sensor.
Preferably, of the present invention in the man-machine interactive system of head-mounted display, described hand held controller bag
Include one or more in finger ring, handle, model gun, glove.
Further, of the present invention in the man-machine interactive system of head-mounted display, described second interactive unit
Also include: be connected with described second RF transceiver and described ultrasonic sensor, be used for processing described second RF transceiver
Second microcontroller of the ultrasonic signal that the data signal received and described ultrasonic sensor receive；
Described second microcontroller has at least four external interrupt, and one of them described external interrupt connects described second
RF transceiver, remaining described external interrupt is respectively by amplifying the modulate circuit described ultrasonic sensor of connection；
One of them described second interactive unit is arranged on the front of described head-mounted display body, remaining described second friendship
Unit is separately mounted to the not ipsilateral of described head-mounted display body mutually.
Preferably, of the present invention in the man-machine interactive system of head-mounted display, described second microcontroller
There are four external interrupt: the first external interrupt, the second external interrupt, the 3rd external interrupt, the 4th external interrupt, wherein,
Described first external interrupt connects described second RF transceiver；
Described second external interrupt connects described first ultrasonic sensor by the first amplification modulate circuit；
Described 3rd external interrupt connects described second ultrasonic sensor by the second amplification modulate circuit；
Described 4th external interrupt connects described 3rd ultrasonic sensor by the 3rd amplification modulate circuit.
Separately, the present invention also provides for a kind of man-machine interaction method for head-mounted display, and described head-mounted display includes head
Wear monitor body and at least one hand held controller, including:
Described hand held controller sends data signal by the first RF transceiver of the first interactive unit, and is sending number
The while of the number of it is believed that or delay the first Preset Time and send ultrasonic signal by ultrasonic transmitter；
Described head-mounted display body receives described data signal, institute by the second RF transceiver of the second interactive unit
The ultrasonic sensor stating the second interactive unit receives described ultrasonic signal；
Described second interactive unit processes described data signal, obtains the control instruction that described hand held controller sends；
The time that described second interactive unit arrives each described ultrasonic sensor according to described ultrasonic signal obtains
Distance between described ultrasonic transmitter and each described ultrasonic sensor；
According to the distance between described ultrasonic transmitter and each described ultrasonic sensor, calculate described hand-held control
Device processed is relative to the locus of described head-mounted display body.
Preferably, the man-machine interaction method for head-mounted display of the present invention, described hand held controller is by the
First RF transceiver of one interactive unit sends data signal, including:
Preset all described hand held controllers and send the shooting sequence of data signal, data signal comprises for labelling institute
State the order information of hand held controller；
The data signal transmission that described hand held controller sends is to hand held controller remaining described；
Remaining described hand held controller receives and identifies the order information in data signal, according to described order information with pre-
If shooting sequence, the described hand held controller of the next one of corresponding shooting sequence is delayed the second Preset Time and is sent data signal；
Or
After remaining described hand held controller receives data signal, according to default shooting sequence, successively time delay Tsafe,
2Tsafe is until nTsafe sends data signal, and wherein, described Tsafe is the 3rd Preset Time, and described n is described hand-held control
The quantity of device.
Preferably, the man-machine interaction method for head-mounted display of the present invention, described believe according to described ultrasound wave
Number arrive time of each described ultrasonic sensor obtain described ultrasonic transmitter and each described ultrasonic sensor it
Between distance, including:
Described data signal is received as time zero, respectively with each described ultrasonic using described second RF transceiver
Wave sensor receives described ultrasonic signal as timing end point, obtains described ultrasonic signal and arrives each described ultrasound wave
The time of sensor；
Arrive from described ultrasonic signal and remove offset time, further according to sound the time of each described ultrasonic sensor
Speed calculates the distance between described ultrasonic transmitter and each described ultrasonic sensor.
Preferably, the man-machine interaction method for head-mounted display of the present invention, described first RF transceiver is sent out
Send data signal to include: attitude detection module, mechanical key, capacitance touch button, capacitance touch draw runner, capacitive touch panel,
The control instruction that one or more in digital flexion sensor, finger navigation sensor produce.
Implement a kind of man-machine interactive system for head-mounted display and the method for the present invention, have the advantages that
Head-mounted display includes head-mounted display body and at least one hand held controller, and man-machine interactive system includes being installed on hand-held control
The first interactive unit on device processed and at least one second interactive unit being installed on head-mounted display body, wherein, first
Interactive unit includes the first RF transceiver and at least one ultrasonic transmitter；Second interactive unit includes the second radio-frequency receiving-transmitting
The ultrasonic sensor that device and at least three are generally aligned in the same plane；First interactive unit transmission data signal and ultrasonic signal are extremely
Second interactive unit, the second interactive unit receives and processes data signal and ultrasonic signal, obtains what hand held controller sent
Control instruction and hand held controller are relative to the locus of head-mounted display body.By implementing the present invention, interactive system is not
Need photographic head, be not afraid of the impact of ambient light, it is not required that take processor and carry out the image procossing of complexity；Reduce merit simultaneously
Consumption, reduces cost.
Accompanying drawing explanation
Below in conjunction with drawings and Examples, the invention will be further described, in accompanying drawing:
Fig. 1 is the structural representation of Simple head-worn display in prior art of the present invention；
Fig. 2 is the structural representation of complicated head-mounted display in prior art of the present invention；
The structural representation of the first interactive unit that Fig. 3 is mounted on hand held controller；
The structural representation of the second interactive unit that Fig. 4 is mounted on head-mounted display body；
Fig. 5 be hand held controller form be the structural representation of finger ring；
Fig. 6 be hand held controller form be the structural representation of handle；
Fig. 7 be hand held controller form be the structural representation of model gun；
Fig. 8 be hand held controller form be the structural representation of glove；
Fig. 9 is the structural representation of the head-mounted display body being provided with multiple second interactive unit；
Figure 10 is the structural representation of the head-mounted display body being provided with second interactive unit；
Figure 11 is the schematic flow sheet of a kind of man-machine interaction method for head-mounted display；
The sequential chart of hand held controller work when Figure 12 is to position single hand held controller；
The sequential chart of this body running of head-mounted display when Figure 13 is to position single hand held controller；
Figure 14 is the structural representation that head-mounted display comprises multiple hand held controller；
The sequential chart of the work of hand held controller 0 and 1 when Figure 15 is to position 2 hand held controllers simultaneously；
The sequential chart of the work of head-mounted display body when Figure 16 is to position 2 hand held controllers simultaneously；
Figure 17 is with the sequential chart of the work of hand held controller 0 and 1 during 2 hand held controllers in direct time expander method location；
Figure 18 is with the sequential chart of the work of head-mounted display body during 2 hand held controllers in direct time expander method location；
Figure 19 is the position relationship illustraton of model between ultrasonic transmitter and ultrasonic receiver 1～3.
Detailed description of the invention
In order to be more clearly understood from the technical characteristic of the present invention, purpose and effect, now comparison accompanying drawing describes in detail
The detailed description of the invention of the present invention.
As shown in Figure 3-4, it is the first embodiment of the present invention.
The open a kind of man-machine interactive system of the present embodiment, this man-machine interactive system is used for head-mounted display, head-mounted display
Include but not limited to: VR (Virtual Reality, virtual reality), AR (Augmented Reality, augmented reality), MR
(Mix reality, mixed reality) etc..
Head-mounted display includes head-mounted display body and at least one hand held controller, and hand held controller is used for controlling head
Wear monitor body, single hand held controller can be used to control head-mounted display body, operate；Or use multiple hand-held control
Device processed cooperates and controls head-mounted display body, operates.
Man-machine interactive system includes that the first interactive unit being installed on hand held controller and at least one are installed on and wears
The second interactive unit on monitor body；It is mutual single to second that first interactive unit sends data signal and ultrasonic signal
Unit, the second interactive unit receives and processes data signal and ultrasonic signal, obtain control instruction that hand held controller sends and
Hand held controller is relative to the locus of head-mounted display body.Individually below to the first interactive unit and the second interactive unit
It is described in detail.
The structural representation of the first interactive unit that Fig. 3 is mounted on hand held controller；
Concrete, the first interactive unit includes: the first microcontroller, the first RF transceiver, drive circuit, at least one
Ultrasonic transmitter, control instruction induction apparatus；Wherein, the first microcontroller connect respectively the first RF transceiver, drive circuit,
Control instruction induction apparatus, the first microcontroller connects ultrasonic transmitter by drive circuit, it will be understood that work as ultrasonic emitting
When device is multiple, corresponding multiple drive circuits.Drive circuit is used for driving ultrasonic transmitter to launch ultrasonic signal；Control refers to
Make induction apparatus for sensing user operation action, produce operational order；First microcontroller is used for controlling drive circuit, first penetrates
Frequently transceiver, control instruction induction apparatus.Preferably, hand held controller is provided with for providing electric energy for the first interactive unit
Battery.
Preferably, control instruction induction apparatus includes attitude detection module, mechanical key, capacitance touch button, capacitance touch
One or more in draw runner, capacitive touch panel, digital flexion sensor, finger navigation sensor.It is appreciated that above several
Plant control instruction induction apparatus and be used for sensing user operation action, produce operational order, such as finger motion state or the installation of user
Mechanical key state etc. on hand held controller.User produces various operation signals, operation letter by control instruction induction apparatus
Number by first microcontroller process after, send through the first RF transceiver, be sent to the second of head-mounted display body
Interactive unit.
Preferably, ultrasonic transmitter is preferably column PVDF thin film ultrasonic transmitter, such as TE Connectivity
Just there is this kind of ultrasonic transmitter of production in company, and tranmitting frequency is 40Khz or 80Khz, and launching angle is omnidirectional 360 degree.
Preferably, in the man-machine interactive system for head-mounted display of the present invention, hand held controller includes finger ring, hands
One or more in handle, model gun, glove, the most as shown in figures 5-9.
As it is shown in figure 5, be the second embodiment of the present invention.
Fig. 5 is the schematic diagram that hand held controller form is finger ring of the present embodiment, and finger ring is provided with ultrasonic transmitter
And button, usually, the position that ultrasonic transmitter is arranged should during using finger mutual omnidistance ultrasonic emitting
The position being blocked all it is difficult between device and ultrasonic sensor.Be appreciated that on finger ring be provided with the first microcontroller, first
RF transceiver and corresponding drive circuit；Button is used for receiving user operation action, produces control information.
As shown in Figure 6, it is the third embodiment of the present invention.
Fig. 6 is the schematic diagram that hand held controller form is handle of the present embodiment.Ultrasonic transmitter it is provided with on handle
With multiple buttons, figure illustrates a material particular, an annular array grid hole is i.e. installed outside ultrasonic transmitter
Guard shield, uses column PVDF piezoelectric membrane ultrasonic transmitter to can be good at theaomni-directional transmission under the protective cover of this form ultrasonic
Ripple, can also ensure that in the case of not affecting location the attractive in appearance of product and protection ultrasonic transmitter do not damage.It is appreciated that
The first microcontroller, the first RF transceiver and corresponding drive circuit it is provided with on finger ring；Button is used for receiving user behaviour
Make action, produce control information.
As it is shown in fig. 7, be the fourth embodiment of the present invention.
Fig. 7 is the schematic diagram that hand held controller form is model gun of the present embodiment.Model gun is provided with super at gun back end
Pinger, and trigger-type button and common key-press A, B.Under VR/AR environment, shooting game is played by such hardware
Extraordinary game experiencing can be obtained, it is possible to realize firearms aiming experience true to nature by the locus of rifle is followed the trail of.
As shown in Figure 8, it is the fifth embodiment of the present invention.
Fig. 8 is the schematic diagram that hand held controller form is glove of the present embodiment.It is provided with ultrasonic on glove at tiger's jaw
Wave launcher, such can ensure that can at major part use hands head-mounted display body in the natural visual field of VR/AR
Preferably receive ultrasonic signal.Additionally each finger is additionally provided with bend sensor, and such as SpectraSymbol company is raw
The bending sensitive resistance produced, or the optical fibre bending sensor being made up of bending sensitive optical fibre, produce when user's digital flexion
Corresponding control signal, is followed the trail of hand position be can be implemented in people's nature visual field of VR/AR by such hardware in and identifies
Various gestures, such as, capture the object of different distance and be placed into other distances.
Fig. 4 is mounted in the structural representation of the second interactive unit of head-mounted display body.
Concrete, the second interactive unit is connected to wear by wireless communication interfaces such as wired or bluetooth/wifi/zigbee
The main system of monitor body, sends the interactive information fully entered to AR/VR main system and processes.It is appreciated that
First RF transceiver of one interactive unit and the second RF transceiver of the second interactive unit should be mutually matched, such as
2.4G RF transceiver common on market, has the NRF24L01 series of NORDIC company, Shanghai link electronics corporation
LT89xx series.The pin that wherein the second RF transceiver and second microcontroller the first external interrupt connect should be packet
Interrupt output pin, above-mentioned NRF24L01 series and LT89xx series all have this pin.Ultrasonic sensor 1～3 is preferably
MEMS silicon microphone, there is the silicon microphone producing the type in Knowles company, and the volume of such sensor is the least, easily collects
Become on head-mounted display.Ultrasonic sensor 1～3 will receive more afterwards by amplifying the conditioning shaping of modulate circuit 1～3
Ultrasonic signal be converted into the level saltus step that the second microcontroller can identify.
Preferably, when the second interactive unit is multiple, one of them second interactive unit is arranged on head-mounted display originally
The front of body, remaining second interactive unit is separately mounted to the not ipsilateral of head-mounted display body.As it is shown in figure 9, Fig. 9 is this
The schematic diagram of embodiment a kind of head-mounted display body.Figure shows the 1 of head-mounted display body, 2,3,4 four faces, wherein, 1
Face is the front of head-mounted display body, and 2,3,4 is the side of head-mounted display body.If head-mounted display body is only arranged
Have second interactive unit, then this second interactive unit is arranged on 1 of head-mounted display body；If head-mounted display body
Be provided only with multiple second interactive unit, then one of them second interactive unit is arranged on 1 of head-mounted display body, remaining
Second interactive unit is separately mounted to the 2 of head-mounted display body, 3,4.
Second interactive unit includes: the second microcontroller, the second RF transceiver, amplification modulate circuit, at least three position
In conplane ultrasonic sensor；Wherein, the second microcontroller connects the second RF transceiver, and the second microcontroller passes through
Amplify modulate circuit and connect ultrasonic sensor respectively.Second microcontroller processes the data signal that the second RF transceiver receives
The ultrasonic signal received with ultrasonic sensor；
Further, the second microcontroller has at least four external interrupt, and one of them external interrupt connects the second radio frequency
Transceiver, remaining external interrupt is respectively by amplifying modulate circuit connection ultrasonic sensor.
Preferably, the second microcontroller has four external interrupt: the first external interrupt, the second external interrupt, outside the 3rd
Portion interrupts, the 4th external interrupt, wherein,
First external interrupt connects the second RF transceiver；
Second external interrupt is amplified modulate circuit by first and is connected the first ultrasonic sensor；
3rd external interrupt is amplified modulate circuit by second and is connected the second ultrasonic sensor；
4th external interrupt connects the 3rd ultrasonic sensor by the 3rd amplification modulate circuit.
As shown in Figure 10, it is the sixth embodiment of the present invention.
Figure 10 is the structural representation of the head-mounted display body of embodiment.Head-mounted display body be provided with three ultrasonic
Wave sensor 1～3, ultrasonic sensor 1～3 is arranged in the plane in direction, the head-mounted display human eye nature visual field, Ke Yi
Use in same plane and arbitrarily arrange, be not limited to the layout shown in figure, according to engineering practice, at 3 ultrasonic sensors
In certain 1 as initial point in the case of, other 2 ultrasonic sensors the most in the x and y direction should with the distance of initial point
Sufficiently large to produce enough resolution, as a example by VR head-mounted display, suitable distance should be between 10～25cm.
As shown in figure 11, it is the seventh embodiment of the present invention.
Figure 11 is the schematic flow sheet of a kind of man-machine interaction method for head-mounted display, and head-mounted display includes wearing
Monitor body and at least one hand held controller, the method includes:
Step S1: hand held controller sends data signal by the first RF transceiver of the first interactive unit, and is sending out
Send data signal simultaneously or delay the first Preset Time and send ultrasonic signal by ultrasonic transmitter；
Preferably, the first RF transceiver transmission data signal includes: attitude detection module, mechanical key, capacitance touch
One or more in button, capacitance touch draw runner, capacitive touch panel, digital flexion sensor, finger navigation sensor are produced
Raw control instruction.
Preferably, when hand held controller only has one, the hand held controller the first radio-frequency receiving-transmitting by the first interactive unit
Device sends data signal, including:
Preset hand held controller and send the shooting sequence of data signal, data signal comprises for labelling hand held controller
Order information；
The data signal transmission that hand held controller sends is to remaining hand held controller；
Remaining hand held controller receives and identifies the order information in data signal, according to order information and default transmitting
Sequentially, the next hand held controller of corresponding shooting sequence is delayed the second Preset Time and is sent data signal；
Refer to Figure 12, shown in Figure 12 is the sequential chart of hand held controller work.Wherein Tw be twice sterically defined
Interval time, according to engineering practice, in order to ensure that this Tw of smoothness followed the trail of locus should be less than 50ms, it is still further preferred that 20
～30ms.When the beginning of each Tw time, first microprocessor controls the first RF transceiver will obtain in time last time Tw
The packet that enters information as of control instruction induction apparatus be transmitted, after time delay Td, control drive circuit by ultrasound wave
Ultrasound wave launched by emitter.According to engineering practice, length and the data package size of Td have relation, it should be the first RF transceiver
Packet send completely after time delay at least 500us for preferably arranging.After completing these work, remaining time first microcontroller
Device can continue to obtain and process the interactive information of other man-machine input hardwares, such as, read the data of attitude measurement module, sweep
Retouch mechanical key, read capacitance touch button/draw runner/panel, the data etc. of bend sensor.
Preferably, as shown in figure 14, Figure 14 is that the structure that the present embodiment head-mounted display comprises multiple hand held controller is shown
It is intended to.Concrete, hand held controller is multiple, is designated as hand held controller 1-n respectively, and n is the quantity of hand held controller.Hand-held control
Device processed sends data signal by the first RF transceiver of the first interactive unit, including:
Preset all hand held controllers and send the shooting sequence of data signal, data signal comprises for the hand-held control of labelling
The order information of device processed；
The data signal transmission that hand held controller sends is to remaining hand held controller；
After remaining hand held controller receives data signal, according to default shooting sequence, successively time delay Tsafe,
2Tsafe is until nTsafe sends data signal, and wherein, Tsafe is the 3rd Preset Time, and n is the quantity of hand held controller.
The process of implementing is: the RF transceiver of hand held controller 0 is first transmission as the method previously described in Tw
Interaction data bag 0 and ultrasonic signal, the RF transceiver of remaining hand held controller node and head-mounted display body is all arranged
For receiving state.After hand held controller 1 receives packet 0, after time delay Tsafe, send interaction data as the method previously described
Bag 1 and ultrasonic signal, send out after time delay Tsafe after then hand held controller 2 receives interaction data bag 1 as the method previously described
Deliver mutual packet 2 and ultrasonic signal, by that analogy.Tsafe when being single channel location between the safety of ultrasonic transmission
Interval.
Wherein should comprise the order information of this hand held controller in interaction data bag 0～n for head-mounted display body
Which the node differentiating work at present with other hand held controller nodes is.The synchronization character of the RF transceiver of all members is (i.e.
Sync word) should be consistent to ensure that certain node can reach to be broadcast to the effect of members when of sending data.
As shown in Figure 15 and Figure 16, Figure 15 and Figure 16 gives the sequential chart simultaneously positioning 2 hand held controllers.Figure 15 institute
Show is the work schedule of hand held controller 0 and 1, and shown in Figure 16 is the work schedule of head-mounted display body.According to Figure 16
T01, T02 and T03 that measured Tt01, Tt02 and Tt03 removal offset time Td obtains can obtain hand held controller 0
Locus, Tt11, Tt12 and Tt13 removal offset time Td obtains T11, T12 and T13 and can obtain the sky of hand held controller 1
Between position.
Especially, the situation of positioning function is only needed when other hand held controller nodes in addition to hand held controller 0
Under, other hand-held control nodes can also after receiving interaction data bag 0 order time delay Tsafe successively according to own node,
2*Tsafe until n*Tsafe then directly transmit ultrasonic signal, now can record in advance each node ultrasonic emitting and
The total linear spacing time of interaction data bag 0 is Tdelay1, Tdelay2 until Tdelayn, and head-mounted display body can pass through
The time of advent of each node is separated, from without other hand held controller nodes again by Tdelay1～n as interval
Send rf data bag as synchronization.
Refer to Figure 17 and Figure 18, Figure 17 and Figure 18 give employing above-mentioned direct time expander method position 2 hand-held controls
The sequential chart of device.Shown in Figure 17 is the work schedule of hand held controller 0 and 1, and shown in Figure 18 is head-mounted display body
Work schedule.Permissible according to T01, T02 and T03 that Tt01, Tt02 and Tt03 removal offset time Td measured by Figure 16 obtains
Obtaining the locus of hand held controller 0, Tt11, Tt12 and Tt13 remove offset time Tdelay1 and obtain T11, T12 and T13
The locus of hand held controller 1 can be obtained.
According to engineering experience, in VR/AR interactive mode, the propagation distance of ultrasound wave is all arranged at below 1m, Tsafe
It is 2 for the node of 10ms～15ms and hand held controller and is obtained in that preferable interactive experience.
Step S2: head-mounted display body receives data signal by the second RF transceiver of the second interactive unit, the
The ultrasonic sensor received ultrasonic signal of two interactive unit；
Refer to Figure 13, shown in Figure 13 is the sequential chart of this body running of head-mounted display.Wherein Tw length is by hand-held control
The transmission interval of device body processed determines.When the second microcontroller receives the packet interrupt signal of the second RF transceiver, open
Open timing, receive interrupt signal with ultrasound wave respectively and respectively obtain 3 times Tt1, Tt2 and Tt3 for end mark；Tt1、Tt2
With Tt3 contains the transmission interval Td of hand held controller the first rf control unit and ultrasonic transmitter, therefore to remove
This fixing offset time respectively obtains T1, T2 and T3.T1, T2 and T3 are multiplied by the velocity of sound respectively i.e. can obtain ultrasonic emitting
Distance between device and ultrasonic receiver 1～3.
Step S3: the second interactive unit processes data signal, obtains the control instruction that hand held controller sends；
Step S4: the time that the second interactive unit arrives each ultrasonic sensor according to ultrasonic signal obtains ultrasound wave
Distance between emitter and each ultrasonic sensor；
Preferably, the man-machine interaction method for head-mounted display of the present invention, arrive each super according to ultrasonic signal
The time of sonic sensor obtains the distance between ultrasonic transmitter and each ultrasonic sensor, including:
Receive data signal as time zero using the second RF transceiver, receive with each ultrasonic sensor respectively
To ultrasonic signal as timing end point, obtain ultrasonic signal and arrive the time of each ultrasonic sensor；
Arrive from ultrasonic signal and the time of each ultrasonic sensor removes offset time, calculate further according to velocimeter
Distance between ultrasonic transmitter and each ultrasonic sensor.
Step S5: according to the distance between ultrasonic transmitter and each ultrasonic sensor, calculate hand held controller
Locus relative to head-mounted display body.
According to the distance between ultrasonic transmitter and each ultrasonic sensor, calculate hand held controller relative to head
Wear the locus of monitor body, including:
As shown in figure 19, it is the position relationship illustraton of model between ultrasonic transmitter and ultrasonic receiver 1～3.Second
Interactive unit includes the first ultrasonic sensor, the second ultrasonic sensor, the 3rd ultrasonic sensor, and note first is ultrasonic respectively
Wave sensor coordinate is R1 (0,0,0), the second ultrasonic sensor coordinate is R2 (x1, y1,0), the 3rd ultrasonic sensor is sat
It is designated as R3 (x2, y2,0) and note ultrasonic transmitter coordinate is that (x, y, z), and UST point is all the time in the positive axis district of z-axis for UST
Territory is movable, if UST point is respectively D0, D1, D2 to the distance of R1, R2, R3 point, then,
The solution equation obtains:
X, y are brought into full scale equation and obtain z, obtain UST point locus (x, y, z).
Engineering practice shows, in an optimal situation, when using this man-machine interactive system to do man-machine interaction, space orientation can
To reach the resolution of 0.1mm level.This method make use of the wireless of hand held controller and AR/VR core processing system dexterously
The signal of communication itself, as time synchronisation signal, both can realize transmission attitude detection module, button, touch key-press/cunning
While the inputted interactive information of bar/hardware such as panel, bend sensor, it is also possible to fixed as the space realizing hand held controller
The synchronizing signal of position.Owing to the location of hand held controller is relative to head-mounted display body, when wearing monitor body appearance
When state and change in location, obtained location data also can be natural there is corresponding change and without extra process so that one
As the complicated attitude that needs between hand held controller and head-mounted display body in man-machine interaction in VR/AR field merge and position
The problem putting fusion has obtained preferable solution.
On the other hand, contrast uses the same domain technical scheme that image recognition realizes, and the present invention has a following advantage:
1. need not photographic head, be not afraid of the impact of ambient light, it is not required that take processor and carry out the image procossing of complexity；
The most low in energy consumption, on the premise of 3.3V powers, the electric current of head-mounted display body can control at below 50mA, hands
The electric current of held controller end can control at below 20mA；
3. hardware is simple, and cost is relatively low.
Above example, only for technology design and the feature of the explanation present invention, its object is to allow person skilled in the art
Scholar will appreciate that present disclosure and implements accordingly, can not limit the scope of the invention.All with right of the present invention want
The equalization asking scope to be done changes and modifies, and all should belong to the covering scope of the claims in the present invention.
Claims (10)
1., for the man-machine interactive system of head-mounted display, described head-mounted display includes head-mounted display body and at least
One hand held controller, it is characterised in that described man-machine interactive system includes the first friendship being installed on described hand held controller
Mutually unit and at least one be installed on the second interactive unit on described head-mounted display body, wherein,
Described first interactive unit includes the first RF transceiver and at least one ultrasonic transmitter；
Described second interactive unit includes the ultrasonic sensor that the second RF transceiver and at least three are generally aligned in the same plane；
Described first interactive unit sends data signal and ultrasonic signal extremely described second interactive unit, and described second is mutual single
Unit receives and processes described data signal and ultrasonic signal, obtains control instruction and described hands that described hand held controller sends
Held controller is relative to the locus of described head-mounted display body.
Man-machine interactive system for head-mounted display the most according to claim 1, it is characterised in that described first mutual
Unit also includes: be connected with described ultrasonic transmitter, for driving described ultrasonic transmitter to launch driving of ultrasonic signal
Galvanic electricity road；
It is connected with described drive circuit and described first RF transceiver, is used for controlling described drive circuit and described first radio frequency
First microcontroller of transceiver.
Man-machine interactive system for head-mounted display the most according to claim 2, it is characterised in that described first mutual
Unit also includes being connected with described first microcontroller, for sensing the control instruction induction apparatus of user operation instruction；
Described control instruction induction apparatus includes attitude detection module, mechanical key, capacitance touch button, capacitance touch draw runner, electricity
Hold one or more in touch panel, digital flexion sensor, finger navigation sensor.
4. according to the arbitrary described man-machine interactive system for head-mounted display of claim 1-3, it is characterised in that described hands
Held controller includes one or more in finger ring, handle, model gun, glove.
Man-machine interactive system for head-mounted display the most according to claim 1, it is characterised in that described second mutual
Unit also includes: is connected with described second RF transceiver and described ultrasonic sensor, is used for processing described second radio frequency receipts
Send out the data signal of device reception and the second microcontroller of the ultrasonic signal of described ultrasonic sensor reception；
Described second microcontroller has at least four external interrupt, and one of them described external interrupt connects described second radio frequency
Transceiver, remaining described external interrupt is respectively by amplifying the modulate circuit described ultrasonic sensor of connection；
One of them described second interactive unit is arranged on the front of described head-mounted display body, remaining described second mutual list
Unit is separately mounted to the not ipsilateral of described head-mounted display body.
Man-machine interactive system for head-mounted display the most according to claim 5, it is characterised in that described second micro-control
Utensil processed has four external interrupt: the first external interrupt, the second external interrupt, the 3rd external interrupt, the 4th external interrupt, its
In,
Described first external interrupt connects described second RF transceiver；
Described second external interrupt connects described first ultrasonic sensor by the first amplification modulate circuit；
Described 3rd external interrupt connects described second ultrasonic sensor by the second amplification modulate circuit；
Described 4th external interrupt connects described 3rd ultrasonic sensor by the 3rd amplification modulate circuit.
7., for the man-machine interaction method of head-mounted display, described head-mounted display includes head-mounted display body and at least
One hand held controller, it is characterised in that including:
Described hand held controller sends data signal by the first RF transceiver of the first interactive unit, and is sending data letter
Number while or delay the first Preset Time and send ultrasonic signal by ultrasonic transmitter；
Described head-mounted display body receives described data signal by the second RF transceiver of the second interactive unit, and described the
The ultrasonic sensor of two interactive unit receives described ultrasonic signal；
Described second interactive unit processes described data signal, obtains the control instruction that described hand held controller sends；
The time that described second interactive unit arrives each described ultrasonic sensor according to described ultrasonic signal obtains described
Distance between ultrasonic transmitter and each described ultrasonic sensor；
According to the distance between described ultrasonic transmitter and each described ultrasonic sensor, calculate described hand held controller
Locus relative to described head-mounted display body.
Man-machine interaction method for head-mounted display the most according to claim 7, it is characterised in that described hand-held control
Device sends data signal by the first RF transceiver of the first interactive unit, including:
Preset all described hand held controllers and send the shooting sequence of data signal, data signal comprises for hands described in labelling
The order information of held controller；
The data signal transmission that described hand held controller sends is to hand held controller remaining described；
Remaining described hand held controller receives and identifies the order information in data signal, according to described order information and preset
Shooting sequence, the described hand held controller of the next one of corresponding shooting sequence is delayed the second Preset Time and is sent data signal；Or
After remaining described hand held controller receives data signal, according to default shooting sequence, successively time delay Tsafe,
2Tsafe is until nTsafe sends data signal, and wherein, described Tsafe is the 3rd Preset Time, and described n is described hand-held control
The quantity of device.
Man-machine interaction method for head-mounted display the most according to claim 7, it is characterised in that described in described basis
The time of the ultrasonic signal each described ultrasonic sensor of arrival obtains described ultrasonic transmitter and each described ultrasound wave
Distance between sensor, including:
Receive described data signal as time zero using described second RF transceiver, pass with each described ultrasound wave respectively
Sensor receives described ultrasonic signal as timing end point, obtains described ultrasonic signal and arrives each described supersonic sensing
The time of device；
Arrive from described ultrasonic signal and remove offset time, further according to velocimeter the time of each described ultrasonic sensor
Calculate the distance between described ultrasonic transmitter and each described ultrasonic sensor.
Man-machine interaction method for head-mounted display the most according to claim 7, it is characterised in that described first penetrates
Frequently transceiver transmission data signal includes: attitude detection module, mechanical key, capacitance touch button, capacitance touch draw runner, electric capacity
The control instruction that one or more in touch panel, digital flexion sensor, finger navigation sensor produce.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610869952.6A CN106293101A (en) | 2016-09-30 | 2016-09-30 | A kind of man-machine interactive system for head-mounted display and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610869952.6A CN106293101A (en) | 2016-09-30 | 2016-09-30 | A kind of man-machine interactive system for head-mounted display and method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106293101A true CN106293101A (en) | 2017-01-04 |
Family
ID=57715936
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610869952.6A Pending CN106293101A (en) | 2016-09-30 | 2016-09-30 | A kind of man-machine interactive system for head-mounted display and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106293101A (en) |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106845008A (en) * | 2017-02-16 | 2017-06-13 | 珠海格力电器股份有限公司 | VR equipment processing method and device |
| CN108563325A (en) * | 2018-03-09 | 2018-09-21 | 深圳市亿境虚拟现实技术有限公司 | virtual reality system data transmission method |
| CN108803861A (en) * | 2017-04-28 | 2018-11-13 | 广东虚拟现实科技有限公司 | A kind of exchange method, equipment and system |
| CN109408128A (en) * | 2018-11-10 | 2019-03-01 | 歌尔科技有限公司 | Split type AR equipment communication means and AR equipment |
| CN109633632A (en) * | 2018-12-26 | 2019-04-16 | 青岛小鸟看看科技有限公司 | One kind wearing display equipment, handle and its location tracking method |
| CN111103980A (en) * | 2019-12-18 | 2020-05-05 | 南京航空航天大学 | VR (virtual reality) environment interaction system and method based on FMCW (frequency modulated continuous wave) |
| WO2020135539A1 (en) * | 2018-12-26 | 2020-07-02 | 青岛小鸟看看科技有限公司 | Positioning method and apparatus for handles in head-mounted display system, and head-mounted display system |
| CN111420391A (en) * | 2020-03-04 | 2020-07-17 | 青岛小鸟看看科技有限公司 | Head-mounted display system and space positioning method thereof |
| CN111565808A (en) * | 2017-12-07 | 2020-08-21 | 威尔乌集团 | Electronic controller with hand holder, housing and finger sensing |
| CN113144592A (en) * | 2021-03-03 | 2021-07-23 | 南京爱奇艺智能科技有限公司 | Interaction method of VR equipment and mobile equipment |
| CN114647312A (en) * | 2018-04-30 | 2022-06-21 | 苹果公司 | Extendable ring device |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN201594136U (en) * | 2009-12-31 | 2010-09-29 | 青岛海尔软件有限公司 | Indoor positioning sensory system |
| CN101890719A (en) * | 2010-07-09 | 2010-11-24 | 中国科学院深圳先进技术研究院 | Robot remote control device and robot system |
| US20150094142A1 (en) * | 2013-09-30 | 2015-04-02 | Sony Computer Entertainment Inc. | Camera based safety mechanisms for users of head mounted displays |
| CN104615237A (en) * | 2013-11-05 | 2015-05-13 | 精工爱普生株式会社 | Image display system, method of controlling image display system, and head-mount type display device |
| CN104965608A (en) * | 2015-07-20 | 2015-10-07 | 杜昊浓 | Space mouse system |
-
2016
- 2016-09-30 CN CN201610869952.6A patent/CN106293101A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN201594136U (en) * | 2009-12-31 | 2010-09-29 | 青岛海尔软件有限公司 | Indoor positioning sensory system |
| CN101890719A (en) * | 2010-07-09 | 2010-11-24 | 中国科学院深圳先进技术研究院 | Robot remote control device and robot system |
| US20150094142A1 (en) * | 2013-09-30 | 2015-04-02 | Sony Computer Entertainment Inc. | Camera based safety mechanisms for users of head mounted displays |
| CN104615237A (en) * | 2013-11-05 | 2015-05-13 | 精工爱普生株式会社 | Image display system, method of controlling image display system, and head-mount type display device |
| CN104965608A (en) * | 2015-07-20 | 2015-10-07 | 杜昊浓 | Space mouse system |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106845008A (en) * | 2017-02-16 | 2017-06-13 | 珠海格力电器股份有限公司 | VR equipment processing method and device |
| CN108803861A (en) * | 2017-04-28 | 2018-11-13 | 广东虚拟现实科技有限公司 | A kind of exchange method, equipment and system |
| CN111565808A (en) * | 2017-12-07 | 2020-08-21 | 威尔乌集团 | Electronic controller with hand holder, housing and finger sensing |
| CN108563325A (en) * | 2018-03-09 | 2018-09-21 | 深圳市亿境虚拟现实技术有限公司 | virtual reality system data transmission method |
| CN114647312A (en) * | 2018-04-30 | 2022-06-21 | 苹果公司 | Extendable ring device |
| CN109408128A (en) * | 2018-11-10 | 2019-03-01 | 歌尔科技有限公司 | Split type AR equipment communication means and AR equipment |
| WO2020135539A1 (en) * | 2018-12-26 | 2020-07-02 | 青岛小鸟看看科技有限公司 | Positioning method and apparatus for handles in head-mounted display system, and head-mounted display system |
| US11294189B2 (en) | 2018-12-26 | 2022-04-05 | Qingdao Pico Technology Co., Ltd. | Method and device for positioning handle in head mounted display system and head mounted display system |
| CN109633632A (en) * | 2018-12-26 | 2019-04-16 | 青岛小鸟看看科技有限公司 | One kind wearing display equipment, handle and its location tracking method |
| CN111103980A (en) * | 2019-12-18 | 2020-05-05 | 南京航空航天大学 | VR (virtual reality) environment interaction system and method based on FMCW (frequency modulated continuous wave) |
| WO2021120971A1 (en) * | 2019-12-18 | 2021-06-24 | 南京航空航天大学 | Fmcw-based vr environment interaction system and method |
| CN111420391A (en) * | 2020-03-04 | 2020-07-17 | 青岛小鸟看看科技有限公司 | Head-mounted display system and space positioning method thereof |
| CN113144592A (en) * | 2021-03-03 | 2021-07-23 | 南京爱奇艺智能科技有限公司 | Interaction method of VR equipment and mobile equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106293101A (en) | A kind of man-machine interactive system for head-mounted display and method | |
| CN105031918B (en) | A kind of man-machine interactive system based on virtual reality technology | |
| CN103877726B (en) | A kind of virtual reality components system | |
| Foxlin et al. | Weartrack: A self-referenced head and hand tracker for wearable computers and portable vr | |
| CN103699219B (en) | A kind of intelligent glasses interactive system and intelligent interactive method | |
| CN103631369B (en) | Electronic equipment and control method | |
| CN110262666A (en) | Augmented reality user interface with touch feedback | |
| CN106598293B (en) | A kind of three-dimensional large space multichannel pen type interactive system | |
| CN205007551U (en) | Human -computer interaction system based on virtual reality technology | |
| CN106405500A (en) | Intelligent robot following method and following system and emission device for following | |
| CN101866539A (en) | Remote control system, remotely-controlled equipment and remote controller | |
| CN106933385B (en) | A kind of implementation method of the low-power consumption sky mouse pen based on three-dimensional ultrasonic positioning | |
| CN106484121A (en) | A kind of motion capture system and method | |
| CN110362231A (en) | The method and device that new line touch control device, image are shown | |
| CN107272892B (en) | Virtual touch system, method and device | |
| TWI549497B (en) | Method, air mouse and smart television to control a cursor | |
| CN104991650A (en) | Gesture controller and virtual reality system | |
| CN206282229U (en) | A kind of man-machine interactive system for head-mounted display | |
| US10908425B2 (en) | Transmission-type head mounted display apparatus, display control method, and computer program | |
| CN106843504A (en) | VR environment realizes the true sense of touch gloves and exchange method for interacting with actual environment | |
| CN107443374A (en) | Manipulator control system and its control method, actuation means, storage medium | |
| CN106873760A (en) | Portable virtual reality system | |
| CN208723929U (en) | A kind of AR remote controler | |
| CN103520921A (en) | Cursor control method and system and control device | |
| US12045388B2 (en) | Finger detection device and method providing increased reliability detecting attitudes of fingers |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination |