CN106873760A - Portable virtual reality system - Google Patents
Portable virtual reality system Download PDFInfo
- Publication number
- CN106873760A CN106873760A CN201510942848.0A CN201510942848A CN106873760A CN 106873760 A CN106873760 A CN 106873760A CN 201510942848 A CN201510942848 A CN 201510942848A CN 106873760 A CN106873760 A CN 106873760A
- Authority
- CN
- China
- Prior art keywords
- image
- virtual reality
- portable
- electronic equipment
- mounted display
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000003287 optical effect Effects 0.000 claims description 60
- 238000009434 installation Methods 0.000 claims description 6
- 230000005236 sound signal Effects 0.000 claims description 5
- 230000005611 electricity Effects 0.000 claims description 4
- 230000002996 emotional effect Effects 0.000 claims description 3
- 230000009467 reduction Effects 0.000 claims description 3
- 238000005259 measurement Methods 0.000 claims 2
- 230000009471 action Effects 0.000 abstract description 7
- 206010034719 Personality change Diseases 0.000 abstract 2
- 238000010586 diagram Methods 0.000 description 9
- 238000000034 method Methods 0.000 description 9
- 238000005516 engineering process Methods 0.000 description 8
- 230000008859 change Effects 0.000 description 5
- 238000006073 displacement reaction Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 230000008030 elimination Effects 0.000 description 3
- 238000003379 elimination reaction Methods 0.000 description 3
- 238000001514 detection method Methods 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 230000035807 sensation Effects 0.000 description 2
- 239000011800 void material Substances 0.000 description 2
- 230000003321 amplification Effects 0.000 description 1
- 230000003190 augmentative effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000001427 coherent effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000000205 computational method Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000003199 nucleic acid amplification method Methods 0.000 description 1
- 238000011017 operating method Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
- G02B27/0172—Head mounted characterised by optical features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
- G06F3/042—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by opto-electronic means
- G06F3/0425—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by opto-electronic means using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected
- G06F3/0426—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by opto-electronic means using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking fingers with respect to a virtual keyboard projected or printed on the surface
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
- G02B2027/0138—Head-up displays characterised by optical features comprising image capture systems, e.g. camera
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
- G02B2027/014—Head-up displays characterised by optical features comprising information/image processing systems
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Optics & Photonics (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
A portable virtual reality system. The portable virtual reality system comprises a portable electronic device, a head-mounted display and an object attitude sensor. The object attitude sensor is used for obtaining attitude change information of the head-mounted display, wherein the portable electronic device calculates virtual reality display image data according to the attitude change information and transmits the display image data to the head-mounted display, so that the head-mounted display performs image display action according to the display image data.
Description
Technical field
The present invention relates to technical field of virtual reality, more particularly to a kind of portable virtual reality system.
Background technology
With the quick evolution of electronics technology and image processing techniques, virtual reality turned into can realize and
The non-one kind ideal for existing only in film, animation.However, in existing virtual reality technology, be mostly
Many sensing elements are configured on head-mounted display apparatus (Head-Mounted Display, HMD), and
The person of needing to use wears head-mounted display apparatus in the space for setting and can realize the display of virtual reality
Effect.And, it is necessary to equipped with the various optical signals such as infrared ray or generating laser in the space that this sets
Generator.Then the attitude of user is detected by arranging in pairs or groups the sensor on head-mounted display apparatus,
To produce three-dimensional virtual reality imagery.
It is noted that in existing virtual reality technology, when user will enjoy the pleasant sensation of virtual reality,
Have to go to pre-set space.Consequently, it is possible to will be had a greatly reduced quality on virtual reality ease of use,
Reduce the wish of user.
The content of the invention
The present invention provides a kind of portable virtual reality system, is achieved removableization virtual reality.
Portable virtual reality system of the invention includes portable electronic equipment, head mounted display and thing
Body attitude transducer.Portable electronic equipment is mutually coupled with head mounted display.Gestures of object sensor point
Not Ou Jie portable electronic equipment and head mounted display, be used to obtain the attitudes vibration letter of head mounted display
Breath, wherein, portable electronic equipment calculates the display image number of virtual reality according to attitudes vibration information
According to, and image data displaying is sent to head mounted display, so that head mounted display is according to display image
Data carry out image and show.
In one embodiment of this invention, above-mentioned portable virtual reality system also includes input equipment, takes
Belt electronic installation also decides whether to show head mounted display according to input equipment by use state
The image of keyboard and the hand image of user.Input equipment can be including keyboard, mouse etc..
In one embodiment of this invention, above-mentioned gestures of object sensor obtains the hand image and keyboard
Image, portable electronic equipment obtains keyboard according to the corresponding relation of the image of hand image and keyboard
By use state.
In one embodiment of this invention, above-mentioned portable electronic equipment obtains hand according to the hand image
The skeleton image of portion's image, and obtain keyboard according to the corresponding relation of skeleton image and the image of keyboard
By use state.
In one embodiment of this invention, above-mentioned framework information includes palm skeleton image and finger skeleton
Image, wherein, portable electronic equipment is according to palm skeleton image and the image of finger skeleton image and keyboard
Overlap condition and the mobile status of palm skeleton image and finger skeleton image decide whether to make to wear
Formula display shows the image and hand image of keyboard.
In one embodiment of this invention, above-mentioned gestures of object sensor includes the light for sending optical signal
Signal generator and for receiving the optical signal to obtain the optical signal receiver of sensitive information, wherein, light
Signal generator is configured in one of portable electronic equipment and head mounted display person, and optical signal connects
Device configuration is received in portable electronic equipment and the another one of head mounted display, optical signal generator sends light
Signal, optical signal receiver then receives optical signal to obtain sensitive information, and gestures of object sensor is according to sensing
Information produces attitudes vibration information.
In one embodiment of this invention, above-mentioned optical signal is the flicker of infrared ray, laser or fixed frequency
Optical signal.
In one embodiment of this invention, above-mentioned gestures of object sensor includes video camera and three dimensional depth
Sensor, video camera and three dimensional depth sensor configuration are on portable electronic equipment.Video camera can be
Such as colour TV camera.
In one embodiment of this invention, above-mentioned colour TV camera and three dimensional depth sensor at least its
One of collection head mounted display image, and portable electronic equipment according to head mounted display image
To obtain attitudes vibration information.
In one embodiment of this invention, above-mentioned colour TV camera also gathers the image of face of user, and
Make portable electronic equipment that the emotional information of user is judged according to image of face.
In one embodiment of this invention, above-mentioned portable electronic equipment further includes audio signal reception device, wherein receiving
Mixer receives multiple voice signals.Portable electronic equipment is compared to produce output for voice signal
Voice signal.
In one embodiment of this invention, above-mentioned portable electronic equipment receives one and instructs, and according to instruction
To drive head mounted display to show virtual input-interface.
In one embodiment of this invention, whether above-mentioned instruction belongs to specific according to the gesture of detection user
Gesture is produced.
Based on above-mentioned, portable virtual reality system provided by the present invention can be by portable electronic equipment
Coordinate head mounted display, reach whenever and wherever possible, it is arbitrary enjoy virtual reality pleasant sensation.
It is that features described above of the invention and advantage can be become apparent, special embodiment below, and coordinate institute
Accompanying drawings are described in detail below.
Brief description of the drawings
Fig. 1 is the structural representation of the portable virtual reality system of one embodiment of the invention.
Fig. 2 is the use operation chart of the portable virtual reality system of one embodiment of the invention.
Fig. 3 A~Fig. 3 C are respectively that the gestures of object of the portable virtual reality system of one embodiment of the invention is passed
The schematic diagram of the different embodiments of sensor.
Fig. 4 A and Fig. 4 B are respectively that the gestures of object of the portable virtual reality system of one embodiment of the invention is passed
The schematic diagram of the configuration mode of sensor.
Fig. 5 is the portable virtual reality system of one embodiment of the invention in display keyboard image and user
Schematic diagram during hand image.
Fig. 6 is the schematic diagram of skeleton image that shows of portable virtual reality system of one embodiment of the invention,
Skeleton image therein can be divided into palm skeleton image and finger skeleton image.
Fig. 7 A~Fig. 7 C are respectively that the display of the portable virtual reality system of one embodiment of the invention is virtually defeated
Schematic diagram during incoming interface.
Fig. 8 is the operating process schematic diagram of the portable virtual reality system of one embodiment of the invention.
Wherein, reference is as follows：
100：Portable virtual reality system
110、310：Portable electronic equipment
120、320：Head mounted display
130、330：Gestures of object sensor
IFO：Attitudes vibration information
IMGD：Image data displaying
USR：User
KB：Keyboard
HD：Hand
331：Optical signal receiver
332、403-1、403-2：Optical signal generator
333、401：Colour TV camera
334、402：Three dimensional depth sensor
410：Sensing region
IR1、IR2：Infrared light-emitting diode
MIC：Audio signal reception device
SCR：Display screen
612：Palm skeleton image
611：Finger skeleton image
701：Virtual selection key
703：Fictitious plan menu
S810~S873：Portable virtual reality system operating procedure
Specific embodiment
Fig. 1 is refer to, Fig. 1 illustrates the structure of the portable virtual reality system of one embodiment of the invention.Such as
Shown in Fig. 1, portable virtual reality system 100 includes portable electronic equipment 110, head mounted display
120 and gestures of object sensor 130.Portable electronic equipment 110 can be by transfer wire or wireless
Mode is coupled to head mounted display 120, gestures of object sensor 130 respectively with portable electronic equipment 110
And head mounted display 120 is mutually coupled.When virtual reality is carried out, gestures of object sensor 130 is used to
Obtain the attitudes vibration information IFO in space of head mounted display 120, and by attitudes vibration information IFO
It is sent to portable electronic equipment 110.Portable electronic equipment 110 can come according to attitudes vibration information IFO
The image data displaying IMGD of virtual reality is calculated, and image data displaying IMGD is sent to is worn
Formula display 120, so as to head mounted display 120 can carry out image according to image data displaying IMGD show
Show.Obtain head mounted display 120 in space attitudes vibration (such as displacement, rotation and it is skewed
State) after information IFO, by three-dimensional computing, such as the Related Computational Methods being related in camera calibration technology,
The image data displaying IMGD of virtual reality is obtained, then by the data display in head mounted display 120
Display picture in.
It is noted that gestures of object sensor 130 can be separately configured in (example on portable electronic equipment 110
As gestures of object sensor 130 and portable electronic equipment 110 become one), or can be separately configured
In head mounted display 120, (such as gestures of object sensor 130 and head mounted display 120 are integrated into one
Body), or two parts and this two parts is respectively arranged at portable electronic equipment 110 can also be included
And on head mounted display 120.The configuration mode being directed to can be using for example it is known installation or
Inline mode is realized.
In an embodiment of the present invention, gestures of object sensor 130 is configured in portable electronic equipment 110
On, by head-mounted display 120 image collection is carried out to obtain the real-time shadow of head mounted display 120
Picture.Then, portable electronic equipment 110 can obtain head according to the real-time imaging of head mounted display 120
The attitudes vibration information IFO of head mounted displays 120, calculates the image data displaying of virtual reality accordingly
IMGD, and image data displaying IMGD is sent to head mounted display 120, so that wear-type shows
The image that device 120 can carry out virtual reality shows.
In still another embodiment of the process, gestures of object sensor 130 is integrated in head mounted display 120,
By sensing the attitudes vibration of head mounted display 120, (for example user's head pose moves, rotates and inclines
The state such as oblique), obtain the attitudes vibration information IFO of head mounted display 120, and the attitude that will be obtained
Change information IFO sends portable electronic equipment 110 to；Portable electronic equipment 110 calculates void accordingly
Intend the image data displaying IMGD of reality, and image data displaying IMGD is sent to head mounted display
120, so that the image that head mounted display 120 can carry out virtual reality shows.Gestures of object sensor 130
Can be such as three-axis gyroscope, three axis accelerometer and three axis magnetometer etc..
In an alternative embodiment of the invention, gestures of object sensor 130 may include two parts, one of them
It is optical signal generator, another part is optical signal receiver.Optical signal generator and optical signal receiver point
It is positioned respectively on head mounted display 120 and portable electronic equipment 110, or is arranged respectively at portable type electricity
In sub-device 110 and head mounted display 120.By the transmission and reception, portable electronic equipment of light
110 can know the real-time attitudes vibration information IFO of head mounted display 120, and calculate display according to this
Image data IMGD.
Implementation detail on gestures of object sensor 130, will be given more detailed in the implementation method after
Description.
Fig. 2 is refer to, Fig. 2 illustrates the use of the portable virtual reality system of one embodiment of the invention.Its
In, user USR wears head mounted display 120, and starts portable electronic equipment (for example, pen
Remember this computer) 110.By gestures of object sensor 130, the appearance of head mounted display 120 can be obtained
State change information, then portable electronic equipment 110 can be according to the attitudes vibration of head mounted display 120 letter
Breath, produces corresponding image data displaying.In the embodiment depicted in figure 2, portable electronic equipment 110
It is exemplified as in notebook computer, but other embodiments, portable electron device 110 can also be panel computer
Or the portable electronic installation such as intelligent mobile phone.
In an embodiment of the present invention, the gestures of object sensor of portable virtual reality system can include light
Signal generator and optical signal receiver；In an alternative embodiment of the invention, gestures of object sensor can be wrapped
Include at least one of video camera and three dimensional depth sensor；In still another embodiment of the process, gestures of object
Sensor may include in optical signal generator, optical signal receiver, video camera and three dimensional depth sensor extremely
Few one.Refer to the portable void that Fig. 3 A~Fig. 3 C, Fig. 3 A~Fig. 3 C illustrate the embodiment of the present invention respectively
Intend the different embodiments of gestures of object sensor in reality system.
In figure 3 a, gestures of object sensor 330 includes that optical signal receiver 331 and optical signal are produced
Device 332.Optical signal receiver 331 is configured in head mounted display 320, and optical signal generator 332 is matched somebody with somebody
Put in portable electronic equipment 310.In Fig. 3 A illustrated embodiments, configuration is in portable electronic dress
The optical signal that the optical signal generator 332 on 310 can produce to be positioned is put, and is configured in wear-type
After optical signal receiver 331 on display 320 senses the optical signal, the relevant information of optical signal is passed
Deliver to portable electronic equipment 310.Portable electronic equipment 310 can be counted according to the relevant information of optical signal
Calculate the attitudes vibration information of head mounted display 320.Consequently, it is possible to portable electronic equipment 310 can enter
The attitudes vibration information of the foundation head mounted display 320 of one step calculates image data displaying, then will display
Image data sends head mounted display 320 to, so as to realize that display is virtual by head mounted display 320
The image of reality.
In figure 3b, gestures of object sensor 330 includes that optical signal receiver 331 and optical signal are produced
Device 332, wherein optical signal generator 332 are configured in head mounted display 320, optical signal receiver 331
Configuration is in portable electronic equipment 310.In Fig. 3 B illustrated embodiments, configuration shows in wear-type
Optical signal generator 332 on device 320 can produce the optical signal for being positioned, and portable electronic is filled
Put after the optical signal receiver 331 on 310 senses optical signal, portable electronic equipment 310 can be according to connecing
The associated information calculation of the optical signal for receiving goes out the attitudes vibration information of head mounted display 320.Such one
Come, portable electronic equipment 310 can further according to the attitudes vibration information meter of head mounted display 320
Image data displaying is calculated, then sends image data displaying to head mounted display 320, so as to pass through head
Head mounted displays 320 realize the image of display virtual real.
In fig. 3 c, gestures of object sensor 330 includes video camera 333 and three dimensional depth sensor 334.
In Fig. 3 C illustrated embodiments, three dimensional depth sensor 334 can be by RF transmitter and infrared ray
Cmos camera is constituted, and is used to detect the limb action of user；Video camera uses colour TV camera
333 (can also be non-colour TV camera in other examples), by imaging and carry out human face recognition and/or body
The mode of body characteristicses identification, picks out the status of user.Additionally, colour TV camera 333 may further be used to distinguish
The countenance of user is known, with the image of face by gathering user so that portable electronic equipment can
The emotional information of user is judged according to image of face.Furthermore, can also play and regard in augmented reality
Interrogate and colour TV camera 333 is used in the related application of call.In other embodiments, regarded by with reference to machine
Feel algorithm, (such as general red, green, blue (RGB) is colored can directly to utilize colour TV camera 333
Video camera), you can to reach the detection of the limb action of user, can so be not required to three dimensional depth sensor
334。
For the configuration mode of gestures of object sensor, Fig. 4 A and Fig. 4 B depict one embodiment of the invention
The configuration mode of middle gestures of object sensor, wherein, gestures of object sensor can be only fitted to portable electronic
Device 40 sensing region 410 (in diagram sensing region be banding, but other shapes also can) on.
Sensing region 410 can be only fitted to the top or lower section of the display screen SCR of portable electronic equipment 40
(see Fig. 4 B).Below with portable electronic equipment as notebook computer as an example, to Fig. 4 A and figure
Configuration mode is illustrated shown in 4B.
In the implementation method shown in Fig. 4 A, gestures of object sensor include optical signal generator (or light letter
Number receiver) 403-1,403-2, colour TV camera 401 and three dimensional depth sensor 402.Believed with light
Number generator 403-1,403-2 are that RF transmitter is example, and it has at least one infrared light emission
Diode IR1, IR2, can be placed in the both sides of sensing region 410, and, it is also possible to it is placed in sensing area
Both sides and middle part (such as colour TV camera 410 and the depth transducer 402 in Fig. 4 A in domain 410
Between small circle shown in), and infrared light beam is distributed in the front of portable electronic equipment e.g., about
The behaviour area of 150 centimetres of user.In this way, the optical signal of the head-mounted display apparatus in behaviour area
Receiver can receive these infrared light beams, and the relevant information of infrared light beam is sent into portable type
Electronic installation.During one kind is implemented, as shown in Figure 4 A, colour TV camera 401 can be placed in sensing
The center in region 410, three dimensional depth sensor 402 can have two camera lenses, and can be respectively disposed on colour
The both sides of video camera 401.
In other embodiments of the present invention, optical signal generator 403-1,403-2 can also produce laser or
Person produces the light flashed with CF, such as LED light signal being related in visible light communication technology.
In an alternative embodiment of the invention, sensing region 410 can also include audio signal reception device MIC.Radio reception
Device MIC can receive at least one voice signal, and be supplied to portable electronic equipment with for being received
Voice signal be compared to eliminate noise, the voice signal after output noise reduction.Audio signal reception device MIC can
Being realized by such as Array Microphone, by the radio reception simultaneously of multigroup microphone, noise is eliminated after comparison,
Equivalent to noise reduction is carried out, the sound of user can more clearly be passed.
For the acquisition of the attitudes vibration information of head-mounted display apparatus, one embodiment of the invention can be by using
The sensing location technology of multiple spot sensing is achieved.During one kind is realized, portable electronic is placed in by providing
Multiple optical sensors and the single spot light being placed on head mounted display on device, or placement is provided
Multiple optical sensors and the single spot light being placed on portable electron device on head mounted display,
It is then based on the known correlation technique such as luminous intensity, trilateration to realize positioning, is derived from wear-type and shows
Show the attitudes vibration information of equipment.Another kind realizes it being to utilize multiple optical signal launch sources, straight according to optical signal
The principle that line advances, the launch angle of optical signal can have different scatterings with the element characteristic of signal projector
Angle, produces corresponding sensing reception value.Accordingly, optical signal receiver can receive corresponding sensing
Reception value.For example, when RF transmitter sends optical signal, its corresponding infrared receiver with
Adjacent infrared receiver can all receive this optical signal, due to the factor of scattering angle, can complete
Multiple spot senses, and then obtains the attitudes vibration information of head-mounted display apparatus.
In an embodiment of the present invention, portable virtual reality system can also have the defeated of hypostazation or virtualization
Enter equipment, it is coupled with portable electronic equipment.Portable electronic equipment can be additionally used according to input equipment
Use state, control head mounted display is aobvious to the hand image of the image of input equipment and user
Show, for example, determine whether the hand shadow of the image and user for making head mounted display show the input equipment
Picture.
It is possible to be inconvenient to see after user USR wears head mounted display 120 referring again to Fig. 2
Input equipment (input equipment is the keyboard KB that portable electronic equipment 110 itself is equipped with the present embodiment)
Relevant position.In this regard, in the portable virtual reality system of one embodiment of the invention, portable electronic dress
Putting 110 can also gather the image of hand HD of keyboard KB and user USR.So, using
When person needs to be input into by keyboard KB, portable electronic equipment 110 can order about head mounted display
The image of the hand HD of the image and user USR of 120 display KB, so as to avoid traditional VR systems
The VR gloves that system need to arrange in pairs or groups special can just be operated.
Still by taking Fig. 2 as an example, because the position of keyboard is usually fixed in notebook computer, it is possible to will
It is directly changed into the position in three dimensions.By sensing region on notebook computer (such as the 410 of Fig. 4 A)
In camera (such as the 401 of Fig. 4 A) obtain hand image after, hand can be converted into three dimensions
In coordinate.Then, hand image (can for example can be used skeleton mode by portable virtual reality system
Present, or stick default textures and present) picture of head-mounted display apparatus is simultaneously displayed on keyboard image
On face.
When user needs the input through keyboard in notebook computer, user is by head-mounted display apparatus
Picture shows can learn finger and each button relative position, so as to can be done as usual as eye is not blocked
Operation keyboard.It is for other input equipments, such as mouse, flying games bar, choke valve etc., then available
The image of the relevant device for recording in advance, then by the technology of machine vision, determine input equipment in space
In position.It is lifting user's experience, when picture shows user's hand image and keyboard image, can
Hand and keyboard image are laminated on picture in translucent mode, as shown in figure 5, even, can also be
Display eliminates image after a period of time.And for other input equipments, be not usually required to be input into word, make
The relative position that user can be operated without seeing finger and button with feel, it is possible to display with
The related position of hand, allows user to touch input equipment with hand.
Diagram is combined further below to being related to input equipment and the image of user's hand to elaborate.
Fig. 5 is refer to, Fig. 5 illustrates the portable virtual reality system of one embodiment of the invention to keyboard image
And the display of user's hand image.Wherein, to lead to the user for wearing head-mounted display apparatus
Cross keyboard to be input into, the portable electronic equipment of the embodiment of the present invention can wear formula by driving head when necessity
Display carries out the display of hand and keyboard image.Hand and keyboard image can be sensed by gestures of object
Device is obtained.Portable electronic equipment can be obtained by judging the corresponding relation of the image of hand image and keyboard
Keyboard by use state, and when portable electronic equipment judge keyboard be in use state when, make
Head mounted display shows the image of hand and keyboard.
Still using keyboard as input equipment as a example by, it uses scene as follows：When user is using input through keyboard text
During word, generally two hands are placed on keyboard, and the displacement of palm portion is little, two finger partial dislocations are frequent,
Now can be with continuously display hand image and keyboard image in display picture；When user plays to play, its
Mode of operation is probably proficiency operation keyboard, and another hand operates mouse, or two hand operation keyboards, or two
Hand is all operating joystick, because the function key of games is generally fixed, so the displacement of palm portion is not
Greatly, and finger part is operated on several fixed keys, now in display picture in can after the set time
Eliminate the display of hand image and keyboard image；When user's both hands all leave keyboard, the hand shadow of user
After with keyboard image non-overlapping more than one section of Preset Time, hand image can be eliminated in display picture
And the display of keyboard image；And, when user's either hand outside keyboard by moving on keyboard (for example originally
One hand operation, in order to carry out word input, another hand is moved on keyboard；Or, two hands of script
All outside keyboard, it is ready for being input into or operates and move on keyboard) when, then hand is shown in display picture
Portion's image and keyboard image.
According to scene used above, can allow portable virtual reality system automatic decision user action and
It is automatic to be shown in picture or the elimination from picture hand image and keyboard image.And on palm image with
Finger image whether there is the judgement details being subjected to displacement, can be according to palm image (the finger shadow in continuous time
Picture) change in location whether there is more than a default distance, when palm image (finger image) position become
Change exceed acquiescence apart from when, represent user palm (or finger) occur it is effective mobile, relatively,
If the change in location of palm image (finger image) be not above acquiescence apart from when, represent user hand
There is no effective movement in the palm (or finger).The appearance and elimination of image in foregoing picture, can be by system certainly
It is dynamic to judge, can also be shown after hot key, voice, gesture etc. are triggered by user or elimination image.Additionally,
System is used to judge the threshold value of palm (finger) displacement, can be set by user or by study
User's input habit and set.
Judgment mode on position relationship between above-mentioned hand image and keyboard image, it is also possible to utilize foundation
Hand image carries out skeleton image that image procossing obtained to carry out.Refer to the present invention one that Fig. 6 is illustrated
The skeleton image that embodiment portable virtual reality system shows.Skeleton image therein can be divided into palm skeleton
Image 612 and finger skeleton image 611.Portable electronic equipment can be according to palm skeleton image 612
And the corresponding relation of finger skeleton image 611 and keyboard image is defeated to be made whether to be carried out by keyboard
The basis for estimation for entering, and decide whether to make head-mounted display apparatus show hand image and keyboard image.
In the portable virtual reality system of one embodiment of the invention, portable electronic equipment can also be received
For driving head mounted display to show the instruction of virtual input-interface.In this embodiment, the instruction passes through
User is produced using the mode of certain gestures, for example, can be set in the storage of portable electronic equipment in advance
Standby middle this certain gestures of storage, when portable electronic equipment is imaged or sensed by gestures of object sensor
After the certain gestures, head mounted display is driven to provide the display picture corresponding to virtual input-interface.Please join
The portable virtual reality system for illustrating the embodiment respectively according to Fig. 7 A~Fig. 7 C, Fig. 7 A~Fig. 7 C shows empty
Intend the process of input interface.In fig. 7, user uses the particular hand for representing startup virtual input-interface
Gesture, and portable electronic equipment such as is imaged or is sensed at the mode by gestures of object sensor obtains the certain gestures
Hand image so that obtain start virtual input-interface instruction.For example, user passes through the particular hand
Gesture opens the software interface for needing operation, and software interface and hand image are shown in picture by head mounted display
Middle confession user viewing.In figure 7b, including multiple virtual selection key 701 virtual input-interface with
And hand image is shown, user can touch button 701 to be held to select using hand image
Capable action.In fig. 7 c, the selection action of the user of corresponding diagram 7B, the fictitious plan quilt of menu 703
It is displayed in head mounted display so that user is further operated.The present invention is to virtual input-interface
There is no special form to limit, it can also be dummy keyboard.
Fig. 8 is refer to, Fig. 8 illustrates a specific behaviour of the portable virtual reality system of one embodiment of the invention
Make process.In step S810, portable virtual reality system is opened,；In step S820, can take
By its gestures of object sensor, (such as optical signal generator and optical signal are received formula virtual reality system
Device) sensed and positioned, and the attitudes vibration information of head-mounted display apparatus is judged according to this；In step
In S830, head mounted display can show coherent video in virtual reality imagery, for example, show notes
The image (or only showing its keyboard image) of this computer, and user's hand is carried out in step S850
Positioning.Then, the certain gestures of correspondence user, in step S850, virtual reality imagery shows master
Menu, and by the selection to primary option menu, occur the menu of model selection in step S860.User
The menu that model selection can be directed to is selected, and selects to enter the typewriter mode (step for representing input word
S871), perform amplification virtual reality program (step S872) or carry out video call (step S873)
Pattern etc..
In sum, the present invention can be taken using portable electronic equipment collocation wear-type electronic installation to set up out
Formula virtual reality system, by portable electronic equipment or in portable electronic equipment and wear-type electronics
Gestures of object sensor is configured between device, consequently, it is possible to significantly simplify the complexity of virtual reality system.And
User need not arrive specific space, and whenever and wherever possible and can follow one's bent and carry out the display of virtual reality
Action, lifts the convenience of virtual reality system, it is believed that can lift the use wish of user.
Although the present invention is disclosed above with embodiment, so it is not limited to the present invention, any affiliated skill
Have usually intellectual in art field, without departing from the spirit and scope of the present invention, when can make it is a little more
Dynamic and retouching, therefore protection scope of the present invention ought be defined depending on the appended claims person of defining.
Claims (13)
1. a kind of portable virtual reality system, it is characterised in that including：
Portable electronic equipment；
Head mounted display, couples the portable electronic equipment；And
Gestures of object sensor, is respectively coupled to the portable electronic equipment and the head mounted display, is used to take
The attitudes vibration information of the head mounted display is obtained,
Wherein, the portable electronic equipment calculates the display shadow of virtual reality according to the attitudes vibration information
As data, and the image data displaying is sent to the head mounted display so that the head mounted display according to
Image is carried out according to the image data displaying to show.
2. portable virtual reality system as claimed in claim 1, it is characterised in that also including input
Equipment, the portable electronic equipment, by use state, decides whether to wear this also according to the input equipment
Formula display shows the image of the input equipment and the hand image of user.
3. portable virtual reality system as claimed in claim 2, it is characterised in that the input equipment
Including keyboard.
4. portable virtual reality system as claimed in claim 3, it is characterised in that the gestures of object
Sensor obtains the hand image and the image of the keyboard, and the portable electronic equipment is according to the hand image
And the corresponding relation of the image of the keyboard, obtain the keyboard by use state.
5. portable virtual reality system as claimed in claim 3, it is characterised in that portable type electricity
Sub-device obtains the skeleton image of the hand image according to the hand image, and according to the skeleton image and
The corresponding relation of the image of the keyboard, obtain the keyboard by use state.
6. portable virtual reality system as claimed in claim 5, it is characterised in that the framework information
Including palm skeleton image and finger skeleton image, wherein the portable electronic equipment is according to the palm skeleton
The overlap condition and the palm skeleton image of image and the finger skeleton image and the image of the keyboard and should
The mobile status of finger skeleton image, the image for deciding whether to make the head mounted display show the keyboard and
The hand image.
7. portable virtual reality system as claimed in claim 1, it is characterised in that the gestures of object
Sensor is included for sending the optical signal generator of optical signal and for receiving the optical signal to be felt
The optical signal receiver of measurement information, the wherein optical signal generator are placed in the portable electronic equipment, the light
Signal receiver is placed in the head mounted display, or, the optical signal generator is placed in the wear-type and shows
Show device, the optical signal receiver is placed in the portable electronic equipment；The gestures of object sensor is according to the sense
Measurement information produces the attitudes vibration information.
8. portable virtual reality system as claimed in claim 7, it is characterised in that the optical signal is
The optical signal of infrared ray, laser or fixed frequency flicker.
9. portable virtual reality system as claimed in claim 1, it is characterised in that the gestures of object
Sensor includes at least one of video camera and three dimensional depth sensor, is used to gather the wear-type and shows
The image of device, the video camera and the three dimensional depth sensor are placed in the portable electronic equipment, the carrying
Formula electronic installation obtains the attitudes vibration information according to the image of the head mounted display.
10. portable virtual reality system as claimed in claim 9, it is characterised in that the video camera is also
The image of face of user is gathered, and the portable electronic equipment is judged that this is used according to the image of face
The emotional information of person.
11. portable virtual reality systems as claimed in claim 1, it is characterised in that portable type electricity
Sub-device also includes audio signal reception device, is used to receive multiple voice signals, and the portable electronic equipment is to the sound
Message number is compared to eliminate noise, and exports the voice signal after noise reduction.
12. portable virtual reality systems as claimed in claim 1, it is characterised in that portable type electricity
Sub-device receive instruction, and according to the instruction with drive the head mounted display show virtual input-interface.
13. portable virtual reality systems as claimed in claim 12, it is characterised in that the instruction according to
Produced according to whether the gesture of the user for detecting belongs to certain gestures.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| TW104141863A TWI596378B (en) | 2015-12-14 | 2015-12-14 | Portable virtual reality system |
| TW104141863 | 2015-12-14 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106873760A true CN106873760A (en) | 2017-06-20 |
Family
ID=59239173
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510942848.0A Pending CN106873760A (en) | 2015-12-14 | 2015-12-16 | Portable virtual reality system |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN106873760A (en) |
| TW (1) | TWI596378B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107357434A (en) * | 2017-07-19 | 2017-11-17 | 广州大西洲科技有限公司 | Information input equipment, system and method under a kind of reality environment |
| TWI648556B (en) * | 2018-03-06 | 2019-01-21 | 仁寶電腦工業股份有限公司 | Slam and gesture recognition method |
| CN111045209A (en) * | 2018-10-11 | 2020-04-21 | 光宝电子(广州)有限公司 | Travel system and method using unmanned aerial vehicle |
| CN112947901A (en) * | 2019-12-11 | 2021-06-11 | 中华电信股份有限公司 | Augmented reality content control system, method and computer storage medium |
| CN114489315A (en) * | 2020-11-12 | 2022-05-13 | 幻景启动股份有限公司 | Image processing system and image processing apparatus |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109495733B (en) | 2017-09-12 | 2020-11-06 | 宏达国际电子股份有限公司 | Three-dimensional image reconstruction method, device and non-transitory computer readable storage medium thereof |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1462929A (en) * | 2002-05-20 | 2003-12-24 | 许旻 | Input system of computers |
| CN103019377A (en) * | 2012-12-04 | 2013-04-03 | 天津大学 | Head-mounted visual display equipment-based input method and device |
| CN103197757A (en) * | 2012-01-09 | 2013-07-10 | 癸水动力（北京）网络科技有限公司 | Immersion type virtual reality system and implementation method thereof |
| CN103534665A (en) * | 2011-04-04 | 2014-01-22 | 英特尔公司 | Keyboard avatar for heads up display (hud) |
| WO2015183621A1 (en) * | 2014-05-30 | 2015-12-03 | Sony Computer Entertainment America Llc | Head mounted display (hmd) system having interface with mobile computing device for rendering virtual reality content |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9255813B2 (en) * | 2011-10-14 | 2016-02-09 | Microsoft Technology Licensing, Llc | User controlled real object disappearance in a mixed reality display |
| TW201447375A (en) * | 2013-06-13 | 2014-12-16 | Hsiu-Chi Yeh | Head wearable electronic device and method for augmented reality |
| US9857971B2 (en) * | 2013-12-02 | 2018-01-02 | Industrial Technology Research Institute | System and method for receiving user input and program storage medium thereof |
-
2015
- 2015-12-14 TW TW104141863A patent/TWI596378B/en not_active IP Right Cessation
- 2015-12-16 CN CN201510942848.0A patent/CN106873760A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1462929A (en) * | 2002-05-20 | 2003-12-24 | 许旻 | Input system of computers |
| CN103534665A (en) * | 2011-04-04 | 2014-01-22 | 英特尔公司 | Keyboard avatar for heads up display (hud) |
| CN103197757A (en) * | 2012-01-09 | 2013-07-10 | 癸水动力（北京）网络科技有限公司 | Immersion type virtual reality system and implementation method thereof |
| CN103019377A (en) * | 2012-12-04 | 2013-04-03 | 天津大学 | Head-mounted visual display equipment-based input method and device |
| WO2015183621A1 (en) * | 2014-05-30 | 2015-12-03 | Sony Computer Entertainment America Llc | Head mounted display (hmd) system having interface with mobile computing device for rendering virtual reality content |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107357434A (en) * | 2017-07-19 | 2017-11-17 | 广州大西洲科技有限公司 | Information input equipment, system and method under a kind of reality environment |
| TWI648556B (en) * | 2018-03-06 | 2019-01-21 | 仁寶電腦工業股份有限公司 | Slam and gesture recognition method |
| CN111045209A (en) * | 2018-10-11 | 2020-04-21 | 光宝电子(广州)有限公司 | Travel system and method using unmanned aerial vehicle |
| CN112947901A (en) * | 2019-12-11 | 2021-06-11 | 中华电信股份有限公司 | Augmented reality content control system, method and computer storage medium |
| CN114489315A (en) * | 2020-11-12 | 2022-05-13 | 幻景启动股份有限公司 | Image processing system and image processing apparatus |
| CN114489315B (en) * | 2020-11-12 | 2023-12-19 | 幻景启动股份有限公司 | Image processing system and image processing apparatus |
Also Published As
| Publication number | Publication date |
|---|---|
| TW201721231A (en) | 2017-06-16 |
| TWI596378B (en) | 2017-08-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106873760A (en) | Portable virtual reality system | |
| CN106575164B (en) | Detection device, detection method, control device, and control method | |
| US8425322B2 (en) | System and method for communicating with a virtual world | |
| JP5256269B2 (en) | Data generation apparatus, data generation apparatus control method, and program | |
| CN108619721A (en) | Range information display methods, device and computer equipment in virtual scene | |
| CN108984087B (en) | Social interaction method and device based on three-dimensional virtual image | |
| EP2132650A2 (en) | System and method for communicating with a virtual world | |
| US20190026950A1 (en) | Program executed on a computer for providing virtual space, method and information processing apparatus for executing the program | |
| CN103019377A (en) | Head-mounted visual display equipment-based input method and device | |
| CN110300274B (en) | Video file recording method, device and storage medium | |
| CN108924412B (en) | Shooting method and terminal equipment | |
| WO2008109299A2 (en) | System and method for communicating with a virtual world | |
| JP6514376B1 (en) | Game program, method, and information processing apparatus | |
| CN109859307A (en) | A kind of image processing method and terminal device | |
| CN110489029A (en) | A kind of icon display method and terminal device | |
| CN113365085B (en) | Live video generation method and device | |
| CN109671034B (en) | Image processing method and terminal equipment | |
| CN109471586B (en) | Keycap color matching method and device and terminal equipment | |
| CN107754316B (en) | Information exchange processing method and mobile terminal | |
| US20200034729A1 (en) | Control Method, Terminal, and System | |
| CN112995687A (en) | Interaction method, device, equipment and medium based on Internet | |
| CN112494958A (en) | Method, system, equipment and medium for converting words by voice | |
| CN115002443B (en) | Image acquisition processing method and device, electronic equipment and storage medium | |
| CN112843703B (en) | Information display method, device, terminal and storage medium | |
| US11054941B2 (en) | Information processing system, information processing method, and program for correcting operation direction and operation amount |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: Taiwan Xindian District, New Taipei City Chinese Po Road No. 6 Applicant after: GIGA-BYTE TECHNOLOGY Co.,Ltd. Address before: Chinese Taiwan Taipei City store Bao Jiang Road No. 6 Applicant before: GIGA-BYTE TECHNOLOGY Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20170620 |
|
| WD01 | Invention patent application deemed withdrawn after publication |