CN107404601A - A kind of apparatus and method for increasing self element of experiencer in virtual reality - Google Patents
A kind of apparatus and method for increasing self element of experiencer in virtual reality Download PDFInfo
- Publication number
- CN107404601A CN107404601A CN201610329153.XA CN201610329153A CN107404601A CN 107404601 A CN107404601 A CN 107404601A CN 201610329153 A CN201610329153 A CN 201610329153A CN 107404601 A CN107404601 A CN 107404601A
- Authority
- CN
- China
- Prior art keywords
- virtual reality
- image
- experiencer
- foreground
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/30—Image reproducers
- H04N13/332—Displays for viewing with the aid of special glasses or head-mounted displays [HMD]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/2224—Studio circuitry; Studio devices; Studio equipment related to virtual studio applications
- H04N5/2226—Determination of depth image, e.g. for foreground/background separation
Abstract
The invention discloses a kind of apparatus and method for increasing self element of experiencer in virtual reality.The outside experience environment of virtual reality is built by green curtain.The experiencer being pointed to using the fish eye camera on virtual implementing helmet in green curtain is shot, and image processing module is partitioned into experiencer's images themselves.The experiencer's images themselves split are merged with the virtual reality scenario of generation all the way, enter gesture recognition module all the way in addition, generate control information.So experiencer not only can see virtual reality scenario in virtual reality picture, and can see the body of oneself, four limbs, both hands, virtual reality scenario can also be manipulated by gesture, and the scene for greatly improving virtual reality incorporates sense.
Description
Technical field
The present invention relates to field of virtual reality, and in particular to a kind of dress for increasing self element of experiencer in virtual reality
Put and method, experiencer is not only can see virtual reality scenario, and oneself can be seen in virtual reality scenario
Body, the four limbs and both hands of oneself, while this system can also allow experiencer to virtual by the gesture identification to experiencer
Reality scene is controlled.
Background technology
Virtual implementing helmet provides the experience of virtual reality scenario immersion on the spot in person for experiencer.But due to void
The characteristics of intending real helmet closure, isolates the body of experiencer with virtual reality scenario and comes, that is, puts on virtual reality head
After helmet, the body of oneself is just can't see, four limbs, is particularly easiest to be also at the both hands within field range, also from the visual field
In disappear, so psychologically can greatly increase itself separation sense with viewing scene, reduce the scene of virtual reality
The sense of reality, reduce that virtual sensation on the spot in person.Simultaneously in order to be manipulated to virtual reality scenario, virtual reality
Helmet generally use remote-control handle because remote-control handle is not a part for body after all, once handle departing from palm because
Sight is limited within virtual implementing helmet, finds handle again with regard to extremely difficult.
In order to improve the authenticity of virtual reality, display system and signal capture system of the equipment supplier in virtual reality
Do substantial amounts of work, such as improved the frame per second of virtual reality device, improve the capture velocity and response speed of signal, but this
A little is all among external equipment angle allows experiencer's involvement virtual reality scenario, enough experiencer could not be allowed psychologically to feel
Oneself is just among scene.In order to allow experiencer to feel oneself among virtual scene it is necessary to allow experiencer in virtual reality
See oneself in scene, it is seen that the body of oneself, it is seen that the four limbs of oneself.In order to solve the drawbacks of remote-control handle, it has been proposed that
Remote-control handle is substituted using gesture identification, remote-control handle is so avoided and departs from the trouble that palm is found again.But this
The problem of a little gesture identification maximums is that hand motion is all to depart from sight, and this equally can also reduce the telepresenc of experiencer.
The method for solving this problem is that experiencer's hand, arm image are merged with the image of virtual reality scenario, forms fusion
Image, so virtual reality scenario can both be manipulated, again experiencer can be allowed to see oneself by virtual reality picture
The action of hand.
The content of the invention
The invention discloses a kind of apparatus and method for increasing self element of experiencer in virtual reality.Taken by green curtain
Build the outside experience environment of virtual reality.Utilize body of the fish eye camera shooting in green curtain on virtual implementing helmet
The person's of testing images themselves, image processing module are partitioned into experiencer's images themselves.The experiencer's images themselves split all the way with
The virtual reality scenario of generation is merged, and enters gesture recognition module all the way in addition, generates control information.So experiencer exists
Virtual reality scenario is not only can see in virtual reality picture, and can see the body of oneself, four limbs, both hands, may be used also
Virtual reality scenario is manipulated by gesture.
A kind of apparatus and method for increasing self element of experiencer in virtual reality, it is characterised in that：Including following step
Suddenly：
Including green curtain system and headgear system：
(1) green curtain system provides green background and illuminator, and structure performs in a radio or TV programme environment；
(2) headgear system includes image capture module, image processing module, display control module, gyroscope, display,
Virtual reality input module；
(3) picture catching system module connection image processing module, image processing module are divided into image segmentation submodule and figure
As identification submodule；
(4) image of image capture module output is divided into prospect and the back of the body by the segmentation submodule of image processing module
Scape, and generate foreground mask；
(5) foreground image that image segmentation submodule is partitioned into is input to image recognition submodule and is identified, and generates hand
Gesture control signal, is input to display control module；
(6) gesture control signal and virtual reality input module of the display control module receiving from image processing module are defeated
The scene signals entered, current virtual reality scenario is generated, receive the head angle signal from gyroscope, according to current virtual
Reality scene generates the virtual reality scenario image of current angular；
(7) foreground image and foreground mask image input display control module, the virtual reality scenario figure with current angular
As fusion, raw row has the display image of three-dimensional virtual reality scenario and three-dimensional foreground image.
The circular green curtain of front and rear six direction up and down, and reasonable disposition light are established in above-mentioned steps 1, builds one
The green background environment of closing.
Image capture module is a camera chain in above-mentioned steps 2.The camera lens of video camera is using the angle of visual field at 150 °
Fish eye lens above.In order to reduce the delay of the gesture control of virtual reality system, the frame per second of video camera is in more than 60fps.
Image segmentation submodule in above-mentioned steps 3 in image procossing is calculated using green algorithm segmentation foreground and background is scratched
Formula is：
Wherein M (x, y) is foreground mask value of the prospect in pixel (x, y), and I represents that the pixel is prospect, and 0 is background.g
(x, y), b (x, y) and r (x, y) difference pixel (x, y) green channels, blue channel and red color channel value, thg, thbAnd thrPoint
Not Wei three passages threshold value.By above formula, foreground image I is obtainedfg(x, y).
Panorama picture of fisheye lens model is utilized in above-mentioned steps 4, makes foreground image and foreground mask respectively to the left and to right avertence
θ is moved, general θ values are 0-4 °, respectively obtain left eye foreground image IfgL(x, y) and right eye foreground image IfgR(x, y), before left eye
Scape mask ML(x, y) and right eye foreground mask MR(x, y).Because left eye foreground image IfgL(x, y) and right eye foreground image IfgR
There is the subtense angle that angle is 2 θ in (x, y), when being respectively displayed on left eye and right eye, form anaglyph, handled by brain, shape
Into stereo-picture.
Image recognition submodule in above-mentioned steps 5 in image processing module is to foreground image Ifg(x, y) carries out image knowledge
Not：
(1) Face Detection is carried out for foreground image, is partitioned into broca scale as IfgSk(x, y)；
(2) to broca scale as IfgSk(x, y) matches with the gesture masterplate in system, finds optimal matching template, and
And meet threshold value, then export corresponding gesture control signal；If Optimum Matching can not meet threshold condition, gesture is not exported
Control signal；
Display control module is according to the virtual reality scenario and gesture control signal of input in above-mentioned steps 6, and generation is currently
Virtual reality scenario；
Display control module divides according to the current virtual reality scenario of input and the head angle signal from gyroscope
Left-eye image I that Sheng Cheng be in current angular virtual reality scenariovrL(x, y) and eye image IvrR(x, y), same IvrL(x, y)
With IvrRThere is the subtense angle that angle is 2 θ in (x, y), synchronously play in left eye right eye, handled by brain respectively, obtain stereogram
Shape；
To the right and left eyes foreground image of input, mask image and right and left eyes virtual reality scenario image enter display control module
Row fusion, obtains display image.Its fusion formula is：
It is synchronous over the display in above-mentioned steps 7 to play IDisplayL(x, y) and IDisplayR(x, y), watched by the helmet,
Three-dimensional virtual reality scenario and three-dimensional prospect are can see simultaneously, i.e., the virtual reality scenario of solid is seen by the helmet,
It can also be seen that the body of oneself, four limbs, feel that oneself seemingly really enters virtual reality scenario.
Brief description of the drawings
Fig. 1 is virtual implementing helmet.
Fig. 2 is each module connection figure of virtual implementing helmet.
Fig. 3 is image processing module process chart.
Fig. 4 is display control module process chart.
Embodiment
Below in conjunction with the accompanying drawings, the present invention will be further described：
The present invention implements to propose a kind of apparatus and method for increasing self element of experiencer in virtual reality.It is performed
Process is as follows：
Including green curtain system and headgear system：
The first step：Green curtain system provides green background and illuminator, and structure performs in a radio or TV programme environment；
The circular green curtain of front and rear six direction up and down, and reasonable disposition light are established, builds the green of a closing
Background environment.
Second step：Headgear system includes image capture module, image processing module, display control module, gyroscope, display
Device, virtual reality input module；
(1) picture catching system module connection image processing module, image processing module are divided into image segmentation submodule and figure
As identification submodule；
Image capture module is a camera chain.The camera lens of video camera uses flake mirror of the angle of visual field more than 150 °
Head.In order to reduce the delay of the gesture control of virtual reality system, it is proposed that the frame per second of video camera is in more than 60fps.
(2) image of image capture module output is divided into prospect and the back of the body by the segmentation submodule of image processing module
Scape, and generate foreground mask；
Image segmentation submodule in image procossing splits foreground and background using green algorithm is scratched, and calculation formula is：
Wherein M (x, y) is foreground mask value of the prospect in pixel (x, y), and 1 represents that the pixel is prospect, and 0 is background.g
(x, y), b (x, y) and r (x, y) difference pixel (x, y) green channels, blue channel and red color channel value, thg, thbAnd thrPoint
Not Wei three passages threshold value.By above formula, foreground image I is obtainedfg(x, y).
Using panorama picture of fisheye lens model, foreground image and foreground mask is set to offset θ to the left and to the right respectively, general θ takes
It is worth for 0-4 °, respectively obtains left eye foreground image IfgL(x, y) and right eye foreground image IfgR(x, y), left eye foreground mask ML(x,
And right eye foreground mask M y)R(x, y).Because left eye foreground image IfgL(x, y) and right eye foreground image IfgRFolder be present in (x, y)
Angle is 2 θ subtense angle, when being respectively displayed on left eye and right eye, forms anaglyph, is handled by brain, forms stereo-picture.
(3) foreground image that image segmentation submodule is partitioned into is input to image recognition submodule and is identified, and generates hand
Gesture control signal, is input to display control module；
Image recognition submodule in image processing module is to foreground image Ifg(x, y) carries out image recognition：
Face Detection is carried out for foreground image, is partitioned into broca scale as IfgSk(x, y)；
To broca scale as IfgSk(x, y) matches with the gesture masterplate in system, finds optimal matching template, and full
Sufficient threshold value, then export corresponding gesture control signal；If Optimum Matching can not meet threshold condition, gesture control is not exported
Signal；
(4) gesture control signal and virtual reality input module of the display control module receiving from image processing module are defeated
The scene signals entered, current virtual reality scenario is generated, receive the head angle signal from gyroscope, according to current virtual
Reality scene generates the virtual reality scenario image of current angular；
Display control module generates current virtual reality field according to the virtual reality scenario and gesture control signal of input
Scape；
Display control module divides according to the current virtual reality scenario of input and the head angle signal from gyroscope
Left-eye image I that Sheng Cheng be in current angular virtual reality scenariovrL(x, y) and eye image IvrR(x, y), same IvrL(x, y)
With IvrRThere is the subtense angle that angle is 2 θ in (x, y), synchronously play in left eye right eye, handled by brain respectively, obtain stereogram
Shape；
To the right and left eyes foreground image of input, mask image and right and left eyes virtual reality scenario image enter display control module
Row fusion, obtains display image.Its fusion formula is：
(5) foreground image and foreground mask image input display control module, the virtual reality scenario figure with current angular
As fusion, raw row has the display image of three-dimensional virtual reality scenario and three-dimensional foreground image.
It is synchronous over the display to play IDisplayL(x, y) and IDisplayR(x, y), watched by the helmet, can see simultaneously
Three-dimensional virtual reality scenario and three-dimensional prospect, i.e., the virtual reality scenario of solid is seen by the helmet, it can also be seen that from
Oneself body, four limbs, feel that oneself seemingly really enters virtual reality scenario.
Exemplary description is carried out to the present invention above in conjunction with accompanying drawing, it is clear that realization of the invention is not by aforesaid way
Limitation, it is or not improved by the present invention as long as employing the various improvement of inventive concept and technical scheme of the present invention progress
Design and technical scheme directly apply to other occasions, within the scope of the present invention.
Claims (10)
- A kind of 1. apparatus and method for increasing self element of experiencer in virtual reality, it is characterised in that：Including green curtain system And headgear system：(1) green curtain system includes green background and illuminator, and structure performs in a radio or TV programme environment；(2) headgear system includes image capture module, image processing module, display control module, gyroscope, display, virtual Real input module；(3) picture catching system module connection image processing module, image processing module is divided into image segmentation submodule and image is known Small pin for the case module；(4) image of image capture module output is divided into foreground and background by the segmentation submodule of image processing module, and Generate foreground mask；(5) foreground image that image segmentation submodule is partitioned into is input to image recognition submodule and is identified, and generates gesture control Signal processed, is input to display control module；(6) display control module receives the gesture control signal from image processing module and the input of virtual reality input module Scene signals, current virtual reality scenario is generated, receive the head angle signal from gyroscope, according to current virtual reality Scene generates the virtual reality scenario image of current angular；(7) foreground image and foreground mask image input display control module, melt with the virtual reality scenario image of current angular Close, raw row has the display image of three-dimensional virtual reality scenario and three-dimensional foreground image.
- 2. a kind of apparatus and method for increasing self element of experiencer in virtual reality according to claim 1, its feature It is：The circular green curtain of front and rear six direction up and down, and reasonable disposition light are established, builds the green background of a closing Environment.
- 3. a kind of apparatus and method for increasing self element of experiencer in virtual reality according to claim 1, its feature It is：Image capture module is a camera chain.The camera lens of video camera uses fish eye lens of the angle of visual field more than 150 °. In order to reduce the delay of the gesture control of virtual reality system, the frame per second of video camera is in more than 60fps.
- 4. a kind of apparatus and method for increasing self element of experiencer in virtual reality according to claim 1, its feature It is：Image segmentation submodule in image procossing splits foreground and background using green algorithm is scratched, and calculation formula is：Wherein M (x, y) is foreground mask value of the prospect in pixel (x, y), and 1 represents that the pixel is prospect, and 0 is background.G (x, Y), b (x, y) and r (x, y) difference pixel (x, y) green channel, blue channel and red color channel value, thg, thbAnd thrRespectively The threshold value of three passages.By above formula, foreground image I is obtainedfg(x, y).
- 5. a kind of apparatus and method for increasing self element of experiencer in virtual reality according to claim 1, its feature It is：Using panorama picture of fisheye lens model, foreground image and foreground mask is set to offset 0 to the left and to the right respectively, general θ values are 0-4 °, respectively obtain left eye foreground image IfgL(x, y) and right eye foreground image IfgR(x, y), left eye foreground mask ML(x, y) and Right eye foreground mask MR(x, y).So left eye foreground image IfgL(x, y) and right eye foreground image IfgRIt is 2 that (x, y), which has angle, θ subtense angle, when being respectively displayed on left eye and right eye, anaglyph is formed, is handled by brain, form stereo-picture.
- 6. a kind of apparatus and method for increasing self element of experiencer in virtual reality according to claim 1, its feature It is：Image recognition submodule in image processing module is to foreground image Ifg(x, y) carries out image recognition：(1) Face Detection is carried out for foreground image, is partitioned into broca scale as IfgSk(x, y)；(2) broca scale is as IfgSk(x, y) matches with the gesture masterplate in system, finds optimal matching template, and meet threshold Value, then export corresponding gesture control signal；If Optimum Matching can not meet threshold condition, gesture control letter is not exported Number.
- 7. a kind of apparatus and method for increasing self element of experiencer in virtual reality according to claim 1, its feature It is：Display control module generates current virtual reality scenario according to the virtual reality scenario and gesture control signal of input.
- 8. a kind of apparatus and method for increasing self element of experiencer in virtual reality according to claim 1, its feature It is：Display control module is according to the current virtual reality scenario of input and the head angle signal from gyroscope, difference Generate the left-eye image I in current angular virtual reality scenariovrL(x, y) and eye image IvrR(x, y), same IvrL(x, y) with IvrRThere is the subtense angle that angle is 2 θ in (x, y), synchronously play in left eye right eye, handled by brain respectively, obtain stereogram Shape.
- 9. a kind of apparatus and method for increasing self element of experiencer in virtual reality according to claim 1, its feature It is：To the right and left eyes foreground image of input, mask image and right and left eyes virtual reality scenario image are carried out display control module Fusion, obtains display image.Its fusion formula is：。
- 10. a kind of apparatus and method for increasing self element of experiencer in virtual reality according to claim 1, its feature It is：It is synchronous over the display to play IDisplayL(x, y) and IDisplayR(x, y), watched, while can see vertical by the helmet The virtual reality scenario of body and three-dimensional prospect, i.e., the virtual reality scenario of solid is seen by the helmet, it can also be seen that oneself Body, four limbs, virtual reality scenario can also be controlled using gesture, feel and oneself seemingly really enter virtually Reality scene.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610329153.XA CN107404601A (en) | 2016-05-18 | 2016-05-18 | A kind of apparatus and method for increasing self element of experiencer in virtual reality |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610329153.XA CN107404601A (en) | 2016-05-18 | 2016-05-18 | A kind of apparatus and method for increasing self element of experiencer in virtual reality |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107404601A true CN107404601A (en) | 2017-11-28 |
Family
ID=60393938
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610329153.XA Pending CN107404601A (en) | 2016-05-18 | 2016-05-18 | A kind of apparatus and method for increasing self element of experiencer in virtual reality |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107404601A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108762508A (en) * | 2018-05-31 | 2018-11-06 | 北京小马当红文化传媒有限公司 | A kind of human body and virtual thermal system system and method for experiencing cabin based on VR |
| CN109413338A (en) * | 2018-09-28 | 2019-03-01 | 北京戏精科技有限公司 | A kind of method and system of scan picture |
| US10877641B2 (en) | 2018-10-18 | 2020-12-29 | Boe Technology Group Co., Ltd. | Image adjustment method, apparatus, device and computer readable storage medium |
| CN112862819A (en) * | 2021-03-17 | 2021-05-28 | 苏州灵目电子科技有限公司 | Device and method for increasing self elements of experiencer in virtual reality |
-
2016
- 2016-05-18 CN CN201610329153.XA patent/CN107404601A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108762508A (en) * | 2018-05-31 | 2018-11-06 | 北京小马当红文化传媒有限公司 | A kind of human body and virtual thermal system system and method for experiencing cabin based on VR |
| CN109413338A (en) * | 2018-09-28 | 2019-03-01 | 北京戏精科技有限公司 | A kind of method and system of scan picture |
| US10877641B2 (en) | 2018-10-18 | 2020-12-29 | Boe Technology Group Co., Ltd. | Image adjustment method, apparatus, device and computer readable storage medium |
| CN112862819A (en) * | 2021-03-17 | 2021-05-28 | 苏州灵目电子科技有限公司 | Device and method for increasing self elements of experiencer in virtual reality |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10645369B2 (en) | Stereo viewing | |
| CN105187723B (en) | A kind of image pickup processing method of unmanned vehicle | |
| US8717352B2 (en) | Tracing-type stereo display apparatus and tracing-type stereo display method | |
| US20180315364A1 (en) | Information Processing Apparatus and Image Generation Method | |
| CN108924540A (en) | Low latency mixed reality head wearable device | |
| CN104615234B (en) | Message processing device and information processing method | |
| TWI633336B (en) | Helmet mounted display, visual field calibration method thereof, and mixed reality display system | |
| CN107404601A (en) | A kind of apparatus and method for increasing self element of experiencer in virtual reality | |
| US11930156B2 (en) | Methods and associated devices and systems for enhanced 2D and 3D vision | |
| US11663737B2 (en) | Information processing apparatus and representative coordinate derivation method | |
| WO2019041614A1 (en) | Head-mounted immersive virtual reality display device and immersive virtual reality display method | |
| CN106980369A (en) | The synthesis of 3rd multi-view video of virtual reality project and output system and method | |
| CN108646776B (en) | Imaging system and method based on unmanned aerial vehicle | |
| TWI775006B (en) | Imaginary face generation method and system, and face recognition method and system using the same | |
| US11232588B2 (en) | Information processing apparatus and device information derivation method | |
| WO2020189450A1 (en) | Device provided with plurality of markers | |
| US11100716B2 (en) | Image generating apparatus and image generation method for augmented reality | |
| CN110244837A (en) | Augmented reality and the experience glasses and its imaging method being superimposed with virtual image | |
| US20230316810A1 (en) | Three-dimensional (3d) facial feature tracking for autostereoscopic telepresence systems | |
| CN107888900A (en) | A kind of projection robot of full parallax Three-dimensional Display and the projecting method of the robot | |
| CN112862819A (en) | Device and method for increasing self elements of experiencer in virtual reality | |
| JP2020181321A (en) | Information processing device and device information derivation method | |
| CN103285585A (en) | Motion sensing fencing interaction device based on internet framework | |
| KR20190053565A (en) | Live stitching method using stitching device | |
| CN111161399B (en) | Data processing method and assembly for generating three-dimensional model based on two-dimensional image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20171128 |