CN115904201A - Virtual reality device, control method and device thereof, and computer storage medium - Google Patents
Virtual reality device, control method and device thereof, and computer storage medium Download PDFInfo
- Publication number
- CN115904201A CN115904201A CN202211436400.8A CN202211436400A CN115904201A CN 115904201 A CN115904201 A CN 115904201A CN 202211436400 A CN202211436400 A CN 202211436400A CN 115904201 A CN115904201 A CN 115904201A
- Authority
- CN
- China
- Prior art keywords
- virtual
- keyboard
- environment
- mouse
- virtual reality
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- User Interface Of Digital Computer (AREA)
Abstract
The application discloses virtual reality equipment, a control method and a control device of the virtual reality equipment and a computer storage medium, and belongs to the technical field of virtual reality. The method comprises the following steps: acquiring an environment image of an external environment of the equipment main body through an image acquisition assembly; carrying out plane detection in the environment image; presenting the detected at least one plane in a virtual environment of a virtual reality device; in response to identifying the keyboard in the environmental image, generating a virtual keyboard at a corresponding location of the keyboard in a virtual environment of the virtual reality device; responsive to the virtual keyboard being placed on a target plane of the at least one plane, establishing a virtual desktop at a corresponding location of the target plane in the virtual environment; acquiring an input signal of a mouse; and generating a selection ray corresponding to the mouse in the virtual environment based on the input signal of the mouse. Therefore, the virtual environment can be controlled through the external mouse and the external keyboard, and the possibility of false touch is reduced.
Description
Technical Field
The present application relates to the field of virtual reality technologies, and in particular, to a virtual reality device, a method and an apparatus for controlling the same, and a computer storage medium.
Background
A virtual reality device is a device that is capable of providing a virtual reality experience to a user. Based on different application scenes, the virtual reality equipment can provide different functions, such as games, movies, offices and the like.
A control method of a virtual reality device for providing a user with an office scene, in which a virtual desktop is first generated in a virtual environment, a virtual keyboard is generated on the virtual desktop, and an image of a hand of the user is photographed by a camera to generate a model of the hand in the virtual environment. When the hand model of the user touches the corresponding key of the virtual keyboard, the information corresponding to the key can be obtained.
However, the above method has a high possibility of being touched by a user by mistake, resulting in low accuracy of the above method.
Disclosure of Invention
The embodiment of the application provides virtual reality equipment, a control method and a control device thereof, and a computer storage medium. The technical scheme is as follows:
according to an aspect of the embodiments of the present application, there is provided a method for controlling a virtual reality device, where the virtual reality device includes a device body, a keyboard, and a mouse, and the device body has an image capture component thereon, the method including:
acquiring an environment image of an environment outside the device main body through the image acquisition assembly;
performing plane detection in the environment image;
presenting the detected at least one plane in a virtual environment of the virtual reality device;
in response to identifying the keyboard in the environmental image, generating a virtual keyboard at a corresponding location of the keyboard in a virtual environment of the virtual reality device;
responsive to the virtual keyboard being placed on a target plane of the at least one plane, establishing a virtual desktop at a corresponding location of the target plane in the virtual environment;
acquiring an input signal of the mouse;
and generating a selection ray corresponding to the mouse in the virtual environment based on the input signal of the mouse.
Optionally, the generating, in the virtual environment, a selection ray corresponding to the mouse based on the input signal of the mouse includes:
acquiring a plane coordinate of the mouse based on the input signal of the mouse;
converting the planar coordinates to world coordinates in the virtual environment;
generating the selection ray based on the world coordinates and an orientation of a virtual camera in the virtual environment, the selection ray passing through the world coordinates and oriented parallel to the orientation of the virtual camera.
Optionally, after generating the selection ray based on the world coordinates and an orientation of a virtual camera in the virtual environment, the method further comprises:
performing collision detection on the selected ray and an object in the virtual environment;
and determining the first object which the selection ray collides with as the object currently selected by the mouse.
Optionally, before the obtaining the input signal of the mouse, the method further includes:
in response to identifying the mouse in the environmental image, generating a virtual mouse at a corresponding location of the mouse in a virtual environment of the virtual reality device.
Optionally, the performing plane detection in the environment image includes:
and carrying out plane detection in the environment image through a random sampling consistency algorithm, a Hough transform algorithm or a region growing clustering algorithm.
Optionally, the generating a virtual keyboard at a corresponding position of the keyboard in the virtual environment of the virtual reality device includes:
detecting whether the keyboard has any one of a plurality of preset characteristics;
when the keyboard has any one characteristic, acquiring a keyboard model corresponding to the any one characteristic;
and loading the keyboard model at the corresponding position of the keyboard in the virtual environment of the virtual reality equipment.
Optionally, the characteristic includes at least one of a preset key arrangement mode, a preset optical mark and a preset appearance.
Optionally, the loading the keyboard model at a corresponding position of the keyboard in the virtual environment of the virtual reality device includes:
determining the position and the posture of the keyboard in the virtual environment through the environment image;
loading the keyboard model in the virtual environment based on the position and the pose.
Optionally, the method further comprises:
generating a first model of the keyboard based on an image of the keyboard when the keyboard does not have the preset features;
loading the first model at a corresponding location of the keyboard in a virtual environment of the virtual reality device.
According to another aspect of the embodiments of the present application, there is provided a control apparatus for a virtual reality device, where the virtual reality device includes a device main body, a keyboard, and a mouse, and the device main body has an image capturing component thereon, and the control apparatus for the virtual reality device includes:
the image acquisition module is used for acquiring an environment image of the external environment of the equipment main body through the image acquisition assembly;
the plane detection module is used for carrying out plane detection in the environment image;
a plane presentation module for presenting the detected at least one plane in a virtual environment of the virtual reality device;
a keyboard generation module for generating a virtual keyboard at a corresponding location of the keyboard in a virtual environment of the virtual reality device in response to identifying the keyboard in the environment image;
a desktop establishing module, configured to, in response to the virtual keyboard being placed on a target plane of the at least one plane, establish a virtual desktop at a corresponding location of the target plane in the virtual environment;
the input acquisition module is used for acquiring an input signal of the mouse;
and the ray establishing module is used for generating a selection ray corresponding to the mouse in the virtual environment based on the input signal of the mouse.
Optionally, the ray establishing module is configured to:
acquiring a plane coordinate of the mouse based on an input signal of the mouse;
converting the planar coordinates to world coordinates in the virtual environment;
generating the selection ray based on the world coordinates and an orientation of a virtual camera in the virtual environment, the selection ray passing through the world coordinates and oriented parallel to the orientation of the virtual camera.
According to another aspect of embodiments of the present application, there is provided a virtual reality device, which includes a processor and a memory, where at least one instruction, at least one program, a set of codes, or a set of instructions is stored in the memory, and the at least one instruction, the at least one program, the set of codes, or the set of instructions is loaded and executed by the processor to implement the control method of the virtual reality device as described above.
According to another aspect of embodiments of the present application, there is provided a computer storage medium having at least one instruction, at least one program, a set of codes, or a set of instructions stored therein, the at least one instruction, the at least one program, the set of codes, or the set of instructions being loaded and executed by a processor to implement the control method of a virtual reality device as described above.
According to another aspect of embodiments herein there is provided a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method provided in the various alternative implementations described above.
The beneficial effects that technical scheme that this application embodiment brought include at least:
at least one detected plane in the external environment is displayed in the virtual environment of the virtual reality equipment, a virtual keyboard corresponding to the keyboard is generated in the virtual environment, then a virtual desktop is established at the position of the plane where the virtual keyboard is placed, and then an input signal of the mouse is converted into a selection ray in the virtual environment, so that the virtual environment can be controlled through the external mouse and the external keyboard, the possibility of false touch is reduced, and the effect of improving the accuracy of the control method of the virtual reality equipment is realized.
In addition, in the method provided by the embodiment of the application, the input signal of the mouse is converted into the corresponding selection ray in the virtual environment, so that the user can directly perform corresponding control in the virtual environment through the commonly used mouse in office without using a special handle for the virtual reality equipment, the cost is saved, the learning cost of the user is reduced, and the user experience is improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
FIG. 1 is a schematic illustration of an implementation environment in an embodiment of the present application;
fig. 2 is a flowchart of a method for controlling a virtual reality device according to an embodiment of the present disclosure;
fig. 3 is a flowchart of a method of another virtual reality device control method provided in an embodiment of the present application;
FIG. 4 is a schematic diagram of a virtual environment in a method provided by an embodiment of the present application;
fig. 5 is a block diagram of a control device of a virtual reality apparatus according to an embodiment of the present application.
With the above figures, there are shown specific embodiments of the present application, which will be described in more detail below. These drawings and written description are not intended to limit the scope of the inventive concepts in any manner, but rather to illustrate the inventive concepts to those skilled in the art by reference to specific embodiments.
Detailed Description
To make the objects, technical solutions and advantages of the present application more clear, embodiments of the present application will be described in further detail below with reference to the accompanying drawings.
The control method of the virtual reality device provided in the embodiment of the present application may be applied to a virtual reality device, as shown in fig. 1, fig. 1 is a schematic diagram of an implementation environment in the embodiment of the present application, the implementation environment may include a virtual reality device, the virtual reality device may include a device main body 11, a keyboard 12, and a mouse 13, and the device main body 11 has an image acquisition component 111.
The device body 11 may be various head-mounted Virtual Reality devices such as Virtual Reality (VR) glasses and Virtual Reality helmets. The apparatus body 11 may be a six-degree-of-freedom (6 DOF) virtual reality device so that the user can have a better experience. The apparatus body 11 has an image pickup assembly 111 thereon, and the image pickup assembly 111 may be directed toward the front of the user wearing the apparatus body 11 so as to acquire an image of the environment in front of the user. The image acquisition assembly 111 may include a visible light camera, a depth camera, and the like. The device body 11 may have a display screen, and a user may obtain an experience of virtual reality by viewing the display screen.
The keyboard 12 may be connected to the device body 11 wirelessly or by wire, and the keyboard 12 may be connected to the device body 11 by bluetooth, for example. The keyboard 12 may be a preset type keyboard, and a pre-established model of the preset type keyboard may be stored in the device main body 11.
The mouse 13 may be a wireless or wired connection with the device body 11, and for example, the mouse 13 may be a bluetooth connection with the device body 11. The mouse 13 may also be a mouse of a preset model, and a pre-established model of the mouse of the preset model may be stored in the device main body 11.
The keyboard 12 and mouse 13 may be located on the desktop D. When the keyboard 12 and the mouse 13 are wirelessly connected to the device body 11, the delay may be within 20 milliseconds to ensure the user experience.
Further, the virtual reality device 10 may further include a virtual reality host that can provide the device body 11 with control information, and processed image data, as a signal source of the device body 11. The virtual reality host may be partially or entirely incorporated in the apparatus body 11, or may be located outside the apparatus body 11. The virtual reality host may have a control component therein for controlling the device body 11.
The control method of the virtual reality device, provided by the embodiment of the application, can provide a virtual-real combined application scene for a user, and the user can use the scene to perform various activities such as office work, entertainment and the like.
Fig. 2 is a flowchart of a method for controlling a virtual reality device according to an embodiment of the present disclosure, where the method may be used for the virtual reality device shown in fig. 1, and the method may include the following steps:
And step 204, responding to the keyboard identified in the environment image, and generating a virtual keyboard at the corresponding position of the keyboard in the virtual environment of the virtual reality device.
And step 206, acquiring an input signal of the mouse.
And step 207, generating a selection ray corresponding to the mouse in the virtual environment based on the input signal of the mouse.
To sum up, according to the control method of the virtual reality device provided in the embodiment of the present application, at least one plane in the detected external environment is displayed in the virtual environment of the virtual reality device, a virtual keyboard corresponding to the keyboard is generated in the virtual environment, then a virtual desktop is established at the position of the plane where the virtual keyboard is placed, and then an input signal of the mouse is converted into a selection ray in the virtual environment, so that the virtual environment can be controlled through the external mouse and the keyboard, the possibility of a false touch is reduced, and the effect of improving the accuracy of the control method of the virtual reality device is achieved.
In addition, in the method provided by the embodiment of the application, the input signal of the mouse is converted into the corresponding selection ray in the virtual environment, so that the user can directly perform corresponding control in the virtual environment through the commonly used mouse in office without using a special handle for the virtual reality equipment, the cost is saved, the learning cost of the user is reduced, and the user experience is improved.
Fig. 3 is a flowchart of another method for controlling a virtual reality device according to an embodiment of the present disclosure, where the method may be used for the virtual reality device shown in fig. 1, and the method may include the following steps:
The image acquisition assembly can continuously acquire the environment image of the external environment of the main body, the environment image is the image of the environment around the user wearing the equipment main body, and the equipment main body can conveniently display the virtual environment of the virtual reality equipment correspondingly based on the image. For example, when the image capturing component faces the front of the device body (the front of the device body may be the same direction as the front of the user wearing the device body), the environment image may reflect the environment in front of the user wearing the device body, and the corresponding content presentation is performed in front of the user in the virtual environment based on the environment image.
The method provided by the embodiment of the application can be applied to a control assembly of the virtual reality device, and the control assembly can be located in a device body or can be located outside the device body. When the control assembly is located outside the device main body, the control assembly and the device main body are in wireless connection or wired connection.
And step 302, carrying out plane detection in the environment image.
The control component can perform plane detection in the environment image acquired by the image acquisition component to detect planes existing in the environment outside the subject. Because it is real environment to detect main part external environment, it is comparatively complicated usually, can have more planes, like desktop, ground, wall and ceiling, and then carry out the plane and detect out a plurality of planes after detecting.
In an exemplary embodiment, the control component can perform plane detection in the environment image through RANdom SAmple Consensus (RANSAC), hough transform, or region growing clustering.
The random sampling consistency algorithm is an algorithm for calculating mathematical model parameters of data according to a group of sample data sets containing abnormal data so as to obtain effective sample data. Random sampling consensus algorithms are widely used in computer vision. When the random sampling consistency algorithm is applied to carry out plane detection, the random sampling consistency algorithm can be circularly used to carry out plane detection aiming at the environment pattern, and when a circulation termination condition (such as a condition that the area of the circulation to the plane is a set minimum threshold value) is reached, the circulation random sampling consistency algorithm is terminated.
The hough transform algorithm is a feature extraction (feature extraction) algorithm, and is widely applied to image analysis (image analysis), computer vision (computer vision), and digital image processing (digital image processing). The hough transform algorithm can be used to discriminate between various features found in the image. In the embodiment of the application, a hough transform algorithm can be applied to detect a plane in an environment image.
The region growing clustering algorithm is a network segmentation algorithm based on region growing, and belongs to a local greedy algorithm. In the embodiment of the application, a region growing clustering algorithm can be applied to detect planes in the environment image.
The specific process of performing plane detection through a random sampling consistency algorithm, a hough transform algorithm or a region growing clustering algorithm may refer to related technologies, and the embodiment of the present application is not described herein again.
The control component may present the detected at least one plane in a virtual environment of the virtual reality device such that the user can see the detected at least one plane in the virtual environment. The control component can display the detected at least one plane at the corresponding position in the virtual environment, and then the at least one plane can reflect the positions of various planes and objects in the real environment, so that the user can know the positions of the various planes in the real environment based on the displayed at least one plane in the virtual environment, the user can determine the plane serving as a desktop during office work based on the displayed plane on one hand, and the user can be prevented from colliding with the planes on the other hand.
In the virtual environment, the plane may be presented in various ways, for example, the plane may be marked by a grid pattern, or the plane may be identified by a model, or the plane may be marked by a dot matrix, or the plane may be marked by a plurality of cross patterns arranged in an array, and the like, which is not limited in this embodiment of the present application.
The control assembly can carry out keyboard detection in the environment image, because the keyboard has a plurality of buttons that the array was arranged, and all has letter or symbol on the button, holistic characteristic is comparatively obvious, when the keyboard appeared in the field of vision of image acquisition assembly, the control assembly can be based on the environment image recognition keyboard that the image acquisition assembly gathered obtained to carry out real-time tracking to the keyboard.
The control component may detect whether the keyboard has any one of a plurality of preset features when the keyboard is detected by the environment image. The characteristics comprise at least one of a preset key arrangement mode, a preset optical mark and a preset appearance. The predetermined characteristic may be one or more predetermined characteristics of the keyboard.
And 305, when the keyboard has any characteristic, acquiring a keyboard model corresponding to the characteristic.
The keyboard model corresponding to each preset feature may be pre-stored in the device main body, and when the recognized keyboard has any feature, the control component may obtain the keyboard model corresponding to the any feature from a local storage of the device main body.
For example, the above feature is a preset optical mark, the optical mark may be specifically a mark of a keyboard model, and when the keyboard detected by the control component has a mark of one of the keyboard models, the keyboard model corresponding to the mark of the keyboard model may be obtained from the local storage.
In an exemplary embodiment, the device body may store the one or more fine models of the keyboard in advance, and the fine models have a higher degree of fineness and may have some corresponding special effects, such as light effects, animation effects, and the like, which is beneficial to improving the user experience.
In addition, the control component may generate a first model of the keyboard based on the identified image of the keyboard and load the first model at a corresponding location of the keyboard in the virtual environment of the virtual reality device when the keyboard does not have the preset features. The fineness of the first model can be lower than the keyboard model stored in the device body in advance so as to reduce the operation pressure of the processing assembly.
And step 306, loading a keyboard model at a position corresponding to the keyboard in the virtual environment of the virtual reality device.
After the control component obtains the keyboard model, the keyboard model can be loaded at the corresponding position of the keyboard in the virtual environment of the virtual reality device. When the image acquisition assembly continuously acquires the environment image, the control assembly can load the keyboard model at a corresponding position (the corresponding position can be the position of the keyboard in the virtual environment) in the virtual environment of the virtual reality equipment in real time, and under the condition, the effect of projecting the keyboard in the real environment to the virtual environment is realized. When a user picks up the keyboard in the real environment and moves the keyboard, the keyboard in the virtual environment moves along with the keyboard.
In an exemplary embodiment, the control component may determine a position and a posture of the keyboard in the virtual environment through the environment image, and load the keyboard model in the virtual environment based on the position and the posture, so as to realize more accurate movement of the keyboard model and improve user experience.
When the control component detects that the virtual keyboard is placed on a target plane of the at least one plane, indicating that the target plane is a plane selected by the user for use as an office desktop, the control component may create a virtual desktop at a corresponding location of the target plane in the virtual environment. The virtual desktop may have a shape similar to that of a desk, or the control unit may store a plurality of models of virtual desktops (the shapes and materials of the models of different virtual desktops are different) in advance, and may select one of the models by the user.
And step 307, ending, in the method provided by the embodiment of the present application, providing a virtual desktop and a virtual keyboard located on the virtual desktop to the user, where the virtual keyboard may be a projection of an entity keyboard in the display environment, and when the user operates the entity keyboard in the display environment, the virtual keyboard may make a corresponding feedback, and the control component may obtain information input by the user and perform corresponding control based on the information. And when the user operates the keyboard of the entity, the possibility of mistaken touch is low, so that the accuracy of the method can be improved.
And 308, responding to the mouse identified in the environment image, and generating a virtual mouse at the corresponding position of the mouse in the virtual environment of the virtual reality device.
The control component can perform mouse recognition in the environment image, and when the mouse is recognized, a virtual mouse is generated at the corresponding position of the mouse in the virtual environment of the virtual reality device. When the image acquisition component continuously acquires the environment image, the control component can load the mouse model at a corresponding position (the corresponding position can be the position of the mouse in the virtual environment) in the virtual environment of the virtual reality device in real time, and under the condition, the effect of projecting the mouse in the real environment to the virtual environment is realized. When a user holds the mouse in a real environment, and moves and operates the mouse, the mouse in the virtual environment moves along with the mouse. Therefore, the effect that the user synchronously operates the mouse in the real environment and the virtual environment is achieved, and the user experience is improved.
The control component can acquire input signals of the mouse through connection with the mouse. The input signal of the mouse may include a position signal of the mouse, and the input signal may further include a key signal.
And 310, acquiring plane coordinates of the mouse based on the input signal of the mouse.
The control component may acquire the plane coordinates of the mouse based on the input signal of the mouse. The plane coordinate may be recorded with a coordinate of the mouse in a desktop plane coordinate system, the plane coordinate system may be a pixel coordinate system on the display screen of the apparatus main body, the range of the coordinate system may be (0,0) to (Sw, sh), sw is a pixel width of the display screen, and Sh is a pixel height of the display screen.
The control component may convert the planar coordinates to world coordinates in the virtual environment. Illustratively, the control component may implement the conversion process through a rendering engine.
The control component may generate the selection ray based on the world coordinates corresponding to the position of the mouse and an orientation of the virtual camera in the virtual environment after the world coordinates are acquired. The orientation of the virtual camera can be determined by the orientation of the device body, the selection ray passes through the world coordinates, and the direction is parallel to the orientation of the virtual camera, that is, the selection ray can be determined by the position of the mouse and the orientation of the virtual camera.
And step 313, performing collision detection on the selected ray and the object in the virtual environment.
The control component, after generating the selected ray, may perform collision detection on the selected ray with an object in the virtual environment. The objects in the virtual environment may be various objects displayed in office, such as one or more screens and three-dimensional models.
The control component may determine the first object that the selection ray hits as the object currently selected by the mouse. In an exemplary embodiment, an object currently selected by a mouse may be highlighted in the virtual environment, and when a certain instruction of the mouse or the keyboard is received (the instruction may be issued when a button of the mouse is clicked or a designated button on the keyboard is clicked), a further operation, such as an opening operation, may be performed on the selected object.
Please refer to fig. 4, fig. 4 is a schematic diagram of a virtual environment in a method provided by an embodiment of the present application, where the virtual environment includes a virtual desktop d, a virtual keyboard k and a virtual mouse m that are located on the virtual desktop d, and the virtual desktop d, the virtual keyboard k and the virtual mouse m are all projections of a desktop, a keyboard and a mouse in a real environment, the virtual environment further includes virtual display screens s1, s2 and s3, which can be used to display office-related content, and in addition, the virtual environment further displays a selection ray L corresponding to the mouse, and the selection ray L can be used to select content displayed by the display screens s1, s2 and s3, or other objects in the virtual environment.
Therefore, the effect of controlling the three-dimensional virtual environment is achieved through operation of the mouse on the plane, a user does not need to use a special virtual reality handle, and equipment cost and learning cost of the user are saved.
To sum up, according to the control method of the virtual reality device provided in the embodiment of the present application, at least one detected plane in the external environment is displayed in the virtual environment of the virtual reality device, the virtual keyboard corresponding to the keyboard is generated in the virtual environment, then the virtual desktop is established at the position of the plane where the virtual keyboard is placed, and then the input signal of the mouse is converted into the selection ray in the virtual environment, so that the virtual environment can be controlled through the external mouse and the keyboard, the possibility of false touch is reduced, and the effect of improving the accuracy of the control method of the virtual reality device is achieved.
In addition, in the method provided by the embodiment of the application, the input signal of the mouse is converted into the corresponding selection ray in the virtual environment, so that the user can directly perform corresponding control in the virtual environment through the commonly used mouse in office without using a special handle for the virtual reality equipment, the cost is saved, the learning cost of the user is reduced, and the user experience is improved.
The following are embodiments of the disclosed apparatus that may be used to perform embodiments of the disclosed methods. For details not disclosed in the embodiments of the apparatus of the present disclosure, refer to the embodiments of the method of the present disclosure.
Fig. 5 is a block diagram of a control apparatus for a virtual reality device according to an embodiment of the present application, where the apparatus may be implemented as part or all of a control component of the virtual reality device through software, hardware, or a combination of the two. The control device 500 of the virtual reality apparatus includes:
an image acquisition module 510, configured to acquire an environment image of an environment outside the apparatus main body through an image acquisition component;
a plane detection module 520, configured to perform plane detection in the environment image;
a plane showing module 530 for showing the detected at least one plane in a virtual environment of the virtual reality device;
a keyboard generation module 540, configured to generate a virtual keyboard in a virtual environment of the virtual reality device in response to identifying the keyboard in the environment image;
a desktop establishing module 550, configured to, in response to the virtual keyboard being placed on a target plane of the at least one plane, establish a virtual desktop at a corresponding position of the target plane in the virtual environment;
an input acquisition module 560, configured to acquire an input signal of the mouse;
and the ray establishing module 570 is used for generating a selection ray corresponding to the mouse in the virtual environment based on the input signal of the mouse.
To sum up, the control device of the virtual reality equipment provided in the embodiment of the present application displays at least one plane in the detected external environment in the virtual environment of the virtual reality equipment, generates the virtual keyboard corresponding to the keyboard in the virtual environment, then establishes the virtual desktop at the position of the plane where the virtual keyboard is placed, and converts the input signal of the mouse into the selection ray in the virtual environment, so that the virtual environment can be controlled through the external mouse and the keyboard, the possibility of false touch is reduced, and the effect of improving the accuracy of the control method of the virtual reality equipment is achieved.
In addition, the control device of the virtual reality equipment provided by the embodiment of the application converts the input signal of the mouse into the corresponding selection ray in the virtual environment, so that the user can directly perform corresponding control in the virtual environment through the commonly used mouse in office without using a special handle for the virtual reality equipment, the cost is saved, the learning cost of the user is reduced, and the user experience is improved.
Optionally, the ray establishing module is configured to:
acquiring a plane coordinate of the mouse based on an input signal of the mouse;
converting the planar coordinates to world coordinates in the virtual environment;
a selection ray is determined based on the world coordinates and an orientation of the virtual camera in the virtual environment, the selection ray passing through the world coordinates and oriented parallel to the orientation of the virtual camera.
Optionally, the control device of the virtual reality apparatus further includes:
the collision detection module is used for performing collision detection on the selected rays and the object in the virtual environment;
and the selection module is used for determining the first object on which the selection ray collides as the currently selected object of the mouse.
Optionally, the control device of the virtual reality apparatus further includes:
and the virtual mouse generating module is used for responding to the mouse identified in the environment image and generating a virtual mouse at the corresponding position of the mouse in the virtual environment of the virtual reality equipment.
Optionally, the plane detection module is configured to:
and carrying out plane detection in the environment image through a random sampling consistency algorithm, a Hough transform algorithm or a region growing clustering algorithm.
Optionally, the keyboard generation module is configured to:
detecting whether the keyboard has any one of a plurality of preset characteristics;
when the keyboard has any characteristic, acquiring a keyboard model corresponding to the any characteristic;
and loading the keyboard model at the corresponding position of the keyboard in the virtual environment of the virtual reality equipment.
Optionally, the characteristic includes at least one of a preset key arrangement mode, a preset characteristic and a preset appearance.
Optionally, the keyboard generation module is configured to:
determining the position and the posture of the keyboard in the virtual environment through the environment image;
the keyboard model is loaded in the virtual environment based on the position and the pose.
Optionally, the control device of the virtual reality apparatus further includes:
the model generation module is used for generating a first model of the keyboard based on the image of the keyboard when the keyboard does not have preset characteristics;
and the model loading module is used for loading the first model at the corresponding position of the keyboard in the virtual environment of the virtual reality equipment.
To sum up, the control device of the virtual reality equipment provided in the embodiment of the present application displays at least one plane in the detected external environment in the virtual environment of the virtual reality equipment, generates the virtual keyboard corresponding to the keyboard in the virtual environment, then establishes the virtual desktop at the position of the plane where the virtual keyboard is placed, and converts the input signal of the mouse into the selection ray in the virtual environment, so that the virtual environment can be controlled through the external mouse and the keyboard, the possibility of false touch is reduced, and the effect of improving the accuracy of the control method of the virtual reality equipment is achieved.
In addition, the control device of the virtual reality equipment provided by the embodiment of the application converts the input signal of the mouse into the corresponding selection ray in the virtual environment, so that the user can directly perform corresponding control in the virtual environment through the commonly used mouse in office without using a special handle for the virtual reality equipment, the cost is saved, the learning cost of the user is reduced, and the user experience is improved.
In addition, an embodiment of the present application further provides a virtual reality device, where the virtual reality device includes a processor and a memory, where the memory stores at least one instruction, at least one program, a code set, or a set of instructions, and the at least one instruction, the at least one program, the code set, or the set of instructions is loaded and executed by the processor to implement the control method of the virtual reality device.
The embodiment of the present application further provides a computer storage medium, where at least one instruction, at least one program, a code set, or an instruction set is stored in the computer storage medium, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the control method of the virtual reality device.
Embodiments of the present application also provide a computer program product or computer program comprising computer instructions stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the method provided in the various alternative implementations described above.
The term "at least one of a and B" in the present application is only one kind of association relationship describing an associated object, and means that three kinds of relationships may exist, for example, at least one of a and B may mean: a exists alone, A and B exist simultaneously, and B exists alone. Similarly, "A, B and at least one of C" means that there may be seven relationships that may represent: a alone, B alone, C alone, A and B together, A and C together, C and B together, and A, B and C together. Similarly, "A, B, C and at least one of D" indicates that there may be fifteen relationships, which may indicate: fifteen cases of a alone, B alone, C alone, D alone, a and B together, a and C together, a and D together, C and B together, D and B together, C and D together, A, B and C together, A, B and D together, A, C and D together, B, C and D together, A, B, C and D together exist.
In this application, the term "first" is used for descriptive purposes only and is not to be construed as indicating or implying relative importance. The term "plurality" means two or more unless explicitly defined otherwise.
In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, where the program may be stored in a computer-readable storage medium, and the storage medium may be a read-only memory, a magnetic disk or an optical disk.
The above description is only exemplary of the present application and should not be taken as limiting, as any modification, equivalent replacement, or improvement made within the spirit and principle of the present application should be included in the protection scope of the present application.
Claims (13)
1. A control method of a virtual reality device is used for the virtual reality device, the virtual reality device comprises a device body, a keyboard and a mouse, an image acquisition assembly is arranged on the device body, and the method comprises the following steps:
acquiring an environment image of an environment outside the device main body through the image acquisition assembly;
performing plane detection in the environment image;
presenting the detected at least one plane in a virtual environment of the virtual reality device;
in response to identifying the keyboard in the environmental image, generating a virtual keyboard at a corresponding location of the keyboard in a virtual environment of the virtual reality device;
responsive to the virtual keyboard being placed on a target plane of the at least one plane, establishing a virtual desktop at a corresponding location of the target plane in the virtual environment;
acquiring an input signal of the mouse;
and generating a selection ray corresponding to the mouse in the virtual environment based on the input signal of the mouse.
2. The method of claim 1, wherein generating the selection ray corresponding to the mouse in the virtual environment based on the input signal of the mouse comprises:
acquiring a plane coordinate of the mouse based on the input signal of the mouse;
converting the planar coordinates to world coordinates in the virtual environment;
generating the selection ray based on the world coordinates and an orientation of a virtual camera in the virtual environment, the selection ray passing through the world coordinates and oriented parallel to the orientation of the virtual camera.
3. The method of claim 2, wherein after generating the selected ray based on the world coordinates and an orientation of a virtual camera in the virtual environment, the method further comprises:
performing collision detection on the selected ray and an object in the virtual environment;
and determining the first object collided by the selection ray as the object currently selected by the mouse.
4. The method of claim 1, wherein before the obtaining the input signal of the mouse, the method further comprises:
in response to identifying the mouse in the environmental image, generating a virtual mouse at a corresponding location of the mouse in a virtual environment of the virtual reality device.
5. The method of claim 1, wherein the performing plane detection in the environment image comprises:
and carrying out plane detection in the environment image through a random sampling consistency algorithm, a Hough transform algorithm or a region growing clustering algorithm.
6. The method of claim 1, wherein generating a virtual keyboard at a corresponding location of the keyboard in a virtual environment of the virtual reality device comprises:
detecting whether the keyboard has any one of a plurality of preset characteristics;
when the keyboard has any one feature, acquiring a keyboard model corresponding to the any one feature;
and loading the keyboard model at the corresponding position of the keyboard in the virtual environment of the virtual reality equipment.
7. The method of claim 6, wherein the characteristic comprises at least one of a predetermined key arrangement, a predetermined characteristic, and a predetermined shape.
8. The method of claim 6, wherein the loading the keyboard model at a corresponding location of the keyboard in the virtual environment of the virtual reality device comprises:
determining the position and the posture of the keyboard in the virtual environment through the environment image;
loading the keyboard model in the virtual environment based on the position and the pose.
9. The method of claim 6, further comprising:
generating a first model of the keyboard based on an image of the keyboard when the keyboard does not have the preset features;
loading the first model at a corresponding location of the keyboard in a virtual environment of the virtual reality device.
10. The utility model provides a controlling means of virtual reality equipment, its characterized in that, virtual reality equipment includes equipment principal, keyboard and mouse, the equipment principal is last to have the image acquisition subassembly, the controlling means of virtual reality equipment includes:
the image acquisition module is used for acquiring an environment image of the external environment of the equipment main body through the image acquisition assembly;
the plane detection module is used for carrying out plane detection in the environment image;
a plane presentation module for presenting the detected at least one plane in a virtual environment of the virtual reality device;
a keyboard generation module for generating a virtual keyboard at a corresponding position of the keyboard in a virtual environment of the virtual reality device in response to identifying the keyboard in the environment image;
a desktop establishing module, configured to, in response to the virtual keyboard being placed on a target plane of the at least one plane, establish a virtual desktop at a corresponding location of the target plane in the virtual environment;
the input acquisition module is used for acquiring an input signal of the mouse;
and the ray establishing module is used for generating a selection ray corresponding to the mouse in the virtual environment based on the input signal of the mouse.
11. The control apparatus of a virtual reality device according to claim 10, wherein the ray establishing module is configured to:
acquiring a plane coordinate of the mouse based on the input signal of the mouse;
converting the planar coordinates to world coordinates in the virtual environment;
generating the selection ray based on the world coordinates and an orientation of a virtual camera in the virtual environment, the selection ray passing through the world coordinates and oriented parallel to the orientation of the virtual camera.
12. A virtual reality device comprising a processor and a memory, the memory having stored therein at least one instruction, at least one program, a set of codes or a set of instructions, the at least one instruction, the at least one program, the set of codes or the set of instructions being loaded and executed by the processor to implement the method of controlling a virtual reality device according to any one of claims 1 to 9.
13. A computer storage medium having stored therein at least one instruction, at least one program, a set of codes, or a set of instructions, which is loaded and executed by a processor to implement a method of controlling a virtual reality device according to any one of claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211436400.8A CN115904201A (en) | 2022-11-16 | 2022-11-16 | Virtual reality device, control method and device thereof, and computer storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211436400.8A CN115904201A (en) | 2022-11-16 | 2022-11-16 | Virtual reality device, control method and device thereof, and computer storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115904201A true CN115904201A (en) | 2023-04-04 |
Family
ID=86474083
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211436400.8A Pending CN115904201A (en) | 2022-11-16 | 2022-11-16 | Virtual reality device, control method and device thereof, and computer storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115904201A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116541923A (en) * | 2023-04-07 | 2023-08-04 | 中国民用航空飞行学院 | VR-based indoor installation foundation positioning method for equipment with support |
-
2022
- 2022-11-16 CN CN202211436400.8A patent/CN115904201A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116541923A (en) * | 2023-04-07 | 2023-08-04 | 中国民用航空飞行学院 | VR-based indoor installation foundation positioning method for equipment with support |
| CN116541923B (en) * | 2023-04-07 | 2023-12-19 | 中国民用航空飞行学院 | VR-based indoor installation foundation positioning method for equipment with support |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11182962B2 (en) | Method and system for object segmentation in a mixed reality environment | |
| KR100851977B1 (en) | Controlling Method and apparatus for User Interface of electronic machine using Virtual plane. | |
| JP4323180B2 (en) | Interface method, apparatus, and program using self-image display | |
| CN110473293B (en) | Virtual object processing method and device, storage medium and electronic equipment | |
| CN104364733A (en) | Position-of-interest detection device, position-of-interest detection method, and position-of-interest detection program | |
| CN112506340B (en) | Equipment control method, device, electronic equipment and storage medium | |
| KR20130099317A (en) | System for implementing interactive augmented reality and method for the same | |
| EP1456806A1 (en) | Device and method for calculating a location on a display | |
| CN108027656B (en) | Input device, input method, and program | |
| US11244511B2 (en) | Augmented reality method, system and terminal device of displaying and controlling virtual content via interaction device | |
| US10372229B2 (en) | Information processing system, information processing apparatus, control method, and program | |
| NZ525717A (en) | A method of tracking an object of interest using multiple cameras | |
| US10030968B2 (en) | Floor estimation for human computer interfaces | |
| CN111766937A (en) | Virtual content interaction method and device, terminal equipment and storage medium | |
| CN111679742A (en) | Interaction control method and device based on AR, electronic equipment and storage medium | |
| EP3714955A1 (en) | Video tracking system and method | |
| CN111083464A (en) | Virtual content display delivery system | |
| JP5565331B2 (en) | Display system, display processing apparatus, display method, and display program | |
| CN111766936A (en) | Virtual content control method and device, terminal equipment and storage medium | |
| CN115904201A (en) | Virtual reality device, control method and device thereof, and computer storage medium | |
| JP2017219942A (en) | Contact detection device, projector device, electronic blackboard system, digital signage device, projector device, contact detection method, program and recording medium | |
| US10304002B2 (en) | Depth-based feature systems for classification applications | |
| CN109643182B (en) | Information processing method and device, cloud processing equipment and computer program product | |
| CN114167997B (en) | Model display method, device, equipment and storage medium | |
| CN106339089B (en) | A kind of interactive action identifying system and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |