WO2019019248A1

WO2019019248A1 - Virtual reality interaction method, device and system

Info

Publication number: WO2019019248A1
Application number: PCT/CN2017/099508
Authority: WO
Inventors: 谢冰; 许秋子
Original assignee: 深圳市瑞立视多媒体科技有限公司
Priority date: 2017-07-28
Filing date: 2017-08-29
Publication date: 2019-01-31

Abstract

Provided are a virtual reality interaction method, a device and a system, applicable to the field of virtual reality technology. The method comprises: firstly, receiving image information from a motion capture camera, and receiving sensing information collected by at least one collector and transmitted through a corresponding virtual scene client; and then acquiring spatial location information and posture information of all users according to the image information and the sensing information; and transmitting the spatial location information and the posture information to all the virtual scene clients, so that each client renders the virtual scene according to the spatial location information, the posture information, and the perspective information of a local user, and displays the virtual scene to the local user. The local user is one of all users. Since two sets of data of optics and inertia are combined, the user is positioned when the users occlude each other.

Description

Virtual reality interaction method, device and system

[0001] The present invention belongs to the field of virtual reality technologies, and in particular, to a virtual reality interaction method, apparatus, and system.

Background technique

[0002] Due to the development of virtual reality and augmented reality personal entertainment devices and the rapid development of computer image processing performance, the demand for multi-person virtual reality interaction or entertainment has become more and more urgent.

[0003] At present, an existing virtual reality interaction technology acquires a plurality of image information of a user, and then acquires spatial position and posture information of the user according to the plurality of image information. Finally, the virtual scene is obtained according to the user's spatial position and posture, and the virtual scene is displayed. In the implementation process, due to the problem of mutual occlusion caused by multi-person interaction, when users are occluded from each other, the problem of the user's spatial position and posture cannot be tracked, and the user cannot be located, resulting in a virtual scene. Rendering failure technical problem

[0004] Existing virtual reality interaction methods cannot locate users when they are occluded from each other.

Problem solution

Technical solution

A first aspect of the embodiments of the present invention provides a virtual reality interaction method, including:

Receiving image information for automatically capturing a camera, and receiving sensing information collected by at least one collector and transmitted through a corresponding virtual scene client;

Acquiring spatial location information and posture information of all users according to the image information and the sensing information; [0008] transmitting the spatial location information and the posture information to all virtual scene clients, so that each And the client renders the virtual scene according to the spatial location information, the posture information, and the perspective information of the local user, and displays the virtual scenario to the local user; the local user is one of all users.

A second aspect of the embodiments of the present invention provides a virtual reality interaction apparatus, including:

[0010] a receiving module, configured to receive image information for automatically capturing a camera, and receive at least one collector for collecting And sensing information transmitted through the virtual scene client;

[0011] an obtaining module, configured to acquire spatial location information and posture information of all users according to the image information and the sensing information;

[0012] a transmission module, configured to transmit the spatial location information and the posture information to all virtual scene clients, so that each of the clients according to the spatial location information, the posture information, and a local user perspective The information renders the virtual scene and displays it to the local user; the local user is one of all users.

[0013] A third aspect of the embodiments of the present invention provides a virtual reality interaction system, the system comprising: at least two motion capture cameras, at least one collector, at least one virtual scene client, at least one helmet display, and a camera Server; among them,

[0014] the motion capture camera is configured to capture image information of a user and transmit the image information to the camera server;

[0015] at least one of the collectors is configured to collect sensing information of the user and transmit the information to the virtual scene client corresponding to the user;

[0016] at least one of the virtual scene clients is configured to receive sensing information of the corresponding collector and transmit the information to the camera server;

[0017] the camera server is configured to acquire spatial location information and posture information of all users according to the image information and the sensing information; and transmit the spatial location information and the posture information to all virtual scenarios a client, so that each of the clients renders a virtual scene according to the spatial location information, the gesture information, and the perspective information of the local user, and displays the virtual scenario to the local user; the local user is one of all users.

A fourth aspect of the embodiments of the present invention provides a computer readable storage medium, where the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the virtual reality interaction method. A step of.

Advantageous effects of the invention

Beneficial effect

[0019] first receiving the image information of the automatic camera, and receiving the sensing information collected by the at least one collector and passing through the corresponding virtual scene client; and then acquiring the spatial location information of all the users according to the image information and the sensing information. Attitude information; then transmit spatial position information and attitude information to all virtual fields The client, so that each virtual scene client renders the virtual scene according to the spatial location information, the posture information, and the perspective information of the local user, and displays the virtual scene to the local user; the local user is one of all users; The positioning 吋 combines the two sets of data of optics and inertia, so that when users occlude each other, they can accurately locate the user.

Brief description of the drawing

DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the embodiments or the prior art description will be briefly described below. Obviously, the drawings in the following description are merely Some embodiments of the present invention may also be used to obtain other drawings based on these drawings without departing from the skilled artisan.

1 is a schematic diagram of an implementation flow of a virtual reality interaction method according to an embodiment of the present invention;

[0022] FIG. 2 is another schematic diagram of an implementation process of a virtual reality interaction method according to an embodiment of the present invention;

3 is a schematic diagram of a virtual reality interaction device according to an embodiment of the present invention;

4 is a schematic diagram of a virtual reality interaction device acquisition module according to an embodiment of the present invention;

[0025] FIG. 5 is a schematic diagram of a second computing module in a virtual reality interaction device acquiring module according to an embodiment of the present invention;

6 is a schematic diagram of a first computing module in a virtual reality interaction device acquiring module according to an embodiment of the present invention;

7 is a schematic structural diagram of a virtual reality interaction system according to an embodiment of the present invention;

8 is a schematic diagram of a server terminal of a virtual reality interactive system according to an embodiment of the present invention.

Embodiments of the invention

[0029] In the following description, for purposes of illustration and description However, it will be apparent to those skilled in the art that the present invention may be practiced in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the invention.

[0030] In order to explain the technical solutions of the present invention, the following description will be made by way of specific embodiments. [0031] FIG. 1 is a schematic flowchart diagram of a first embodiment of a virtual reality interaction method according to an embodiment of the present invention. For ease of description, only parts related to the embodiment of the present invention are shown, which are as follows:

[0032] Step 101: Receive image information of an automatic camera, and receive sensing information collected by at least one collector and transmitted through a corresponding virtual scene client.

[0033] In a specific implementation, the execution body of the embodiment may be a server for capturing a camera (also referred to as a camera server). In virtual reality interaction, the number of virtual scene clients is the same as the number of users. In the existing virtual reality interaction, the virtual scene is mainly based on games. It can be understood that the virtual scene in the embodiment of the present invention is not limited to a game, but may also be a virtual scene in other application fields, such as a live room, educational training, military exercises, and the like.

[0034] In addition, in the virtual reality interaction based on the optical motion capture technology, the optical imaging system (multiple camera) can be used to identify the active optics attached to the observed object (one or more, human or weapon) (or Passive optics) Marks the image of the marker point through the image capture system of the camera, and then transmits it to the server (referred to as the camera server) via the network (wired, wireless, USB, etc.). The camera server receives image information of the automatic camera capture, wherein the received image information may include position coordinate information of all users in the virtual scene. The server identifies the observed object based on the position coordinate information, acquires the location information of the user, and realizes positioning of the user. It can be understood that if the server is to achieve positioning of the user, the received image information of the same user must come from two different camera cameras.

[0035] On the other hand, the collector may specifically be an inertial navigation unit such as a gyroscope attached to the user, and after acquiring the speed and acceleration information of the user through the gyroscope, the user may be sent to the user by wire or wirelessly, such as Bluetooth. For the corresponding client, one user corresponds to a virtual scene client. The sensor then transmits the sensor information to the server of the camera. The sensing information may include speed and acceleration information of all users, and the acceleration information may be specifically six-axis acceleration. The client can be a backpack host, and the user can be carried on the back by the user, so that the user can perform the virtual interaction and can get rid of the constraint of the traditional wire and expand the activity space.

[0036] Step 102: Acquire spatial location information and posture information of all users according to the image information and the sensing information. The gesture information includes the face orientation of all users.

[0037] After the server receives the image information of the automatic camera and the sensor information from the collector, the server can According to these two pieces of information, the spatial location information and posture information of the user are calculated.

[0038] Step 103: The spatial location information and the posture information are transmitted to all the virtual scene clients, so that each virtual scene client renders the virtual scene according to the spatial location information, the posture information, and the perspective information of the local user, and displays the virtual scene to the local user.

[0039] wherein the local user is one of all users. According to the foregoing description, one user corresponds to one client (one user carries a backpack host), and the user corresponding to the client is a local user. The server can transmit spatial location information and posture information of all users to each client through the network. After receiving the spatial location information and posture information of all users, each client can combine the perspective information of the local user to render a virtual scene suitable for the perspective of the local user and display it to the local user through the helmet worn by the user.

[0040] In the virtual reality interaction method of the embodiment of the present invention, the server realizes the tracking and positioning of the user by combining the image information of the camera and the sensor information collected by the collector, and then the physical location information of each user is obtained. The gesture information is mapped into the virtual space created by the virtual scene client and the server's graphics engine, thereby completing the actual interaction. Since the user is positioned, the image information collected by the camera and the sensor information collected by the collector are used to accurately locate the user even if multiple people interact with each other, thereby avoiding interaction between multiple people. The occlusion causes the problem that the optical marker points are lost and cannot be located.

[0041] It should be noted that, since the user continuously moves during the virtual interaction process, the system also needs to collect the image information of the next frame and the next engraved sensing information, and acquire the next engraving of the user. The spatial position information and the posture information, and the virtual scene is updated according to the actual motion state of the user, and the immersive feeling of the interaction is realized. After the step 103 is performed, the step 101 can be continued.

[0042] According to the foregoing description, the image information of the automatic camera capture includes position coordinate information of all users, and the sensor information from the collector may include: speed and acceleration information of all users. Then, in step 102, the spatial position information and the posture information of the user are calculated, which may be: filtering the position coordinate information, the speed and the acceleration information to obtain spatial position information and posture information of all users. Hereinafter, a detailed description will be made by the embodiment of Fig. 2.

2 is a schematic flowchart of a second embodiment of a virtual reality interaction method according to an embodiment of the present invention. For ease of description, only parts related to the embodiment of the present invention are shown, which are as follows: [0044] Step 201: Receive image information of an automatic camera, and receive sensing information collected by at least one collector and transmitted through a corresponding virtual scene client.

[0045] Step 202: Perform filter processing on the position coordinate information, the speed and the acceleration information to obtain spatial position information and posture information of all users.

[0046] wherein the position coordinate information of the automatic camera capture may include: current coordinate position information and historical coordinate position information; wherein the historical coordinate position information is coordinate position information acquired by the camera capture history. Then, in calculating the user's spatial position information and posture information, there are two ways of operation:

[0047] One is: Regardless of whether occlusion occurs or not, the two pieces of data (ie, position coordinate information, velocity and acceleration information) are used in performing the positioning calculation of the user's spatial position information and posture information, and the position coordinate information used by the 位置 is used. It may be the current coordinate position information (no occlusion 出现 appears) or historical coordinate position information (occlusion 出现 appears). In this case, in calculating the spatial position information and the posture information of the user, specifically, the speed and acceleration information, the current position coordinate information or the historical position coordinate information are filtered to obtain spatial position information and posture information of all users.

[0048] The other is: Positioning is performed using a set of data (ie, current coordinate position information) in the absence of occlusion. When an occlusion 出现 occurs, two sets of data (ie, historical coordinate position information, velocity and acceleration information) are used for positioning. Therefore, before starting the positioning, it is necessary to determine whether the spatial position information and the posture information of all users can be calculated according to the current position coordinate information; it should be noted here that when the occlusion 出现 occurs, there may be no current position coordinate information (currently The position coordinate information is none or there is a part of the position coordinate information but the position information cannot be located by using the part coordinate position information. In this case, after judging that the user can be located according to the current position coordinate information, the spatial position information and the posture information of all the users are directly calculated according to the current position coordinate information. When it is judged that the user cannot be located according to the current position coordinate information, it is necessary to calculate the spatial position information and the posture information of all the users based on the historical position coordinate information and the speed and acceleration information of the user.

[0049] Specifically, the spatial position information and the posture information of all users are calculated according to the current position coordinate information, and the specific operation manner may include, for example, B1-1 to B1-3, as follows:

[0050] B1-1. Extracting two-dimensional coordinate information of the plurality of marker points from the current position coordinate information.

[0051] Wherein, the plurality of marking points may be the optical spheres of the rigid body in the optical motion tracking technology, and the optical tracking system needs to configure the tracked objects with the optical spheres, and the layout is geometrically distributed. The combination of these light balls is configured The rigid body recognized by the system, such as a person in a virtual scene, can be represented by multiple light spheres, changing the spatial position of multiple light spheres, which is different for the tracking system.

[1-22] B1-2. Calculating three-dimensional coordinate information of the plurality of marked points according to the two-dimensional coordinate information. Step B1-2 is specifically implemented to match the key points of the image acquired by the plurality of motion capture cameras with the engraving, and then the two-dimensional coordinate information of the matched marked points is converted into the same by the principle of triangulation. Three-dimensional coordinate information of the space.

[0053] B1-3. Acquire spatial position information and posture information of all users according to the three-dimensional coordinate information of the plurality of marked points and a preset algorithm.

[0054] Specifically, the spatial position information and the posture information of the all users are calculated according to the historical position coordinate information and the speed and acceleration information of the user, and the specific operation manner includes, for example, step C1-1 and step C1-2.

[0055] C1-1. Predicting current position coordinate information based on historical position coordinate information.

[0056] C1-2. Calculate the spatial position information and posture information of all users based on the predicted current position coordinate information and the velocity and acceleration information.

[0057] Step C1-1 may be specifically: predicting current position coordinate information according to historical position coordinate information and historical speed and acceleration information. Step C1-2 may be specifically: calculating spatial position information and posture information of all users according to the predicted current position coordinate information and the current speed and acceleration information.

[0058] Step 203: The spatial location information and the posture information are transmitted to all the virtual scene clients, so that each virtual scene client renders the virtual scene according to the spatial location information, the posture information, and the perspective information of the local user, and displays the virtual scene to the local user.

[0059] In the virtual reality interaction method of the embodiment of the present invention, after the server locates the user, the server can use the image information collected by the camera and the sensor information collected by the collector, so even if the multi-person interaction occurs, the occlusion can be Accurate positioning of the user, avoiding the problem of multi-person interaction and mutual occlusion leading to loss of optical marker points.

[0060] The above two embodiments describe the virtual reality interaction method in detail. The apparatus using the above virtual reality interaction method will be described in detail below with reference to the accompanying drawings. It should be noted that the description and definition of some terms are used. If it has been described in detail in the virtual reality interaction method, it will not be described again in the device embodiment. [0061] In order to implement the above-mentioned virtual reality interaction method, an embodiment of the present invention further provides a virtual reality interaction device, which may be, for example, a server that captures a camera. As shown in FIG. 3, the virtual reality interaction device 30 includes a receiving module 310, an obtaining module 320, and a transmitting module 330.

[0062] The receiving module 310 receives the image information of the automatic camera, and receives the sensing information collected by the at least one collector and transmitted through the corresponding virtual scene client.

[0063] The obtaining module 320 is configured to acquire spatial location information and posture information of all users according to the image information and the sensing information.

[0064] The transmission module 330 transmits the spatial location information and the posture information to all the virtual scene clients, so that each client renders the virtual scene according to the spatial location information, the posture information, and the perspective information of the local user, and displays the virtual scene to the local user; The user is one of all users.

[0065] In a specific implementation, since the user continuously moves during the virtual interaction process, the system further needs to execute the receiving module to collect the image information of the next frame and the next engraved sensing information to obtain the next user. The spatial position information and the posture information are engraved, and the virtual scene is updated according to the actual movement state of the user and the immersive feeling of the virtual interaction is realized. Therefore, when the transmission module 330 completes the corresponding function, the receiving module 310 needs to be triggered to continue working.

[0066] In a specific implementation, the image information includes location coordinate information of all users, and the sensing information includes speed and acceleration information of all users, and the obtaining module 320 is specifically configured to: filter the position coordinate information, the speed, and the acceleration information to obtain Spatial location information and pose information for all users.

[0067] In a specific implementation, the position coordinate information includes current coordinate position information and historical coordinate position information; the obtaining module 320 is further specifically configured to: filter the speed and acceleration information, the current position coordinate information, or the historical position coordinate information to obtain all User's spatial location information and posture information.

[0068] wherein, as shown in FIG. 4, the position coordinate information includes current coordinate position information and historical coordinate position information; the obtaining module 320 includes a determining module 321, a first calculating module 322, and a second calculating module 323.

[0069] The determining module 321, is configured to determine whether the spatial position information and the posture information of all the users can be calculated according to the current position coordinate information.

[0070] The first calculating module 322 is configured to calculate spatial location information and posture information of all users according to the current location coordinate information when the determination result of the determining module is 吋.

[0071] The second calculating module 323 is configured to: when the determining result of the determining module is negative, according to the historical location coordinate letter The user and the user's speed and acceleration information calculate the spatial position information and attitude information of all users.

[0072] wherein, as shown in FIG. 5, the second calculation module 323 includes a prediction module 3231 and a first posture information calculation module 3232.

[0073] a prediction module 3231, configured to predict current location coordinate information according to historical location coordinate information;

[0074] The first posture information calculation module 3232 is configured to calculate spatial position information and posture information of all users according to the predicted current position coordinate information and the speed and acceleration information.

[0075] Wherein, as shown in FIG. 6, the first calculation module 322 includes an extraction module 3221, a three-dimensional coordinate information calculation module 3222, and a second posture information calculation module 3223.

[0076] The extracting module 3221 is configured to extract two-dimensional coordinate information of the plurality of marked points from the current position coordinate information.

[0077] The three-dimensional coordinate information calculation module 3222 is configured to calculate three-dimensional coordinate information of the plurality of marked points according to the two-dimensional coordinate information.

[0078] The second posture information calculation module 3223 is configured to acquire spatial position information and posture information of all users according to the three-dimensional coordinate information of the plurality of marked points and a preset algorithm.

[0079] wherein the posture information includes face orientations of all users.

[0080] The virtual reality interaction device of the embodiment of the present invention realizes the tracking and positioning of the user by combining the image information of the camera and the sensor information collected by the collector, and then the physical location information of each user is The gesture information is mapped into a virtual space created by the client and the graphics engine of the interaction device to complete the actual interaction. Because the user is positioned, the image information collected by the camera and the sensor information collected by the collector are used, even if the interaction of multiple people is blocked, the interactive device can accurately locate the user, avoiding multi-person interaction and mutual interaction. The occlusion between the two causes the problem that the optical marker point is lost and cannot be located.

Correspondingly, the present invention further provides a virtual reality interaction system, the system comprising: a plurality of camera, at least one collector, at least one virtual scene client, at least one helmet display, and the above description of the embodiment Interactive device. Wherein, the collector can be implemented by a gyroscope, and the interaction device can be, for example, a server that captures the camera. One user corresponds to one virtual scene client, one helmet display, and at least one collector. In the following, the interactive system will be described in detail through FIG.

[0082] As shown in FIG. 7, is a schematic diagram of an embodiment of a virtual reality interactive system of the present invention. The virtual reality interaction system includes: a camera 13 , a camera 12 and a camera 13 , an Ethernet router 2 , a server 3. WIFI router 4, virtual scene client 51 and virtual scene client 52, helmet display 61 and helmet display 62, gyroscope 71 and gyroscope 72.

[0083] Among them, the camera 16 , the camera 12 and the camera 13 are used to capture image information of the user and transmit it to the server 3 of the camera through the Ethernet router 2 . At the same time, the gyroscope 71 and the gyroscope 72 also respectively collect the sensing information of the corresponding user and transmit it to the server 3 through the WIFI router 4.

[0084] The server 3 acquires spatial location information and posture information of all users according to the received image information and sensing information; and transmits the spatial location information and the posture information to the client 51 and the client 52. After receiving the spatial location information and posture information of all users, the client 51 renders the virtual scene in conjunction with the perspective information of the local user and displays it to the local user through the helmet display 61. Similarly, after receiving the spatial location information and posture information of all users, the client 52 renders the virtual scene in combination with the perspective information of the local user and displays it to the local user through the helmet display 62. The helmet display and the client computer are connected through an HDMI (High Definition Multimedia Interface) interface, and the gyroscope and the client are connected via Bluetooth.

In summary, the embodiment of the present invention firstly receives the image information of the automatic camera capture and collects the sensor information transmitted by the client of the virtual scene; and then acquires the spatial location of all users according to the image information and the sensor information. Information and attitude information; finally transmitting the spatial location information and the posture information to the client of the virtual scene, so that the client renders the virtual scene according to the spatial location information, the posture information, and the perspective information of the local user and displays it to the local user; The two sets of data with the inertia, so that the user occludes each other, achieving high accuracy of positioning the user.

It should be understood that the size of the serial number of each step in the foregoing embodiment does not mean the order of execution sequence, and the execution order of each process should be determined by its function and internal logic, and should not be constituted by the implementation process of the embodiment of the present invention. Any restrictions.

[0087] wherein, as shown in FIG. 8, the camera server includes a processor 80, a memory 81, and a computer program 82 stored in the memory 81 and operable on the processor 80, such as a virtual reality interactive program. The processor 80 executes the computer program 82 to implement the steps in the various embodiments of the virtual reality interaction method described above, such as steps 101 through 103 shown in FIG. Alternatively, processor 80 executes computer program 82 to implement the functions of the various modules/units in the various apparatus embodiments described above, such as the functions of modules 310 through 330 shown in FIG.

[0088] Illustratively, computer program 82 can be partitioned into one or more modules/units, one or more The modules/units are stored in memory 81 and executed by processor 80 to complete the present invention. The one or more modules/units may be a series of computer program instructions that are capable of performing a particular function, which is used to describe the execution of computer program 82 in the camera server. For example, the computer program 82 can be divided into a receiving module 310, an obtaining module 320, and a transmitting module 330 (modules in a virtual device), and the specific functions of each module are as follows:

[0089] The receiving module 310 is configured to receive image information for automatically capturing a camera, and receive sensing information collected by the at least one collector and transmitted through the corresponding virtual scene client;

[0090] The obtaining module 320 is configured to acquire spatial location information and posture information of all users according to the image information and the sensing information.

[0091] The transmission module 330 is configured to transmit the spatial location information and the posture information to all the virtual scene clients, so that each client renders the virtual scene according to the spatial location information, the posture information, and the perspective information of the local user, and displays the virtual scene to the local user. ; The local user is one of all users.

[0092] The camera server may be a computing device such as a desktop computer, a notebook, a palmtop computer, and a cloud server. The camera server may include, but is not limited to, a processor 80, a memory 81. It will be understood by those skilled in the art that FIG. 8 is merely an example of a camera server, and does not constitute a limitation of the server server. It may include more or less components than those illustrated, or some components may be combined, or different components, such as The camera server may also include an input and output device, a network access device, a bus, and the like.

The processor 80 may be a central processing unit (CPU), or may be another general-purpose processor, a digital signal processor (DSP), or an application specific integrated circuit (Application Specific Integrated).

Circuit, ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, etc. The general purpose processor can be a microprocessor or the processor can be any conventional processor or the like.

[0094] The memory 81 may be an internal storage unit of the camera server, such as a hard disk or a memory of a camera server. The memory 81 may also be an external storage device of the camera server, such as a plug-in hard disk equipped on the camera server, a smart memory card (SMC), and a secure digital (SD) card. , flash card (Flash

Card) and so on. Further, the memory 81 may further include an internal storage list of the camera server. The element also includes an external storage device. The memory 81 is used to store the computer program and other programs and data required by the camera server. The memory 81 can also be used to temporarily store data that has been output or is about to be output.

[0095] It will be clearly understood by those skilled in the art that, for convenience and brevity of description, only the division of each functional unit and module described above is exemplified. In practical applications, the above functions may be assigned differently according to needs. The functional unit and the module are completed, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit, and the integrated unit may be implemented by hardware. Formal implementation can also be implemented in the form of software functional units. In addition, the specific names of the respective functional units and modules are only for the purpose of facilitating mutual differentiation, and are not intended to limit the scope of protection of the present application. For the specific working process of the units and modules in the foregoing system, reference may be made to the corresponding processes in the foregoing method embodiments, and details are not described herein again.

[0096] In the above embodiments, the descriptions of the various embodiments are all focused on, and the parts that are not detailed or described in the specific embodiments may be referred to the related descriptions of other embodiments.

[0097] Those of ordinary skill in the art will appreciate that the elements and algorithm steps of the various examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the solution. A person skilled in the art can use different methods for implementing the described functions for each particular application, but such implementation should not be considered to be beyond the scope of the present invention.

[0098] In the embodiments provided by the present invention, it should be understood that the disclosed device/terminal device and method may be implemented in other manners. For example, the device/terminal device embodiment described above is merely illustrative. For example, the division of the module or unit is only a logical function division, and the actual implementation may have another division manner, for example, multiple units. Or components may be combined or integrated into another system, or some features may be omitted or not implemented. Alternatively, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be in electrical, mechanical or other form.

[0099] The unit described as a separate component may or may not be physically distributed as a unit The displayed components may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.

[0100] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0101] The integrated module/unit, if implemented in the form of a software functional unit and sold or used as a standalone product, may be stored in a computer readable storage medium. Based on such understanding, the present invention implements all or part of the processes in the foregoing embodiments, and may also be completed by a computer program to instruct related hardware. The computer program may be stored in a computer readable storage medium. After the program is executed by the processor, the steps of the various method embodiments described above can be implemented. . Wherein, the computer program comprises computer program code, and the computer program code may be in the form of source code, object code form, executable file or some intermediate form. The computer readable medium can include

: any entity or device capable of carrying the computer program code, recording medium, u disk, mobile hard disk

, Disk, CD, computer memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier signals, telecommunications signals, and software distribution media. It should be noted that the content contained in the computer readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in a jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, computer readable media Does not include electrical carrier signals and telecommunication signals.

The embodiments described above are only intended to illustrate the technical solutions of the present invention, and are not intended to be limiting; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art will understand that The technical solutions described in the foregoing embodiments are modified, or some of the technical features are equivalently replaced; and the modifications or substitutions do not deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. It should be included in the scope of protection of the present invention.

Claims

Claim

[Claim 1] A virtual reality interaction method, which is used for interaction between a real user and a virtual scene, including:

Receiving image information of the automatic camera capture, and receiving sensing information collected by at least one collector and transmitted through the corresponding virtual scene client;

Obtaining spatial location information and posture information of all users according to the image information and the sensing information;

Transmitting the spatial location information and the gesture information to all virtual scene clients, so that each of the virtual scene clients renders a virtual scene according to the spatial location information, the posture information, and the perspective information of the local user, and displays the virtual scene. To the local user; the local user is one of all users.

[Claim 2] The virtual reality interaction method according to claim 1, wherein the image information includes position coordinate information of all users, and the sensor information includes speed and acceleration information of all users, and the basis The step of acquiring the spatial location information and the posture information of all users by using the image information and the sensing information is specifically:

The position coordinate information, the speed and acceleration information are subjected to filtering processing to obtain spatial position information and posture information of all the users.

The virtual reality interaction method according to claim 2, wherein the position coordinate information includes current coordinate position information and historical coordinate position information; and the position coordinate information, the speed and The step of performing the filtering process on the acceleration information to obtain the spatial position information and the posture information of all the users is further specifically: filtering the speed and acceleration information, the current position coordinate information or the historical position coordinate information to obtain Spatial location information and posture information of all users

The virtual reality interaction method according to claim 2, wherein the position coordinate information includes current coordinate position information and historical coordinate position information; and the position coordinate information, the speed and The step of performing acceleration processing on the acceleration information to obtain spatial position information and posture information of all the users includes: Determining whether the spatial location information and the posture information of the all users can be calculated according to the current location coordinate information;

If the determination result is yes, calculating spatial position information and posture information of all the users according to the current position coordinate information;

If the result of the determination is no, the spatial position information and the posture information of the all users are calculated based on the historical position coordinate information and the speed and acceleration information of the user.

[Claim 5] The virtual reality interaction method according to claim 4, wherein the calculating the spatial position information and the posture information of the all users according to the historical position coordinate information and the speed and acceleration information The steps include:

Predicting current position coordinate information according to the historical position coordinate information;

The spatial position information and the posture information of all the users are calculated based on the predicted current position coordinate information and the speed and acceleration information.

The method of claim 4, wherein the calculating the spatial location information and the posture information of the all users according to the current location coordinate information comprises:

Extracting two-dimensional coordinate information of the plurality of marking points from the current position coordinate information; calculating three-dimensional coordinate information of the plurality of marking points according to the two-dimensional coordinate information; and according to the three-dimensional coordinate information of the plurality of marking points The preset algorithm acquires spatial location information and posture information of all the users.

[Claim 7] The virtual reality interaction method according to claim 1, wherein the gesture information includes a face orientation of the all users.

[Claim 8] A virtual reality interaction device, configured to: interact between a real user and a virtual scene, including:

a receiving module, configured to receive image information for automatically capturing a camera, and receive sensing information collected by at least one collector and transmitted through a corresponding virtual scene client;

An obtaining module, configured to acquire spatial location information and posture information of all users according to the image information and the sensing information;

a transmission module, configured to transmit the spatial location information and the posture information to the client Ending, so that each of the clients renders a virtual scene according to the spatial location information, the posture information, and the perspective information of the local user, and displays the virtual scene to the local user; the local user is one of all users.

The virtual reality interaction device according to claim 8, wherein the image information includes location coordinate information of all users, and the sensing information includes speed and acceleration information of all users, and the acquiring module is specifically configured to:

The virtual reality interactive device according to claim 9, wherein the position coordinate information includes current coordinate position information and historical coordinate position information; and the obtaining module is further specifically used for:

And filtering the speed and acceleration information, the current position coordinate information or the historical position coordinate information to obtain spatial position information and posture information of all users according to claim 9, wherein the virtual reality interaction device according to claim 9 The location coordinate information includes current coordinate location information and historical coordinate location information. The acquiring module includes:

a determining module, configured to determine whether the spatial location information and the posture information of the all users can be calculated according to the current location coordinate information;

a first calculating module, configured to: when the determining result of the determining module is 吋, calculate spatial position information and posture information of all users according to the current position coordinate information; and second calculating module, configured to: when The judgment result of the module is NO, and the spatial position information and the posture information of the all users are calculated according to the historical position coordinate information and the speed and acceleration information of the user.

The virtual reality interaction device according to claim 11, wherein the second calculation module comprises:

a prediction module, configured to predict current position coordinate information according to the historical position coordinate information; a first attitude information calculation module, configured to use the predicted current position coordinate information to And the speed and acceleration information calculates spatial position information and posture information of the all users.

The virtual reality interaction device according to claim 11, wherein the first calculation module comprises:

An extraction module, configured to extract two-dimensional coordinate information of the plurality of marked points from the current position coordinate information;

a three-dimensional coordinate information calculation module, configured to calculate three-dimensional coordinate information of the plurality of mark points according to the two-dimensional coordinate information;

The second posture information calculation module is configured to acquire spatial position information and posture information of all the users according to the three-dimensional coordinate information of the plurality of marked points and a preset algorithm.

[Claim 14] A virtual reality interactive system, the system comprising: a plurality of camera, at least one collector, at least one virtual scene client, at least one helmet display, and a camera server;

The camera is configured to capture image information of a user and transmit it to the server server; at least one of the collectors is configured to collect sensing information of the user and transmit the information to the virtual scene client corresponding to the user. ;

At least one of the virtual scene clients is configured to receive sensing information of the corresponding collector and transmit the information to the camera server;

The camera server is configured to acquire spatial location information and posture information of all users according to the image information and the sensing information; and transmit the spatial location information and the posture information to all the virtual scene clients Ending, so that each of the virtual scene clients renders a virtual scene according to the spatial location information, the posture information, and the perspective information of the local user, and displays the virtual scene to the local user through the helmet display; One of the users.

[Claim 15] A computer readable storage medium storing a computer program, wherein the computer program is executed by a processor, implementing the method of any one of claims 1 to The steps of the virtual reality interaction method.