TWI804031B - Multi-human body sensing system combining virtual and reality, method, equipment, and storage medium - Google Patents

Abstract

A multi-human body sensing system, method, device and storage medium combining virtual and reality, which belong to the field of human-computer interaction. The system includes: a computer equipment, a simulation vehicle connected to the computer equipment, n simulation firearms and a display device. The simulation vehicle includes a driver's seat and n passenger seats. The display device is used to display the virtual environment screen and n aiming points. The computer equipment controls the virtual vehicle to change a driving direction in the virtual environment in response to a first driving operation on a first peripheral component, and controls the virtual vehicle to change a driving speed in the virtual environment in response to a second driving operation on a second peripheral component, and simulates to shoot the virtual object aimed at the i-th aiming point in the virtual environment, in response to a shooting operation on an i-th simulated firearm among the n simulated firearms.

Description

Multi-body sensory system, method, device and storage medium combining virtual and reality

The present application relates to the field of human-computer interaction, in particular to a multi-human sensory system, method, device and storage medium combining virtual and reality.

In real life, offline somatosensory games are popular in various entertainment venues, such as racing games and shooting games that can be seen everywhere in game halls.

In related technologies, the game modes between players in the same somatosensory game are similar. For example, in a racing game, player A and player B are racers, and the game mode is to control a simulated vehicle to compete. In a shooting game, player A and player B Players B are all snipers, and the game method is to control simulated firearms to shoot.

In related technologies, the man-machine dialogue mode in the same motion sensing game is single, the types of player operation instructions received by the same motion sensing game system are few, and the degree of freedom of motion sensing games is small.

This application provides a multi-body sensory system, method, device and storage medium combining virtual and reality, which can increase the man-machine dialogue mode. The technical solution is as follows:

According to one aspect of the present application, a multi-person sensory system combining virtual and reality is provided. The system includes: computer equipment, simulation vehicles connected to the computer equipment, n simulation firearms and display devices, where n is a positive integer;

The simulation vehicle includes a driver's seat and n passenger seats, the driver's seat is provided with a first peripheral component and a second peripheral component, the first peripheral component is used to control the driving direction, and the second peripheral component is used to control the driving speed;

The display device is used to display the virtual environment picture provided by the computer equipment and n aiming points, the virtual environment picture is the picture of the virtual environment where the virtual vehicle corresponding to the simulated vehicle is located;

The computer equipment is used to control the virtual vehicle to change the driving direction in the virtual environment in response to the first driving operation on the first peripheral component; to control the virtual vehicle to change the driving direction in the virtual environment in response to the second driving operation on the second peripheral component Changing the driving speed in the virtual environment; responding to the shooting operation on the i-th simulated firearm among the n simulated firearms, shooting the virtual object aimed at by the i-th aiming point among the n aiming points in the virtual environment.

According to one aspect of the present application, a multi-body sensory method combining virtual and reality is provided, the method is executed by a computer device, and the method includes:

Displaying a virtual environment picture and n aiming points through the display device, the virtual environment picture is a picture of the virtual environment where the virtual vehicle corresponding to the simulation vehicle is located;

In response to a first driving operation on the first peripheral component, controlling the virtual vehicle to change its driving direction in the virtual environment;

In response to a second driving operation on the second peripheral component, controlling the virtual vehicle to change its driving speed in the virtual environment;

In response to the shooting operation on the i-th simulated firearm among the n simulated firearms, shoot the virtual object aimed at by the i-th aiming point among the n aiming points in the virtual environment.

According to one aspect of the present application, a multi-body sensory device combining virtual and reality is provided, and the device includes:

The display module is used to display the virtual environment picture and n aiming points through the display device, the virtual environment picture is the picture of the virtual environment where the virtual vehicle corresponding to the simulation vehicle is located;

a control module, configured to control the virtual vehicle to change the driving direction in the virtual environment in response to the first driving operation on the first peripheral component;

The control module is also used to control the virtual vehicle to change the driving speed in the virtual environment in response to the second driving operation on the second peripheral component;

The shooting module is also used to respond to the shooting operation on the i-th simulated firearm among the n simulated firearms, and shoot the virtual object aimed at by the i-th aiming point among the n aiming points in the virtual environment.

According to another aspect of the present application, a computer-readable storage medium is provided, the storage medium stores a computer program, and the computer program is loaded and executed by a processor to realize the above-mentioned multi-body sensation method combining virtual and reality.

According to another aspect of the present application, a computer program product or computer program is provided, the computer program product or computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the above-mentioned multi-person sensory method combining virtual and reality.

The beneficial effects brought by the technical solutions provided by the embodiments of the present application at least include:

By constructing a multi-body sensory system that combines virtuality and reality, the multi-body sensory system that combines virtuality and reality includes: computer equipment, simulated vehicles, n simulated firearms and display devices, and the multi-body sensory system that combines virtuality and reality The system improves the degree of freedom of the somatosensory game, enriches the man-machine dialogue mode of players in the same somatosensory game, and realizes that different players can play different roles in the same somatosensory game.

In order to make the purpose, technical solution and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present application, and are not intended to limit the present application.

First, a brief introduction to the nouns involved in the embodiments of this application:

Multi-person sensory system combining virtual and reality: refers to the somatosensory system used for mutual cooperation between multiple players. In one embodiment, the multi-human sensory system combining virtual and reality includes computer equipment, simulated vehicles connected to the computer equipment, n simulated firearms and display devices, where n is a positive integer; in one embodiment, the simulated vehicle Tools include a driver's seat and n passenger seats. In one embodiment, the driving seat is provided with a first peripheral component and a second peripheral component, the first peripheral component is used to control the driving direction, and the second peripheral component is used to control the driving speed.

In one embodiment, FIG. 1 shows a schematic diagram of a multi-human sensory system combining virtual and reality in an exemplary embodiment of the present application, wherein the multi-human sensory system combining virtual and reality shown in FIG. 1 includes a simulation load Tool 101, display device 102, and n simulated firearms 103. Optionally, simulated vehicles 101 include but are not limited to: cars, off-road vehicles, karts, pickup trucks, motorcycles, speedboats, etc. The simulated vehicle 101 includes a driver's seat and n passenger seats. In one embodiment, the driving position is provided with a simulated steering wheel (first peripheral component) and simulated pedals (second peripheral component), wherein the simulated steering wheel is used to change the driving direction of the virtual vehicle in the virtual environment, and the simulated pedals are used to change the driving direction of the virtual vehicle in the virtual environment. It is used to change the driving speed of the virtual vehicle in the virtual environment.

The display device 102 is used to display a virtual environment image and n aiming points, wherein the n simulated firearms correspond to the n aiming points one by one. Schematically, the display device is a wrap-around screen.

The n simulated firearms 103 are used to shoot virtual objects in the virtual environment. Optionally, the simulated firearms include but not limited to: pistols, submachine guns, shotguns, assault rifles, sniper rifles, crossbows, machine guns, etc. The types of the n simulated firearms 103 may be the same or different.

Virtual Environment: is the virtual environment displayed (or provided) by the application when running on the terminal. The virtual environment can be a simulation environment of the real world, a semi-simulation and semi-fictional environment, or a purely fictitious environment. The virtual environment may be any one of a two-dimensional virtual environment, a 2.5-dimensional virtual environment and a three-dimensional virtual environment, which is not limited in this application. The following embodiments are described with an example that the virtual environment is a three-dimensional virtual environment.

Optionally, the virtual environment may provide a fighting environment for virtual characters. Exemplarily, in a battle royale game, at least one virtual object conducts a single game in a virtual environment, and the virtual object avoids attacks launched by enemy units and dangers in the virtual environment (such as poisonous gas circles, swamps, etc.) etc.) to achieve the purpose of surviving in the virtual environment, when the life value of the virtual object in the virtual environment is zero, the life of the virtual object in the virtual environment ends, and the virtual object that survives successfully at the end is the winner.

Virtual object: refers to the movable object in the virtual environment. The movable object can be a virtual character, a virtual animal, an animation character, etc., for example: a character or an animal displayed in a three-dimensional virtual environment. Optionally, the virtual object is a three-dimensional model created based on animation skeleton technology. Each virtual object has its own shape and volume in the three-dimensional virtual environment, and occupies a part of the space in the three-dimensional virtual environment.

Fig. 2 is a block diagram of a multi-body sensory system combining virtuality and reality in an exemplary embodiment of the present application, wherein the multi-body sensory system 200 combining virtuality and reality includes a computer device 201, a simulation vehicle 101, and n simulated firearms 103 and display device 102.

In one embodiment, the simulation vehicle 101 includes a driver's seat and n passenger seats, the driver's seat is provided with a first peripheral component and a second peripheral component, the first peripheral component is used to control the driving direction, and the second peripheral component The components are used to control the driving speed; optionally, the simulation vehicle 101 includes a simulated vehicle, the first peripheral component includes a steering wheel, and the second peripheral component includes an accelerator and a brake; or, the simulation vehicle 101 includes a simulated aircraft, and the first peripheral component includes a steering wheel. The peripheral components include at least one of a flight rocker and a rudder, and the second peripheral component includes an throttle rocker.

In one embodiment, each of the n simulated firearms 103 includes at least one of a gyroscope, a vibration motor, a battery, a circuit board, and a trigger. In one embodiment, in response to the circuit board obtaining the location data of the simulated firearm through the gyroscope, the circuit board sends the location data to the computer device; in response to the circuit board obtaining the information on whether the simulated firearm is fired through the trigger, the development board will The shooting data is sent to the computer equipment; the vibration motor is connected with the circuit board, and in response to the circuit board confirming shooting, the circuit board controls the vibration motor to generate vibration. The battery is connected with the circuit board and the vibration motor to provide power for the circuit board and the vibration motor. In one embodiment, a communication chip is arranged on the circuit board to support the circuit board to send and receive data.

In one embodiment, the display device 102 displays the virtual environment picture provided by the computer device 201 and n aiming points. Optionally, the display device includes n+1 VR (Virtual Reality, virtual reality) helmets, or is set in Simulate a semi-surround screen in front of the vehicle or a surround screen around the sides, or, a projector.

In one embodiment, the computer device 201 displays a virtual environment picture and n aiming points through the display device 102, the virtual environment picture is a picture of the virtual environment where the virtual vehicle corresponding to the simulated vehicle is located; in response to the first peripheral component The first driving operation on the computer device 201 controls the virtual vehicle to change the driving direction in the virtual environment; in response to the second driving operation on the second peripheral component, the computer device 201 controls the virtual vehicle to change the driving speed in the virtual environment ; In response to the shooting operation on the i-th simulated firearm among the n simulated firearms, the computer device 201 shoots the virtual object aimed at by the i-th aiming point among the n aiming points in the virtual environment.

Optionally, the computer device 201 is implemented as a terminal or a server. An application program supporting a virtual environment is installed and run on the computer device 201 . The application program can be any one of three-dimensional map program, military simulation program, horizontal shooting, horizontal adventure, horizontal pass, horizontal strategy, virtual reality application program, and augmented reality program.

It is worth noting that in FIG. 2, the computer equipment 201 is connected to the simulated vehicle 101, and the n simulated firearms 103 are connected to the display device 102 respectively, wherein the connection mode can be implemented as all wired/wireless connections or partial wired/wireless connections.

Optionally, the number of computer devices 201 can be more or less. For example, there may be only one computer device 201 , or there may be dozens or hundreds of computer devices 201 , or more. The embodiment of the present application does not limit the number and type of computer devices 201 .

In order to improve the player's human-computer interaction experience, Fig. 1 is a schematic diagram of a multi-body sensory system combining virtual and reality in an exemplary embodiment of the present application. The system includes: computer equipment, simulation vehicles 101 connected to the computer equipment, A display device 102 and n simulated firearms 103, where n is a positive integer.

Schematically, FIG. 1 shows a multi-person sensory system combining virtual reality and reality according to an exemplary embodiment of the present application. The system includes a simulated vehicle 101 , a display device 102 and n simulated firearms 103 .

The instructions for the simulation vehicle 101 are as follows:

In one embodiment, the simulation vehicle 101 includes a driver's seat and n passenger seats, the driver's seat is provided with a first peripheral component and a second peripheral component, the first peripheral component is used to control the driving direction, The second peripheral component is used to control the driving speed. Optionally, the simulated vehicle 101 includes a simulated vehicle, the first peripheral component includes a steering wheel, and the second peripheral component includes an accelerator and a brake; or, the simulated vehicle 101 includes a simulated aircraft, and the first peripheral component includes a flight joystick and At least one of the rudders, the second peripheral component includes a throttle rocker; optionally, the simulated vehicle 101 includes a simulated ship, the first peripheral component includes a rudder or paddles or sails, and the second peripheral component includes a throttle and a sail . It is worth noting that the present application does not limit the type, size, shape, etc. of the simulation vehicle 101 , in fact, the simulation vehicle 101 can be implemented as any type of vehicle.

In one embodiment, the simulated vehicle 101 includes, but is not limited to: at least one of simulated cars, simulated off-road vehicles, simulated karts, simulated pickup trucks, simulated motorcycles, and simulated speedboats.

With reference to FIG. 1 , the simulated vehicle 101 is implemented as a simulated off-road vehicle. The simulated vehicle 101 in FIG. 1 includes a driver's seat and 3 passenger seats, which are specifically assigned as co-pilot, rear left, and rear right. The first peripheral component is the steering wheel of the simulated off-road vehicle, and the second peripheral component is the accelerator and brake of the simulated off-road vehicle. The steering wheel of the simulated off-road vehicle is used to control the driving direction of the simulated off-road vehicle, and the accelerator and brake of the simulated off-road vehicle are used to control the driving speed of the simulated off-road vehicle.

Schematically, FIG. 3 shows a first peripheral component and a second peripheral component provided by an exemplary embodiment of the present application. Wherein, the first peripheral component is a steering wheel 301 , and the second peripheral component includes an accelerator pedal 302 and a brake pedal 303 .

In one embodiment, at least one of the driver's seat and the n passenger seats in the simulation vehicle 101 is provided with a vibration feedback component. Wherein, the vibration feedback component is used to generate vibration on the simulation vehicle 101 when the virtual vehicle in the virtual environment collides with an obstacle. Schematically, the collision between the virtual vehicle and the obstacle includes three possible states: collision occurs, collision exists and collision leaves. When the collision occurs between the virtual vehicle and the obstacle in the virtual environment, the vibration feedback component generates a strong vibration on the simulation vehicle 101; when the collision exists between the virtual vehicle and the obstacle in the virtual environment , the vibration feedback component generates ordinary vibration on the simulation vehicle 101; when the virtual vehicle in the virtual environment collides with the obstacle and leaves the state, the vibration feedback component does not generate vibration on the simulation vehicle 101.

Wherein, the vibration feedback component generates vibrations on the driver's seat and at least one of the n passenger seats on the simulation vehicle 101 . Or, the vibration feedback component generates vibrations on the trunk of the simulation vehicle 101 .

In one embodiment, when the virtual vehicle collides with the "wall" in the virtual environment, the computer device generates a collision event between the virtual vehicle and the "wall", and controls the vibration feedback component to generate corresponding vibrations.

It is worth noting that different obstacles are different due to the settings of the virtual environment, that is, there are more than one type of obstacle, different vibration intensities are different due to different collision obstacles, and different vibration intensities are different due to different types of collision events. The intensity of the shock varies depending on the driving speed of the virtual vehicle in the virtual environment at the time of the collision.

In one embodiment, when the computer device detects that the surface of the vehicle collides with the obstacle, it is in the state of collision, and when the computer device detects that the surface of the vehicle is colliding with the obstacle, it is in the state of collision. When the computer device detects that the surface of the vehicle is no longer colliding with the obstacle, it is in the collision-away state.

Schematically, FIG. 4 shows three of the six sides of the carrier. Schematically, the three sides are the front 403 of the carrier, the left side 401 of the carrier and the top 402 of the carrier. When the computer equipment detects that the left side 401 or the top surface 402 or the front side 403 of the vehicle collides with the obstacle, it is a state of collision; When the vehicle is colliding, it is the state of collision existence. When the computer device detects that the left side 401 or the top surface 402 or the front side 403 of the vehicle no longer collides with the obstacle, it is the state of collision leaving.

The description for the display device 102 is as follows:

In one embodiment, the display device 102 is configured to display a virtual environment picture provided by a computer device and n aiming points, and the virtual environment picture is a picture of a virtual environment where a virtual vehicle corresponding to a simulated vehicle is located.

Optionally, the display device 102 includes n+1 VR helmets; wherein, the n VR helmets are in one-to-one correspondence with the n aiming points, that is, except that the VR helmet corresponding to the driving seat of the simulation vehicle 101 does not display the aiming points, other The VR headsets corresponding to the passenger seats all display aiming points.

Optionally, the display device 102 includes a semi-surround screen or a surround screen arranged in front of the simulation vehicle 101; optionally, the display device 102 includes a projector.

With reference to FIG. 1, FIG. 1 shows that the display device 102 is a surrounding screen on the side of the vehicle, and the display device 102 shows a picture of the virtual environment where the virtual vehicle corresponding to the simulation vehicle 101 is located. In FIG. 1, the simulation off-road The car is on the surrounding road at the foot of the mountain, and the virtual environment includes hillsides, surrounding roads, trees, etc.

In one embodiment, the width of the half-surrounding screen is not less than the width of the simulation vehicle 101, and the height of the half-surrounding screen is not less than the height of the simulation vehicle. The width of the half-surrounding screen refers to the distance between the two vertical sides of the half-surrounding screen. The height of the half-surround screen refers to the distance between the two horizontal sides of the half-surround screen. Optionally, both the half-surround screen and the simulation vehicle 101 are placed on the ground, and the half-surround screen half-surrounds the simulation vehicle in a half-surround posture. With reference to FIG. 1 , the display device 102 (half-surround screen) is wider than the simulation vehicle. The width of the tool 101 and the height of the display device 102 (half-surrounding screen) are greater than the height of the simulation vehicle 101, and the display device 102 (half-surrounding screen) half-encloses the simulation vehicle 101 in a half-surrounding posture.

Detailed description for n simulated firearms 103:

In one embodiment, n simulated firearms 103 are used to shoot virtual objects aimed at in the virtual environment by n aiming points corresponding to n passenger seats.

In one embodiment, when the i-th simulated firearm among the n simulated firearms 103 performs a shooting operation, the i-th simulated firearm is a virtual object aimed at the i-th aiming point corresponding to the i-th passenger seat in the virtual environment to shoot.

Optionally, the n simulated firearms 103 include but are not limited to: at least one of pistols, submachine guns, shotguns, assault rifles, sniper rifles, crossbows, and machine guns. The types of the n simulated firearms may be the same or different.

In one embodiment, each of the n simulated firearms 103 includes at least one of a gyroscope, a vibration motor, a battery, a circuit board, and a trigger. In one embodiment, in response to the circuit board obtaining the location data of the simulated firearm through the gyroscope, the circuit board sends the location data to the computer device; in response to the circuit board obtaining the information on whether the simulated firearm is fired through the trigger, the development board will The shooting data is sent to the computer equipment; the vibration motor is connected with the circuit board, and in response to the circuit board confirming shooting, the circuit board controls the vibration motor to generate vibration. The battery is connected with the circuit board and the vibration motor to provide power for the circuit board and the vibration motor. In one embodiment, a communication chip is arranged on the circuit board to support the circuit board to send and receive data.

With reference to FIG. 1 , the simulated firearm is shown as an assault rifle.

Schematically, FIG. 5 shows a simulated firearm according to an exemplary embodiment of the present application. In FIG. 5 , the player is manipulating the simulated firearm 501 for a shooting game, and the shooting game screen is displayed on the screen. Wherein, the simulated firearm 501 is wirelessly connected to the screen. In one embodiment, the circuit board uses the UDP (User Datagram Protocol) protocol to transmit packet information in the local area network of the simulated firearm and computer equipment to realize wireless transmission.

Finally, the computer device 201 is described in detail:

In one embodiment, the computer device 201 is connected to the simulated vehicle 101 , n simulated firearms 103 and the display device 102 respectively. The simulation vehicle 101 includes a driver's seat and n passenger seats, the driver's seat is provided with a first peripheral component and a second peripheral component, the first peripheral component is used to control the driving direction, and the second peripheral component is used to control the driving speed ;

Among them, the computer device 201 is used to control the virtual vehicle to change the driving direction in the virtual environment in response to the first driving operation on the first peripheral component; in response to the second driving operation on the second peripheral component, control the virtual The vehicle changes the driving speed in the virtual environment; in response to the shooting operation on the i-th simulated firearm in the n simulated firearms 103, shoot the virtual object aimed at the i-th aiming point in the virtual environment among the n aiming points .

In one embodiment, the first driving operation refers to the operation of controlling the virtual vehicle to change the driving direction in the virtual environment, and the second driving operation refers to the operation of controlling the virtual vehicle to change the driving speed in the virtual environment.

Optionally, at least one of the n simulated firearms 103 has a wireless communication connection with the computer device 201; and/or, at least one of the n simulated firearms 103 has a wired communication connection with the computer device 201.

Optionally, the simulation vehicle 101 establishes a wireless communication connection with the computer device 201 , or, the simulation vehicle 101 establishes a wired communication connection with the computer device 201 .

In one embodiment, the n simulated firearms 103 transmit data with the computer device 201 through the UDP protocol.

In one embodiment, the simulation vehicle 101 performs data transmission with the computer device 201 through USB (Universal Serial Bus, universal serial bus) HID (Human Interface Device, human-computer interaction device) protocol communication.

The working principle of the above-mentioned multi-person sensory system combining virtual and reality is as follows:

The player in the driver's seat controls the driving direction of the virtual vehicle in the virtual environment through the first peripheral component of the simulation vehicle 101, and controls the direction of the virtual vehicle in the virtual environment through the second peripheral component of the simulation vehicle 101. At the driving speed, the player in the passenger position uses n simulated firearms 103 to shoot at virtual objects aimed at in the virtual environment, and finally the display device 102 displays a picture of the virtual environment where the virtual vehicle is located.

To sum up, by constructing a multi-body sensory system that combines virtuality and reality, the multi-body sensory system that combines virtuality and reality includes: computer equipment, simulated vehicles, n simulated firearms and display devices, which enriches the player's experience in The man-machine dialogue mode in the same somatosensory game realizes that different players can play the game through different operation methods under the same somatosensory game.

The above-mentioned multi-human sensory system combining virtual and reality also includes a vibration feedback component. When the virtual vehicle in the virtual environment collides with an obstacle, the vibration feedback component on the simulation vehicle vibrates, which realizes the vibration feedback between the virtual vehicle and the obstacle. The simulation of the collision of obstacles brings the sense of collision in the virtual environment to reality, realizing the immersive experience of players in the virtual environment.

The above-mentioned multi-body sensory system combining virtual and reality improves the degree of freedom of the game, thereby enhancing the player's human-computer interaction experience and meeting the growing entertainment needs of the player.

Fig. 6 is a flowchart of a multi-body sensory method combining virtuality and reality according to an exemplary embodiment of the present application. The method is executed by a computer device in the above-mentioned multi-body sensory system combining virtuality and reality. The method includes:

Step 620 , displaying the virtual environment picture and n aiming points through the display device;

Wherein, the virtual environment picture is a picture of the virtual environment where the virtual vehicle corresponding to the simulation vehicle is located;

In one embodiment, the display device includes n+1 VR helmets, and the computer device displays the virtual environment picture and the i-th aiming point among the n aiming points through the i+1th VR helmet in the n+1 VR helmets .

In one embodiment, the display device includes a surround screen or a projector; the computer device displays a virtual environment picture and n aiming points through the surround screen or a projector, and the n aiming points have different graphic styles, and the graphic styles include shapes, At least one of color, transparency, and numbering.

Wherein, the virtual environment picture and the n aiming points are on the interface of the same display device.

Schematically, FIG. 7 shows a diagrammatic view of an aiming point in an exemplary embodiment of the present application. Wherein, the pictorial style of the aiming point is displayed as a crosshair, and the corresponding passenger seat number is displayed on each sight.

In one embodiment, in response to the computer device receiving the aiming data sent by the simulated firearm, the computer device displays a virtual environment picture and n aiming points through a display device.

In one embodiment, the aiming data may include the Euler angle of rotation of the gyroscope in the simulated firearm, the magnification of the aiming point, the demagnification of the aiming point, and the like. Among them, a gyroscope is set on the simulated firearm, and the gyroscope is used to monitor the degree of deviation of the player controlling the simulated firearm, and based on the degree of deviation of the simulated firearm, Euler angles are generated, and the Euler angle is used to describe the scope on the simulated firearm s position. In one embodiment, the aiming data is a quaternion corresponding to the Euler angle of the gyroscope on the firearm.

In one embodiment, the n simulated firearms are provided with unique numbers, for example, the number of simulated firearm 1 is aim01, and the number of simulated firearm 2 is aim02.

In one embodiment, in response to the computer device receiving the aiming information sent by the simulated firearm, the computer device displays n aiming points through the display device, which is implemented by the following code: //Use the udp module of nodejs to obtain the data sent by the specified ip const dgram = require('dgram'); const dgramServer = dgram. createSocket('udp4'); dgramServer.on('message', (message, rinfo) => {; const IP = rinfo.address; const msg = message; //If the ip in the UDP port is aim01IP, report the parameters reported by udp to the computer device if (IP === aim01Ip) { const data = [msg.readFloatLE(3), msg.readFloatLE(7), msg.readFloatLE(11), msg.readFloatLE(15), message[2]]; socketData.aim01 = data; }; ... }; // Listen to the socket aimiMove event Vars.socket.on('aimiMove', (data) => { this. aimMgr(data); }); ... // Simulated firearm 1 if(data.aim01.length > 0){ //offset value const posX = 100; //Quaternion data of gyroscope let quat = new Quat(data.aim01[0],-data.aim01[2],-data.aim01[3],data.aim01[1]); const offset = new Vec3(0, 0, 1); //Convert to 3D vector Vec3. transformQuat(offset, offset, quat); //Three-dimensional vector multiplied by 1200 offset. multiplyScalar(1200); const aimPos = new Vec3(offset.x,offset.y,0); // coordinate boundary detection const pos = aimBoundary(aimPos); //Assign a value for the front sight (2D) this.aim01.setPosition(pos.x+posX,pos.y,0);

Step 640, in response to the first driving operation on the first peripheral component, control the virtual vehicle to change the driving direction in the virtual environment;

Wherein, the first driving operation refers to a driving operation performed on the simulated vehicle to control the driving direction of the simulated vehicle.

In one embodiment, the simulated vehicle includes a simulated vehicle, and in response to the first driving operation on the steering wheel on the simulated vehicle, the computer device controls the virtual vehicle to change the driving direction in the virtual environment;

In one embodiment, the simulated vehicle includes a simulated aircraft, and in response to the first driving operation of the flight stick and/or rudder on the simulated aircraft, the computer device controls the virtual aircraft to change the driving direction in the virtual environment.

In one embodiment, the computer device receives the steering wheel steering data of the simulated vehicle by the following code, wherein the steering data includes the quaternion of the Euler angle of the steering wheel rotation. const g = require('logitech-g29'); //Initial value of the steering wheel const wheelOptions = { autocenter: [0.2, 0.2], debug: false, range: 900, }; // connect the steering wheel g.connect(wheelOptions, (err) => { if (err) return; let baseWheel = 50; g.connect(wheelOptions, (err) => { if (err) return; g. connect({ autocenter: true }) // Listen to the player's steering operation event, and if it occurs, send the steering wheel's steering data to the computer device g.on('wheel-turn', function(val) { io. emit('wheel-turn', [val, baseWheel - val]); })

Step 660, in response to the second driving operation on the second peripheral component, control the virtual vehicle to change the driving speed in the virtual environment;

Wherein, the second driving operation refers to the driving operation performed on the simulated vehicle to control the driving speed of the simulated vehicle.

In one embodiment, the simulated vehicle includes a simulated vehicle, and in response to a second driving operation on the accelerator and/or brakes on the simulated vehicle, the computer device controls the virtual vehicle to change its driving speed in the virtual environment.

In one embodiment, the simulated vehicle includes a simulated aircraft, and in response to the second driving operation on the accelerator stick on the simulated aircraft, the computer device controls the virtual aircraft to change the driving speed in the virtual environment.

In one embodiment, the computer device receives the forward data of the accelerator and brake on the simulated vehicle by the following code, wherein the forward data includes acceleration data corresponding to the acceleration generated by the accelerator pedal and deceleration data corresponding to the deceleration generated by the brake. //Monitor the player's braking operation event, if it occurs, send the braking deceleration data to the computer device g.on('shifter-gear', function(val) { io. emit('shifter-gear', val); }); //Monitor the player's acceleration operation event, if it occurs, send the acceleration data of the accelerator to the computer device g.on('pedals-gas', function(val) { io. emit('pedals-gas', val); }) ... })

In one embodiment, the computer device calculates the coordinates of the virtual vehicle in the virtual environment from the forward data and the steering data generated in step 640, which is specifically implemented by the following code: // If hit a wall, don't move forward if(this. carFrontOnWall) return; speed = this.forwardData-this.brakeData;//Calculate the difference between acceleration data and deceleration data to get forward data speedBase = speed*this.maxSpeedBase/this.fullSpeed/2;//Convert the forward data into the forward distance of the virtual vehicle in the virtual environment if(speed<0) { speed = 0; } // If it is not a wall or an obstacle, move the virtual vehicle in the virtual environment according to the forward distance this.carPosition.x = this.carPosition.x + Number(((Math.sin(2*Math.PI/360*(90+this.carEulerAngles.y))*speedBase).toFixed(3))); this.carPosition.z = this.carPosition.z - Number(((Math.sin(2*Math.PI/360*(this.carEulerAngles.y))*speedBase).toFixed(3))); this.node.setWorldPosition(this.carPosition.x,this.carPosition.y,this.carPosition.z);

There is a mapping relationship between the advancing distance of the virtual vehicle in the virtual environment and the advancing data generated by the simulated vehicle. Optionally, the forward distance is speed, the forward distance of the virtual vehicle in the virtual environment is speedBase, the maximum forward distance of the virtual vehicle in the virtual environment is maxSpeedBase, the maximum forward data generated by the simulated vehicle is fullSpeed, and the mapping relationship is speedBase = speed* maxSpeedBase/fullSpeed/2.

Wherein, the steering data refers to the steering angle generated by the steering wheel of the simulated vehicle. Optionally, the steering angle is the Euler angle. Optionally, the formula for mapping the forward distance to the coordinates of the virtual vehicle in the virtual environment is:

carPosition.x = carPosition.x + sin(2* PI/360*(90+carEulerAngles.y))*speedBase) (1)

In the above formula (1), carPosition.x on the left side of the equation is the x coordinate of the virtual vehicle after steering, carPosition.x on the right side of the equation is the x coordinate of the virtual vehicle before steering, and carEulerAngles.y is the Euler angle generated by the steering wheel rotation.

carPosition.z = carPosition.z - sin(2*PI/360*carEulerAngles.y)*speedBase(2)

In the above formula (2), carPosition.z on the left side of the equation is the z coordinate of the virtual vehicle after turning, and carPosition.z on the right side of the equation is the z coordinate of the virtual vehicle before turning.

In one embodiment, in response to the collision between the virtual vehicle and the virtual object in the virtual environment, the computer device controls the vibration feedback component on the simulation vehicle to generate vibration.

Specifically implemented by the following code: // Mount the collision event for the collision body carFrontCollider.on('onTriggerStay', this.onTrigger, this); carFrontCollider.on('onTriggerEnter', this.onTrigger, this); carFrontCollider.on('onTriggerExit', this.onTrigger, this); // Get the value when the collision event occurs onTrigger(event){ const eNode = event. otherCollider. node; const eName = eNode._name; const scoreTips = this.aim00Scoretips; //If the current collision is not a wall or an obstacle, execute the forward or backward logic. Otherwise, the prompt cannot move if(event.type === 'onTriggerEnter' || event.type === 'onTriggerStay'){ if(eName.indexOf('Wall') === 0 || eName.indexOf('jeep') === 0 || eName.indexOf('Outside_Target_07') === 0){ if(event.selfCollider.node._name.indexOf('Front')===0){ this.carFrontOnWall = true; } if(event.selfCollider.node._name.indexOf('Rear')===0){ this.carRearOnWall = true; } ...

The hardware setting of the vibration feedback component is realized by the following code: const SerialPort = require('serialport'); const getCRC = require('./libs/crc16'); // Connect the vibration module plugged into the com5 port SHOCK = new SerialPort('COM5', { autoOpen: true, baudRate: 38400 }); // Listen to the corresponding events of the com5 port SHOCK.on('close', (line) => console.log(`> Close, the vibrator failed`, line)); SHOCK.on('error', (err) => console.log(`> Error, the vibrator failed`, err.message)); //Write 1500 (0x5dc) speed for motor No. 1 let buffer = Buffer.from([1, 0x3, rpm === 0 ? 0 : 0x1, 0, 0, 4, 0, 0]); buffer.writeUInt16LE(0x5dc, 3); let crc = getCRC(buffer, buffer.length - 2); buffer.writeUInt16BE(crc, 6); SHOCK.write(buffer);

Step 680: In response to the shooting operation on the i-th simulated firearm among the n simulated firearms, shoot the virtual object aimed at by the i-th aiming point among the n aiming points in the virtual environment.

In one embodiment, in response to a shooting operation on the i-th simulated firearm among the n simulated firearms, the computer device shoots the virtual object aimed at by the i-th aiming point among the n aiming points in the virtual environment.

In one embodiment, the shooting operation includes, but is not limited to: starting shooting, stopping shooting, adjusting shooting distance, replacing dummy clips, and the like. In one embodiment, the shooting data generated by the shooting operation is the shooting instruction aim01[4] of the simulated firearm 1 .

In response to the shooting operation on the i-th simulated firearm in the n simulated firearms, shoot the virtual object aimed at the i-th aiming point in the virtual environment among the n aiming points, optionally, n=4, i= 1, implemented by the following code: //When aim01[4]=1, trigger the shooting logic if(data.aim01[4] === 1){ this.fire('aim01'); }

In one embodiment, the computer device shoots the virtual object aimed at by the first aiming point in the virtual environment, which is implemented by the following code: // Temporary 3D vector const screenP_mouse = new Vec3(); // 2D coordinates to 3D coordinates this.canvas_2d.camera.worldToScreen(screenP_mouse, aimNode.worldPosition); // generate ray this.mainCamera.screenPointToRay(screenP_mouse.x,screenP_mouse.y, this._ray_mouse); // Get the nodes contained in the ray const raycaseList = PhysicsSystem.instance.raycast(this._ray_mouse); // process node if(raycaseList){ const all = PhysicsSystem. instance. raycastResults; all.forEach((r, index) => { // Obtain the position according to the property of r.distance if(r.distance > 250) return; // Judge the separate logic based on the node name if(r._collider.node.name === 'target' ... ) { //... } }) }

As mentioned above, the computer equipment converts the two-dimensional coordinates of the aiming point into the three-dimensional coordinates of the aiming point in the virtual environment, and when receiving the command to start shooting, emits a ray along the direction of the camera in the virtual environment and the aiming point in the virtual environment, When the ray collides with the virtual object in the virtual environment, the computer equipment obtains the virtual object, and displays the performance characteristics of the virtual object when the virtual object is obtained on the display device. Optionally, the performance characteristics of the virtual object are bullet holes appearing in the virtual object, explosion of the virtual object, disappearance of the virtual object, and the like.

To sum up, the multi-body sense method combining virtual and reality executed by computer equipment, the computer equipment controls the display device to display the virtual environment picture and the aiming point, the computer equipment controls the virtual vehicle to change the driving direction and driving speed in the virtual environment, The computer device realizes the shooting of the virtual object aimed at the i-th aiming point in the virtual environment among the n aiming points, which enriches the man-machine dialogue mode of players in the same somatosensory game, and realizes different players in the same somatosensory game The game can be played in different ways.

The above-mentioned multi-body sensory method combining virtualization and display improves the degree of freedom of the game, thereby enhancing the player's human-computer interaction experience and meeting the growing entertainment needs of the player.

In order to realize the management and allocation of the driver's seat and the passenger seat, based on the optional embodiment shown in FIG. 6 , FIG. 8 shows a multi-body sensory method combining virtual and reality provided by an exemplary embodiment of the present application. Step 620 Previously also included the following steps:

Step 611: Send the room information of the virtual environment, the driver bit entry graphic code and the n passenger bit entry graphic codes to the administrator terminal;

In one embodiment, the multi-person sensory system combining virtual and reality further includes an administrator terminal. Wherein, the administrator terminal is used for receiving data sent by the computer equipment or sending data to the computer equipment. Optionally, the administrator terminal receives the room information sent by the computer device, and displays the status of at least one room on the terminal interface; optionally, the administrator terminal sends the information of the virtual character selected by the player to the computer device, and displays the status of at least one room on the terminal interface. Display the avatar information selected by the player.

In one embodiment, the computer device sends the room information of the virtual environment, the driver bit entry graphic code and the n passenger bit entry graphic codes to the administrator terminal.

In one embodiment, in response to the computer equipment sending the room information of the virtual environment to the administrator terminal, the administrator terminal displays the administrator interface. Schematically, FIG. 9 shows the administrator interface of an exemplary embodiment of the present application. Schematic diagram, wherein, the administrator interface displays a zoomed-out window 901 of the room in progress and a detailed window 902 of the ready room, where n=3, that is, 3 passenger seats are shown in FIG. 9 .

Wherein, the zoom-out window 901 of the room in progress shows the control items of "room in progress: No. 917" and "close". In one embodiment, in response to the administrator touching the "close" control item, the administrator terminal sends an instruction to close room No. 917 to the computer device, and the computer device executes the operation of closing room No. 917.

Among them, the detailed window 902 of the preparation room shows "preparation room: No. 918", four optional seat control items of room No. 918 "driver-1P, copilot-2P, left rear-3P and right rear-4P" and "obsolete " control item, in response to the administrator touching the "driver-1P" control item, the computer device sends the driving bit graphic code to the administrator terminal; 4P” control item, the computer device sends the passenger bit graphic code to the administrator terminal; in response to the administrator’s touch of the “void” control item, the administrator terminal sends the command to cancel Room 918 to the computer device, and the computer The device performs the operation of voiding room 918.

In one embodiment, in response to the computer equipment sending the driver bit entry graphic code and the n passenger bit entry graphic codes to the administrator terminal, the administrator terminal displays the administrator interface. Schematically, FIG. 10 shows the A schematic diagram of the interface of the administrator terminal displaying the co-pilot-2P entry graphic code of an exemplary embodiment, as shown in Figure 10, on the basis of Figure 9, Figure 10 adds the passenger bit entry graphic code of "Co-pilot-2P" 903 , the interface diagram of the administrator terminal displaying the graphic codes corresponding to the driver-1P, the left rear-3P and the right rear-4P on the administrator interface is similar to that in FIG. 9 , and will not be repeated here.

Step 612 : Receive the first joining request triggered by scanning the driving bit entry graphic code by the first terminal; in response to the first joining request, add the virtual character corresponding to the first account logged in by the first terminal to the virtual vehicle in the virtual environment on the driver's seat;

The first joining request refers to a request from the player to join the driving seat when the player has completed account login on the first terminal, that is, the player requests to drive the simulation vehicle. The first account logged in by the first terminal refers to the account logged in by the first player on the terminal, which is used to identify the first player and send/receive information about the first player.

In one embodiment, the computer device receives the first joining request triggered by the scanning of the driving bit entry graphic code by the first terminal, and in response to the first joining request, the computer device adds the virtual character corresponding to the first account logged in by the first terminal to the in the driver's seat of a virtual vehicle within a virtual environment.

Step 613: Receive the second joining request triggered by the second terminal scanning the passenger bit entry graphic code; in response to the second joining request, add the virtual character corresponding to the second account logged in by the second terminal to the virtual vehicle in the virtual environment on the passenger seat.

The second join request refers to a request to join the passenger seat when the player has completed the account login on the second terminal, that is, the player requests to become a passenger of the simulated vehicle and control the simulated firearm.

The second account logged in by the second terminal refers to the account logged in by the second player on the terminal, which is used to identify the second player and send/receive information about the second player.

In one embodiment, the computer device receives the second joining request triggered by the second terminal scanning the passenger bit entry graphic code; in response to the second joining request, the computer device adds the virtual character corresponding to the second account logged in by the second terminal to the on the passenger seat of the virtual vehicle within the virtual environment.

Schematically, FIG. 11 shows the administrator interface displayed on the administrator terminal after the computer device adds the virtual character corresponding to the second account logged in by the second terminal to the passenger seat of the virtual vehicle in the virtual environment.

Wherein, the account information area 1101 displays the second account information added to the copilot-2P position of the virtual vehicle in the virtual environment, including the second account's avatar, name, room number and passenger seat of the virtual vehicle. The room information area 1102 displays the information of the started room, including "room number" information, "location" information, "start time" information, "game friend" avatar information and "score" information.

In one embodiment, the player needs to choose to join the room after scanning the graphic code in the above steps. Schematically, FIG. 12 shows a flow chart of a player joining a room in an exemplary embodiment of the present application. Joining a room includes the following situations :

First case:

Step 1210, if there is no target room information, the player cannot enter the target room;

When the player touches the control item of the target room, the player terminal interface displays no room information, that is, the room is not open, and the player cannot enter the room.

Second case:

Step 1220, if the state of the target room is waiting state, the player enters the target room;

When the state of the target room touched by the player is a room in the waiting state, step 1221 is executed.

Step 1221, when the character selected by the player does not conflict with the selected character in the room, the player enters the room;

When the character selected by the player does not conflict with the selected character in the room, the player enters the room.

Step 1222, if the user information has been authorized, the player completes the selection of the character in the room;

When the user information is authorized, the player completes the selection of the character in the room.

Step 1223, if the user information is not authorized, the player has not completed the selection of the character in the room;

When the user information is not authorized, the player has not completed the selection of the character in the room.

Step 1224, the player enters the target room multiple times;

When the player enters the room multiple times, it is judged according to the state of the player's current character selection. If the player has completed the selection of the character in the room, the interface shown in Figure 11 will be displayed on the player terminal interface; After completing the selection of characters in other rooms, the player cannot enter this room.

Step 1225, when a role conflict occurs, the player terminal interface prompts to rescan;

When the character selected by the player conflicts with the selected character in the room, the player cannot enter the room, and the player terminal interface will prompt the player to re-enter the room or re-scan the graphic code.

The third case:

Step 1230, when the state of the target room is in progress, the player cannot enter the target room;

When the state of the target room touched by the player is a room in progress, go to step 1231 .

Step 1231, when the player has entered the room, re-establish the connection;

When the player has entered the room, the player terminal re-establishes the connection with the multi-person sensory system combining virtual and reality.

Step 1232, when the player has not entered the room, the connection cannot be re-established;

When the player does not enter the room, the player cannot establish a connection with the multi-person sensory system that combines virtual and reality.

Fourth case:

Step 1240, when the state of the target room is an end state, the player cannot enter;

When the player touches a room whose room state is the end state, go to step 1241 .

Step 1241, when the player has entered the room, re-establish the connection;

When the player has entered the room, the player terminal re-establishes the connection with the multi-person sensory system combining virtual and reality.

Step 1242, when the game ends normally, the player terminal displays the score list;

When the room status is the normal ending status of the game, the player terminal displays the score list interface.

Step 1243, when the game is terminated, the player terminal does not display the score list;

When the room status is the game terminated state, the player terminal cannot display the score list interface.

Step 1244, when the player has not entered the room, the connection cannot be re-established;

When the player does not enter the room, the player cannot establish a connection with the multi-person sensory system that combines virtual and reality.

Fifth case:

Step 1250, when the player enters the target room overtime, the player cannot enter;

When the player enters the room timeout, the player cannot enter the room and needs to seek help from the administrator.

To sum up, sending/receiving room-related information, virtual character information, player account information, etc. to the administrator terminal through the computer equipment provides a way for the administrator terminal to quickly join the game under the premise of safety and controllability .

In one embodiment, FIG. 13 shows a structural block diagram of a multi-body sensing device combining virtual and reality in an exemplary embodiment of the present application, wherein the device includes:

The display module 1303 is used to display the virtual environment picture and n aiming points through the display device, the virtual environment picture is the picture of the virtual environment where the virtual vehicle corresponding to the simulation vehicle is located;

The control module 1304 is configured to control the virtual vehicle to change the driving direction in the virtual environment in response to the first driving operation on the first peripheral component;

The control module 1304 is further configured to control the virtual vehicle to change its driving speed in the virtual environment in response to the second driving operation on the second peripheral component;

The shooting module 1305 is configured to respond to the shooting operation on the i-th simulated firearm among the n simulated firearms, and shoot the virtual object aimed at by the i-th aiming point among the n aiming points in the virtual environment.

In an optional embodiment, the simulated vehicle includes a simulated vehicle.

In an optional embodiment, the control module 1304 is further configured to control the virtual vehicle to change the driving direction in the virtual environment in response to the first driving operation on the steering wheel of the simulated vehicle.

In an optional embodiment, the simulated vehicle includes a simulated aircraft.

In an optional embodiment, the control module 1304 is also used to control the virtual aircraft to change the driving direction in the virtual environment in response to the first driving operation of the flight stick and/or rudder on the simulated aircraft.

In an optional embodiment, the control module 1304 is further configured to control the virtual vehicle to change its driving speed in the virtual environment in response to the second driving operation on the accelerator and/or brake on the simulated vehicle.

In an optional embodiment, the control module 1304 is also used to control the virtual aircraft to change the driving speed in the virtual environment in response to the second driving operation on the accelerator stick on the simulated aircraft.

In an optional embodiment, the display device includes n+1 VR helmets.

In an optional embodiment, the display module 1303 is also configured to display the virtual environment image and the i-th aiming point among the n aiming points through the i+1th VR helmet among the n+1 VR helmets.

In an optional embodiment, the display device includes a surround screen or a projector;

In an optional embodiment, the display module 1303 is also used to display a virtual environment picture and n aiming points through a surround screen or a projector. The n aiming points have different graphic styles, and the graphic styles include shapes, colors At least one of , transparency, and numbering.

In an optional embodiment, the multi-body sensing device combining virtual and reality further includes a sending module 1301 and a processing module 1302 .

In an optional embodiment, the sending module 1301 is used to send the room information of the virtual environment, the driving bit entry graphic code and the n passenger bit entry graphic codes to the administrator terminal;

In an optional embodiment, the processing module 1302 is used to receive the first joining request triggered by the first terminal scanning the driving bit entry graphic code; in response to the first joining request, the processing module 1302 is also used to send the first The virtual character corresponding to the first account logged in by the terminal is added to the driver's seat of the virtual vehicle in the virtual environment;

In an optional embodiment, the processing module 1302 is also used to receive the second joining request triggered by the second terminal scanning the passenger bit entry graphic code; in an optional embodiment, the processing module 1302 is also used to In response to the second joining request, the virtual character corresponding to the second account logged in by the second terminal is added to the passenger seat of the virtual vehicle in the virtual environment.

In summary, by constructing a multi-body sensory device combining virtual and reality, the device controls the display device to display the virtual environment picture and aiming point, the device controls the virtual vehicle to change the driving direction and speed in the virtual environment, the device Realize the shooting of the virtual object aimed at the i-th aiming point in the virtual environment among the n aiming points, which enriches the man-machine dialogue mode of players in the same somatosensory game, and realizes that different players in the same somatosensory game can pass through Different modes of operation to play the game.

The above-mentioned multi-body sensory device combining virtual and display improves the degree of freedom of the game, further enhances the player's human-computer interaction experience, and satisfies the growing entertainment needs of the player.

FIG. 14 is a schematic structural diagram of a server provided by an embodiment of the present application. Specifically: the server 1400 includes a central processing unit (Central Processing Unit, CPU) 1401, a system including a random access memory (Random Access Memory, RAM) 1402 and a read-only memory (Read-Only Memory, ROM) 1403 memory 1404, and a system bus 1405 connecting the system memory 1404 and the central processing unit 1401. The computer equipment 1400 also includes a basic input/output system (I/O system, Input/Output system) 1406 for explaining the transmission of information between various devices in the computer, and for storing the operating system 1413, application programs 1414 and other program modules 1415 large storage area for equipment 1407 .

The basic input/output system 1406 includes a display 1408 for displaying information and an input device 1409 such as a mouse and a keyboard for a user to input information. Wherein both the display 1408 and the input device 1409 are connected to the central processing unit 1401 through the input/output controller 1410 connected to the system bus 1405 . The basic input/output system 1406 may also include an input/output controller 1410 for receiving and processing input from a number of other devices such as a keyboard, mouse, or electronic stylus. Similarly, input/output controller 1410 also provides output to a display screen, printer, or other type of output device.

The bulk storage device 1407 is connected to the central processing unit 1401 through a bulk storage controller (not shown) connected to the system bus 1405 . Large storage device 1407 and its associated computer-readable storage media provide non-volatile storage for server 1400 . That is to say, the large storage area device 1407 may include a computer-readable storage medium (not shown) such as a hard disk or a Compact Disc Read-Only Memory (CD-ROM) drive.

Without loss of generality, computer-readable storage media may include computer storage media and communication storage media. Computer storage media includes volatile and nonvolatile, removable and non-removable storage media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media include RAM, ROM, Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM) ), flash memory or other solid-state storage technologies, CD-ROM, Digital Versatile Disc (DVD) or other optical storage, cassettes, tapes, disk storage or other magnetic storage devices. Certainly, those skilled in the art know that the computer storage media is not limited to the above-mentioned ones. The above-mentioned system memory 1404 and large storage area device 1407 may be collectively referred to as memory.

According to various embodiments of the present application, the server 1400 can also run on a remote computer connected to the network through a network such as the Internet. That is to say, the server 1400 can be connected to the network 1412 through the network interface unit 1411 connected to the system bus 1405, or in other words, the network interface unit 1411 can also be used to connect to other types of networks or remote computer systems (not shown).

The present application also provides a computer-readable storage medium, wherein at least one instruction, at least one program, code set or instruction set is stored in the storage medium, and the at least one instruction, the at least one program, the code set or the instruction The set is loaded and executed by the processor to realize the multi-body sensory method combining virtual and reality provided by the above method embodiments.

The present application provides a computer program product or computer program. The computer program product or computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the multi-body sensory method combining virtual and reality provided by the above method embodiments.

The serial numbers of the above embodiments of the present application are for description only, and do not represent the advantages and disadvantages of the embodiments.

Those of ordinary skill in the art can understand that all or part of the steps for realizing the above-mentioned embodiments can be completed by hardware, and can also be completed by instructing related hardware through a program. The program can be stored in a computer-readable storage medium, The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, and the like.

The above are only exemplary implementations of the present application. It should be pointed out that for those skilled in the art, some improvements and modifications can be made without departing from the principle of the application, and these improvements and modifications should also be considered as For the scope of protection of this application.

101: Simulation vehicle 102: display device 103: Simulation firearms 200: Multi-body sensory system combining virtual and reality 201:Computer equipment 301: steering wheel 302: accelerator pedal 303: brake pedal 401: left side of the vehicle 402:Top surface of the vehicle 403: The front of the vehicle 501: Simulation firearms 620, 640, 660, 680: steps 611, 612, 613: steps 901: Shrink the window 902: Detailed window 903: bit entry graphics code 1101: account information area 1102: room information area 1210, 1220, 1221, 1222, 1223, 1224, 1225: steps 1230, 1231, 1232, 1240, 1241, 1242, 1243, 1244, 1250: steps 1301: send module 1302: processing module 1303: display module 1304: control module 1305: Shooting module 1400: server 1401: central processing unit 1402: random access memory 1403: ROM 1404: System Memory 1405: System bus 1406:I/O system 1407:Large Storage Area Equipment 1408:Display 1409: input device 1410: Input/Output Controller 1411: Network interface unit 1412: network 1413: operating system 1414: application 1415: Other program modules

Fig. 1 is a schematic diagram of a multi-person sensory system combining virtual and reality provided by an exemplary embodiment of the present application.

Fig. 2 is a schematic diagram of a multi-person sensory system combining virtual reality and reality provided by an exemplary embodiment of the present application.

Fig. 3 is a schematic diagram of a first peripheral component and a second peripheral component provided by an exemplary embodiment of the present application.

Fig. 4 is a schematic diagram of a virtual vehicle provided by an exemplary embodiment of the present application.

Fig. 5 is a schematic diagram of a simulated firearm provided by an exemplary embodiment of the present application.

Fig. 6 is a flow chart of a multi-person sensory method combining virtual and reality provided by an exemplary embodiment of the present application.

Fig. 7 is a schematic diagram of an aiming point provided by an exemplary embodiment of the present application.

Fig. 8 is a flow chart of a multi-person sensory method combining virtual and reality provided by another exemplary embodiment of the present application.

Fig. 9 is a schematic diagram of an administrator interface provided by an exemplary embodiment of the present application.

Fig. 10 is a schematic diagram of an administrator interface provided by another exemplary embodiment of the present application.

Fig. 11 is a schematic diagram of an administrator interface provided by another exemplary embodiment of the present application.

Fig. 12 is a flowchart of a player joining a room provided by an exemplary embodiment of the present application.

Fig. 13 is a structural block diagram of a multi-body sensory device combining virtual and reality provided by an exemplary embodiment of the present application.

Fig. 14 shows a structural block diagram of a computer device provided by an exemplary embodiment of the present application.

101: Simulation vehicle

102: display device

103: Simulation firearms

200: Multi-body sensory system combining virtual and reality

201:Computer equipment

Claims (20)

A multi-person sensory system combining virtuality and reality, wherein the system includes: computer equipment, simulation vehicles respectively connected to the computer equipment, n simulation firearms and display devices, where n is a positive integer; The tool includes a driver's seat and n passenger seats, the driver's seat is provided with a first peripheral component and a second peripheral component, the first peripheral component is used to control the driving direction, and the second peripheral component is used for Control the driving speed; the display device is located outside the simulation vehicle, and is used to display the virtual environment picture and n aiming points provided by the computer equipment, and the virtual environment picture is the virtual environment picture corresponding to the simulation vehicle. A picture of the virtual environment where the vehicle is located; the computer device is configured to control the virtual vehicle to change the driving direction in the virtual environment in response to the first driving operation on the first peripheral component; respond For the second driving operation on the second peripheral component, control the virtual vehicle to change the driving speed in the virtual environment; respond to the shooting operation on the i-th simulated firearm among the n simulated firearms , shooting the virtual object aimed at by the i-th aiming point in the virtual environment among the n aiming points, where i is a positive integer not greater than n. The system according to claim 1, wherein the simulated vehicle includes a simulated vehicle, the first peripheral component includes a steering wheel, and the second peripheral component includes an accelerator and a brake; or, the simulated vehicle includes For simulating an aircraft, the first peripheral component includes at least one of a flight stick and a rudder, and the second peripheral component includes a throttle stick. The system according to claim 1, wherein at least one of the driver's seat and the n passenger seats is provided with a vibration feedback component. The system according to claim 1, wherein at least one of the n simulated firearms has established a wireless communication connection with the computer device; or, at least one of the n simulated firearms is connected to all the simulated firearms The computer device establishes a wired communication connection. The system according to claim 1, wherein the display device includes n+1 virtual reality VR helmets; or, the display device includes a half-surround screen or a surround screen arranged in front of the simulation vehicle screen; or, the display device includes a projector. The system according to claim 5, wherein the display device includes a half-surround screen arranged in front of the simulation vehicle; the width of the half-surround screen is not smaller than the width of the simulation vehicle, and the The height of the semi-surrounding screen is not less than the height of the simulated vehicle. A multi-body sensory method combining virtuality and reality, wherein the method is executed by the computer equipment in any of the systems of claims 1 to 6, and the method includes: displaying the virtual environment picture and The n aiming points, the virtual environment picture is the picture of the virtual environment where the virtual vehicle corresponding to the simulated vehicle is located; in response to the first driving operation on the first peripheral component, control the The virtual vehicle changes the driving direction in the virtual environment; in response to the second driving operation on the second peripheral component, the virtual vehicle is controlled to change the driving speed in the virtual environment; in response to the n The shooting operation on the i-th simulated firearm among the simulated firearms is to shoot the virtual object aimed at by the i-th aiming point among the n aiming points in the virtual environment. The method according to claim 7, wherein the simulated vehicle comprises a simulated vehicle; The controlling the virtual vehicle to change the driving direction in the virtual environment in response to the first driving operation on the first peripheral component includes: responding to the first driving operation on the steering wheel on the simulated vehicle The operation is to control the virtual vehicle to change the driving direction in the virtual environment. The method according to claim 7, wherein the simulated vehicle includes a simulated aircraft; and in response to the first driving operation on the first peripheral component, controlling the virtual vehicle in the virtual environment Changing the driving direction includes: controlling the virtual aircraft to change the driving direction in the virtual environment in response to the first driving operation of the flight stick and/or rudder on the simulated aircraft. The method according to claim 7, wherein the simulated vehicle includes a simulated vehicle; and in response to a second driving operation on the second peripheral component, controlling the virtual vehicle in the virtual environment Changing the driving speed includes: controlling the virtual vehicle to change the driving speed in the virtual environment in response to a second driving operation on the accelerator and/or brake on the simulated vehicle. The method according to claim 7, wherein the simulated vehicle includes a simulated aircraft; and in response to the second driving operation on the second peripheral component, controlling the virtual vehicle in the virtual environment Changing the driving speed includes: controlling the virtual aircraft to change the driving speed in the virtual environment in response to the second driving operation on the throttle rocker on the simulated aircraft. The method according to any one of claim items 7 to 11, wherein the display device includes n+1 VR helmets; displaying a virtual environment picture and n aiming points through the display device includes: through the The i+1 th VR helmet among the n+1 VR helmets displays the virtual environment image and the i th aiming point among the n aiming points. The method according to any one of claims 7 to 11, wherein the display device includes a surround screen or a projector; displaying a virtual environment picture and n aiming points through the display device includes: A screen or a projector displays the virtual environment picture and the n aiming points, and the n aiming points have different graphic styles, and the graphic styles include at least one of shape, color, transparency and number. The method according to any one of claims 7 to 11, wherein the method further includes: sending the room information of the virtual environment, the driving bit entry graphic code and n passenger bit entry graphic codes to the administrator terminal ; Receive the first join request triggered by the first terminal scanning the driving bit entry graphic code; respond to the first join request, add the virtual character corresponding to the first account logged in by the first terminal to the virtual character on the driver’s seat of the virtual vehicle in the environment; receiving a second joining request triggered by the second terminal scanning the passenger bit entry graphic code; in response to the second joining request, connecting the second terminal to The virtual character corresponding to the logged-in second account is added to the passenger seat of the virtual vehicle in the virtual environment. A multi-body sensory device combining virtual and reality, which includes: The display module is used to display the virtual environment picture and n aiming points through the display device, the virtual environment picture is the picture of the virtual environment where the virtual vehicle corresponding to the simulation vehicle is located; the control module is used to respond to the first The first driving operation on a peripheral component is used to control the virtual vehicle to change the driving direction in the virtual environment; the control module is also used to control the virtual vehicle in response to the second driving operation on the second peripheral component. The vehicle changes the driving speed in the virtual environment; the shooting module is also used to respond to the shooting operation on the i-th simulated firearm among the n simulated firearms, and adjust the i-th aiming point among the n aiming points A virtual object aimed at in the virtual environment is shot. The device according to claim 15, wherein the simulated vehicle includes a simulated vehicle; the control module is further configured to control the virtual vehicle to operate on the simulated vehicle in response to the first driving operation on the steering wheel on the simulated vehicle. change the driving direction in the virtual environment described above. The device according to claim 15, wherein the simulated vehicle includes a simulated aircraft; the control module is further configured to respond to the first driving operation of the flight stick and/or rudder on the simulated aircraft, The virtual aircraft is controlled to change the driving direction in the virtual environment. The device according to claim 15, wherein the simulated vehicle includes a simulated vehicle; the control module is further configured to respond to a second driving operation on the accelerator and/or brake on the simulated vehicle, The virtual vehicle changes travel speed in the virtual environment. A computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program is loaded and executed by a processor to realize the combination of virtual and reality as described in any one of claims 7 to 14 A multi-person sensory approach. A computer program product, including computer programs/instructions, wherein, when the computer programs/instructions are executed by a processor, the multi-body sensory method for combining virtual and reality described in any one of claims 7 to 14 is realized.

Images