WO2024060799A1

WO2024060799A1 - Multi-device collaboration method, and client

Info

Publication number: WO2024060799A1
Application number: PCT/CN2023/106607
Authority: WO
Inventors: 郑亚; 冯艳妮; 董亚龙; 胡潇; 魏记
Original assignee: 华为云计算技术有限公司
Priority date: 2022-09-23
Filing date: 2023-07-10
Publication date: 2024-03-28
Also published as: CN115581913A

Abstract

The present application relates to the technical field of terminals. Provided are a multi-device collaboration method, and a client. The method comprises: a first AR client presenting a first AR scene, wherein the first AR scene is a first collaborative scene in an AR mode; a first VR client presenting a first VR scene, wherein the first VR scene is a first collaborative scene in a VR mode; when a first real location of the first AR scene comprises a first object, the first VR client presenting the first object at a first virtual location of the first VR scene, wherein the first real location and the first virtual location are the same location in the first collaborative scene; and/or when a second virtual location of the first VR scene comprises a second object, the first AR client presenting the second object at a second real location of the first AR scene, wherein the second virtual location and the second real location are the same location in the first collaborative scene. The technical solution provided in the present application can realize collaboration between an AR client and a VR client.

Description

Multi-device collaboration method and client

This application claims priority to the Chinese patent application submitted to the State Intellectual Property Office on September 23, 2022, with application number 202211168295.4 and the application name "Multi-device collaboration method and client", the entire content of which is incorporated into this application by reference. middle.

Technical field

This application relates to the field of Internet technology, and in particular to a multi-device collaboration method and client.

Background technique

With the continuous development of clients and Internet technology, multiple clients can collaborate with each other to meet the needs of multi-person interaction such as games.

In the existing technology, multiple clients can all be (virtual reality, VR) clients. Each VR client can access the virtual collaboration scene and present the collaboration scene in VR mode to the user. When a virtual location in the collaborative scene includes a virtual object, each VR client can present the object at the virtual location in the respective VR scene, so that users of each VR client can see the object. However, the collaboration method between multiple VR clients is relatively simple and the user experience is very poor.

Contents of the invention

In view of this, this application provides a multi-device collaboration method and client, which can realize collaboration between augmented reality (AR) clients and VR clients, breaking the barriers between AR scenes and VR scenes.

In order to achieve the above object, in a first aspect, an embodiment of the present application provides a multi-device collaboration method, which is characterized in that the method includes:

The first AR client presents a first AR scene, and the first AR scene is a first collaborative scene in AR mode;

The first VR client presents a first VR scene, and the first VR scene is the first collaborative scene in VR mode;

When the first real position of the first AR scene includes a first object, the first VR client presents the first object at a first virtual position of the first VR scene, the first real position and the first virtual position is the same position in the first collaborative scene; and/or, when the second virtual position of the first VR scene includes a second object, the first AR client is in that The second object is presented in a second real position of the first AR scene, and the second virtual position and the second real position are the same position in the first collaborative scene.

The first VR scene may be a virtual scene obtained by processing the first AR scene.

In this embodiment of the present application, the first AR client and the first VR client can access the same first collaboration scene, the first AR client presents the first AR scene, and the first VR client presents the first VR scene, that is, The first AR client and the first VR client can collaboratively present the first collaborative scene, where the first VR scene can be a virtual scene obtained by processing the first AR scene. When the first real position of the first AR scene includes the first object, the first VR client presents the first object at the first virtual position of the first VR scene, that is, the first VR client can display the first AR The first object in the scene is synchronously presented in the first VR scene, so that the user can see the first object from the first AR scene from the first VR client. When the second virtual position of the first VR scene includes the second object, the first AR client presents the second object at the second real position of the first AR scene, that is, the first AR client can place the first VR The second object in the scene is synchronously presented in the first AR scene, so that the user can see the second object from the first VR scene from the first AR client. Therefore, the first AR client and the first VR client can present the object in the first collaborative scene from the opposite end, so that the user of the first AR client and the user of the first VR client can see the object, realizing the AR client Collaboration and interoperability with VR clients.

In some implementations, the first object in the first AR scene may be a real object, a combined virtual and real object, or a virtual object, and the first object presented in the first VR scene may be a virtual object. In some implementations, the first object in the first AR scene may be a user of the first AR client, and the first object presented in the first VR scene may be an avatar corresponding to the user. Among them, if the first object is an object that moves synchronously with the first AR client (for example, the first AR client is tied to the first object, or the first object carries the first AR client), then the location information of the first object with the first AR client The position information may be the same, and the rotation information of the first object and the rotation information of the first AR client may be the same. If the first object is not an object that moves synchronously with the first AR client, the first AR client can generate or collect at least one of the position information and rotation information of the first object.

In some implementations, the second object in the first VR scene may be a virtual object, and the second object in the first AR scene may also be a virtual object. Wherein, the first VR client can receive user input, generate or collect at least one of position information and rotation information of the second object.

In some implementations, the local coordinate system of the first AR client may be the third coordinate system, the local coordinate system of the first VR client may be the fourth coordinate system, and the first AR client and the first VR client are both located together. The common coordinate system may include a second coordinate system. The first AR client and the first VR client can realize the conversion of the position information of the first object and the second object in the third coordinate system and the fourth coordinate system through the second coordinate system. Taking the first object as an example, the first AR client can present the first object at the first real position in the first AR scene based on the position information of the first object in the third coordinate system, and the first VR client can present the first object based on the position information of the first object in the third coordinate system. The position information of the first object in the fourth coordinate system presents the first object in the first VR scene.

In some embodiments, the first AR client and the first VR client can realize the conversion of the rotation information of the first object and the second object in the third coordinate system and the fourth coordinate system through the second coordinate system. . Taking the first object as an example, the first AR client can present the first object at the first real position in the first AR scene based on the rotation information and position information of the first object in the third coordinate system. The first VR client The client can present the first object in the first VR scene based on the rotation information and position information of the first object in the fourth coordinate system, so that the angle of the first object in the first AR scene is consistent with the rotation of the first AR client. The angles match, and the angle of the first object in the first VR scene matches the rotation angle of the first VR client, thereby improving the authenticity and user experience of presenting the first object.

In some embodiments, the method further includes any one or more of the following:

When a first event corresponding to the first object occurs in the first AR scene, the first VR client presents the first event in the first VR scene; or,

When a second event corresponding to the second object occurs in the first VR scene, the first AR client presents the second event in the first AR scene.

In some embodiments, the first event includes any one or more of the following:

a status update event of the first object, or,

The motion event of the first object; or,

Interaction events between the first object and other objects.

In some implementations, the motion event of the first object includes the movement of the first object from the first real position to a third real position in the first AR scene, and the first VR client The first VR scene presents the first event, including:

The first VR client presents the movement process of the first object moving from the first virtual position in the first VR scene to a third virtual position, and the third virtual position and the third real position are the same position in the first collaborative scene.

In some embodiments, the second event includes any one or more of the following:

The status update event of the second object; or,

a motion event of the second object; or,

Interaction events between the second object and other objects.

By synchronously presenting at least one of the first event and the second event, the first VR client and the first AR client improve the authenticity of the game scene collaboratively presented by the first VR client and the first AR client, thereby improving the user experience.

In some implementations, the motion event of the second object includes the movement of the second object from the second virtual position to a fourth virtual position in the first VR scene, and the first AR client The first AR scene presents the second event, including:

The first AR client presents the movement process of the second object moving from the second real position in the first AR scene to a fourth real position, the fourth real position and the fourth virtual The location is the same location in the first collaborative scene.

In some implementations, the first object is a real object or a virtual combined object, and the first VR client is Presenting the first object in the first virtual position of the first VR scene includes:

The first VR client presents the virtual image corresponding to the first object in the first virtual position of the first VR scene.

In some implementations, the second object is a virtual object, and the first AR client presents the second object at a second real location of the first AR scene, including:

The first AR client presents the second object or a combined virtual and real image corresponding to the second object in the second real position of the first AR scene.

In some implementations, the first object is a user of the first AR client, and/or the second object is a player-controlled character controlled by the first VR client. That is to say, the user of the first AR client can interact with the player-controlled character controlled by the first VR client in the first AR scene, and the player-controlled character controlled by the first VR client can interact with the player-controlled character controlled by the first VR client in the first VR scene. Users of an AR client can see and interact with each other, enabling users of VR clients and users of AR clients to see and interact with each other in the same collaborative scene, breaking the barriers between AR scenes and VR scenes.

In some implementations, the first AR scene includes at least part of the real scene, and the first VR scene includes a virtual scene obtained by processing the at least part of the real scene.

In some implementations, the first VR scene may be a virtual scene obtained by processing the real scene in the first AR scene through a view synthesis model such as a neural radiance fields (NeRF) network. Among them, the view synthesis model can re-model and render the actual scene, thereby outputting a virtual scene corresponding to the real scene.

In some implementations, before the first AR client presents the first AR scene, and the first AR scene is the first collaborative scene of the AR mode, the method further includes:

The first AR client accesses the first collaborative scene based on the real position of the first AR client in the first coordinate system, and the first VR client accesses the first collaborative scene based on the location of the first VR client. The real position in the first coordinate system is connected to the first collaborative scene, where the real position of the first AR client in the first coordinate system is the same as the real position of the first VR client in the first coordinate system. The real position in the coordinate system is in the same AOI area; or,

The first AR client accesses the first collaborative scene based on the real position of the first AR client in the first coordinate system, and the first VR client accesses the first collaborative scene based on the location of the first VR client. The virtual position in the first coordinate system is connected to the first collaborative scene, where the real position of the first AR client in the first coordinate system is the same as the real position of the first VR client in the first coordinate system. The virtual positions in the coordinate system are in the same AOI area.

The first coordinate system may be a common coordinate system where the first AR client and the first VR client are located. The first AR client can access the first collaborative scene based on the real position of the first AR client in the first coordinate system, and the first VR client can access the first collaborative scene based on the real position of the first VR client in the first coordinate system or The virtual location is connected to the first collaboration scene, which enables the first AR client and the first VR client to access the same first collaboration scene whether they are at the same location in the real world, realizing cross-region and cross-device collaboration. Improved the flexibility and user experience of multi-device collaboration.

In some embodiments, the method further comprises:

The first AR client presents a third object in the first AR scene, the first VR client presents a virtual image corresponding to the third object in the first VR scene, and the third The object is an object included in the second AR scene presented by the second AR client, and the third object is a real object or a combination of virtual and real objects.

In some embodiments, the method further includes:

The first AR client presents a fourth object in the first AR scene, the first VR client presents the fourth object in the first VR scene, and the fourth object is a second Objects included in the second VR scene presented by the VR client, and the fourth object is a virtual object.

In some implementations, the first AR client presents the second object in a second real-world location of the first AR scene, including:

The first AR client obtains the position of the second object in a second coordinate system, and the second coordinate system is a public coordinate system;

The first AR client converts the position of the second object in the second coordinate system into the second real position of the second object in the third coordinate system. The third coordinate is the local coordinate system corresponding to the first AR client.

In some implementations, the first VR client presents the first virtual location in the first VR scene. Objects, including:

The first VR client obtains the position of the first object in a second coordinate system, and the second coordinate system is a public coordinate system;

The first VR client converts the position of the first object in the second coordinate system into the first virtual position of the first object in the fourth coordinate system. The fourth coordinate is the local coordinate system corresponding to the first VR client.

In some embodiments, the method further includes:

The first AR client converts the first real position of the first object in the third coordinate system into the position of the first object in the second coordinate system;

The first AR client sends the position of the first object in the second coordinate system to the service server;

The first VR client obtains the position of the first object in the second coordinate system, including:

The first AR client obtains the position of the first object in the second coordinate system from the service server.

In some embodiments, the method further includes:

The first VR client converts the second virtual position of the second object in the fourth coordinate system into the position of the second object in the second coordinate system;

The first VR client sends the position of the second object in the second coordinate system to the service server;

The first AR client obtains the position of the second object in the second coordinate system, including:

The first VR client obtains the position of the second object in the second coordinate system from the business server.

In a second aspect, embodiments of the present application provide a multi-device collaboration method, applied to the first AR device, and the method includes:

Presenting a first AR scene, the first AR scene being a first collaborative scene in AR mode;

When the second virtual position of the first VR scene includes a second object, the second object is presented at a second real position of the first AR scene, and the second virtual position and the second real position are the The same position in the first collaborative scene;

Wherein, the first VR scene is a scene presented by the first VR client, and the first VR scene is the first collaborative scene in VR mode.

In some embodiments, the method further includes:

The status update event of the second object; or,

The motion event of the second object; or,

Interaction events between the second object and other objects.

In a third aspect, embodiments of the present application provide a multi-device collaboration method, applied to the first VR device. The method includes:

Presenting a first VR scene, the first VR scene being a first collaborative scene in VR mode;

When the first real position of the first AR scene includes the first object, the first object is presented at the first virtual position of the first VR scene, and the first real position and the first virtual position are the The same position in the first collaborative scene;

Wherein, the first AR scene is a scene presented by a first AR client, and the first AR scene is the first collaborative scene in AR mode.

In some embodiments, the method further includes:

When a first event corresponding to the first object occurs in the first AR scene, the first event is presented in the first VR scene.

In some embodiments, the first event includes any one or more of the following:

a status update event of the first object, or,

The motion event of the first object; or,

Interaction events between the first object and other objects.

In a fourth aspect, embodiments of the present application provide a system including a first AR device and a first VR device;

When the first real position of the first AR scene includes a first object, the first VR client The first virtual position of the scene presents the first object, and the first real position and the first virtual position are the same position in the first collaborative scene; and/or, when the first VR scene When the second virtual position includes a second object, the first AR client presents the second object at a second real position of the first AR scene, and the second virtual position and the second real position are The same location in the first collaborative scene.

In the fifth aspect, embodiments of the present application provide a device that has the function of realizing the client behavior in any possible implementation of the first aspect, or the function of realizing the client behavior in any possible implementation of the second aspect. . Functions can be implemented by hardware, or by hardware executing corresponding software. Hardware or software includes one or more modules or units corresponding to the above functions. For example, a transceiver module or unit, a processing module or unit, an acquisition module or unit, etc.

In a sixth aspect, embodiments of the present application provide a client, including: a memory and a processor. The memory is used to store a computer program; the processor is used to execute any of the methods described in the first aspect when calling the computer program or The method described in any one of the second aspects.

In a seventh aspect, embodiments of the present application provide a chip system. The chip system includes a processor, the processor is coupled to a memory, and the processor executes a computer program stored in the memory to implement any of the above-mentioned aspects of the first aspect. The method described in one aspect or the method described in any one of the second aspects.

The chip system may be a single chip or a chip module composed of multiple chips.

In an eighth aspect, embodiments of the present application provide a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the method described in any one of the first aspects or any one of the second aspects is implemented. method described in the item.

In a ninth aspect, embodiments of the present application provide a computer program product. When the computer program product is run on a client, it causes the client to execute the method described in any one of the first aspects or any one of the second aspects. method described.

It can be understood that the beneficial effects of the above-mentioned second aspect to the ninth aspect can be referred to the relevant description in the above-mentioned first aspect, and will not be described again here.

Description of the drawings

Figure 1 is a schematic structural diagram of a client provided by an embodiment of the present application;

Figure 2 is a schematic structural diagram of a server provided by an embodiment of the present application;

Figure 3 is a schematic diagram of a game scene provided by an embodiment of the present application;

Figure 4 is a schematic diagram of another game scene provided by an embodiment of the present application;

Figure 5 is a schematic structural diagram of a multi-device collaboration system provided by an embodiment of the present application;

Figure 6 is a schematic structural diagram of another multi-device collaboration system provided by an embodiment of the present application;

Figure 7 is a schematic structural diagram of another multi-device collaboration system provided by an embodiment of the present application;

Figure 8 is a flow chart of a multi-device collaboration method provided by an embodiment of the present application;

Figure 9 is a schematic diagram of another game scene provided by an embodiment of the present application;

Figure 10 is a flow chart of a method of accessing a game scene provided by an embodiment of the present application;

Figure 11 is a schematic diagram of a login interface provided by an embodiment of the present application;

Figure 12 is a schematic diagram of a camp selection interface provided by an embodiment of the present application;

Figure 13 is a schematic diagram of a game lobby interface provided by an embodiment of the present application;

Figure 14 is a flow chart of a method for presenting an AR scene provided by an embodiment of the present application;

FIG15 is a flowchart of a method for presenting a VR scene provided in an embodiment of the present application;

FIG16 is a flow chart of a method for presenting an AR scene provided in an embodiment of the present application;

Figure 17 is a flow chart of a method for presenting a VR scene provided by an embodiment of the present application;

Figure 18 is a schematic diagram of an AR scene provided by an embodiment of the present application;

Figure 19 is a schematic diagram of a VR scene provided by an embodiment of the present application;

Figure 20 is a flow chart of a method for presenting game objects provided by an embodiment of the present application;

Figure 21 is a flow chart of another method of presenting game objects provided by an embodiment of the present application;

Figure 22 is a flow chart of another method of presenting game objects provided by an embodiment of the present application;

Figure 23 is a schematic diagram of another AR scene provided by an embodiment of the present application;

Figure 24 is a schematic diagram of another VR scene provided by an embodiment of the present application;

Figure 25 is a flow chart of an object interaction method provided by an embodiment of the present application;

Figure 26 is a schematic diagram of another AR scene provided by an embodiment of the present application;

Figure 27 is a schematic diagram of another VR scene provided by an embodiment of the present application.

Detailed ways

The multi-device collaboration method provided in the embodiment of the present application can be applied to the client and the server.

In some embodiments, clients may include AR clients and VR clients. The VR client can use computer technology to simulate and generate a three-dimensional virtual world that provides users with visual, auditory, tactile and other sensory simulations. Users can naturally interact with the virtual world with the help of special input/output devices. The AR client can overlay the virtual world with the real world, making it difficult for users to distinguish between real and fake. Users can interact with the virtual world and/or the real world.

In some embodiments, AR clients and VR clients can be implemented as mobile phones, tablets, wearable devices (such as smart glasses, smart helmets, etc.), vehicle-mounted devices, laptops, ultra-mobile personal computers (ultra-mobile personal computers), etc. computer (UMPC), netbook, personal digital assistant (personal digital assistant, PDA) and other devices. The embodiments of this application do not place any restrictions on the specific types of AR clients and VR clients.

Please refer to FIG. 1 , which is a schematic structural diagram of a client 100 provided by this application. The client 100 may include a processor A110, an external memory interface 120, an internal memory 121, an antenna 1, a communication module A160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone interface 170D, a button 190, a motor 191, and a camera 193 , display screen 194, and subscriber identification module (subscriber identification module, SIM) card interface 195, etc.

It can be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the client 100 . In other embodiments of the present application, the client 100 may include more or fewer components than shown in the figures, or combine some components, or split some components, or arrange different components. The components illustrated may be implemented in hardware, software, or a combination of software and hardware.

The processor A 110 may include one or more processing units. For example, the processor A 110 may include an application processor (application processor, AP), a modem processor, a graphics processing unit (GPU), an image signal Processor (image signal processor, ISP), controller, memory, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural-network processing unit, NPU) etc. Among them, different processing units can be independent devices or integrated in one or more processors.

Among them, the controller may be the nerve center and command center of the client 100. The controller can generate operation control signals based on the instruction operation code and timing signals to complete the control of fetching and executing instructions.

A memory may also be provided in the processor A 110 for storing instructions and data. In some embodiments, the memory in processor A 110 is cache memory. This memory can hold instructions or data that have just been used or recycled by processor A 110. If processor A 110 needs to use the instruction or data again, it can be called directly from the memory. Repeated accesses are reduced and the waiting time of processor A 110 is reduced, thus improving the efficiency of the system.

The communication module A 160 can be used for communication between various internal modules of the client 100, or communication between the client 100 and other external electronic devices. Exemplarily, if the client 100 communicates with other electronic devices via a wireless connection, the communication module A 160 can provide solutions for wireless communications including 2G/3G/4G/5G applied on the client 100, and/or, the communication module A 160 can provide solutions for wireless communications including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared technology (IR), etc. applied on the client 100.

In some embodiments, the antenna 1 of the client 100 is coupled with the communication module A 160 so that the client 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), broadband Code division multiple access (wideband code division multiple access, WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (long term evolution, LTE), BT, GNSS, WLAN, NFC , FM, and/or IR technology, etc. The GNSS may include global positioning system (GPS), global navigation satellite system (GLONASS), Beidou navigation satellite system (BDS), quasi-zenith satellite system (quasi -zenith satellite system (QZSS) and/or satellite based enhancement system (satellite based augmentation systems (SBAS).

The client 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor A 110 may include one or more GPUs that execute program instructions to generate or change display information.

The display screen 194 is used to display images, videos, etc. Display 194 includes a display panel. The display panel can use a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active matrix organic light emitting diode or an active matrix organic light emitting diode (active-matrix organic light emitting diode). emitting diode (AMOLED), flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, quantum dot light emitting diode (QLED), etc. In some embodiments, the client 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

The client 100 can implement the shooting function through the ISP, camera 193, video codec, GPU, display screen 194 and application processor.

The ISP is used to process the data fed back by the camera 193. For example, when taking a photo, the shutter is opened, the light is transmitted to the camera sensor through the lens, the optical signal is converted into an electrical signal, and the camera sensor passes the electrical signal to the ISP for processing, and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on image noise, brightness, and skin color. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, the ISP may be provided in the camera 193.

Camera 193 is used to capture still images or video. The object passes through the lens to produce an optical image that is projected onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to convert it into a digital image signal. ISP outputs digital image signals to DSP for processing. DSP converts digital image signals into standard RGB, YUV and other format image signals. In some embodiments, the client 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

In some embodiments, the client 100 can superimpose the virtual image with the image captured by the camera 193, and display the superimposed image on the display screen 194, thereby realizing a display effect that combines virtual and real such as augmented reality. Or, in other embodiments, the display screen 194 is a transparent display screen, the client 100 can project a virtual image on the display screen 194, and when the user's line of sight passes through the display screen 194, the projected virtual image can be Superimpose it with real images to achieve a display effect that combines virtual and real images such as augmented reality.

The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the client 100. The external memory card communicates with the processor A 110 through the external memory interface 120 to implement the data storage function. Such as saving music, videos, etc. files in external memory card.

Internal memory 121 may be used to store computer executable program code, which includes instructions. The processor A 110 executes instructions stored in the internal memory 121 to execute various functional applications and data processing of the client 100 . The internal memory 121 may include a program storage area and a data storage area. Among them, the stored program area can store an operating system, at least one application program required for a function (such as a sound playback function, an image playback function, etc.). The storage data area can store data created during use of the client 100 (such as audio data, phone book, etc.). In addition, the internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

The client 100 can implement audio functions through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone interface 170D, and the application processor. Such as music playback, recording, etc.

The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. Audio module 170 may also be used to encode and decode audio signals. In some embodiments, the audio module 170 may be provided in the processor A 110, or some functional modules of the audio module 170 may be provided in the processor A 110.

Speaker 170A, also called "speaker", is used to convert audio electrical signals into sound signals. Client 100 can listen to music through speaker 170A, or listen to hands-free calls.

Receiver 170B, also called "earpiece", is used to convert audio electrical signals into sound signals. When the client 100 answers a call or a voice message, the voice can be heard by bringing the receiver 170B close to the human ear.

Microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending When sending voice information, the user can speak with the mouth close to the microphone 170C and input the sound signal to the microphone 170C. The client 100 may be provided with at least one microphone 170C. In other embodiments, the client 100 may be provided with two microphones 170C, which in addition to collecting sound signals, may also implement a noise reduction function. In other embodiments, the client 100 can also be equipped with three, four or more microphones 170C to collect sound signals, reduce noise, identify sound sources, and implement directional recording functions, etc.

The headphone interface 170D is used to connect wired headphones. The headphone interface 170D can be a USB interface, or a 3.5mm open mobile terminal platform (OMTP) standard interface, or a Cellular Telecommunications Industry Association of the USA (CTIA) standard interface.

Please refer to FIG. 2 , which is a schematic structural diagram of a server 200 provided by this application. The server 200 may include a memory 210, a processor B 220, a communication module B 230, and the like.

Processor B 220 may include one or more processing units, and memory 210 may be used to store program code and data. In the embodiment of the present application, the processor B 220 can execute computer execution instructions stored in the memory 210 for controlling and managing the actions of the server 200.

The communication module B 230 can be used to implement communication between the server 200 and other devices such as the client 100.

It should be understood that, except for the various components or modules listed in Figure 2, the embodiment of the present application does not specifically limit the structure of the server 200. In other embodiments of the present application, the server 200 may also include more or less components than shown in the figures, or combine some components, or split some components, or arrange different components. The components illustrated may be implemented in hardware, software, or a combination of software and hardware.

In order to facilitate understanding of the technical solutions in the embodiments of the present application, the application scenarios of the embodiments of the present application are first introduced below.

Client-server is an important service architecture pattern. In this architecture model, the client can connect to the server through the network, the client can request network services from the server, and the server can respond to the request to provide the client with the requested data. Multiple clients can communicate through the server to present collaborative scenarios to users. Among them, the collaborative scene can be a scene presented by multiple clients at the same time. In some implementations, multiple clients can simultaneously present at least part of the information in the collaborative scene, so that users of multiple clients can see at least part of the information at the same time. In some implementations, when any client among the multiple clients performs any operation on the scene, resulting in any change in the scene, other clients among the multiple clients will also synchronize. Present this change. In some embodiments, collaborative scenarios may include game scenarios, entertainment scenarios, or conference scenarios. Of course, in practical applications, collaborative scenarios may also include more or fewer scenarios. For example, the collaborative scenarios may also include other applications such as industrial production. field scene.

Taking a game-based collaborative scenario as an example, each user's client can obtain game data from the server, and based on the game data, present the user with a game scene, as well as one or more player characters and other objects distributed in the game scene. The game scene can be used to indicate the environment, area, terrain, building, etc. where the game is located. Each object in the game scene can interact in the game scene. In some implementations, the object may include an object controlled by the user, such as a player character (PC). In some implementations, the object may include a non-user-controlled object, such as a non-player character (NPC). In some implementations, the object may include other objects that can interact with the user, such as various game props such as vehicles, tools, treasure chests, etc. In some embodiments, at least one of the interactive objects is a user-controlled object. The user can control an object in the game scene to interact with another object, such as the user's player-controlled character dancing with another user's player-controlled character. , the user's player-controlled character opens the treasure chest, the user's player-controlled character and another user's player-controlled character drive the same vehicle together, etc. In some implementations, the interactive objects may not be objects controlled by the user. For example, the game scene includes two animals: "wolf" and "sheep". Both animals are not controlled by the user. When the "wolf" encounters After reaching "Sheep", you can hunt "Sheep".

In some implementations, each user's client corresponds to different directions and positions in the game scene. The position corresponding to the client can be understood as the position of the user of the client in the game scene, and the direction corresponding to the client can be understood as the direction in which the user of the client observes the game scene. When the client presents a game scene to the user, the client may, based on the position and direction corresponding to the client, present to the user at least part of the game scene corresponding to the position and the direction and one of the at least part of the game scene. or multiple objects, that is, based on the position of the user of the client in the game scene and the direction in which the user observes, the user is presented with at least part of the game scene and objects observed by the user at the position and from the direction. .

For example, in a multiplayer battle game, the game screen displayed by client A can be as shown in Figure 3. The game screen of client A The picture shown in the third-person perspective can be understood as the game scene that the user of Client A can see at the current position and direction. The game picture of Client A includes the player-controlled character A corresponding to Client A 301 , the player-controlled character B 302 and the treasure box 303 corresponding to another client B, wherein the player-controlled character B 302 is located in the northwest direction in front of the player-controlled character A 301, and the treasure box 303 is located in the northeast direction in front of the player-controlled character A 301. The game screen displayed by another client B can be as shown in Figure 4. The game screen of client B is a third-person perspective screen, which can be understood as the game scene that the user of client B can see at the current position and direction. The game screen of client B includes the game scene corresponding to client B. The player controls character B 302 and the treasure box 303. The treasure box 303 is located in the due east direction of the player-controlled character B 302. Comparing Figure 3 and Figure 4, we can see that although Client A and Client B are in the same game scene, the direction and position corresponding to Client A are different from the direction and position corresponding to Client B. Therefore, the game of Client A The screen is also different from the game screen of client B. If the user of client B controls the movement of player-controlled character B 302, the game screen of client A can also simultaneously display the movement process of player-controlled character B 302, that is, player-controlled character A 301 observes the movement of player-controlled character B 302. If the player controls character B 302 and opens the treasure box 303, then the process of the player controlling character B 302 to open the treasure box 303 and the status change of the treasure box 303 can be displayed simultaneously on the game screen of client A and the game screen of client B, that is, Player-controlled character A 301 observed player-controlled character B 302 opening the treasure chest 303 .

It can be understood that the game screens shown in Figures 3 and 4 are both from a third-person perspective. Therefore, the game screen shown in Figure 3 includes the player-controlled character A 301 corresponding to client A. The game shown in Figure 4 The screen includes the player-controlled character B 302 corresponding to client B. However, if the game screens shown in Figures 3 and 4 are both from the first-person perspective, then the game screen shown in Figure 3 does not include the character B 302 corresponding to client A. The player controls character A 301. The game screen shown in Figure 4 does not include the player-controlled character B 302 corresponding to client B. This is more in line with the visual experience that humans cannot see themselves under normal circumstances, making users feel immersed in the scene. gaming experience.

Please refer to Figure 5. This embodiment of the present application provides a multi-device collaboration system, which can realize the aforementioned collaboration scenario. The system can include multiple clients 100 and servers 200. Each client 100 and the server 200 are connected via network 300.

The multiple clients 100 include an AR client 510 and a VR client 520 . The AR client 510 may include a mobile phone 530 and smart glasses 540 , and the VR client 520 may include a smart large screen 550 and a laptop 560 . It should be noted that in actual applications, the AR client 510 and the VR client 520 can also include more or less devices, and the mobile phone 530 and the smart glasses 540 can also be used as the VR client 520 and the smart large screen 550 And laptop 560 can also serve as AR client 510. The AR client 510 can present an AR scene to the user, and the VR client 520 can present a VR scene to the user. The AR scene and the VR scene can be the same collaborative scene respectively, with different presentation methods on the AR client 510 and the VR client 520 , AR scenes can be virtual and real scenes superimposed on real scenes and virtual scenes, and VR scenes can be virtual scenes. In some implementations, for the same object, the object can be a virtual object, a real object, or a combination of virtual and real objects in the AR scene, and the object can be a virtual object in the VR scene. For example, the AR scene includes a real person, and the person can be presented as a virtual humanoid image corresponding to the real person in the VR scene. For another example, the AR scene includes a virtual and real horse composed of a real support frame and a virtual projection. The horse can be presented as a virtual horse in the VR scene. For another example, the AR scene includes a virtual humanoid character. Since the humanoid character is also a virtual object, the humanoid character is also a virtual humanoid character when presented in the VR scene.

The server 200 can provide multiple clients 100 with data required for multi-device collaboration. For example, the server 200 can provide each client 100 with location information and direction information corresponding to each client 100 , provide each client 100 with scene data for presenting a collaborative scene, and provide each client 100 with the collaborative scene. The status data of the objects in the object, etc.

In some implementations, the server 200 may be omitted, and various functions of the server 200 may be integrated on at least one of the multiple clients 100 . In some embodiments, any client of the plurality of clients 100 (for example, any AR client 510) can serve as both a client and a server, and other clients of the plurality of clients 100 can communicate with any of the clients 100. A client is connected through the network 300, and any client can provide data required for multi-device collaboration to multiple clients 100 in the same or similar manner as the server 200. For example, still taking a game-based collaboration scenario as an example, multiple clients 100 include multiple mobile phones 530 and laptops 560, where the laptop 560 serves as both a client and a server, and the user of the laptop 560 creates a In the game scene, multiple mobile phones 530 serve as clients to access the game scene. Users of each mobile phone 530 and laptop 560 can play games in the game scene created by the laptop 560.

Please refer to FIG. 6 , which is a structural diagram of a multi-device collaboration system provided by an embodiment of the present application. The multiple device collaboration system may include a client 100, a gateway 600, and a server 200.

The gateway 600 is used to implement communication between the client 100 and the server 200 . The gateway 600 may be a network device in the aforementioned network 300.

The client 100 can establish a communication connection with the server 200 through the gateway 600, and the client 100 can communicate with the server 200 through the communication connection.

In some implementations, the gateway 600 may be a service mesh-based gateway. A service mesh is an infrastructure layer that handles communication between services. A service mesh can be composed of a series of lightweight network proxies.

In some embodiments, the communication connection may be a long connection, that is, once established, the communication connection can be maintained for a long time or permanently, so that when the client 100 communicates with the server 200 subsequently, there is no need to establish a new communication connection again, saving connection requests. Reduce time and resource consumption and reduce communication delay.

In some implementations, the gateway 600 can be deployed in a container based on docker to achieve dynamic expansion and contraction to meet message routing and forwarding in massive request scenarios.

In some implementations, the server 200 may include a first synchronization module 210, a second synchronization module 220, and an official service module 230.

The first synchronization module 210 may be used to implement area of interest (AOI) synchronization of multiple clients 100 in the first coordinate system. Among them, AOI can be used to indicate regional geographical entities in the map, such as a residential area, a university, an office building, an industrial park, a comprehensive shopping mall, a hospital, a scenic spot or a stadium, etc. Taking a residential area as an example, the AOI of the residential area can be used to indicate the outline and area of the residential area. The first coordinate system may be a coordinate system in the real world. The first coordinate system is the public coordinate system.

In some implementations, the first coordinate system may include a GPS coordinate system.

In some implementations, the first synchronization module 210 can serialize the latitude and longitude data in the map through the geohash spatial index algorithm, thereby dividing the map into different areas, and each area can be used as an AOI area.

In some embodiments, the first synchronization module 210 may determine the AOI area where the client 100 is located, and the client 100 included in each AOI area. In some embodiments, an AOI area may correspond to one or more collaborative scenes, and a collaborative scene may correspond to one or more AOI areas. When a client is in an AOI area, it may access the collaborative scene corresponding to the AOI area.

For example, in a multiplayer battle game, the server 200 configures three AOI areas through the first synchronization module 210, urban area A, urban area B, and urban area C. Each urban area can be used as an independent game scene. When the client 100 is in urban area A, the client 100 can access the game scene.

The second synchronization module 220 may be used to implement AOI synchronization of multiple clients 100 in the second coordinate system, and the synchronization accuracy of the second synchronization module 220 may be higher than the synchronization accuracy of the first synchronization module 210 . The second synchronization module 220 can be specifically used to more accurately determine the position of the client 100 in the collaborative scene, as well as the positions of different clients 100 in the collaborative scene, and then detect whether a certain client 100 is present with other clients 100 Related content, and/or whether to send content related to the client 100 to other clients 100 so that other clients 100 can present content related to the client 100 . The second coordinate system may be a real-world coordinate system. The second coordinate system may be a common coordinate system.

In some implementations, the second coordinate system may be a UTM coordinate system.

In some implementations, the second synchronization module 220 can implement AOI synchronization of multiple clients 100 in the second coordinate system through a nine-square grid algorithm.

The public service module 230 may include one or more public services, such as terrain resource services, event bus services, etc. The public service module 230 may provide one or more public services to one or more clients 100 based on at least one of the first synchronization module 210 and the second synchronization module 220 .

In some embodiments, at least part of the data in the server 200 can be stored in memory, thereby reducing or avoiding complex addressing operations when obtaining the at least part of the data, improving the efficiency of obtaining the at least part of the data, and thereby improving the efficiency of obtaining the at least part of the data. Equipment collaboration efficiency.

Please refer to FIG. 7 , which is a structural diagram of a multi-device collaboration system provided by an embodiment of the present application. Among them, the system shown in FIG. 7 is based on the system shown in FIG. 6 , and the first synchronization module 210 is described in detail.

The first synchronization module 210 may include a regional service module 211, a regional management module 212 and a public access module 213.

The area service module 211 can be used to provide multiple divided AOI areas, including the location, boundary, size of each AOI area, and the positional relationship between any two AOI areas.

The regional management module 212 can serve as a regional configuration center to manage the AOI regions included in the regional service module 211, including regional division, load balancing, elastic computing, etc.

In some implementations, the area management module 212 may be used to divide the map into regions in the first coordinate system for different collaborative scenarios, thereby achieving physical and logical isolation between different collaborative scenarios.

The public access module 213 can be used to access one or more public service functions based on multiple AOI areas in the regional service module 211, such as chat function, friend function, log monitoring function, etc.

The technical solution of the present application will be described in detail below with specific examples. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

It should be noted that the following only takes the first collaborative scenario as a game scenario as an example to illustrate the multi-device collaboration method provided by the embodiment of the present application. When the first collaborative scenario is another type of scenario, each client and each server described below can implement multi-device collaboration in a manner similar to or the same as that in the game scenario.

Please refer to Figure 8, which is a flowchart of a multi-device collaboration method provided in an embodiment of the present application. It should be noted that the method is not limited to Figure 8 and the specific sequence described below. It should be understood that in other embodiments, the order of some steps in the method can be interchanged according to actual needs, or some steps can be omitted or deleted. The method includes the following steps:

S801: The first AR client and the first VR client access the same game scene.

The first AR client and the first VR client can access the same game scene, so that the game scene can be presented to respective users, so that the respective users can play in the game scene.

Among them, the way in which the first AR client and the first VR client access the same game scene can also refer to the relevant descriptions in the following Figures 10 to 13.

It should be noted that S801 is an optional step. For example, in some implementations, the first AR client and the first VR client may always remain in a state of accessing the game scene.

S802: The first AR client presents the first AR scene, and the first VR client presents the first VR scene.

Among them, the first AR scene can be a game scene in AR mode, and the first VR scene can be a game scene in VR mode. That is, the first AR scene and the first VR scene can be the same game scene, and the results are presented in different modes in the first AR client and the first VR client respectively.

In some implementations, the first AR scene includes at least part of a realistic scene. In some implementations, the at least part of the real scene may be a scene captured by a camera of the first AR client.

In some embodiments, the first VR scene may be a virtual scene corresponding to the first AR scene. In some embodiments, the first VR scene may be a virtual scene obtained by processing the real scene included in the first AR scene. In some embodiments, the first VR scene may be a virtual scene obtained by processing the real scene in the first AR scene through a view synthesis model such as a NeRF network. The view synthesis model may remodel and render the realization scene, thereby outputting a virtual scene corresponding to the real scene.

For example, the first AR scene and the first VR scene may be as shown in Figure 9. The first AR scene can be a real scene captured by the camera of the first AR client in real time, including real buildings, corridors, trees, etc. The first VR scene is a virtual scene obtained by processing the first AR scene, including virtual buildings corresponding to real buildings, virtual corridors corresponding to real corridors, and virtual trees corresponding to real trees. And it can be understood that since the first AR scene and the first VR scene are scenes presented by different clients, and since the corresponding positions and angles of different clients are different, the specific contents included in the first AR scene and the first VR scene The content also varies. The first AR scene and the first VR scene in Figure 9 can be understood as different game scenes seen by users in two opposite directions.

The way in which the first AR client presents the first AR scene can also refer to the relevant description in Figure 14 below, and the way in which the first VR client presents the first VR scene can also refer to the relevant description in Figure 15 below.

S803. When the first real position of the first AR scene includes the first object, the first VR client presents the first object at the first virtual position of the first VR scene, and the first real position and the first virtual position are the game scene. at the same position in; and/or, when When the second virtual position of the first VR scene includes the second object, the first AR client presents the second object at the second real position of the first AR scene, and the second virtual position and the second real position are the same in the game scene. Location.

When the first real position of the first AR scene includes the first object, the first VR client presents the first object at the first virtual position of the first VR scene, that is, the first VR client can display the first AR The first object in the scene is synchronously presented in the first VR scene, so that the user can see the first object from the first AR scene from the first VR client. When the second virtual position of the first VR scene includes the second object, the first AR client presents the second object at the second real position of the first AR scene, that is, the first AR client can place the first VR The second object in the scene is synchronously presented in the first AR scene, so that the user can see the second object from the first VR scene from the first AR client. Therefore, both the first AR client and the first VR client can present the object in the game scene from the opposite end, so that both the user of the first AR client and the user of the first VR client can see the object, realizing the AR client Collaboration and interoperability with VR clients.

For example, in the first AR scene shown in FIG9 , the real staircase includes virtual character A 701 and virtual character B 702, and virtual character B 702 is in front of virtual character A 701. In the first VR scene shown in FIG9 , the same position on the virtual staircase also includes virtual character A 701 and virtual character B 702, and virtual character B 702 is in front of virtual character A 701.

It should be noted that since the first AR scene is a scene that combines virtuality and reality, the objects in the first AR scene may be virtual objects, real objects, or objects that combine virtuality and reality. Real objects may be real objects in the first AR scene, such as real people, animals, objects, buildings, etc., such as real buildings, stairs, and trees in the first AR scene shown in Figure 9. The virtual object can be an image superimposed on the real scene or an image projected in the real scene, such as virtual character A 701 and virtual character B 702 in the first AR scene as shown in Figure 9. The virtual combination object may be an object that actually exists in the first AR scene and the image on the display screen, or it may be an object that actually exists in the first AR scene and a virtual image projected on the object. For example, the first AR scene includes a realistic torch frame, the first AR client projects a virtual image of the flame on the torch frame, and the torch frame and the virtual image of the flame are combined into a virtual and real torch. . Since the first VR scene is a virtual scene, the objects in the first VR scene may be virtual objects.

In some embodiments, the first object may be an object controlled by the first AR client, for example, the first object may be a player-controlled character controlled by the first AR client. Of course, in practical applications, the first object may not be an object controlled by the first AR client, for example, the first object may be a user of the first AR client, a real animal or object in the first AR scene, etc.

In some implementations, the second object may be an object controlled by the first VR client. For example, the second object may be a player-controlled character controlled by the first VR client. Of course, in actual applications, the second object may not be an object controlled by the first VR client. For example, the second object may be a character controlled by a non-player.

The first object may be the user of the first AR client. For example, when the user of the first AR client is in the first real position of the first AR scene, the first VR client may be in the first real position of the first VR scene. The first object is presented in the virtual position. For example, the first object of the first VR scene can be a virtual image corresponding to the user of the first AR client, which can enable the user of the first VR client to see the first object in the first VR scene. A virtual image corresponding to the user of an AR client; the second object is a player-controlled character controlled by the first VR client, and the first AR client presents the second object at the second real position of the first AR scene, which can make the first AR The user of the client can see the player-controlled character controlled by the first VR client in the first AR scene. Therefore, multiple users from multiple different types of clients can interact with each other.

In some implementations, the first object in the first AR scene may be a real object or a virtual combined object, and the first VR client may present the virtual image corresponding to the first object at the first virtual position of the first VR scene. . In some implementations, the first object in the first AR scene is a virtual object, and the first VR client can present the virtual image of the first object in the first VR scene.

In some implementations, the second object in the first VR scene is a virtual object, and the first AR client can present the second object or a combination of virtual and real objects corresponding to the second object in the second real position of the first AR scene. image.

In some implementations, when the first event corresponding to the first object occurs in the first AR scene, the first VR client can also present the first event in the first VR scene. That is, the first VR client can A first event occurring in the first AR scene is presented synchronously with the first AR client. For example, when the first object in the first AR scene is a real object and the first event occurs in the real object, the first event can also occur in the virtual image of the first object in the first VR scene, and the first event can also occur. The event is presented to the first VR scene in the first VR client. The first VR client and the first AR client present the first event synchronously, This improves the authenticity of the first VR client and the first AR client in collaboratively presenting the game scene, thereby improving the user experience.

In some implementations, the first event may be at least one of a status update event of the first object, a movement event of the first object, and an interaction event between the first object and other objects.

In some implementations, a status update event of the first object may be used to indicate a status change of the first object.

For example, the status update event of the first object can be used to indicate changes in the first object's body shape, body shape, hairstyle, clothing, etc., the first object's health value, physical strength value, endurance value, skill cooling time, remaining number of props, tasks Numerical changes such as completion progress. It should be noted that the state of the first object and the specific content of the state change can be determined by relevant technical personnel such as the game developer. The embodiment of the present application does not limit the state of the first object and the specific content of the state change.

In some implementations, the first object motion event can be used to indicate the motion process of the first object, such as speed, direction, motion route, and so on. In some embodiments, the motion event of the first object includes the movement of the first object from a first real position in the first AR scene to a third real position, and the first VR client can present the first object from the first VR scene. The first virtual position in , the movement process of moving to the third virtual position, and the third virtual position and the third real position are the same position in the game scene.

In some implementations, interaction events between the first object and other objects may be used to indicate the interaction process between the first object and other objects.

The other objects may be any interactive objects, and the interaction method between the first object and any object may differ based on the object. For example, if the object that interacts with the first object is a player-controlled character or a non-player-controlled character, then the interaction method may include dialogue, dancing, attack, assistance, etc.; or, if the object that interacts with the first object is a real object, the interaction method may include Including operating the object; or, the object interacting with the first object is an animal, and the interaction method may include stroking, feeding, driving, attacking, riding, etc. It should be noted that the interactive objects in the game scene and the interaction methods for different objects can be determined by relevant technical personnel such as game developers. The embodiments of this application do not specifically limit the types of interactive objects and the corresponding interaction methods.

In some implementations, when a second event corresponding to the second object occurs in the first VR scene, the first AR client may present the second event corresponding to the second object in the first AR scene, that is, the second event corresponding to the second object occurs in the first VR scene. A VR client can present a second event occurring in the first VR scene synchronously with the first AR client. By synchronously presenting the first event, the first VR client and the first AR client improve the authenticity of the game scene collaboratively presented by the first VR client and the first AR client, thereby improving the user experience.

In some implementations, the second event may be at least one of a status update event of the second object, a movement event of the second object, and an interaction event between the second object and other objects. For the second event, please refer to the description related to the first event and will not be described again here.

It is understood that the first event and the second event may also include more or less events.

Among them, the way in which the first VR client presents the first object at the first virtual position of the first VR scene, and the way in which the motion event of the first object is presented, can also refer to the relevant descriptions in the following Figures 16, 19 and 20; the way in which the first AR client presents the second object at the second real position of the first AR scene, and the way in which the motion event of the second object is presented, can also refer to the relevant descriptions in the following Figures 17-18 and 21; the way in which the first AR client and the first VR client present the state change event of the first object can also refer to the relevant descriptions in the following Figures 22-25.

In this embodiment of the present application, the first AR client and the first VR client can access the same game scene, the first AR client presents the first AR scene, and the first VR client presents the first VR scene, that is, the first The AR client and the first VR client can collaboratively present a game scene, where the first VR scene can be a virtual scene obtained by processing the first AR scene. When the first real position of the first AR scene includes the first object, the first VR client presents the first object at the first virtual position of the first VR scene, that is, the first VR client can display the first AR The first object in the scene is synchronously presented in the first VR scene, so that the user can see the first object from the first AR scene from the first VR client. When the second virtual position of the first VR scene includes the second object, the first AR client presents the second object at the second real position of the first AR scene, that is, the first AR client can place the first VR The second object in the scene is synchronously presented in the first AR scene, so that the user can see the second object from the first VR scene from the first AR client. Therefore, the first AR client and the first VR client can present the object in the game scene from the opposite end, so that the user of the first AR client and the user of the first VR client can see the object, realizing the realization of the AR client and VR Collaboration and interoperability between clients.

Please refer to FIG. 10 , which is a flow chart of a method for accessing a game scene provided by an embodiment of the present application. It should be noted that this method is not limited to the specific sequence shown in Figure 10 and below. It should be understood that in other embodiments, the order of some steps of the method can be exchanged with each other according to actual needs, or some of the steps can be It can also be omitted or deleted. The method includes the following steps:

S1001, client starts.

The user can start the client by clicking on the icon of the client in the terminal device. Of course, in actual applications, the client can also be started in other ways. The embodiments of this application do not limit the way in which the client is started.

It should be noted that S1001 is an optional step. When S1001 is omitted, the client can reside in the terminal device, thereby improving the efficiency of the client accessing the game scene.

S1002, the client updates local resources.

In order to ensure stable operation of the client, the client can query whether there is updated data when starting up. If there is updated data, the updated data is obtained and the local resources are updated based on the updated data. It is understandable that the client can also update the local resources at other times, or not actively update the resources. S1002 is an optional step.

In some implementations, the client can also verify the security and integrity of local resources. If the verification passes, continue to perform subsequent steps. Otherwise, the client can repair the resource and continue to perform subsequent steps.

S1003, the client performs permission verification.

A client can check whether at least one of the permissions required for the client to run is granted. If you continue to perform subsequent steps, if one or more permissions have not been authorized, you can obtain the one or more permissions and then continue to perform subsequent steps. In some embodiments, the at least one permission includes positioning permission, permission to call the network, permission to read and write storage devices, permission to read and write photo albums, permission to obtain device information of the terminal device, permission to call the microphone and camera, etc. . Of course, in practical applications, the at least one permission may also include more or fewer permissions. The embodiment of this application does not limit the number of permissions verified by the client and the specific types of permissions.

In some implementations, the client can also enable at least one function on the terminal device. The at least one function may be used to ensure proper functioning of the client. In some implementations, the at least one function may include a positioning function, a network function, and the like. Of course, in actual applications, the at least one function may also include more or less functions. The embodiments of this application do not limit the number and specific types of functions enabled by the client.

It should be noted that S1003 is an optional step. When S1003 is omitted, the efficiency of client login can be improved.

S1004, the client logs in to the business server.

In some implementations, the client can obtain the first login credential information of the SMS server from the SMS server through a network request such as a hypertext transfer protocol (HTTP) request, and obtain it from the business server based on the first login credential information. The second login credential information of the business server.

Among them, the login credential information is used to prove that the user name and password provided by the client are correct. The client avoids frequently providing user names and passwords to the server by providing login credential information to the server. In some implementations, the login credential information can be a token, and the token can be a string generated by the server. In some implementations, the first login credential information may be a character string generated by the SMS server, and the second login credential information may be a character string generated by the business server.

For example, the client name is "Starlight Tower". The user opens "Starlight Tower" on the mobile phone, thereby displaying the login interface shown in Figure 11, which includes a mobile phone number input box 901, a verification code input box 902, a verification code refresh button 903, a login button 904 and a registration button 905. The user can enter a pre-registered mobile phone number in the mobile phone number input box 901, then click the verification code refresh button 903, enter the obtained verification code into the verification code input box 902, and then click the login button 904 to complete the login. Of course, if the user has not registered yet, he can click the registration button 905 to complete the registration and then log in. It can be understood that in actual applications, the client can also log in to the business server in other ways, and the embodiment of the present application does not limit the way in which the client logs in to the business server.

S1005, the client obtains role data from the business server.

The client can obtain role data from the business server and select the role.

The client can obtain role data from the business server based on the second login credential information. If there is no role data corresponding to the client in the business server, it is possible that the user of the client has not created a role, so the client can create a role and thereby generate role data.

Among them, the role data can be role-related data. In some implementations, role data may include role index information and role names.

It can be understood that in actual applications, the client can also obtain role data from the business server through other means, and the role data is not limited to the role index information and role name mentioned above. The embodiment of the present application does not limit the client to obtain role data from the server and the specific content included in the role data.

S1006: The client selects a game scene and a method of accessing the game scene.

In some implementations, the client can establish a transmission control protocol (TCP) connection with the service server. When the TCP connection is successful, the lobby scene is entered. The lobby scene can include one or more game scenes, and the client can choose to access any one of the game scenes. When the client selects the game scene to access, it can further choose to access the game scene through AR or VR. If the client chooses to access the game scene through AR, the client can access the game scene as an AR client. If the client chooses to access the game scene through VR, the client can access the game scene as a VR client. For example, as shown in Figure 10, the lobby scene includes the Jet Arena scene and the classic scene. When the client selects the Jet Arena scene, it can further select AR Arena or VR Arena. When the client selects a classic scene, it can further select an AR classic scene or a VR classic scene.

For example, when the user successfully logs in based on the login interface shown in Figure 11, the mobile phone can display the camp selection interface shown in Figure 12. The camp selection interface includes a blue camp button 906 and a red camp button 907 . The user can join the blue camp by clicking the blue camp button 906, or click the red camp button 907 to join the red camp. After the user selects a camp, the game lobby interface as shown in Figure 13 can be displayed. The game lobby interface includes user information 913 such as the user's avatar, name, and level, and a game map 908, and the game map 908 includes 5 games. Scenes, the user can click on any of the game scenes to select the game scene. In Figure 13, the game scene 909 currently located in the middle of the game map 908 displays the user's avatar, which means that the user currently selects the game scene 909. The game lobby interface also includes an access method switching button 910. The user can click the access method switching button 910 to choose to access the game scene through AR or VR. For example, the current access method switching button 910 displays "AR", which means that the game scene is currently accessed through AR. The game lobby interface also includes a backpack button 911. The user can click the backpack button 911 to open a virtual backpack, which can include virtual game props and assets, etc. The game lobby interface also includes a message button 912. When the user clicks the message button 912, one or more messages can be displayed to the user. These can be system messages from the server or messages from other users.

It is understandable that, in actual applications, the client may also select the game scene and/or the method of accessing the game scene in other ways, and the embodiments of the present application do not limit the method of selecting the game scene and accessing the game scene by the client. For example, in other embodiments, the client may first select the method of accessing the game scene, and then select the game scene to be accessed. Alternatively, in other embodiments, the client is configured to act only as an AR client or a VR client, so it is not necessary to select the method of accessing the game scene in S1006.

Please refer to FIG. 14 , which is a flow chart of a method for presenting an AR scene provided by an embodiment of the present application. It should be noted that this method is not limited to the specific sequence shown in Figure 14 and below. It should be understood that in other embodiments, the order of some steps of the method can be exchanged with each other according to actual needs, or some of the steps can be It can also be omitted or deleted. The method includes the following steps:

S1401: A first AR client sends first positioning information collected by the first AR client to a resource server.

The first positioning information may be used to indicate the position of the first AR client in the first coordinate system. In some implementations, the first positioning information may be used to indicate the AOI area where the first AR client is located, such as the country, province, city or administrative region where the first AR client is located. In some implementations, the first coordinate system is a GPS coordinate system, and the first positioning information may be the first GPS coordinates.

In some implementations, the first AR client may also send first range information to the resource server, and the first range information may be used to indicate the range of the game scene presented by the first AR client.

S1402: The resource server returns the first digital resource to the first AR client based on the first positioning information.

The resource server may obtain the first digital resource from the stored digital resource collection based on the first positioning information, and return the first digital resource to the first AR client. Wherein, the first digital resource can be used to present at least part of the scene in the game scene.

In some embodiments, the resource server may, based on the first positioning information and the first range information, retrieve the digital resource from the stored digital resource. Get the first digital resource in the collection.

S1403. The first AR client sends the first positioning information and the first environment image collected by the first AR client to the positioning server.

The first AR client may send the first positioning information and the first environment image collected by the first AR client to the positioning server, thereby requesting the positioning server to perform more accurate positioning of the first AR client.

Wherein, the first AR client can photograph the scene where the first AR client is located through the camera, thereby obtaining the first environment image.

In some implementations, the positioning server may be a visual positioning service (visual positioning service) server.

In some implementations, the first AR client compresses the first environment image, thereby sending the compressed first environment image to the positioning server. By compressing the first environment image, the amount of data required to be transmitted can be reduced, and positioning efficiency and real-time positioning can be improved.

S1404: The positioning server returns the second positioning information to the first AR client.

The positioning server may determine the second positioning information based on the first environment image and the first positioning information, and return the second positioning information to the first AR client. The second positioning information may indicate the first AR client more accurately than the first positioning information. end position. In some implementations, the second positioning information may be used to indicate the specific location of the first AR client in the area indicated by the first positioning information.

The second positioning information may be used to indicate the position of the first AR client in the second coordinate system. In some implementations, the second coordinate system is a universal transverse mercator grid system (UTM) coordinate system, and the second positioning information may be the first UTM coordinates.

In some implementations, the positioning server may also determine the first rotation information based on the first environment image and the first positioning information, and return the first rotation information to the first AR client. The first rotation information may be used to indicate the rotation angle of the first AR client in the second coordinate system.

S1405: The first AR client converts the second positioning information into third positioning information.

The first AR client can perform coordinate system conversion on the second positioning information, thereby obtaining the third positioning information in the third coordinate system. The third positioning information may be used to indicate the position of the first AR client in a third coordinate system, and the third coordinate system may be a coordinate system with the camera of the first AR client as a reference. In some implementations, the third coordinate system can be understood as the local coordinate system of the first AR client, and the third positioning information can be understood as the local coordinates of the first AR client.

In some implementations, the first AR client may convert the first rotation information into the second rotation information. The second rotation information may be used to indicate the rotation angle of the first AR client in the third coordinate system.

S1406: The first AR client loads the first digital resource based on the third positioning information and presents the first AR scene.

The first AR client can load the first digital resource based on the third positioning information, thereby presenting the first AR scene, so that the user of the first AR client can accurately see objects at various locations around him based on the location, that is, see The first AR scene that combines reality and reality.

In some implementations, the first AR client can load the first digital resource based on the third positioning information and the second rotation information, so that each object in the first AR scene matches the rotation angle of the camera of the first AR client. , improve the authenticity and user experience of the first AR scene.

Please refer to FIG. 15 , which is a flow chart of a method for presenting a VR scene provided by an embodiment of the present application. It should be noted that this method is not limited to the specific sequence shown in Figure 15 and below. It should be understood that in other embodiments, the order of some steps of the method can be exchanged with each other according to actual needs, or some of the steps can be It can also be omitted or deleted. The method includes the following steps:

S1501. The first VR client sends preset fourth positioning information to the resource server.

The fourth positioning information may be used to indicate the position of the first VR client in the first coordinate system. In some implementations, the fourth positioning information may be used to indicate the AOI area where the first VR client is located, such as the country, province, city or administrative region where the first VR client is located. In some implementations, the first coordinate system is a GPS coordinate system, and the fourth positioning information may be a second GPS coordinate.

In some implementations, the first VR client may also send second range information to the resource server, and the second range information may be used to indicate the range of the game scene presented by the first AR client.

In some implementations, the fourth positioning information may be obtained by the first VR client receiving a user submission. For example, the first VR client is located in Shanghai, but it can still receive the GPS coordinates submitted by the user and belongs to Beijing, so that for the resource server, the first VR client is the client located in Beijing.

S1502: The resource server returns the second digital resource to the first VR client based on the fourth positioning information.

The resource server may obtain the second digital resource from the stored digital resource collection based on the fourth positioning information, and return the second digital resource to the first VR client. Wherein, the second digital resource can be used to present at least part of the scene in the game scene.

In some implementations, the resource server may obtain the second digital resource from the stored digital resource set based on the fourth positioning information and the second range information.

S1503. The first VR client converts the preset fifth positioning information into sixth positioning information.

The fifth positioning information may indicate the position of the first VR client more accurately than the fourth positioning information. In some implementations, the fifth positioning information may be used to indicate the specific location of the first VR client in the AOI area indicated by the fourth positioning information.

The fifth positioning information may be used to indicate the position of the first VR client in the second coordinate system. In some implementations, the second coordinate system is a UTM coordinate system, and the fifth positioning information may be the second UTM coordinates.

In some implementations, the fifth positioning information may be stored in a local file of the first VR client, or the fifth positioning information may be obtained from the service server. In some implementations, the position indicated by the fifth positioning information can be understood as the birth point, rebirth point, refresh point, etc. of the first VR client in the first area.

The first AR client can perform coordinate system conversion on the fifth positioning information, thereby obtaining the sixth positioning information in the fourth coordinate system. The sixth positioning information may be used to indicate the position of the first VR client in a fourth coordinate system, and the fourth coordinate system may be a coordinate system with the first VR client as a reference. In some implementations, the fourth coordinate system can be understood as the local coordinate system of the first VR client, and the sixth positioning information can be understood as the local coordinates of the first AR client.

S1504. The first VR client loads the second digital resource based on the sixth positioning information, thereby presenting the first VR scene.

The first VR client can load the second digital resource based on the sixth positioning information, thereby presenting the first VR scene, so that the user of the first VR client can accurately see objects at various locations around him based on the location, that is, see to the virtual first VR scene.

In some implementations, the first VR client can load the second digital resource based on the sixth positioning information and the third rotation information, so that each object in the first VR scene matches the rotation angle of the camera of the first VR client. , improve the authenticity and user experience of the first VR scene. The third rotation information can be obtained by the first VR client receiving a user submission.

In the method shown in Figure 15 above, the first VR client can obtain the second digital resource through the preset fourth positioning information, which makes it possible that even in reality, the first VR client is not in the same position as the first AR client. The same AOI area can also be collaborated with the first AR client. It can be understood that in other implementations, if in reality, the first VR client and the first AR client are in the same AOI area, then the first VR client can also follow a similar process as the first AR client. way, the fourth positioning information is obtained through collection.

Combining Figures 14 and 15, it can be seen that since the first AR client obtains the first digital resource from the resource server based on the first positioning information, the first VR client obtains the second digital resource from the resource server based on the fourth positioning information, and The first positioning information and the fourth positioning information are both in the first coordinate system, so for the resource server, the first AR client and the first VR client are in the same coordinate system, so the first AR client and the first VR client The terminal can present the same game scene. And because the first AR client loads the first digital resource based on the third positioning information to present the first AR scene, the first VR client loads the second digital resource based on the sixth positioning information to present the first VR scene, and the third The positioning information is obtained by converting the second positioning information in the second coordinate system. The sixth positioning information is obtained by converting the fifth positioning information in the second coordinate system. The second positioning information and the fifth positioning information belong to The same coordinate system, so for the resource server, the first AR client and the first VR client are both in the same coordinate system, which makes the first AR scene presented by the first AR client and the first VR client The first VR scene presented by the client can be synchronized. For example, the character corresponding to the first AR client and the character corresponding to the first VR client are relative in the game scene, then when the character corresponding to the first AR client is in the first When you see the character corresponding to the first VR client in the AR scene, the character corresponding to the first VR client can also see the character corresponding to the first AR client in the first VR scene, that is, the character corresponding to the first AR client The characters corresponding to the first VR client can interact with each other in the game scene.

Please refer to Figure 16, which is a flow chart of a method for collaboratively presenting game objects provided by an embodiment of the present application. This method can be used to implement: when the first real position of the first AR scene includes the first object, the first VR client in the first VR scene A virtual location presents the first object, and the method can also be used to present motion events of the first object. Among them, taking the first object in the first real position of the first AR scene as a real user of the first AR client at the first real position (that is, the first object is a real object) as an example, the corresponding positioning of the first AR client The information is the positioning information corresponding to the first object. It should be noted that this method is not limited to the specific order described in Figure 16 and below. It should be understood that in other embodiments, the order of some steps of the method can be exchanged with each other according to actual needs, or some of the steps can be It can also be omitted or deleted. The method includes the following steps:

S1601. The service server sends the second positioning information corresponding to the first object to the first VR client.

The first object may be at a first realistic location in the first AR scene. The business server can send the second positioning information of the first object in the second coordinate system to the first VR client. Since to the business server, the first AR client and the first VR client are in the second coordinate system, therefore The first VR client can present the first object in the first VR scene based on the second positioning information.

The second positioning information may correspond to the first real position of the first object in the first AR scene.

In some implementations, the service server may send the first corresponding first rotation information to the first AR client.

In some implementations, the service server may send at least one of the second positioning information and the first rotation information to the first VR client by broadcasting.

S1602: The first VR client converts the second positioning information into seventh positioning information.

The first VR client can perform coordinate system conversion on the second positioning information, thereby obtaining seventh positioning information in the fourth coordinate system.

The seventh positioning information may indicate the position of the first object in the fourth coordinate system, and the seventh positioning information may also be understood as the local coordinates of the first object.

In some implementations, the first VR client may convert the first rotation information into fourth rotation information. The fourth rotation information may be the rotation angle of the first object in the fourth coordinate system, or may be understood as the fourth rotation information. The rotation information indicates the local rotation angle of the first object on the first VR client.

S1603. The first VR client presents the first object at the first virtual position in the first VR scene indicated by the seventh positioning information.

In some implementations, the first VR client may display the first object based on the fourth rotation information in the first VR scene.

In some implementations, the first VR client may display the first object based on the fourth rotation information at the first virtual position in the first VR scene indicated by the seventh positioning information.

S1604, the first AR client controls the first object to move in the first AR scene. The moved first object is in the third real position in the first AR scene, and the positioning information corresponding to the third real position is the new third real position. Positioning information.

In some embodiments, the user of the first AR client can hold or wear the first AR client. When the user moves in the first AR scene, the first AR client will also be driven to move. The user and the first AR client The movement of the end can be understood as the movement of the first object. Of course, in practical applications, the first object can also move in the first AR scene in other ways.

In some implementations, the first AR client controls the movement of the first object in the first AR scene, and the rotation information corresponding to the moved first object is the new second rotation information.

S1605: The first AR client converts the new third positioning information into new second positioning information.

The first AR client can perform coordinate system conversion on the new third positioning information, thereby obtaining new second positioning information in the second coordinate system. Since the first AR client and the first AR client are both in the second coordinate system for the business server, the first VR client can present the motion status of the first object in the first VR scene based on the new second positioning information. .

In some implementations, the first AR client may convert the new second rotation information into new first rotation information.

S1606: The first AR client sends new second positioning information to the service server.

In some implementations, the first AR client may send new first rotation information to the service server.

S1607: The service server sends new second positioning information to the first VR client.

In some implementations, the service server may send new first rotation information to the first VR client.

S1608: The first VR client converts the new second positioning information into the new seventh positioning information.

In some implementations, the first VR client may convert the new first rotation information into new fourth rotation information.

S1609: The first VR client presents the first object at the third virtual position in the first VR scene indicated by the new seventh positioning information.

The third virtual position in the first VR scene may correspond to the third real position in the first AR scene. The third virtual position and the third real position are the same position in the game scene.

In some implementations, the first VR client may present the first object in the first VR scene based on the new fourth rotation information.

In some implementations, the first VR client may present the first object based on the new fourth rotation information at a third virtual position in the first VR scene indicated by the new seventh positioning information.

In this embodiment of the present application, when the first object moves from the first real position in the first AR scene to the third real position in the first VR scene, the first object also moves from the first virtual position in the first VR scene. Moving to the third virtual position in the first VR scene, that is, the instantaneous position of the first object in the first AR client and the position changes at different times are visible to the first VR client. It can be understood that the motion status of the first object can be presented synchronously by executing at least part of the above-mentioned steps S1601-S1609 multiple times.

Please refer to Figure 17, which is a flow chart of a method for collaboratively presenting game objects provided by an embodiment of the present application. This method can be used to implement: when the second virtual position of the first VR scene includes a second object, the first The AR client presents the second object in the second real-world position of the first AR scene, and this method can also be used to present motion events of the second object. The second object is the player-controlled character corresponding to the first VR client (that is, the second object is a virtual object), and the positioning information corresponding to the first VR client is the positioning information corresponding to the second object. It should be noted that this method is not limited to the specific order described in Figure 17 and below. It should be understood that in other embodiments, the order of some steps of the method can be exchanged with each other according to actual needs, or some of the steps can be exchanged. It can also be omitted or deleted. The method includes the following steps:

S1701. The service server sends the fifth positioning information corresponding to the second object to the first AR client.

The second object may be in a second virtual location in the first VR scene. The business server may send the fifth positioning information of the second object in the second coordinate system to the first AR client. Since to the business server, the first AR client and the first VR client are both in the second coordinate system, therefore The first AR client can present the second object in the first AR scene based on the fifth positioning information.

The fifth positioning information may correspond to the second virtual position of the second object in the first VR scene.

In some implementations, the service server may send fifth rotation information to the first AR client. The first VR client may convert the third rotation information to obtain the fifth rotation information, and send the fifth rotation information to the business server. The fifth rotation information may indicate the rotation angle of the second object in the second coordinate system. .

In some implementations, the service server may send at least one of the fifth positioning information and the fifth rotation information to the first AR client by broadcasting.

S1702: The first AR client converts the fifth positioning information into the eighth positioning information.

The first AR client may perform coordinate system conversion on the fifth positioning information to obtain the eighth positioning information in the third coordinate system.

The eighth positioning information may indicate the position of the second object in the third coordinate system, and the eighth positioning information may also be understood as the local coordinates of the second object.

In some implementations, the first AR client may convert the fifth rotation information into sixth rotation information. The sixth rotation information may be the rotation angle of the second object in the third coordinate system, or it may be understood that the sixth rotation information indicates the rotation angle of the second object local to the first AR client.

S1703. The first AR client presents the second object at the second real position in the first AR scene indicated by the eighth positioning information.

In some implementations, the first AR client may present the second object based on the sixth rotation information at a second real-world location in the first AR scene.

In some implementations, the first AR client may present the second object based on the sixth rotation information at a second real-world position in the first AR scene indicated by the eighth positioning information.

S1704, the first VR client controls the second object to move in the first VR scene. The moved second object is in the fourth virtual position in the first VR scene, and the positioning information corresponding to the fourth virtual position is the new sixth Positioning information.

In some implementations, the first VR client can receive movement instructions submitted by the user through peripheral devices such as keyboard, mouse, handle, remote control, microphone, smart glasses, helmet, and somatosensory equipment, and then based on the movement instructions, the second VR client Objects move in the first VR scene. In some embodiments, the type of movement instruction may include one or more types of key operations, touch operations, gesture operations, eye movement operations, and brain waves. Of course, in actual applications, the first VR client can also receive movement instructions submitted by the user through other methods, or can control the movement of the second object in the first VR scene through other methods.

In some implementations, the first VR client controls the second object to move in the first VR scene, and the rotation information corresponding to the moved second object is the new third rotation information.

S1705: The first VR client converts the new sixth positioning information into new fifth positioning information.

The first VR client can perform coordinate system conversion on the new sixth positioning information, thereby obtaining new fifth positioning information in the second coordinate system. Since the first AR client and the first AR client are both in the second coordinate system for the business server, the first AR client can present the motion status of the second object in the first AR scene based on the new fifth positioning information. .

In some implementations, the first VR client may convert the new third rotation information into new fifth rotation information.

S1706: The first VR client sends new fifth positioning information to the service server.

In some implementations, the first VR client may also send new fifth rotation information to the service server.

S1707: The service server sends new fifth positioning information to the first AR client.

In some implementations, the service server may also send new fifth rotation information to the first AR client.

S1708: The first AR client converts the new fifth positioning information into the new eighth positioning information.

In some implementations, the first AR client may also convert the new fifth rotation information into new sixth rotation information.

S1709: The first AR client presents the second object at the fourth real position in the first AR scene indicated by the new eighth positioning information.

The fourth real position in the first AR scene may correspond to the fourth virtual position in the first VR scene.

In some implementations, the first AR client may present the second object at a fourth real-world location in the first AR scene based on the new sixth rotation information.

In some implementations, the first AR client may present the second object based on the new sixth rotation information at a fourth real-world position in the first AR scene indicated by the new eighth positioning information.

In this embodiment of the present application, when the second object moves from the second virtual position in the first VR scene to the fourth virtual position in the first VR scene, the second object also moves from the second real position in the first AR scene. Moving to the fourth real position in the first AR scene, that is, the instantaneous position of the second object in the first VR client and the position changes at different times are visible to the first AR client. It can be understood that the motion status of the second object can be presented synchronously by executing at least part of the above-mentioned steps S1701-S1709 multiple times.

Combining the methods shown in Figure 16 and Figure 17 above, the instantaneous position of the first object in the first AR client and the position changes at different times are visible to the first VR client, and the second object in the first VR client The instantaneous position of the object and its position changes at different moments are visible to the first AR client. Therefore, when the user of the first AR client sees the virtual character corresponding to the first VR client in the first AR scene, the user of the first VR client can also see the virtual character corresponding to the first AR client in the first VR scene. The virtual characters, that is, the virtual characters controlled by the real user of the first AR client and the real user of the first VR client, can interact with each other in the game scene.

For example, Figure 18 is the first AR scene, and Figure 19 is the first VR scene.

The first AR scene shown in Figure 18 is a game scene presented by the first AR client from a first-person perspective. The first AR scene includes the second object 1601 corresponding to the first VR client, but does not include the first object 1602 corresponding to the first AR client. That is to say, the user of the first AR client can see other users playing together, but cannot see the user himself. The second object 1601 presented in the first AR scene is a virtual object.

The first VR scene shown in FIG. 19 is a game scene presented by the first VR client from a third-person perspective. The first VR scene includes both the second object 1601 corresponding to the first VR client and the first object 1602 corresponding to the first AR client, wherein the first object 1602 is a virtual object corresponding to the user of the first AR client. That is, the user of the first AR client can see other users playing together, and can also see the user himself.

When the user of the first VR client controls the second object 1601 to move in the first VR scene, the first AR client also The movement process of the second object 1601 is presented in the first AR scene. As shown in Figure 18, the second object 1601 moves from a position close to the left in the first AR scene to a middle position.

When the user of the first AR client (ie, the first object 1602) moves in the first AR scene, the first VR client also presents the movement process of the first object 1602 in the first VR scene. As shown in Figure 19, the first object 1602 moves from a position near the right to a position near the middle of the first VR scene.

Please refer to Figure 20, which is a flowchart of a method for collaboratively presenting game objects provided in an embodiment of the present application. The method can be used to implement: when the first real position of the first AR scene includes the first object, the first VR client presents the first object at the first virtual position of the first VR scene. The method can also be used to present motion events of the first object. Among them, the first object is an object that is not controlled by the first AR client and the first VR client. For example, the first object can be a non-player-controlled character, or the first object can be a vehicle in the first AR scene or other objects that are not controlled by the first AR client. It should be noted that the method is not limited to Figure 20 and the specific order described below. It should be understood that in other embodiments, the order of some steps in the method can be interchanged according to actual needs, or some steps can be omitted or deleted. The method includes the following steps:

S2001. The first AR client obtains ninth positioning information corresponding to the first object. The ninth positioning information corresponds to the third real position of the first object in the first AR scene.

The ninth positioning information may be used to indicate the position of the first object in the third coordinate system.

In some implementations, the first AR client may obtain seventh rotation information corresponding to the first object. The seventh rotation information may be used to indicate the rotation angle of the first object in the third coordinate system.

In some implementations, the first object is an object such as a real vehicle in the first AR scene that is not controlled by the first AR client. Then the first AR client obtains the ninth positioning information corresponding to the first object in a manner similar to The method of obtaining the second positioning information corresponding to the first AR client is the same or similar, and the method of obtaining the seventh rotation information by the first AR client can be the same as or similar to the method of obtaining the first rotation information corresponding to the first AR client. . In other embodiments, the first object is a virtual object not controlled by the first AR client, then at least one of the ninth positioning information and the seventh rotation information corresponding to the first object can be determined by the first object. Generated by preset movement strategies. Of course, in practical applications, the first AR client can also obtain at least one of the ninth positioning information and the seventh rotation information corresponding to the first object through other methods.

S2002: The first AR client converts the ninth positioning information into the tenth positioning information.

The tenth positioning information may be used to indicate the position of the first object in the second coordinate system.

In some implementations, the first AR client may convert the seventh rotation information into the eighth rotation information. The eighth rotation information may be used to indicate the rotation angle of the first object in the second coordinate system.

S2003: The first AR client sends the tenth positioning information to the service server.

In some implementations, the first AR client may also send eighth rotation information to the service server.

S2004: The service server sends the tenth positioning information to the first VR client.

In some implementations, the service server may also send the eighth rotation information to the first VR client.

S2005, the first VR client converts the tenth positioning information into the eleventh positioning information.

The eleventh positioning information may be used to indicate the position of the first object in the fourth coordinate system.

In some implementations, the first VR client may convert the eighth rotation information into ninth rotation information. The ninth rotation information may be used to indicate the rotation angle of the first object in the fourth coordinate system.

S2006: The first VR client presents the first object at the first virtual position in the first VR scene indicated by the eleventh positioning information.

In some implementations, the first VR client may present the first object based on the ninth rotation information in the first VR scene.

In some implementations, the first VR client may present the first object based on the ninth rotation information at the first virtual position in the first VR scene indicated by the eleventh positioning information.

S2007: The first AR client acquires new ninth positioning information after the movement of the first object. The new ninth positioning information corresponds to the third real position of the first object after movement in the first AR scene.

In some implementations, the first AR client may obtain new seventh rotation information corresponding to the movement of the first object after the movement.

In some implementations, the first object is an object such as a real vehicle in the first AR scene that is not controlled by the first AR client, and the first object can move on its own. In other implementations, the first object is not controlled by the first AR client virtual object, then the first object moves based on the preset movement strategy corresponding to the first object. Of course, in practical applications, the first object may also move based on other control methods. The embodiments of this application do not limit the reasons for the movement of the first object.

S2008, the first AR client converts the new ninth positioning information into the new tenth positioning information.

In some implementations, the first AR client may convert the new seventh rotation information into new eighth rotation information.

S2009: The first AR client sends new tenth positioning information to the service server.

In some implementations, the first AR client may send new eighth rotation information to the service server.

S2010: The service server sends new tenth positioning information to the first VR client.

In some implementations, the service server may send new eighth rotation information to the first VR client.

S2011, the first VR client converts the new tenth positioning information into the new eleventh positioning information.

In some implementations, the first VR client may convert the new eighth rotation information into new ninth rotation information.

S2012: The first VR client presents the first object at the third virtual position in the first VR scene indicated by the new eleventh positioning information.

In some implementations, the first VR client may present the first object based on the new ninth rotation information in the first VR scene.

In some implementations, the first VR client may present the first object based on the new ninth rotation information at a third virtual position in the first VR scene indicated by the new eleventh positioning information.

In this embodiment of the present application, when the first object moves from the first real position in the first AR scene to the third real position in the first VR scene, the first object also moves from the first virtual position in the first VR scene. Moving to the third virtual position in the first VR scene, also in time, the instantaneous position of the first object in the first AR client and the position changes at different times are visible to the first VR client. It can be understood that the motion status of the first object can be presented synchronously by executing at least part of the above-mentioned steps S2001-S2012 multiple times.

Please refer to Figure 21, which is a flow chart of a method for collaboratively presenting game objects provided by an embodiment of the present application. This method can be used to implement: when the second virtual position of the first VR scene includes a second object, the first The AR client presents the second object in the second real-world position of the first AR scene, and this method can also be used to present motion events of the second object. Among them, the second object is an object that is not controlled by the first VR client. For example, the second object can be a non-player controlled character, etc. It should be noted that this method is not limited to the specific order described in Figure 21 and below. It should be understood that in other embodiments, the order of some steps of the method can be exchanged with each other according to actual needs, or some of the steps can be It can also be omitted or deleted. The method includes the following steps:

S2101. The first VR client obtains twelfth positioning information corresponding to the second object. The twelfth positioning information corresponds to the second virtual position of the second object in the first VR scene.

Wherein, the twelfth positioning information may be used to indicate the position of the second object in the fourth coordinate system.

In some implementations, the first VR client may obtain the tenth rotation information corresponding to the second object. The tenth rotation information may be used to indicate the rotation angle of the second object in the fourth coordinate system.

In some embodiments, the second object is a virtual object that is not controlled by the first VR client, then at least one of the twelfth positioning information and the tenth rotation information corresponding to the second object can be generated by a preset motion strategy corresponding to the second object. Of course, in actual applications, the first VR client can also obtain at least one of the twelfth positioning information and the tenth rotation information corresponding to the second object by other means.

S2102: The first VR client converts the twelfth positioning information into the thirteenth positioning information.

The thirteenth positioning information may be used to indicate the position of the second object in the second coordinate system.

In some implementations, the first VR client may convert the tenth rotation information into eleventh rotation information, and the eleventh rotation information may be used to indicate the rotation angle of the second object in the second coordinate system.

S2103. The first VR client sends the thirteenth positioning information corresponding to the second object to the service server.

In some implementations, the first VR client may send eleventh rotation information to the service server.

S2104: The service server sends the thirteenth positioning information corresponding to the second object to the first AR client.

In some implementations, the service server may send the eleventh rotation information to the first AR client.

S2105: The first AR client converts the thirteenth positioning information into the fourteenth positioning information.

The fourteenth positioning information may be used to indicate the position of the second object in the third coordinate system.

In some implementations, the first AR client may convert the eleventh rotation information into the twelfth rotation information. in, The twelfth rotation information may be used to indicate the rotation angle of the second object in the third coordinate system.

S2106: The first AR client presents the second object at the second real position in the first AR scene indicated by the fourteenth positioning information.

In some implementations, the first AR client may present the second object based on the twelfth rotation angle at a second real-world location in the first AR scene.

In some implementations, the first AR client may present the second object based on the twelfth rotation angle at the second real position in the first AR scene indicated by the fourteenth positioning information.

S2107. The first VR client obtains new twelfth positioning information after the movement of the second object. The new twelfth positioning information corresponds to the fourth virtual position of the second object after movement in the first VR scene.

In some implementations, the first VR client obtains new tenth rotation information corresponding to the second object after the movement.

In some implementations, the second object is a virtual object not controlled by the first VR client, then the second object moves based on the preset movement strategy corresponding to the second object. Of course, in practical applications, the second object may also move based on other control methods. The embodiments of this application do not limit the reasons for the movement of the second object.

S2108: The first VR client converts the new twelfth positioning information into the new thirteenth positioning information.

In some implementations, the first VR client may convert the new tenth rotation information into new eleventh rotation information.

S2109: The first VR client sends new thirteenth positioning information to the service server.

In some implementations, the first VR client may send new eleventh rotation information to the service server.

S2110: The service server sends new thirteenth positioning information to the first AR client.

In some implementations, the service server may send new eleventh rotation information to the first AR client.

S2111. The first AR client converts the new thirteenth positioning information into the new fourteenth positioning information.

In some implementations, the first AR client may further convert the new eleventh rotation information into new twelfth rotation information.

S2112. The first AR client presents the second object at the fourth real position in the first AR scene indicated by the new fourteenth positioning information.

In some implementations, the first AR client may present the second object at a fourth real-world location in the first AR scene based on the new twelfth rotation information.

In some implementations, the first AR client may present the second object based on the new twelfth rotation information at a fourth real-world position in the first AR scene indicated by the new fourteenth positioning information.

In this embodiment of the present application, when the second object moves from the second virtual position in the first VR scene to the fourth virtual position in the first VR scene, the second object also moves from the second real position in the first AR scene. Moving to the fourth real position in the first AR scene, also in time, the instantaneous position of the second object in the first VR client and the position changes at different times are visible to the first AR client. It can be understood that the motion status of the second object can be presented synchronously by performing at least part of the above-mentioned steps S2101-S2112 multiple times.

Please refer to FIG. 22 , which is a flow chart of a method for collaboratively presenting game objects provided by an embodiment of the present application. It can be understood that the following takes the first object in the game scene as an example to illustrate the manner in which the AR client and the VR client collaboratively present the first object and the status update event of the first object. When the first collaborative scene is other type of collaborative scene, or for other objects (including the second object) other than the first object, the AR client and the VR client can collaboratively present the object and the state update event of the object in a similar or identical manner. . It should be noted that this method is not limited to the specific order described in Figure 22 and below. It should be understood that in other embodiments, the order of some steps of the method can be exchanged with each other according to actual needs, or some of the steps can be exchanged. It can also be omitted or deleted. The method includes the following steps:

S2201. The business server sends the identity information corresponding to the first object to the first AR client and the first VR client.

In some embodiments, the business server can determine whether the first object is within the range indicated by the first range information based on the positions of the first object and the first AR client in the second coordinate system and the first range information corresponding to the first AR client, and if so, send the identity information corresponding to the first object to the first AR client.

In some implementations, the business server may determine whether the first object is in the second range information based on the positions of the first object and the first VR client in the second coordinate system and the second range information corresponding to the first VR client. Within the indicated range, if the identity information corresponding to the first object is sent to the first VR client.

Of course, in practical applications, the service server may also determine whether to send the identity information of the first object to the first AR client and/or the first VR client through other methods.

In some implementations, the identity information of the first object may include at least one of a user ID, a name, and a role ID corresponding to the first object. Of course, in practical applications, the identity information of the first object may also include more or less information indicating the identity of the first object.

S2202: The first AR client and the first VR client obtain resource data corresponding to the first object based on the identity information corresponding to the first object.

The resource data corresponding to the first object can be used to present the first object. In some implementations, the resource data may include an avatar corresponding to the first object. In some implementations, the resource data may include props, assets, status, etc. corresponding to the first object. Of course, in actual applications, the resource data may also include more or less data for presenting the first object.

In some implementations, both the first AR client and the first VR client locally store at least part of the resource data corresponding to the first object, then the first AR client can locally obtain at least part of the resource data from the first AR client. Data, the first VR client can obtain at least part of the resource data locally from the first VR client.

In some implementations, the resource server stores at least part of the resource data corresponding to the first object, and then the first AR client and the first VR client can obtain the at least part of the resource data from the resource server.

S2203. The first AR client and the first VR client present the first object based on the resource data corresponding to the first object.

In some implementations, the first AR client and the first VR client may render and display the virtual image corresponding to the first object, thereby presenting the first object. In some implementations, the first object is a real object in the first AR scene, and what is presented by the first AR client is the first object photographed by the first AR client through the camera, or the user of the first AR client can The first object is directly seen, so the first AR client may not present the virtual image corresponding to the first object.

In some implementations, the first AR client and the first VR client may display identity information corresponding to the first object.

In some implementations, the first AR client and the first VR client can display the status of the first object, such as health value, mana value, stamina value, endurance value, remaining items, etc.

S2204: When a status update event of the first object is triggered, the service server notifies the first AR client and the first VR client of the status event of the first object.

In some implementations, the status update event of the first object may be an event triggered in the first AR scene. For example, the first object is a real user corresponding to the first AR client. The real user obtains a prop in the first AR scene, that is, the prop balance of the first object changes; or, the first object is a real user in the first AR scene. For the treasure box in the scene, if the real user corresponding to the first AR client opens the treasure box, the treasure box can change from an unopened state to an open state.

In some implementations, the status update event of the first object may be an event triggered in the first VR scene. For example, the first object is a treasure box, and the player corresponding to the first VR client controls the character to open the treasure box in the first VR scene.

S2205. The first AR client and the first VR client update the status of the first object, that is, present the status update event of the first object.

The first AR client can update the first object presented in the first AR scene, and the first VR client can update the first object presented in the first VR scene.

In some implementations, the first object is a real object, and the first AR client can capture the state change of the first object through the camera, or the user of the first AR client can directly see the state change of the first object, so the first AR client can capture the state change of the first object through the camera. An AR client may not update the state of the first object.

In the embodiment of the present application, the first AR client and the first VR client can obtain the identity information corresponding to the first object, and obtain the resource data corresponding to the first object based on the identity information. The first AR client can present the first object in the first AR scene, and the first VR client can present the first object in the first VR scene, so that the user corresponding to the first AR client and the first VR client can both see the image and status of the first object. When the status update event of the first object is triggered, the first AR client can update the status of the first object in the first AR scene, and the first VR client can update the status of the first object in the first VR scene, so that the user corresponding to the first AR client and the first VR client can both see the status change of the first object, thereby improving the accuracy of the collaborative presentation of the first object.

It can be understood that the business server can send the identity corresponding to the second object to the first AR client and the first VR client. Information, the first AR client and the first VR client can obtain the resource data corresponding to the second object based on the identity information corresponding to the second object, and present the second object based on the resource data corresponding to the second object, so that the first AR Both the user corresponding to the client and the first VR client can see the image and status of the second object. When the status update event of the second object is triggered, the business server can notify the first AR client and the first VR client of the status event of the second object, and the first AR client and the first VR client can present the second object. status update event, thereby updating the status of the second object, so that both the user corresponding to the first AR client and the first VR client can see the status change of the second object, which improves the accuracy of collaborative presentation of the second object.

For example, the first AR scene presented by the first AR client is as shown in Figure 23, and the first VR scene presented by the first VR client is as shown in Figure 24. Among them, the first object is a real user of the first AR client, and the first object is a virtual image in the first VR scene, the second object is a player-controlled character corresponding to the first VR client, and the second object is in the first AR Both the scene and the first VR scene are virtual images.

As can be seen from Figure 23, the upper left corner of the first AR scene also includes identity information such as the avatar A 1603 and character name A 1604 corresponding to the first object 1602, and also includes status information such as the health value A 1605 corresponding to the first object 1602. . The top of the head of the second object 1601 in the first AR scene also includes identity information such as the avatar B 1606 and character name B 1607 corresponding to the second object 1601, and also includes status information such as the health value B 1608 corresponding to the second object 1601. . As can be seen from Figure 24, the upper left corner of the first VR scene also includes identity information such as the avatar B 1606 and character name B 1607 corresponding to the second object 1601, and also includes status information such as the health value B 1608 corresponding to the second object 1601. . The top of the head of the first object 1602 in the first VR scene also includes identity information such as the avatar A 1603 and character name A 1604 corresponding to the first object 1602, and also includes status information such as the health value A 1605 corresponding to the first object 1602. . And the health value A 1605 and other status information corresponding to the first object 1602 is consistent in the first AR scene shown in Figure 23 and the first VR scene shown in Figure 24, and the health value B 1608 and so on corresponding to the second object 1601 The status information is also consistent in the first AR scene shown in FIG. 23 and the first VR scene shown in FIG. 24 .

Please refer to the first AR scene before status update on the left side of Figure 23 and the first VR scene before status update on the left side of Figure 24 . Among them, the health value A 1605 corresponding to the first object 1602 is in a incomplete state, and the health value B 1608 corresponding to the second object 1601 is in a full state.

Please continue to refer to the first AR scene after the status update on the right side of Figure 23 and the first VR scene after the status update on the right side of Figure 24 . When the first object 1602 uses the health value restoration item in the first AR scene, both the first AR scene and the first VR scene can update the health value A 1605 of the first object 1602. In the first AR scene after the status update on the right side of Figure 23 and the first VR scene after the status update on the right side of Figure 24, the health value A 1605 of the first object 1602 is both updated to a full state. When the second object 1601 triggers the health value elimination prop in the first VR scene, both the first AR scene and the first VR scene can update the health value B 1608 of the second object 1601. The status update on the right side of Figure 23 In the first AR scene after the state update and the first VR scene after the status update on the right side of Figure 24, the health value A 1605 of the second object 1601 is updated to the incomplete state.

It should be noted that this embodiment of the present application only uses FIG. 23 and FIG. 24 to describe the possible ways in which the first object and the second object may be presented in the first AR scene and the first VR scene, the status update event of the first object, and the second object. The state update event of the first object and the second object will be described without limiting the manner of the first object and the second object, the state update event of the first object, and the state update event of the second object.

Please refer to FIG. 25 , which is a flow chart of an object interaction method provided by an embodiment of the present application. It can be understood that the interaction method between the first object and the second object will be described below by taking the first object shooting at the second object in the game scene as an example. It should be noted that this method is not limited to the specific order described in Figure 25 and below. It should be understood that in other embodiments, the order of some steps of the method can be exchanged with each other according to actual needs, or some of the steps can be It can also be omitted or deleted. The method includes the following steps:

S2501. The first AR client responds to the attack command of the first object and sends the bullet information corresponding to the first object to the business server.

In some implementations, the first object is an object controlled by the user corresponding to the first AR client, and the first AR client can receive attack instructions submitted by the user.

In some implementations, the first object may not be an object controlled by the user, and the first AR client may obtain the attack instruction from the preset interaction strategy corresponding to the first object. Of course, in actual applications, the first AR client can also obtain the attack instructions of the first object through other methods.

In some implementations, the attack instruction of the first object may be used to indicate bullet information corresponding to the first object.

In some embodiments, the bullet information may include the bullet ID, the position of the bullet in the second coordinate system, and the firing direction of the bullet in the second coordinate system. Of course, in practical applications, the bullet information may also include more or less information, such as one or more of bullet type, bullet flight speed, bullet flight distance, etc.

S2502: The business server sends the bullet information corresponding to the first object to the first VR client.

In some implementations, the service server may send the bullet information corresponding to the first object to the first VR client by broadcasting.

In some implementations, the service server may be based on the second positioning information corresponding to the first AR client (ie, the position of the first AR client in the second coordinate system) and the fifth positioning information corresponding to the first VR client (ie, the position of the first AR client in the second coordinate system). That is, the position of the first VR client in the second coordinate system), determine whether the first VR client is included around the first AR client, and if so, send the bullet information corresponding to the first object to the first VR client.

In some implementations, the business server can perform meshing on the game scene based on the second coordinate system, determine the position of the first VR client in the second coordinate system, and determine whether the first AR client is in the second coordinate system. In the nine-square grid with the position as the center, that is, whether the first AR client is surrounded by the first VR client, when the first AR client is surrounded by the first VR client, the business server can send a message to the first VR client. Bullet information corresponding to the first object.

S2503: The first VR client presents the bullet shot by the first object in the first VR scene based on the bullet information corresponding to the first object.

In some embodiments, the first VR client may localize the bullet information corresponding to the first object, including converting the position of the bullet in the second coordinate system into the position of the bullet in the fourth coordinate system, and converting the firing direction of the bullet in the second coordinate system into the firing direction of the bullet in the fourth coordinate system. The first VR client presents the bullet shot by the first object in the first VR scene based on the bullet ID, the position of the bullet in the fourth coordinate system, and the firing direction of the bullet in the fourth coordinate system.

In some implementations, the first VR client can instantiate a bullet fired by the first object and simulate the trajectory of the bullet. In some ways, the first VR client may present the auditory and/or visual effects of the first object's shooting.

In some implementations, the first AR client can also present the bullet shot by the first object in the first AR scene based on the bullet information corresponding to the first object in the same or similar manner as the first VR client in S2503. .

S2504: When the bullet fired by the first object hits the second object, the first VR client sends the identity information of the second object to the business server.

In some embodiments, if the second object is in the trajectory of a bullet fired by the first object, the bullet hits the second object.

In some implementations, the first VR client can detect whether the bullet fired by the first object hits the second object through a collision detection algorithm. Of course, in practical applications, the first VR client can also detect whether the bullet fired by the first object hits the second object through other methods.

It can be understood that in actual applications, the business server or the first AR client can also determine whether the bullet fired by the first object hits the second object.

S2505: The business server obtains the health value change information corresponding to the second object based on the shooting result of the first object.

In some implementations, the health value change information corresponding to the second object may include the health value change amount and/or the changed health value.

S2506: The business server sends health value change information corresponding to the second object to the first AR client and the first VR client.

In some implementations, the service server may be based on the second positioning information corresponding to the first AR client (ie, the position of the first AR client in the second coordinate system) and the fifth positioning information corresponding to the first VR client (ie, the position of the first AR client in the second coordinate system). That is, the position of the first VR client in the second coordinate system), determine whether the first VR client is surrounded by the first AR client, and if so, send the health value change information corresponding to the second object to the first AR client. .

S2507. The first AR client and the first VR client update the health value of the second object based on the health value change information corresponding to the second object.

The first AR client can present the result of the reduction of the health value corresponding to the second object in the first AR scene, and can also present the auditory effect and/or the visual effect when the health value is reduced. The first VR client can present the result of the second presentation corresponding to the reduced health value in a VR scene, and can also present the auditory effect and/or visual effect when the health value is reduced.

It can be understood that when the business server obtains the bullet information corresponding to the first object, it can send the information to the first VR client according to the The client sends the bullet information in a similar or identical manner, and sends the bullet information corresponding to the first object to more clients, thereby detecting whether the first object hits each of the more clients. When the business server obtains the life value change information corresponding to the second object, it can send the life value change information to more clients in a similar or identical manner to the first AR client, so that The further client may present a state change event of a health value change of the second object.

In some implementations, in response to the second object's attack instruction, the first VR client may send bullet information corresponding to the second object to the business server. The business server sends the bullet information corresponding to the second object to the first AR client. The first AR client presents the bullet fired by the second object in the first AR scene based on the bullet information corresponding to the second object. When the bullet fired by the second object hits the first object, the first AR client sends the identity information of the first object to the business server. The business server obtains the health value change information corresponding to the first object based on the shooting result of the second object. The business server sends health value change information corresponding to the first object to the first AR client and the first VR client. The first AR client and the first VR client update the health value of the first object based on the health value change information corresponding to the first object.

In the above embodiment, when the first AR client and the first VR client access the same game scene, when the first real position of the first AR scene includes the first object, the first VR client A manner in which a first object is presented in a first virtual location of a VR scene, and when a second virtual location of the first VR scene includes a second object, the first AR client presents the first object in a second real location of the first AR scene. Two object methods are described. However, it is understandable that in actual applications, the clients accessing the game scene may also include more AR clients and/or more VR clients.

In other implementations of any of the above embodiments, the first AR client, the first VR client and the second AR client access the same game scene. When the second AR device presents the When the fifth real position includes the third object, the first AR client can present the third object at the fifth real position in the first AR scene, and the first VR client can present it at the fifth virtual position in the first VR scene. The third object, wherein the fifth real position and the fifth virtual position are the same position in the game scene.

In other implementations of any of the above embodiments, when the third event corresponding to the third object occurs in the second AR scene, the first VR client can present the second event in the first VR scene, and the first AR The client can present a third event in the first AR scene.

In other implementations of any of the above embodiments, the third event may include at least one of a status update event of the third object, a movement event of the third object, and an interaction event between the first object and other objects.

In other implementations of any of the above embodiments, the first AR client, the first VR client and the second VR client access the same game scene, and when the second VR scene presented by the second VR device When the sixth virtual position includes the fourth object, the first AR client can present the fourth object at the sixth real position in the first AR scene, and the first VR client can present it at the sixth virtual position in the first VR scene. The fourth object, wherein the sixth real position and the sixth virtual position are the same position in the game scene.

In other implementations of any of the foregoing embodiments, when a fourth event corresponding to a fourth object occurs in the second AR scene, the first VR client may present the fourth event in the first VR scene, and the first AR client may present the fourth event in the first AR scene.

In other implementations of any of the above embodiments, the fourth event may include at least one of a state update event of a fourth object, a fourth object motion event, and an interaction event between the fourth object and other objects.

That is, the first AR client can present the second object from the first VR scene and the third object from the second AR scene in the first AR scene, and the first VR client can present it in the first VR scene. The first object from the first AR client and the fourth object from the second VR scene realize multi-type and multi-client collaborative presentation of the same game scene, and multiple users can play the same game scene through multiple different types of clients. Meet, interact and play with each other in the game scene.

For example, the first AR scene as shown in Figure 26 also includes a third object 1603 from the second AR scene, where the third object 1603 is a real user corresponding to the second AR client. In the first VR scene as shown in Figure 27, a third object 1603 from the second AR scene is also included, and the third object 1603 is a virtual image corresponding to the real user.

Based on the same inventive concept, embodiments of the present application also provide a client. The client includes: a memory and a processor. The memory is used to store the computer program; the processor is used to execute the method described in the above method embodiment when the computer program is called.

The client provided in this embodiment can execute the above method embodiments, and its implementation principles and technical effects are similar, and will not be repeated here. narrate.

Based on the same inventive concept, an embodiment of the present application further provides a chip system, which includes a processor coupled to a memory, and the processor executes a computer program stored in the memory to implement the method described in the above method embodiment.

Embodiments of the present application also provide a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the method described in the above method embodiment is implemented.

An embodiment of the present application also provides a computer program product. When the computer program product is run on a client, the client implements the method described in the above method embodiment when executed.

If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, this application can implement all or part of the processes in the methods of the above embodiments by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. The computer program When executed by a processor, the steps of each of the above method embodiments may be implemented. Wherein, the computer program includes computer program code, which may be in the form of source code, object code, executable file or some intermediate form. The computer-readable storage medium may at least include: any entity or device capable of carrying computer program code to the camera device/client, recording media, computer memory, read-only memory (ROM), random access memory (random access memory, RAM), electrical carrier signals, telecommunications signals, and software distribution media. For example, U disk, mobile hard disk, magnetic disk or CD, etc.

In the above embodiments, each embodiment is described with its own emphasis. For parts that are not detailed or documented in a certain embodiment, please refer to the relevant descriptions of other embodiments.

Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each specific application, but such implementations should not be considered beyond the scope of this application.

In the embodiments provided in this application, it should be understood that the disclosed devices/devices and methods can be implemented in other ways. For example, the apparatus/equipment embodiments described above are only illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or units. Components may be combined or may be integrated into another system, or some features may be ignored, or not implemented. On the other hand, the coupling or direct coupling or communication connection between each other shown or discussed may be through some interfaces, indirect coupling or communication connection of devices or units, which may be in electrical, mechanical or other forms.

It will be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integers, steps, operations, elements and/or components but does not exclude one or more other The presence or addition of features, integers, steps, operations, elements, components and/or collections thereof.

It will also be understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

As used in this specification and the appended claims, the term "if" may be interpreted as "when" or "once" or "in response to determining" or "in response to detecting" depending on the context. ". Similarly, the phrase "if determined" or "if [the described condition or event] is detected" may be interpreted, depending on the context, to mean "once determined" or "in response to a determination" or "once the [described condition or event] is detected ]" or "in response to detection of [the described condition or event]".

In addition, in the description of this application and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

Reference in this specification to "one embodiment" or "some embodiments" or the like means that a particular feature, structure or characteristic described in connection with the embodiment is included in one or more embodiments of the application. Therefore, the phrases "in one embodiment", "in some embodiments", "in other embodiments", "in other embodiments", etc. appearing in different places in this specification are not necessarily References are made to the same embodiment, but rather to "one or more but not all embodiments" unless specifically stated otherwise. The terms “including,” “includes,” “having,” and variations thereof all mean “including but not limited to,” unless otherwise specifically emphasized.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present application, but not to limit it; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: The technical solutions described in the foregoing embodiments can still be modified, or some or all of the technical features can be equivalently replaced; and these modifications or substitutions do not deviate from the essence of the corresponding technical solutions from the technical solutions of the embodiments of the present application. scope.

Claims

A multi-device collaboration method, characterized in that the method includes:

The first AR client presents a first AR scene, and the first AR scene is a first collaborative scene in AR mode;

The first VR client presents a first VR scene, and the first VR scene is the first collaborative scene in VR mode;

When the first real position of the first AR scene includes a first object, the first VR client presents the first object at a first virtual position of the first VR scene, the first real position and the first virtual position is the same position in the first collaborative scene; and/or, when the second virtual position of the first VR scene includes a second object, the first AR client is in that The second object is presented in a second real position of the first AR scene, and the second virtual position and the second real position are the same position in the first collaborative scene.
The method according to claim 1, characterized in that the method further includes any one or more of the following:

When a first event corresponding to the first object occurs in the first AR scene, the first VR client presents the first event in the first VR scene; or,

When a second event corresponding to the second object occurs in the first VR scene, the first AR client presents the second event in the first AR scene.
The method of claim 2, wherein the first event includes any one or more of the following:

a status update event of the first object, or,

The motion event of the first object; or,

Interaction events between the first object and other objects.
The method according to claim 3, characterized in that the motion event of the first object includes the first object moving from the first real position in the first AR scene to a third real position, and the first VR client presents the first event in the first VR scene, comprising:

The first VR client presents the movement process of the first object moving from the first virtual position in the first VR scene to a third virtual position, the third virtual position and the third virtual position. The real location is the same location in the first collaborative scene.
The method according to any one of claims 2-4, characterized in that the second event includes any one or more of the following:

The status update event of the second object; or,

The motion event of the second object; or,

Interaction events between the second object and other objects.
The method of claim 5, wherein the motion event of the second object includes the second object moving from the second virtual position to a fourth virtual position in the first VR scene, so The first AR client presents the second event in the first AR scene, including:

The first AR client presents the movement process of the second object moving from the second real position in the first AR scene to a fourth real position, the fourth real position and the fourth virtual The location is the same location in the first collaborative scene.
The method according to any one of claims 1 to 6, characterized in that the first object is a real object or a virtual combined object, and the first VR client is at the first virtual position of the first VR scene. Presenting the first object includes:

The first VR client presents the virtual image corresponding to the first object in the first virtual position of the first VR scene.
The method according to any one of claims 1 to 7, characterized in that the second object is a virtual object, and the first AR client presents the second object at a second real position of the first AR scene. Objects, including:

The first AR client presents the second object or a combined virtual and real image corresponding to the second object in the second real position of the first AR scene.
The method according to any one of claims 1 to 8, characterized in that the first object is a user of the first AR client, and/or the second object is a user controlled by the first VR client. The player controls the character.
The method according to any one of claims 1 to 9, characterized in that the first AR scene includes at least part of a real scene, and the first VR scene includes a virtual image obtained by processing the at least part of the real scene. Scenes.
The method according to any one of claims 1 to 10, characterized in that, before the first AR client presents the first AR scene, and the first AR scene is the first collaborative scene of the AR mode, the method Also includes:

The first AR client accesses the first collaborative scene based on the real position of the first AR client in the first coordinate system, and the first VR client accesses the first collaborative scene based on the location of the first VR client. The real position in the first coordinate system is connected to the first collaborative scene, where the real position of the first AR client in the first coordinate system is the same as the real position of the first VR client in the first coordinate system. The real position in the coordinate system is in the same AOI area; or,

The first AR client accesses the first collaborative scene based on the real position of the first AR client in the first coordinate system, and the first VR client accesses the first collaborative scene based on the location of the first VR client. The virtual position in the first coordinate system is connected to the first collaborative scene, where the real position of the first AR client in the first coordinate system is the same as the real position of the first VR client in the first coordinate system. The virtual positions in the coordinate system are in the same AOI area.
The method according to any one of claims 1-11, characterized in that the method further includes:

The first AR client presents a third object in the first AR scene, and the first VR client presents a virtual image corresponding to the third object in the first VR scene. The third object is an object included in the second AR scene presented by the second AR client, and the third object is a real object or a combination of real and virtual objects.
The method according to any one of claims 1-12, characterized in that the method further includes:

The first AR client presents a fourth object in the first AR scene, the first VR client presents the fourth object in the first VR scene, and the fourth object is a second Objects included in the second VR scene presented by the VR client, and the fourth object is a virtual object.
The method according to any one of claims 1 to 13, characterized in that the first AR client presents the second object at a second real position of the first AR scene, comprising:

The first AR client obtains the position of the second object in a second coordinate system, and the second coordinate system is a public coordinate system;

The first AR client converts the position of the second object in the second coordinate system into the second real position of the second object in a third coordinate system, where the third coordinate system is a local coordinate system corresponding to the first AR client.
The method of claim 14, further comprising:

The first VR client converts the second virtual position of the second object in the fourth coordinate system into the position of the second object in the second coordinate system. The fourth coordinate is the local coordinate system corresponding to the first VR client;

The first VR client sends the position of the second object in the second coordinate system to the business server;

The first AR client obtains the position of the second object in the second coordinate system, including:

The first VR client obtains the position of the second object in the second coordinate system from the business server.
The method according to any one of claims 1 to 15, characterized in that the first VR client presents the first object in the first virtual position of the first VR scene, including:

The first VR client obtains the position of the first object in a second coordinate system, and the second coordinate system is a public coordinate system;

The first VR client converts the position of the first object in the second coordinate system into the first virtual position of the first object in the fourth coordinate system. The fourth coordinate is the local coordinate system corresponding to the first VR client.
The method of claim 16, further comprising:

The first AR client converts the first real position of the first object in the third coordinate system into the position of the first object in the second coordinate system. The third coordinate is the local coordinate system corresponding to the first AR client;

The first AR client sends the position of the first object in the second coordinate system to the business server;

The first VR client obtains the position of the first object in the second coordinate system, including:

The first AR client obtains the position of the first object in the second coordinate system from the service server.
A multi-device collaboration method, characterized in that it is applied to a first AR device, and the method includes:

Presenting a first AR scene, the first AR scene being a first collaborative scene in AR mode;

When the second virtual position of the first VR scene includes a second object, the second object is presented at a second real position of the first AR scene, and the second virtual position and the second real position are the The same position in the first collaborative scene;

Wherein, the first VR scene is a scene presented by a first VR client, and the first VR scene is the VR mode. The first collaborative scene.
The method of claim 18, further comprising:

When a second event corresponding to the second object occurs in the first VR scene, the first AR client presents the second event in the first AR scene.
The method of claim 19, wherein the second event includes any one or more of the following:

A status update event of the second object; or

The motion event of the second object; or,

Interaction events between the second object and other objects.
A multi-device collaboration method, characterized in that it is applied to the first VR device, and the method includes:

Presenting a first VR scene, the first VR scene being a first collaborative scene in VR mode;

When the first real position of the first AR scene includes the first object, the first object is presented at the first virtual position of the first VR scene, and the first real position and the first virtual position are the The same position in the first collaborative scene;

Wherein, the first AR scene is a scene presented by a first AR client, and the first AR scene is the first collaborative scene in AR mode.
The method according to claim 21, characterized in that, the method further includes:

When a first event corresponding to the first object occurs in the first AR scene, the first event is presented in the first VR scene.
The method of claim 22, wherein the first event includes any one or more of the following:

a status update event of the first object, or,

The motion event of the first object; or,

Interaction events between the first object and other objects.
A client, characterized in that it includes: a memory and a processor, the memory is used to store a computer program; the processor is used to execute the method described in any one of claims 18-20 or the method described in any one of claims 21-23 when calling the computer program.
A computer-readable storage medium with a computer program stored thereon, characterized in that when the computer program is executed by a processor, the method as claimed in any one of claims 18-20 or any of claims 21-23 is implemented. The method described in one item.
A computer program product, characterized in that, when the computer program product is run on an electronic device, it causes the electronic device to execute the method according to any one of claims 18-20 or the method according to any one of claims 21-23. method described in one item.