WO2019109828A1

WO2019109828A1 - Ar service processing method, device, server, mobile terminal, and storage medium

Info

Publication number: WO2019109828A1
Application number: PCT/CN2018/117486
Authority: WO
Inventors: 王晓振
Original assignee: 广州市动景计算机科技有限公司
Priority date: 2017-12-04
Filing date: 2018-11-26
Publication date: 2019-06-13
Also published as: CN108114471B; CN108114471A

Abstract

An AR service processing method, a device, a server, a mobile terminal, and a storage medium. The AR service processing method comprises: receiving geographic location information sent by a mobile terminal, and if it is determined that AR service location information matching the geographic location information has been stored locally, sending an image acquisition instruction to the mobile terminal; receiving image data acquired and returned by the mobile terminal according to the image acquisition instruction; and if the returned image data matches image data corresponding to the AR service location information, sending to the mobile terminal AR service data corresponding to the AR service location information. The invention associates a geographic location, an AR service scene image, and an AR service, such that the AR technology is effectively applied to location-related game applications, thereby improving the user experience.

Description

AR service processing method, device, server, mobile terminal and storage medium

Cross reference

The present application claims the following priority: Application No.: 201711261596.0, entitled: "AR Business Processing Method, Apparatus, Server, and Mobile Terminal", Chinese Patent Application, which is incorporated herein by reference. content.

Technical field

The embodiments of the present invention relate to the field of computer technologies, and in particular, to an AR service processing method, apparatus, server, mobile terminal, and storage medium.

Background technique

AR (Augmented Reality) technology is a new technology that integrates real world information and virtual world information "seamlessly", which is difficult to experience in a certain time and space of the real world (for example, Visual information, sound information, etc.), superimposed on the real information after the simulation, the real environment and the virtual object are superimposed in the same picture or space at the same time.

With the development of AR technology, it is becoming a trend to apply AR technology to various application scenarios. In all applications, game applications account for a large proportion of them, so how to deal with AR technology related data, so that AR technology can be effectively applied to some game applications, for example, location-related In game applications, to enhance the user's gaming experience, it is becoming a research hotspot of AR technology applications.

Summary of the invention

In view of this, an embodiment of the present invention provides an AR service processing method, apparatus, server, mobile terminal, and storage medium, so that AR technology can be effectively applied to location-related processing by processing data related to AR technology. In the game class application.

According to a first aspect of the present invention, an AR service processing method includes: receiving geographic location information sent by a mobile terminal, and if it is determined that an AR service location information that matches the geographical location information is locally stored, Sending an image acquisition instruction to the mobile terminal; receiving image data collected by the mobile terminal according to the image acquisition instruction; and if the returned image data matches image data corresponding to the AR service location information, And transmitting, to the mobile terminal, AR service data corresponding to the AR service location information.

According to a second aspect of the embodiments of the present invention, another AR service processing method is provided, including: acquiring geographic location information of a mobile terminal, transmitting the geographical location information to a server; and receiving the server to confirm that the storage is An image acquisition instruction sent after the geographic location information matches the geographical location information of the AR service; collecting image data according to the image acquisition instruction, transmitting the image data to the server; and receiving the image data by the server The AR service data sent after the image data corresponding to the AR geographic location information is matched; and the AR service display is performed according to the AR service data.

According to a third aspect of the present invention, an AR service processing apparatus includes: a first receiving module, configured to receive geographic location information sent by a mobile terminal, and if it is determined that the local storage is matched with the geographic location information, The AR service location information is sent to the mobile terminal, and the second receiving module is configured to receive the image data collected by the mobile terminal according to the image collection instruction; the first sending module is configured to: And if the returned image data matches the image data corresponding to the AR service location information, the AR service data corresponding to the AR service location information is sent to the mobile terminal.

According to a fourth aspect of the present invention, there is provided another AR service processing apparatus, including: a first acquiring module, configured to acquire geographic location information of a mobile terminal, and send the geographical location information to a server; a module, configured to receive an image collection instruction sent by the server after confirming that the geographic location information of the AR service that matches the geographical location information is stored; and a second sending module, configured to collect image data according to the image collection instruction, Sending the image data to the server; the fourth receiving module is configured to receive the AR service data that is sent by the server after confirming that the image data matches the image data corresponding to the AR geographic location information; And a module, configured to perform AR service presentation according to the AR service data.

According to a fifth aspect of the embodiments of the present invention, a server is provided, including: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface complete each other through the communication bus The memory is configured to store at least one executable instruction that causes the processor to perform an operation corresponding to the AR service processing method of the first aspect.

According to a fifth aspect of the embodiments of the present invention, a mobile terminal is provided, including: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface complete each other through the communication bus The memory is configured to store at least one executable instruction that causes the processor to perform an operation corresponding to the AR service processing method of the second aspect.

According to a sixth aspect of the present invention, there is provided a computer readable storage medium carrying one or more computer instruction programs, the computer instruction program being executed by one or more processors, the one or A plurality of processors execute the AR service processing method as described in the first aspect.

According to a sixth aspect of the present invention, there is provided a computer readable storage medium carrying one or more computer instruction programs, the computer instruction program being executed by one or more processors, the one or A plurality of processors execute the AR service processing method as described in the second aspect.

It can be seen from the above technical solution that when the AR service is implemented, the geographical location information and the image data need to be combined, wherein the image data should be an image of a location indicated by the geographical location information, such as an image of an office foreground, and the image can be described. Implementation scenario of the AR service. If the geographical location indicated by the geographical location information does not belong to a location where the AR service can be set, or if the geographical location indicated by the geographical location information belongs to a location where the AR service can be set, the implementation of the AR service does not exist. The scenario (without matching image data) cannot implement the corresponding AR service. It can be seen that, by using the embodiment of the present invention, the geographic location, the AR service scenario image, and the AR service are associated, so that the AR technology can be effectively applied to the game-related application related to the location, thereby improving the user experience.

DRAWINGS

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the embodiments or the description of the prior art will be briefly described below. Obviously, the drawings in the following description are only It is a few embodiments described in the embodiments of the present invention, and other drawings can be obtained according to the drawings for those skilled in the art.

1 is a flow chart showing the steps of an AR service processing method according to Embodiment 1 of the present invention;

2 is a flow chart showing the steps of an AR service processing method according to Embodiment 2 of the present invention;

3 is a flow chart showing the steps of an AR service processing method according to Embodiment 3 of the present invention;

4 is a flow chart showing the steps of an AR service processing method according to Embodiment 4 of the present invention;

FIG. 5 is a flowchart of steps of an AR service processing method according to Embodiment 5 of the present invention; FIG.

FIG. 6 is a structural block diagram of an AR service processing apparatus according to Embodiment 6 of the present invention; FIG.

FIG. 7 is a structural block diagram of an AR service processing apparatus according to Embodiment 7 of the present invention; FIG.

8 is a schematic structural diagram of a server according to Embodiment 8 of the present invention;

FIG. 9 is a schematic structural diagram of a mobile terminal according to Embodiment 9 of the present invention.

Detailed ways

For a better understanding of the technical solutions in the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. The embodiments are only a part of the embodiments of the embodiments of the invention, and not all of the embodiments. All other embodiments obtained by those skilled in the art should be within the scope of protection of the embodiments of the present invention based on the embodiments in the embodiments of the present invention.

The specific implementation of the embodiments of the present invention is further described below in conjunction with the accompanying drawings.

Embodiment 1

Referring to FIG. 1, a flow chart of steps of an AR service processing method according to a first embodiment of the present invention is shown.

This embodiment describes the AR service processing method provided by the embodiment of the present invention from the perspective of the server. The AR service processing method in this embodiment includes the following steps:

Step S102: Receive geographic location information sent by the mobile terminal, and if it is determined that the AR service location information that matches the geographical location information is stored locally, send an image collection instruction to the mobile terminal.

In this embodiment, if the mobile terminal wants to trigger the AR service, the mobile terminal needs to report the geographical location information to the server first. Through the geographical location information, the server can determine whether the mobile terminal is currently in the use scenario position of the AR service. The higher the accuracy of the geographic location information, the better the effect. Preferably, the accuracy of the geographic location information can be controlled within 10 meters, and more preferably, within 3 meters.

The AR service may be any suitable service that can display the AR effect and/or interact with the AR virtual object.

The AR service location information that is locally stored by the server and matches the geographic location information may be the same information as the geographic location information, such as latitude and longitude coordinates, or may be specific location information generated according to the geographic location information, for example, the XX company front desk (this In this case, when the server receives the geographical location information of the mobile terminal, the server needs to convert the geographic location information into a corresponding specific location, and the like.

If the server determines that the mobile terminal is currently located in the use scenario of the AR service, the image acquisition instruction is sent to the mobile terminal to instruct the mobile terminal to collect the image of the current location for subsequent scene comparison.

Step S104: Receive image data collected by the mobile terminal according to the image acquisition instruction.

After receiving the image acquisition instruction, the mobile terminal controls the camera in the mobile terminal to perform image collection at the current location, and uploads the collected image data to the server. The server receives the image data uploaded by the mobile terminal, and performs subsequent comparison of the usage scenarios.

Step S106: If the returned image data matches the image data corresponding to the AR service location information, the AR service data corresponding to the AR service location information is sent to the mobile terminal.

The server stores, in addition to the AR service location information, image data corresponding to the AR service location information, where the image data is image data of the usage scenario of the AR service, and may include one or more sheets (two and two) The image data of the image of the scene is used. In addition, the server also stores logic code for processing the AR service to implement functions such as receiving, processing, and transmitting information and data.

And determining, according to the geographic location information and the image data sent by the mobile terminal, that the current geographic location and the usage scenario of the mobile terminal match the AR service, and sending the AR service data to the mobile terminal. The AR service data includes but is not limited to: an AR virtual object, an AR virtual item, and operable data for the AR virtual object.

With the embodiment, when the AR service is implemented, the geographic location information and the image data need to be combined, wherein the image data should be an image of a location indicated by the geographic location information, such as an image of an office foreground, and the image can describe the AR. The implementation scenario of the business. If the geographical location indicated by the geographical location information does not belong to a location where the AR service can be set, or if the geographical location indicated by the geographical location information belongs to a location where the AR service can be set, the implementation of the AR service does not exist. The scenario (without matching image data) cannot implement the corresponding AR service. It can be seen that, by using the embodiment, the geographic location, the AR service scene image, and the AR service are associated, so that the AR technology can be effectively applied to the game-related application related to the location, thereby improving the user experience.

Embodiment 2

Referring to FIG. 2, a flow chart of steps of an AR service processing method according to Embodiment 2 of the present invention is shown.

This embodiment describes the AR service processing method provided by the embodiment of the present invention from the perspective of the server.

The AR service processing method in this embodiment includes the following steps:

Step S202: The server receives the geographical location information sent by the mobile terminal, and if it determines that the AR service location information that matches the geographical location information is stored locally, sends an image collection instruction to the mobile terminal.

The server locally stores data required for performing the AR service, including but not limited to: AR service location information, image data corresponding to the AR service location information, AR service data, user information, AR service progress information, and the like.

It should be noted that, when the mobile terminal user uses the AR service for the first time, the foregoing data may not exist. In this case, the user may be prompted to set the AR service, and the location and the usage scene image of the AR service are set by the mobile terminal. Save the set information and data upload server. Among them, it is recommended to use the scene image to collect multiple images from different angles, or to use a video stream to provide a more flexible acquisition angle for the user in subsequent image comparison.

In this step, after receiving the geographical location information of the mobile terminal, the server compares the geographical location information with the locally stored AR service location information to determine whether the current location of the mobile terminal is an AR service location, and if it is an AR service. And sending an image collection instruction to the mobile terminal, instructing the mobile terminal to perform image collection of the usage scenario of the AR service; if not for the AR service location, returning the prompt information to the mobile terminal to prompt the user of the mobile terminal to make an error or prompt The AR service is set up. In another possible manner, if the AR service location is not, that is, the server determines that the AR service location information that matches the geographical location information is not locally stored, the AR service location information is searched and stored. The geographical location information is closest to the AR service location information; the prompt information including the closest AR service location information is sent to the mobile terminal. This is because although there is no AR service at the current location where the mobile terminal is located, when there is a possibility that there is an AR service within a certain distance range, the mobile terminal can choose to participate. Therefore, the corresponding location information of the AR service can be sent to the mobile terminal, and the user of the mobile terminal is guided to participate in the AR service at the location. Optionally, the navigation application may be invoked on the mobile terminal side to navigate the user of the mobile terminal according to the current geographic location information of the mobile terminal and the closest AR service location information.

Step S204: The server receives the image data collected by the mobile terminal according to the image acquisition instruction.

The mobile terminal collects an image of the location according to the image acquisition instruction and uploads corresponding image data, and the server receives the image data uploaded by the mobile terminal.

Step S206: The server determines whether the image data returned by the mobile terminal matches the locally stored image data corresponding to the AR service location information; if yes, step S208 is performed; if not, step S214 is performed.

The image data of the usage scenario of the AR service is stored locally, that is, the image data corresponding to the AR service location information, and the comparison with the image data returned by the mobile terminal can determine whether the mobile terminal is currently in the usage scenario of the AR service. .

In a feasible manner, the server performs image description processing on the image data returned by the mobile terminal to obtain a text description of the image data. The image data stored locally by the server corresponding to the AR service location information is also a textual description of the stored image data, that is, an image description. The server compares the text description returned by the mobile terminal with the stored image description. By performing image description processing on the image data, the data storage burden on the server side is alleviated, and the comparison efficiency of the image data is also improved.

Specifically, the following steps may be used to determine whether the image data returned by the mobile terminal matches the image data corresponding to the AR service location information:

Performing image description processing on the returned image data to obtain a text description corresponding to the image data; if the text description matches the image description corresponding to the AR service location information, determining the image corresponding to the returned image data and the AR service location information The data matches.

The image description process includes: after segmenting the image, the image is divided into several regions, including objects and backgrounds of different features, which may include certain shapes, such as rectangles, circles, curves, and regions of any shape; after the segmentation is completed, Data, symbols, formal languages are used to represent these regions with different characteristics, and a textual description of the image is generated. The description of the image area can be divided into a description of the area itself and a relationship and structure between the areas, including descriptions of various forms such as lines, curves, regions, and geometric features. In the embodiment of the present invention, a person skilled in the art may perform image description processing on the image represented by the image data returned by the mobile terminal by using any suitable image description model and algorithm, which is not limited in the embodiment of the present invention.

Step S208: If the server determines that the returned image data matches the image data corresponding to the AR service location information, the AR service data corresponding to the AR service location information is sent to the mobile terminal.

The AR service data includes: AR service feature data, and the AR service feature data includes at least one of the following: an AR virtual object, an AR virtual item, and operable data for the AR virtual object. The AR virtual object may be any suitable two-dimensional image and/or special effect, three-dimensional image and/or special effect that can be displayed in the mobile terminal, or may be a video or the like, such as a three-dimensional virtual plant or a virtual animal; the AR virtual item may be Any suitable props capable of operating on the AR virtual object, such as a virtual kettle for watering virtual plants, a virtual shovel for shoveling virtual plants, and a virtual de-worm for de-worming virtual plants Insecticide, etc.; the operational data for the AR virtual object is related to the operation and current state of the AR virtual object. For example, if the virtual plant is watered up to five times a day, the initial corresponding to the watering is performed every day. The operational data is 5, when a user waters the virtual plant 1 time, the operational data is updated to 4, and so on. Through the AR virtual object, the AR virtual prop, and the operable data for the AR virtual object, functions such as effective interaction between the user and the virtual object and status update and viewing of the virtual object can be realized in the AR service.

However, the AR service characteristic data may further set other suitable feature data according to actual needs by the person skilled in the art; and, in addition to the foregoing AR service feature data, the AR service data may further include, but not limited to, AR service progress data, AR business user data, and so on.

After receiving the corresponding AR service data, the mobile terminal performs rendering and display of the AR virtual object based on the image acquired by the image acquisition device in real time. Of course, the AR virtual props can also be synchronously rendered and displayed, and the corresponding information prompts can be performed according to the operable data for the AR virtual object.

Step S210: The server receives operation information that the user sends the AR virtual object through the AR virtual item in the AR service data, and updates the operable data of the AR virtual object according to the operation information.

After the mobile terminal performs the corresponding AR service presentation according to the AR service data, the user of the mobile terminal can operate the AR virtual object through the AR virtual item, and feedback the operation information to the server in real time. After receiving the operation information, the server will update the operational data of the AR virtual object according to these operations. Optionally, the AR virtual object and/or the AR virtual tool may also be updated according to the updated operational data, and the updated AR virtual object and/or the AR virtual tool is sent to the mobile terminal for updating by the mobile terminal.

It should be noted that, for a social game application, multiple users participate in the same application through their respective mobile terminals, and these users can operate the same AR virtual object simultaneously or non-simultaneously through their respective mobile terminals, such as simultaneous use. The virtual kettle waters the same virtual plant; or, while one user waters the virtual plant using a virtual kettle through its mobile terminal, another user uses the virtual shovel to move the same virtual plant through its mobile terminal. The operation information is reported to the server, and the server updates the state of the virtual plant in real time according to the operation information and feeds back to the plurality of mobile terminals.

That is, when the mobile terminal includes a plurality of, the server receives operation information of at least one operation performed by the at least one of the plurality of mobile terminals on the AR virtual object by the AR virtual item, according to the at least one operation. The operation information updates the operational data of the corresponding AR virtual object. Alternatively, the AR virtual object and/or the AR virtual item can also be updated simultaneously.

It can be understood from the above that the plurality of mobile terminals may be a plurality of mobile terminals that simultaneously participate in the AR service.

Step S212: The server synchronizes the updated operable data to the mobile terminal, and returns to step S210 to continue the execution.

As described above, for a certain mobile terminal, the server may update the operable data for the AR virtual object according to the operation information uploaded by the mobile terminal, and then synchronize the updated operational data to the mobile terminal. However, in addition to this, the AR virtual object and/or the AR virtual prop can be updated. In this case, the updated AR virtual object and/or the synchronously updated AR virtual object and/or The updated AR virtual item to the mobile terminal.

When the mobile terminal includes more than one, the updated AR virtual object and/or the updated AR virtual item are synchronized to the plurality of mobile terminals while the updated operational data is synchronized to the plurality of mobile terminals. The plurality of mobile terminals display operation information of an operation performed on the AR virtual object, and user information of the mobile terminal corresponding to the operation. For example, user XX waters the XX plant once, user YY fertilizes the XX plant once, and so on.

It can be understood from the above that the mobile terminal includes a plurality of mobile terminals that can simultaneously participate in the AR service.

Thereby, synchronization between AR services in which multiple people participate is achieved.

Step S214: If the server determines that the image data returned by the mobile terminal does not match the image data corresponding to the AR service location information, the image data corresponding to the AR service location information is sent to the mobile terminal, to prompt the mobile terminal to send the image according to the image. Data is used for image acquisition.

When the image data returned by the mobile terminal does not match the image data stored locally by the server, the angle of the image captured by the user of the mobile terminal may be incorrect. Therefore, the image data stored in the server may be sent to the mobile terminal for the mobile terminal. After the user's reference, shoot from the correct angle. By transmitting image data for user reference, the accuracy of the image acquisition by the user using the mobile terminal can be improved, and the user experience is improved.

However, it should be understood by those skilled in the art that, in actual use, if the server determines that the image data returned by the mobile terminal does not match the image data corresponding to the AR service location information, other appropriate processing, such as a prompt error, may be performed.

Embodiment 3

Referring to FIG. 3, a flow chart of steps of an AR service processing method according to Embodiment 3 of the present invention is shown.

This embodiment describes an AR service processing method provided by an embodiment of the present invention from the perspective of a mobile terminal.

Step S302: Acquire geographical location information of the mobile terminal, and send the geographical location information to the server.

When the mobile terminal wants to trigger the AR service, it needs to first send its own geographical location information to the server, so that the server can determine whether the mobile terminal is currently in the use scenario of the AR service. The higher the accuracy of the geographic location information, the better the effect.

Step S304: Receive an image collection instruction sent by the server after confirming the geographical location information.

Specifically, determining, in step S304, the geographic location information may be whether the AR service geographic location information that matches the geographic location information is stored in the confirmation server.

The server stores information related to the AR service, including but not limited to: AR service location information, image data corresponding to the AR service location information, AR service data, user information, AR service progress information, and the like.

After receiving the geographical location information sent by the mobile terminal, and determining that the geographical location information matches the AR service location information stored in the server, the server sends an image collection instruction to the mobile terminal.

For the user of the mobile terminal that uses the AR service for the first time, the location and image of the usage scenario of the AR service can be set by the mobile terminal, and the information or data of the set location and image is sent to the server for storage. The server determines whether it is a setting type message or an application type message according to a message sent by the mobile terminal, thereby determining a subsequent operation. For example, if the message of the setting type is used, the corresponding information or data uploaded by the mobile terminal is saved; if the message of the application type is used, the solution described in this embodiment is executed.

Step S306: Collect image data according to the image acquisition instruction, and send the collected image data to the server.

After receiving the image capturing instruction, the mobile terminal controls the image capturing device in the mobile terminal, such as the camera, to capture the scene at the current location, and uploads the captured image to the server for the server to identify the usage scene of the AR service. Judge.

Step S308: Receive AR service data sent by the server after confirming the image data.

Specifically, in step S308, the confirmation image data may be to confirm whether the image data matches the image data corresponding to the AR geographic location information.

If both the geographical location information and the image data are confirmed by the server, it indicates that the current location and scene of the mobile terminal are matched with the corresponding AR service, and then the AR service data is sent to the mobile terminal, so that the mobile terminal can be based on the local image acquisition device in real time. The captured images are used to render, display, and interact with AR business data.

The AR service data received by the mobile terminal from the server includes, but is not limited to, AR service feature data, and the AR service feature data includes at least one of the following: an AR virtual object, an AR virtual item, and an operable for the AR virtual object. data. However, the AR service characteristic data may further set other suitable feature data according to actual needs by the person skilled in the art; and, in addition to the foregoing AR service feature data, the AR service data may further include, but not limited to, AR service progress data, AR business user data, and so on.

Step S310: Perform AR service display according to the AR service data.

Specifically, in the foregoing step S310, performing AR service display according to the AR service data, specifically includes:

The AR service data is rendered and rendered on the real scene image currently collected by the mobile terminal to implement the AR service display.

After receiving the AR service data, the mobile terminal will render and draw corresponding AR service data, such as AR virtual objects and AR virtual items, based on the collected real-time images, and display them.

Embodiment 4

Referring to FIG. 4, a flow chart of steps of an AR service processing method according to Embodiment 4 of the present invention is shown.

This embodiment describes the AR service processing method provided by the embodiment of the present invention from the perspective of the mobile terminal.

Step S402: The mobile terminal receives a trigger instruction of the AR service.

The triggering instruction may be a triggering instruction triggered by an AR application when the mobile terminal is started in the mobile terminal, or may be a triggering instruction generated when the AR service option in the conventional application is triggered, or may be an instruction generated by other appropriate manners to trigger the generated instruction. The embodiment of the present invention does not limit the triggering manner, the generating manner and the specific form of the triggering instruction.

Step S404: The mobile terminal acquires the geographic location information of the mobile terminal according to the triggering instruction, and sends the geographical location information to the server.

In this embodiment, the accuracy of setting the geographic location information of the mobile terminal is within 10 meters, and preferably within 3 meters.

Step S406: The mobile terminal receives an image collection instruction sent by the server after confirming the geographical location information.

Specifically, determining, in step S406, the geographic location information may be whether the AR service geographic location information matching the geographic location information is stored in the confirmation server.

Optionally, if the server determines that there is no geographical location information sent by the mobile terminal, the server may further search for the AR service location information that is closest to the geographical location information from the stored AR service location information; and send the most The prompt information of the proximity AR service location information is used to guide the user of the mobile terminal to participate in the AR service at the location. In this case, the mobile terminal can invoke the navigation application to navigate the user of the mobile terminal according to the current geographic location information of the mobile terminal and the closest AR service location information. Certainly, if the server determines that there is no geographical location information sent by the mobile terminal, the server may also prompt an error message or prompt the user to set information about the AR service, and the like.

Step S408: The mobile terminal collects image data according to the image acquisition instruction, and sends the collected image data to the server.

Step S410: The mobile terminal receives the AR service data that is sent by the server after confirming the image data.

Specifically, in step S410, the confirmation image data may be to confirm whether the image data matches the image data corresponding to the AR geographic location information.

The AR service data includes: AR service feature data, and the AR service feature data includes at least one of the following: an AR virtual object, an AR virtual item, and operable data for the AR virtual object.

Step S412: The mobile terminal performs AR service presentation according to the AR service data.

Specifically, in the foregoing step S412, the mobile terminal performs the AR service display according to the AR service data, and specifically includes:

For example, after receiving the AR service data sent by the server, the AR virtual object and the AR virtual item are rendered and drawn on the image captured by the mobile terminal in real time, and the like.

Step S414: The mobile terminal receives the update data sent by the server after updating the AR service data, updates the local AR service data according to the update data, and performs AR service display.

In one aspect, for the current mobile terminal, the AR virtual object is operated by using the AR virtual prop according to the operable data, and the operation information of the operation is sent to the server; and further, the receiving server is configured according to the operation information. The updated data stored in the server is updated, and the AR service is displayed according to the updated data.

Optionally, the server may also update the AR virtual object and/or the AR virtual item according to the updated operational data. Based on this, the mobile terminal may receive the updated AR virtual object and/or the AR virtual prop in addition to the updated operable data, and perform AR service display according to the updated data.

On the other hand, for the case of multiple mobile terminals, it is possible that the user of the current mobile terminal does not operate the AR virtual object, but the user of the other mobile terminal performs the operation, or the user of the current mobile terminal and other mobile terminals. The user has operated on the AR virtual object. In either case, the AR service for multi-user participation needs to send corresponding update data to multiple mobile terminals to update the AR service data and update the AR service presentation. Based on this, the current mobile terminal may receive the update data sent by the server, and the update data may include only the updated operational data, or may include the updated AR virtual object and the updated operational data. And/or the updated AR virtual item, the update data is generated by the server according to the operation information of the operation of the AR virtual object by using the AR virtual item by the plurality of mobile terminals including the current mobile terminal; and performing the AR service according to the updated data. Show.

Specifically, the multiple mobile terminals may be multiple mobile terminals that participate in the AR service at the same time; correspondingly, the update data may be at least by the server according to at least one of the plurality of mobile terminals participating in the AR service including the mobile terminal. An operation information generated by a mobile terminal to operate an AR virtual object using an AR virtual prop.

In addition, each mobile terminal may display operation information of operations performed by a plurality of mobile terminals participating in the AR service on the AR virtual object, and user information of the mobile terminal corresponding to the operation. For example, user A fertilizes the XX plant once, user B waters the XX plant once, user C waters the XX plant once, and so on.

Specifically, operation information of an operation performed by any one of the plurality of mobile terminals on the AR virtual object, and user information of the mobile terminal corresponding to the operation may be displayed on each mobile terminal.

Thereby, the update and display of the AR service of the multi-user reference is realized.

Embodiment 5

Referring to FIG. 5, a flow chart of steps of an AR service processing method according to Embodiment 5 of the present invention is shown.

This embodiment describes an AR service processing method provided by an embodiment of the present invention by taking an AR game of a virtual plant as an example from the perspective of interaction between the mobile terminal and the server.

Step S502: The mobile terminal A sets the use location and usage scenario of the AR game for breeding the virtual plant, and uploads the corresponding geographical location information and the scene image data to the server.

The mobile terminal A starts or enters the AR game through the corresponding account. When the AR game is used for the first time, such as the AR game of the virtual plant, the server does not store any data related to the AR game, and the mobile terminal needs to be uploaded and uploaded to the server.

For example, after the mobile terminal A starts up, it sends its GPS data to the server, and then starts the camera according to the image acquisition instruction sent by the server, and sends the camera video stream to the server.

Step S504: The server receives and saves the geographical location information and image data uploaded by the mobile terminal A, generates initialization data of the AR game, and sends the initialization data to the mobile terminal A.

For example, after receiving the GPS data of the mobile terminal A, the server first checks whether the GPS data is stored. In the case of the first use, the GPS data does not exist in the server, saves the GPS data, and sends the image collection to the mobile terminal A. The instruction instructs the mobile terminal A to upload image data; after waiting for the mobile terminal A to upload the video stream, the data of the video stream is associated with the GPS data; after the server obtains sufficient image data from the video stream, the image data and the GPS data are correspondingly The initialization data of the AR game for breeding the virtual plant is generated and generated, and the initialization data is transmitted to the mobile terminal A. Alternatively, the saved image data may be an original image, or feature information of the image, or a textual description of the image.

In general, since the image in the video stream is used to generate an entry for the AR game, it is necessary to acquire sufficient data of the use scene of the AR game through the image. For example, if you use the company front desk to make an AR game portal, you need a continuous image from the left to the right of the company's front desk, so that users who participate in the AR game can normally enter the game when they take images of the company's front desk from different angles. The initialization data of the AR game includes at least an AR virtual object and an AR virtual item. For example, the initialization data of the AR game for breeding the virtual plant in this embodiment may include: an initial 3D model of the plant image, and an interactive setting such as a watering kettle. .

In addition, if there is no GPS data sent by the mobile terminal A in the server, the server may further search for the location closest to the current location of the mobile terminal A according to the stored GPS data and the GPS data of the mobile terminal A, and the GPS data according to the location. A prompt message is sent to the mobile terminal A to prompt the mobile terminal A to the location of the nearest AR game to guide the user to participate in the AR game.

In addition, the AR game data corresponding to each GPS data is also stored in the server, wherein the AR game data includes the user id, the AR game progress, the AR game feature data, and the like. The AR game feature data may be AR game virtual objects, virtual items, AR game operational data, for example, how many pests are on the virtual plant, how much nutrients the plant needs, how much water the plant needs, and the like.

Step S506: The mobile terminal A receives the initialization data of the AR game, and displays the AR game interface according to the initialization data.

After receiving the initialization data returned by the server, the AR game for breeding the virtual plant in the mobile terminal A draws the initialization data onto the image based on the image currently captured by the camera to display the AR game interface for the user.

After the initialization process of the AR game is completed, a plurality of mobile terminals including the mobile terminal A can directly enter the AR game for breeding the virtual plant through the GPS data and the image data of the use scene.

Step S508: The mobile terminal B triggers the AR game, acquires the geographical location information of the mobile terminal B, and sends the information to the server.

For example, the mobile terminal B transmits GPS data to the server.

It should be noted that, in this embodiment, the mobile terminal B is taken as an example to describe the process of performing AR games by other mobile terminals except the mobile terminal A that performs the initialization described above in the case of multiple mobile terminals, and those skilled in the art should understand that In actual use, any mobile terminal participating in the AR game including the mobile terminals A and B can perform the AR game operation of the present embodiment with reference to the operation of the mobile terminal B.

Step S510: The server receives the geographical location information of the mobile terminal B, and matches the geographical location information stored in the server. If the matching is successful, step S512 is performed; if the matching fails, the prompt information is sent to the mobile terminal B, indicating an error.

Step S512: The server sends an image acquisition instruction to the mobile terminal B.

After receiving the GPS data of the mobile terminal B, the server checks whether the GPS data exists in the server, and if so, whether the GPS data has corresponding image data. If there is image data corresponding to the GPS data, the server sends the data to the mobile terminal B. The image acquisition instruction instructs the mobile terminal B to acquire an image, and waits for the mobile terminal B to transmit the acquired image, such as an image in the collected video stream.

Step S514: The mobile terminal B collects an image of the location according to the image acquisition instruction, and sends the image to the server.

For example, after receiving the image acquisition instruction, the mobile terminal B starts the camera and sends the camera video stream to the server.

Step S516: The server receives the image data of the image sent by the mobile terminal B, and matches the image data corresponding to the geographical location information stored in the server. If the matching is successful, step S518 is performed; if the matching fails, the mobile terminal B is performed. Sending a prompt message prompting the user of the mobile terminal B whether there is an AR game usage scenario.

After waiting for the mobile terminal B to return the video stream, the server compares the image data in the video stream with the stored image data.

If the comparison is successful, the operation of the subsequent step S518 is performed; if the comparison fails, the scene prompt is sent to the mobile terminal B. For example, the company front desk is the game entrance, the user of the mobile terminal B does not recognize the server after the company front desk is photographed, but the GPS data of the mobile terminal B has been matched, then the server will prompt the user of the mobile terminal B to take a closer shot, or the server The image data of the company front desk stored in the middle is extracted from the user of the mobile terminal B by a few frames, and the user of the mobile terminal B is prompted to photograph the company front desk according to the angle in the image data.

Step S518: The server sends the AR game data to the mobile terminal B.

In the above step S518, the AR game data transmitted to the mobile terminal B is the AR game data corresponding to the geographical location information.

If the image data in the video stream is successfully compared with the stored image data, the game data stored in the server is transmitted back to the mobile terminal B.

Step S520: The mobile terminal B presents the AR game interface according to the AR game data.

After waiting for the server to pass back the game data, the mobile terminal B presents the game interface to the user according to the game data.

The AR game interface is an interface combining an AR virtual object and a real scene, wherein the AR virtual object is from the AR game data sent by the server, and the real scene is from the camera framing of the mobile terminal B.

Step S522: The mobile terminal B receives the AR game operation performed by the user through the AR game interface, and transmits the operation information to the server.

Step S524: The server updates the AR game data according to the information of the operation, and transmits to the mobile terminals A and B.

Step S526: The mobile terminals A and B receive the updated AR game data and update the game interface.

The AR virtual object responds according to the operation of the user of the mobile terminal B, and the mobile terminal B receives the operation of the user. If the operation is an effective operation of the AR game, the information corresponding to the operation is uploaded to the server, and the server receives the corresponding information. Thereafter, the AR game data is updated and transmitted to the mobile terminal A while being transmitted back to the mobile terminal B. After receiving the updated data, the mobile terminals A and B update the respective AR game data based on the updated data.

For example, in the AR game for breeding virtual plants in this embodiment, the user clicks on the kettle icon to water the virtual plant; the AR game receives the information that the kettle icon is clicked, and sends an event corresponding to the click of the kettle icon to the server, for example, “ The server receives the "water" event. If the virtual plant lacks water for the AR game at this time, the virtual plant water shortage is 4 after clicking the kettle image; the server sends the water shortage to the mobile terminal A and B. Corresponding virtual plant 3d model; AR game receives 3d model change message, acquires new 3d model data, and draws in the game interface of mobile terminal A and B respectively.

If there are other users participating in the AR game, the server will send the updated AR game data to a plurality of mobile terminals that are simultaneously accessed.

Through the present embodiment, an AR game for an office plant is constructed. First, a user uploads the location data of the AR game using the mobile terminal, and starts a camera to scan a specific location corresponding to the location data, such as a company front desk, to initialize the entire AR game. Then, other users use their respective mobile terminals to upload location data and launch the camera to scan the location again, such as the company front desk, to enter the AR game. At the same time, the AR game has the characteristics of multi-person collaboration and sociality, which can effectively meet the social needs of users and enhance the user experience.

It should be noted that the AR service processing solution provided by the embodiment of the present invention is applicable not only to the game application scenario, but also to other AR service scenarios related to the geographic location information.

Embodiment 6

Referring to FIG. 6, a block diagram of a structure of an AR service processing apparatus according to Embodiment 6 of the present invention is shown.

The AR service processing apparatus of this embodiment may be configured in a server, where the AR service processing apparatus includes: a first receiving module 602, configured to receive geographical location information sent by the mobile terminal, and if it is determined that the local geographical location information is stored The matched AR service location information is sent to the mobile terminal, and the second receiving module 604 is configured to receive the image data collected by the mobile terminal according to the image collection instruction. The first sending module 606 is configured to return When the image data matches the image data corresponding to the AR service location information, the AR service data corresponding to the AR service location information is transmitted to the mobile terminal.

Optionally, the apparatus provided by the embodiment of the present invention further includes:

a judging module, configured to perform image description processing on the image data returned by the mobile terminal, and obtain a text description corresponding to the image data; if the text description matches an image description corresponding to the AR service location information, determine the returned image The data matches the image data corresponding to the AR service location information.

Optionally, the AR service data includes: AR service feature data, and the AR service feature data includes at least one of the following: an AR virtual object, an AR virtual item, and operable data for the AR virtual object.

Optionally, the AR service processing apparatus of this embodiment further includes: a first update module 608, configured to: after the first sending module 606 sends the AR service data to the mobile terminal, receive the user sent by the mobile terminal by using the AR virtual item to the AR The operation information of the virtual object is operated, and the operable data of the AR virtual object is updated according to the operation information; the synchronization module 610 is configured to synchronize the updated operational data to the mobile terminal.

Optionally, the first update module 608 is further configured to update the AR virtual object and/or the AR virtual item according to the updated operational data; the synchronization module 610 is further configured to synchronize the updated operational data to the mobile terminal, Synchronize the updated AR virtual object and/or the updated AR virtual item to the mobile terminal.

Optionally, the mobile terminal includes a plurality of; the first update module 608 is configured to receive, by using at least one of the plurality of mobile terminals, operation information of the at least one operation performed by the user on the AR virtual object by using the AR virtual item, according to The operation information of the at least one operation updates the operable data of the corresponding AR virtual object; the synchronization module 610 is configured to synchronize the updated AR virtual object and/or while synchronizing the updated operational data to the plurality of mobile terminals. Or updated AR virtual items to multiple mobile terminals.

Specifically, the multiple mobile terminals may be multiple mobile terminals that participate in the AR service at the same time.

Optionally, the first sending module 606 is further configured to: if the image data returned by the mobile terminal does not match the image data corresponding to the AR service location information, send the image data corresponding to the AR service location information to the mobile terminal to prompt the mobile The terminal performs image acquisition based on the transmitted image data.

Optionally, the first sending module 606 is further configured to: if it is determined that the AR service location information that matches the geographic location information is not locally stored, look up the stored AR service location information and the closest to the geographic location information. AR service location information; sending prompt information including the closest AR service location information to the mobile terminal.

The AR service processing apparatus in this embodiment is used to implement the corresponding server-side AR service processing method in the foregoing multiple method embodiments, and has the beneficial effects of the corresponding method embodiments, and details are not described herein again.

Example 7

Referring to FIG. 7, a block diagram of a structure of an AR service processing apparatus according to Embodiment 7 of the present invention is shown.

The AR service processing apparatus of this embodiment may be disposed in any suitable mobile terminal, and the AR service processing apparatus includes: a first obtaining module 702, configured to acquire geographic location information of the mobile terminal, and send the geographical location information to the server. The third receiving module 704 is configured to receive an image collection instruction sent by the server after confirming the geographical location information; the second sending module 706 is configured to collect image data according to the image collection instruction, and send the image data to the server; The fourth receiving module 708 is configured to receive AR service data that is sent by the server after confirming the image data, and the display module 710 is configured to perform AR service display according to the AR service data.

Specifically, the confirming the geographic location may be: determining whether the AR service geographic location information that matches the geographical location information is stored in the server; the determining the image data may be determining whether the image data corresponds to the AR geographic location information. The image data matches.

Optionally, the display module 710 is configured to render and render the AR service data on the real application scene image currently collected by the mobile terminal to implement the AR service display.

Optionally, the AR service data includes: AR service feature data, where the AR service feature data includes at least one of the following: an AR virtual object, an AR virtual item, and operable data for the AR virtual object.

Optionally, the AR service processing apparatus of this embodiment further includes: a third sending module 712, configured to use the AR virtual prop to the AR according to the operable data after the displaying module 710 performs the AR service display according to the AR service data. The virtual object operates and sends operational information of the operation to the server.

Optionally, the AR data processing apparatus of the embodiment further includes: a fifth receiving module 714, configured to: after the third sending module 712 sends the operation information of the operation to the server, receive the update data sent by the server, where The update data includes, in addition to the updated operational data, an updated AR virtual object and/or an updated AR virtual item, the update data being performed by the server according to multiple movements including the current mobile terminal. The operation information generated by the terminal to operate the AR virtual object by using the AR virtual item is generated. The second update module 716 is configured to perform the AR service display according to the update data.

Specifically, the multiple mobile terminals may be multiple mobile terminals that participate in the AR service at the same time; correspondingly, the update data is used by the server according to the multiple mobile terminals that refer to the AR service, including the current mobile terminal. The operation information generated by the at least one mobile terminal to operate the AR virtual object using the AR virtual item is generated.

Optionally, the display module 710 is further configured to display operation information of operations performed by the multiple mobile terminals on the AR virtual object on the mobile terminal, and user information of the mobile terminal corresponding to the operation.

Specifically, the display module 710 may display, on the mobile terminal, operation information of the operation performed by any one of the plurality of mobile terminals on the AR virtual object, and user information of the mobile terminal corresponding to the operation.

The AR service processing apparatus of this embodiment is used to implement the AR service processing method of the corresponding mobile terminal in the foregoing multiple method embodiments, and has the beneficial effects of the corresponding method embodiments, and details are not described herein again.

Example eight

Referring to FIG. 8, a schematic structural diagram of a server according to Embodiment 8 of the present invention is shown. The specific embodiment of the present invention does not limit the specific implementation of the server.

As shown in FIG. 8, the server may include a processor 802, a Communications Interface 804, a memory 806, and a communication bus 808.

among them:

Processor 802, communication interface 804, and memory 806 complete communication with one another via communication bus 808.

The communication interface 804 is configured to communicate with other servers or mobile terminals.

The processor 802 is configured to execute the program 810. Specifically, the related steps in the foregoing embodiment of the AR service processing method on the server side may be performed.

In particular, program 810 can include program code, the program code including computer operating instructions.

The processor 802 may be a central processing unit CPU, or an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention. The mobile terminal includes one or more processors, which may be the same type of processor, such as one or more CPUs; or may be different types of processors, such as one or more CPUs and one or more ASICs.

The memory 806 is configured to store the program 810. Memory 806 may include high speed RAM memory and may also include non-volatile memory, such as at least one disk memory.

The program 810 may be specifically configured to: the processor 802: receive the geographic location information sent by the mobile terminal, and if it is determined that the AR service location information that matches the geographic location information is stored locally, send an image collection instruction to the mobile terminal. Receiving image data collected according to the image acquisition instruction returned by the mobile terminal; if the returned image data matches the image data corresponding to the AR service location information, transmitting the AR corresponding to the AR service location information to the mobile terminal Business data.

In an optional implementation manner, the program 810 is further configured to cause the processor 802 to determine whether the returned image data matches the image data corresponding to the AR service location information by:

Performing image description processing on the returned image data, and acquiring a text description corresponding to the image data; if the text description matches an image description corresponding to the AR service location information, determining the returned image data and the The image data corresponding to the AR service location information is matched.

In an optional implementation manner, the AR service data includes: AR service feature data, where the AR service feature data includes at least one of the following: an AR virtual object, an AR virtual item, and operable data for the AR virtual object.

In an optional implementation manner, the program 810 is further configured to: the processor 802 is configured to receive, by the mobile terminal, operation information that is operated by the user by using the AR virtual item to operate the AR virtual object, and update the AR virtual object according to the operation information. Operation data; synchronously updated operational data to the mobile terminal.

In an optional implementation, the program 810 is further configured to cause the processor 802 to update the AR virtual object and/or the AR virtual item according to the updated operational data; and the operational data after the synchronization update to the mobile terminal At the same time, the updated AR virtual object and/or the updated AR virtual item are synchronized to the mobile terminal.

In an optional implementation manner, the mobile terminal includes multiple; the program 810 is further configured to enable the processor 802 to receive an operation of the AR virtual object by the AR virtual prop by the mobile terminal, and update the AR virtual object according to the operation. When the data is operable, receiving operation information of at least one operation performed by the at least one of the plurality of mobile terminals by the user on the AR virtual object by the AR virtual item, and updating the corresponding AR virtual according to the operation information of the at least one operation The operable data of the object; the program 810 is further configured to cause the processor 802 to synchronize the updated AR virtual object and/or the updated AR virtual prop to the synchronously updated operational data to the plurality of mobile terminals The plurality of mobile terminals.

In an optional implementation manner, the program 810 is further configured to: when the processor 802 does not match the image data corresponding to the returned AR and the AR service location information, send the image data corresponding to the AR service location information to The mobile terminal is configured to prompt the mobile terminal to perform image collection according to the transmitted image data.

In an optional implementation, the program 810 is further configured to enable the processor 802 to search from the stored AR service location information if it is determined that the AR service location information that matches the geographic location information is not stored locally. The AR service location information closest to the geographic location information; the prompt information including the closest AR service location information is sent to the mobile terminal.

For the specific implementation of the steps in the program 810, refer to the corresponding steps in the foregoing embodiment of the AR service processing method on the server side and the corresponding description in the unit, and details are not described herein. A person skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working process of the device and the module described above may be referred to the corresponding process description in the foregoing method embodiment, and details are not described herein again.

With the embodiment, when the AR service is implemented, the geographic location information and the image data need to be combined, wherein the image data should be an image of a location indicated by the geographic location information, such as an image of an office foreground, and the image can describe the AR. The implementation scenario of the business. If the geographical location indicated by the geographical location information does not belong to a location where the AR service can be set, or if the geographical location indicated by the geographical location information belongs to a location where the AR service can be set, the implementation of the AR service does not exist. The scenario (without matching image data) cannot implement the corresponding AR service. It can be seen that, by using the embodiment of the present invention, the geographic location, the AR service scenario image, and the AR service are associated, so that the AR technology can be effectively applied to the game-related application related to the location, thereby improving the user experience.

Example nine

Referring to FIG. 9, a schematic structural diagram of a mobile terminal according to Embodiment 9 of the present invention is shown. The specific embodiment of the present invention does not limit the specific implementation of the mobile terminal.

As shown in FIG. 9, the mobile terminal can include a processor 902, a communications interface 904, a memory 906, and a communications bus 908.

among them:

Processor 902, communication interface 904, and memory 906 complete communication with one another via communication bus 908.

Communication interface 904 is for communicating with other mobile terminals or servers.

The processor 902 is configured to execute the program 910, and specifically, the related steps in the foregoing embodiment of the AR service processing method on the mobile terminal side.

In particular, program 910 can include program code, the program code including computer operating instructions.

The processor 902 may be a central processing unit CPU, or an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention. The mobile terminal includes one or more processors, which may be the same type of processor, such as one or more CPUs; or may be different types of processors, such as one or more CPUs and one or more ASICs.

The memory 906 is configured to store the program 910. Memory 906 may include high speed RAM memory and may also include non-volatile memory, such as at least one disk memory.

The program 910 may be specifically configured to enable the processor 902 to: obtain geographic location information of the mobile terminal, send the geographic location information to the server, and receive an image collection instruction sent by the server after confirming the geographical location information; The acquisition instruction collects image data, and sends the image data to the server; the AR service data sent by the server after confirming the image data; and the AR service presentation according to the AR service data.

Specifically, the performing the AR service display according to the AR service data includes: rendering and rendering the AR service data on the real application scene image currently collected by the mobile terminal to implement the AR service display.

In an optional implementation manner, the program 910 is further configured to: after the processor 902 performs the AR service presentation according to the AR service data, use the AR virtual item to operate the AR virtual object according to the operable data, and The operational information of the operation is sent to the server.

In an optional implementation, the program 910 is further configured to: after receiving the AR service presentation according to the AR service data, the processor 902 receives the update data sent by the server, where the update data includes the updated operation. In addition to the data, the updated AR virtual object and/or the updated AR virtual item are further included, and the update data is operated by the server to use the AR virtual item to operate the AR virtual object according to the plurality of mobile terminals including the current mobile terminal. The operation information is generated; and the AR service is displayed according to the update data.

In an optional implementation manner, the program 910 is further configured to enable the processor 902 to display operation information of operations performed by the plurality of mobile terminals on the AR virtual object on the mobile terminal, and the user of the mobile terminal corresponding to the operation. information.

For the specific implementation of the steps in the program 910, reference may be made to the corresponding steps in the foregoing embodiment of the AR service processing method on the mobile terminal side, and the corresponding description in the unit is not described herein. A person skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working process of the device and the module described above may be referred to the corresponding process description in the foregoing method embodiment, and details are not described herein again.

Example ten

The embodiment of the present specification further provides a computer readable storage medium carrying one or more computer instruction programs thereon, the one or more processors executing when the computer instruction program is executed by one or more processors The AR service processing method described in any of the above embodiments.

It should be noted that the various components/steps described in the embodiments of the present invention may be split into more components/steps according to the needs of the implementation, or two or more components/steps or partial operations of the components/steps may be combined into one. New components/steps to achieve the objectives of embodiments of the present invention.

The above method according to an embodiment of the present invention may be implemented in hardware, firmware, or implemented as software or computer code that may be stored in a recording medium such as a CD ROM, a RAM, a floppy disk, a hard disk, or a magneto-optical disk, or implemented by The network downloads computer code originally stored in a remote recording medium or non-transitory machine readable medium and stored in a local recording medium so that the methods described herein can be stored using a general purpose computer, a dedicated processor or programmable Such software processing on a recording medium of dedicated hardware such as an ASIC or an FPGA. It will be understood that a computer, processor, microprocessor controller or programmable hardware includes storage components (eg, RAM, ROM, flash memory, etc.) that can store or receive software or computer code, when the software or computer code is The two-dimensional code data processing method described herein is implemented when the processor or hardware accesses and executes. Moreover, when a general purpose computer accesses code for implementing the two-dimensional code data processing method shown herein, execution of the code converts the general purpose computer into a special purpose computer for executing the two-dimensional code data processing method shown herein.

Those of ordinary skill in the art will appreciate that the elements and method steps of the various examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the solution. A person skilled in the art can use different methods to implement the described functions for each particular application, but such implementation should not be considered to be beyond the scope of the embodiments of the invention.

The above embodiments are only used to illustrate the embodiments of the present invention, and are not intended to limit the embodiments of the present invention, and those skilled in the art can also make various kinds without departing from the spirit and scope of the embodiments of the present invention. Variations and modifications, therefore, all equivalent technical solutions are also within the scope of the embodiments of the present invention, and the scope of patent protection of the embodiments of the present invention should be defined by the claims.

Claims

An augmented reality AR service processing method, comprising:

Receiving geographical location information sent by the mobile terminal, if it is determined that the AR service location information that matches the geographical location information is stored locally, sending an image collection instruction to the mobile terminal;

Receiving image data collected by the mobile terminal according to the image acquisition instruction;

And if the returned image data matches the image data corresponding to the AR service location information, the AR service data corresponding to the AR service location information is sent to the mobile terminal.
The method according to claim 1, wherein it is judged whether the returned image data matches the image data corresponding to the AR service location information by:

Performing image description processing on the returned image data, and acquiring a text description corresponding to the image data;

And if the text description matches the image description corresponding to the AR service location information, determining that the returned image data matches the image data corresponding to the AR service location information.
The method according to claim 1 or 2, wherein the AR service data comprises: AR service feature data, the AR service feature data comprising at least one of: an AR virtual object, an AR virtual item, and the AR virtual The actionable data of the object.
The method of claim 3, wherein after the sending the AR service data corresponding to the AR service location information to the mobile terminal, the method further comprises:

Receiving, by the mobile terminal, operation information that the user operates the AR virtual object by using the AR virtual item, and updating the operable data of the AR virtual object according to the operation information;

Synchronizing the updated operational data to the mobile terminal.
The method of claim 4 wherein the method further comprises:

Updating the AR virtual object and/or the AR virtual item according to the updated operational data;

Simultaneously updating the updated AR virtual object and/or the updated AR virtual item to the mobile terminal while synchronizing the updated operational data to the mobile terminal.
The method according to claim 5, wherein the number of the mobile terminals participating in the AR service simultaneously is plural;

And the receiving, by the mobile terminal, the operation information that is performed by the user by using the AR virtual item to operate the AR virtual object, and updating the operable data of the AR virtual object according to the operation information, including:

Receiving, by the at least one of the plurality of mobile terminals, operation information of the at least one operation performed by the user on the AR virtual object by using the AR virtual item, and updating the operation information according to the operation information of the at least one operation Operational data of the AR virtual object;

And the synchronizing the updated AR virtual object and/or the updated AR virtual item to the mobile terminal, including:

Simultaneously updating the updated AR virtual object and/or the updated AR virtual item to the plurality of mobile terminals while synchronizing the updated operational data to the plurality of mobile terminals.
The method of claim 1 wherein the method further comprises:

And if the returned image data does not match the image data corresponding to the AR service location information, sending image data corresponding to the AR service location information to the mobile terminal, to prompt the mobile terminal to send according to the The image data is subjected to image acquisition.
The method of claim 1 wherein the method further comprises:

If it is determined that the AR service location information that matches the geographical location information is not stored locally, the AR service location information that is closest to the geographic location information is searched from the stored AR service location information;

Sending prompt information including the closest AR service location information to the mobile terminal.
An augmented reality AR service processing method, comprising:

Obtaining geographic location information of the mobile terminal, and sending the geographical location information to the server;

Receiving an image collection instruction sent by the server after confirming that the geographic location information of the AR service matching the geographical location information is stored;

Acquiring image data according to the image acquisition instruction, and transmitting the image data to the server;

Receiving, by the server, AR service data that is sent after confirming that the image data matches image data corresponding to the AR geographic location information;

Performing an AR service display according to the AR service data.
The method of claim 9, wherein the performing the AR service presentation according to the AR service data comprises:

The AR service data is rendered and rendered on the real scene image currently collected by the mobile terminal to implement the AR service display.
The method according to claim 9, wherein the AR service data comprises: AR service feature data, the AR service feature data comprising at least one of: an AR virtual object, an AR virtual item, and an AR virtual object. Operational data.
The method of claim 11, wherein after the performing the AR service presentation according to the AR service data, the method further comprises:

And operating, according to the operable data, the AR virtual object by using the AR virtual item, and transmitting operation information of the operation to the server.
The method according to claim 11, wherein after the transmitting the operation information of the operation to the server, the method further comprises:

Receiving update data sent by the server, where the update data includes an updated AR virtual object and/or an updated AR virtual item, in addition to the updated operational data, the update data is Generating, by the server, operation information for operating the AR virtual object by using the AR virtual prop by at least one of the plurality of mobile terminals that participate in the AR service, including the mobile terminal;

The AR service is displayed according to the updated data.
The method of claim 13 wherein the method further comprises:

And displaying, on the mobile terminal, operation information of the operation performed by any one of the plurality of mobile terminals on the AR virtual object, and user information of the mobile terminal corresponding to the operation.
An augmented reality AR service processing device, comprising:

The first receiving module is configured to receive the geographic location information sent by the mobile terminal, and if it is determined that the AR service location information that matches the geographic location information is stored locally, send an image collection instruction to the mobile terminal;

a second receiving module, configured to receive image data collected by the mobile terminal according to the image capturing instruction;

The first sending module is configured to: if the returned image data matches the image data corresponding to the AR service location information, send the AR service data corresponding to the AR service location information to the mobile terminal.
The apparatus of claim 15 wherein said apparatus further comprises:

a judging module, configured to perform image description processing on the returned image data, and obtain a text description corresponding to the image data; if the text description matches an image description corresponding to the AR service location information, determine the return The image data matches the image data corresponding to the AR service location information.
The apparatus according to claim 15 or 16, wherein the AR service data comprises: AR service feature data, the AR service feature data comprising at least one of: an AR virtual object, an AR virtual item, and the AR virtual The actionable data of the object.
The apparatus of claim 17 wherein said apparatus further comprises:

a first update module, configured to: after the first sending module sends the AR service data to the mobile terminal, receive operation information that the user sent by the mobile terminal to operate the AR virtual object by using the AR virtual item Updating the operable data of the AR virtual object according to the operation information;

And a synchronization module, configured to synchronize the updated operational data to the mobile terminal.
The device according to claim 18, wherein

The first update module is further configured to update the AR virtual object and/or the AR virtual item according to the updated operational data;

The synchronization module is further configured to synchronize the updated AR virtual object and/or the updated AR virtual item to the mobile terminal while synchronizing the updated operational data to the mobile terminal.
The apparatus according to claim 19, wherein the number of the mobile terminals participating in the AR service simultaneously is plural;

The first update module is configured to receive, by using at least one of the plurality of mobile terminals, operation information of the at least one operation performed by the user on the AR virtual object by using the AR virtual item, according to the The operation information of the at least one operation updates the operable data of the corresponding AR virtual object;

The synchronization module is configured to synchronize the updated AR virtual object and/or the updated AR virtual item to the plurality of mobile terminals while synchronizing the updated operational data to the plurality of mobile terminals.
The device according to claim 15, wherein the first sending module is further configured to: if the returned image data does not match the image data corresponding to the AR service location information, The image data corresponding to the information is sent to the mobile terminal to prompt the mobile terminal to perform image collection according to the transmitted image data.
The device according to claim 15, wherein the first sending module is further configured to: if it is determined that the AR service location information that matches the geographical location information is not stored locally, from the stored AR service location information Finding AR service location information that is closest to the geographic location information; and transmitting, to the mobile terminal, prompt information including the closest AR service location information.
An augmented reality AR service processing device, comprising:

a first acquiring module, configured to acquire geographic location information of the mobile terminal, and send the geographical location information to a server;

a third receiving module, configured to receive an image collection instruction sent by the server after confirming that the geographic location information of the AR service that matches the geographical location information is stored;

a second sending module, configured to collect image data according to the image capturing instruction, and send the image data to the server;

a fourth receiving module, configured to receive, by the server, AR service data that is sent after confirming that the image data matches image data corresponding to the AR geographic location information;

And a display module, configured to perform AR service presentation according to the AR service data.
The device according to claim 23, wherein

The display module is configured to render and render the AR service data on the real scene image currently collected by the mobile terminal to implement the AR service display.
The apparatus according to claim 23, wherein the AR service data comprises: AR service feature data, the AR service feature data comprising at least one of: an AR virtual object, an AR virtual item, and an AR virtual object. Operational data.
The device of claim 25, wherein the device further comprises:

a third sending module, configured to: after the displaying module performs AR service presentation according to the AR service data, perform operation on the AR virtual object by using the AR virtual item according to the operable data, and Operational information of the operation is sent to the server.
The device of claim 25, wherein the device further comprises:

a fifth receiving module, configured to: after the third sending module sends the operation information of the operation to the server, receive update data sent by the server, where the update data includes an updated operation In addition to the data, the updated AR virtual object and/or the updated AR virtual item are further included, and the update data is used by the server according to the plurality of mobile terminals including the mobile terminal and simultaneously participating in the AR service. Operation information generated by the at least one mobile terminal operating the AR virtual object using the AR virtual item;

The second update module is configured to perform AR service presentation according to the update data.
The device according to claim 27, wherein the display module is further configured to display, on the mobile terminal, an operation of the operation performed by any one of the plurality of mobile terminals on the AR virtual object Information, and user information of the mobile terminal corresponding to the operation.
A server includes: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface complete communication with each other through the communication bus;

The memory is configured to store at least one executable instruction, the executable instruction causing the processor to perform an operation corresponding to the AR service processing method according to any one of claims 1-8.
A mobile terminal includes: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface complete communication with each other through the communication bus;

The memory is for storing at least one executable instruction that causes the processor to perform an operation corresponding to the AR service processing method of any one of claims 9-14.
A computer readable storage medium carrying one or more computer program programs, executed by one or more processors, the one or more processors executing any one of claims 1-8 The AR service processing method described in the item.
A computer readable storage medium carrying one or more computer program programs, executed by one or more processors, the one or more processors executing any one of claims 9-14 The AR service processing method described in the item.