WO2019019974A1

WO2019019974A1 - Augmented reality interaction system, method and device

Info

Publication number: WO2019019974A1
Application number: PCT/CN2018/096679
Authority: WO
Inventors: 李先锋
Original assignee: 广州市动景计算机科技有限公司
Priority date: 2017-07-25
Filing date: 2018-07-23
Publication date: 2019-01-31
Also published as: CN109298776A; CN109298776B

Abstract

Disclosed are an augmented reality interaction system, method and device. The system comprises at least one server and a plurality of clients, wherein the server determines a client to participate in interaction and an actual interaction area; a plurality of clients or some of the clients perform interactive operation on one or more predetermined virtual objects displayed, by means of augmented reality, within the actual interactive area captured via the cameras on their respective screens, wherein the predetermined virtual object is limited by the server to be displayed in the actual interaction area; and the display location of the predetermined virtual object is synchronized by the server in real time to the screens of the plurality of clients participating in the interaction. Therefore, by means of limiting the participants and participation ranges of an interaction activity, the user's sense of participation in the interactive activity is promoted, and feasibility is provided for the implementation of complex interactive content, thereby enhancing the level of fun during interaction and further improving the user's interactive experience.

Description

Augmented reality interactive system, method and device

Technical field

The present invention relates to the field of information technology, and in particular, to an augmented reality interaction system, method and apparatus.

Background technique

With the increasing processing speed and rendering capabilities of mobile devices, people are increasingly using virtual objects (VO) displayed on the mobile terminal for learning, communication or gaming. Especially with the popularization of Augmented Reality (AR) technology, people are more exposed to various virtual objects that can interact with them. The AR technology enhances the reality by superimposing computer-generated virtual objects, scenes, or system prompt information into real scenes. But how to combine AR technology and virtual objects to improve the user's interactive experience is currently the main problem.

Take the AR virtual electronic red envelope game on the market as an example. As shown in Figure 1, the user hides the AR red envelope in any position in the real environment through the "small red envelope" function. The current user or other users can use the "find red envelope" function. View the red packets displayed in a certain area near you on your own terminal device. The user scans the possible red envelope hiding position through the camera of the terminal device, and if the scanned position is correct, the red envelope is obtained.

Through the above-mentioned red envelope-seeking red envelope function, although it can bring a pleasure to the user to a certain extent, the whole red envelope transmission and reception scene is single and boring, and the user participation feeling is not strong. Moreover, there is still a lack of good interaction between different users, between users and red envelopes, between red envelopes and the real world, and it is difficult to bring an impressive interactive experience to users.

Therefore, there is a need for an augmented reality interaction that can bring interest and interaction to the user.

Summary of the invention

The present invention provides an augmented reality interaction scheme, which utilizes augmented reality technology to expand an interactive operation mode for one or more predetermined virtual objects, which can enhance the interactive interest and enhance the user's interactive experience.

According to an aspect of the present invention, an augmented reality interactive system is provided, the system may include at least one server and a plurality of clients, wherein the server may determine a plurality of clients participating in the interaction; the server may also determine the actual interaction Interactive area; multiple clients or some of them are able to interact with one or more predetermined virtual objects that are capable of augmented reality display within the actual interactive area captured by their cameras on their respective screens. The predetermined virtual object may be displayed by the server in the actual interaction area, and the display position of the predetermined virtual object may be synchronized by the server in real time to the screen of the plurality of clients participating in the interaction.

Thus, by defining the number of participants and the actual scope of the AR interaction, it provides feasibility for the implementation of complex interactive scenarios.

The server can determine multiple clients participating in the interaction in a variety of ways. For example, the server may determine multiple clients participating in the interaction according to the qualification conditions specified by the specific or any one or more clients of the plurality of clients, and may determine the multiple clients participating in the interaction according to the predefined conditions of the system. It is also possible to determine a plurality of clients participating in the interaction according to other suitable qualifications.

The qualifications may include, but are not limited to, identifying a client connected to a particular wireless local area network as a plurality of clients participating in the interaction; and/or identifying a client entering the same group of friends as a plurality of clients participating in the interaction; And/or the client that enters the same virtual room is identified as multiple clients participating in the interaction. The server may determine a plurality of clients participating in the interaction by satisfying the qualifications of one or several or any of the above.

Thus, the clients participating in the interaction can be flexibly determined according to the specific application.

Alternatively, the server can determine the actual interaction area in which the interaction takes place in a number of ways. For example, the server may determine the actual interaction area for interaction according to the positioning information or the shooting information of the specific or any one or more of the plurality of clients; and/or the server may also determine the interaction according to the predefined conditions of the system. The actual interaction area; and/or the server can also determine the actual interaction area for interaction based on the coverage of the same WLAN that multiple clients connect to.

As a result, the actual interaction area can be determined by selecting an appropriate method according to the specific application, and the flexibility of interaction can be further improved.

The motion of one or more predetermined virtual objects can be affected by the server or the clients participating in the interaction. For example, one or more predetermined virtual objects move in accordance with a route specified by the server; and/or one or more predetermined virtual objects may also change motion lines in response to virtual operations performed by a plurality of clients or some of them. .

In this way, by the virtual operation of the virtual object by the user, one or more predetermined virtual objects can interact with the user according to the user's idea, which brings more interest to the user.

A server or multiple clients participating in an interaction can interact with one or more predetermined virtual objects. For example, one or more predetermined virtual objects may be issued to a specific interaction area by a specific one or more of the plurality of clients participating in the interaction, and other clients may display one or more of the augmented reality displays in the actual interaction area. Schedule virtual objects to interact.

In this way, the interaction of different identities further enriches the interactive scene and enhances the user experience.

According to another aspect of the present invention, an augmented reality interaction method is provided, which may include: the current client joins an interaction to participate in conforming to the qualification of the participation interaction; and displays the actual interaction area to be interactive on the screen. And displaying the one or more predetermined virtual objects within the actual interactive area of the display, wherein the predetermined virtual objects are displayed by the server within the actual interactive area, and the display positions of the predetermined virtual objects are synchronized in real time to other customers participating in the interaction On the side of the screen.

Alternatively, the qualifications may be defined by the current client, and/or one or more other clients participating in the interaction, and/or by a server implementing the interaction.

Optionally, the qualifying condition may include: identifying a client connected to a specific wireless local area network as a client participating in the interaction; and/or identifying the client entering the same group of friends as a client participating in the interaction; and/or Clients entering the same virtual room are identified as participating clients.

Optionally, the actual interaction area for the interaction may be determined according to the positioning information or the shooting information of the current client and/or one or more other clients; or the actual interaction area for the interaction may be determined according to the predefined conditions of the system; The actual interaction area for interaction can be determined based on the coverage of the same WLAN connected to the interacting client. The way in which the actual interaction area is determined is not limited.

Optionally, the method may further comprise: performing an interaction operation on the augmented reality display one or more predetermined virtual objects in the actual interaction area.

Optionally, the motion of one or more predetermined virtual objects is affected by the virtual operations performed by the current client. For example, the current client issues one or more predetermined virtual objects to the actual interaction area, and the other clients perform further interaction operations on one or more predetermined virtual objects of the augmented reality display in the actual interaction area. Or one or more other clients issue one or more predetermined virtual objects to the actual interaction area, and the current client performs further interaction operations on one or more predetermined virtual objects of the augmented reality display in the actual interaction area.

According to still another aspect of the present invention, an augmented reality interaction device is further provided, including: an interaction joining unit, configured to add an interaction to be participated when the current client meets the qualification condition of the participation interaction; and the interactive area display unit is configured to Displaying on the screen the actual interaction area to be interactive; and a virtual object display unit for displaying the one or more predetermined virtual objects in the actual interactive area displayed, wherein the predetermined virtual object is limited by the server in the actual interaction area Display, and the display position of the predetermined virtual object is synchronized in real time to the screen of other clients participating in the interaction.

Optionally, the qualification is defined by the current client, and/or one or more other clients participating in the interaction, and/or by a server implementing the interaction, and the qualifications include: connecting to a particular wireless local area network The client is deemed to be the client participating in the interaction; and/or the client entering the same group of friends is identified as the client participating in the interaction; and/or the client entering the same virtual room is identified as the client participating in the interaction.

Optionally, the interactive area display unit may determine the actual interaction area for interacting according to the positioning information or the shooting information of the current client and/or one or more other clients; or determine the actual interaction according to the predefined conditions of the system. Interactive area; the actual interaction area for interaction can also be determined based on the coverage of the same WLAN connected to the interacting client.

Optionally, the apparatus may further comprise an interaction operation unit for performing an interaction operation on the augmented reality display one or more predetermined virtual objects in the actual interaction area.

Optionally, the interactive operation unit may be further configured to issue one or more predetermined virtual objects to the actual interaction area; the interaction operation unit may also be used to interact with one or more predetermined virtual objects issued by themselves or other clients. operating.

According to still another aspect of the present invention, there is also provided a graphical user interface system on a portable electronic device having a touch screen display, comprising: an interactive area display interface, in response to the current client joining the interaction to participate, at the current client The screen displays the actual interactive area for interaction; the virtual object interactive interface, in the actual interactive area of the display, augmented reality displays one or more predetermined virtual objects for interactive operations.

According to still another aspect of the present invention, a computing device is provided, comprising: a processor; and a memory having stored thereon executable code that, when executed by the processor, causes the processor to perform the method described above.

According to still another aspect of the present invention, there is also provided a non-transitory machine readable storage medium having stored thereon executable code for causing a processor to perform the method described above when executable code is executed by a processor of an electronic device .

The augmented reality interaction scheme of the embodiment of the present invention combines the AR technology and the virtual object, and expands the interaction operation mode of one or more predetermined virtual objects through certain interaction rules, thereby increasing the interest of the interactive operation. Give users an impressive interactive experience.

DRAWINGS

The above and other objects, features, and advantages of the present invention will become more apparent from the aspects of the embodiments of the invention. The same parts.

Figure 1 shows an example of an existing virtual electronic red envelope.

2 shows a schematic system diagram of an augmented reality interactive system in which the present invention may be implemented.

FIG. 3 shows a schematic flow chart of an augmented reality interaction method in accordance with one embodiment of the present invention.

4 shows a schematic block diagram of an augmented reality interactive device in accordance with one embodiment of the present invention.

Figure 5 shows a schematic block diagram of a computing device in accordance with one embodiment of the present invention.

Detailed ways

Preferred embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While the preferred embodiment of the present invention is shown in the drawings, it is understood that the invention may be embodied in various forms and not limited by the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.

With the development of augmented reality (AR) technology, augmented reality technology has a wide range of applications in areas such as cutting-edge weapons, aircraft development and development, data model visualization, virtual training, entertainment and art. Through the augmented reality technology, the virtual object that does not exist in the real environment can be accurately placed in the real environment, and the virtual object and the real environment are integrated by the display device, and the user is presented with a new environment with realistic sensory effects, thereby making the user Users experience experiences that are often unfeasible or difficult to feel in the real world.

While the operation and interaction of virtual objects broadens the user's cognitive boundaries, the interaction of users with virtual objects is still subject to many limitations, given current state of the art limitations. At present, there is still a lack of a solution that can effectively promote interaction between multiple users, between users and virtual objects, between virtual objects and the real world, and it is difficult to bring an impressive interactive experience to users.

In view of this, the present invention proposes an augmented reality interaction scheme. First, by defining a range of people and interactions participating in the interaction, the augmented reality display scene of the virtual object is determined, and secondly, by allowing the participant to interact with the virtual object, The augmented reality display of virtual objects is more interesting and gives users an impressive interactive experience. The augmented reality interaction scheme associated with the interactive personnel and the scope of interaction of the present invention is applicable to various scenarios, such as business, games, and/or social activities, which can further enhance the participants' good interaction.

The augmented reality interaction scheme of the present invention will be specifically described below with reference to FIGS. 2 through 5 and the embodiments. Description will first be made in conjunction with system block diagram 2 for implementing embodiments of the present invention.

As shown in FIG. 2, the system includes at least one server 20 and a plurality of terminal devices 10. The terminal device 10 can implement information transceiving with the server 20 via the network 40. The server 20 can obtain the content required by the terminal device 10 by accessing the database 30 and enable each terminal device 10 to display one or more predetermined virtualities of the augmented reality display within the actual interactive area captured via its camera on its respective screen. The object interacts.

Network 40 may be a network for information transfer in a broad sense, and may include one or more communication networks, such as a wireless communication network, the Internet, a private area network, a local area network, a metropolitan area network, a wide area network, or a cellular data network. In one embodiment, network 40 may also include a satellite network, thereby transmitting GPS signals of terminal device 10 to server 20.

The terminal device 10 is any suitable electronic device (e.g., a client for accessing a particular networked game) that is available to access the server, preferably a portable electronic device having a camera and display screen, including but not limited to a smart phone, tablet Computer or other portable client, etc. Indirect interactions can also be implemented between the plurality of terminal devices 10 through the server.

Although a plurality of terminal devices 10-1...N and a single server 20 and database 30 are shown in the figure, one or a part of the terminal devices are selected for description (for example, the terminal device 10-1) in the following description. However, those skilled in the art should understand that the above 1...N terminal devices are intended to represent a large number of terminal devices existing in a real network, and the illustrated single server 20 and database 30 are intended to indicate that the technical solution of the present invention relates to a server and a database. Operation. The specific numbered terminal devices, as well as the individual servers and databases, are described at least for convenience of description, and do not imply limitations on specific types or locations.

The system shown in FIG. 2 can be implemented as an augmented reality interactive system, including at least one server and multiple clients, and the server can determine multiple clients participating in the interaction and the actual interaction area for interacting, multiple clients or Some of the clients are able to interact with one or more predetermined virtual objects of the augmented reality display within the actual interactive area captured by their cameras on their respective screens. Wherein, the predetermined virtual object can be displayed by the server in the actual interactive area, and the display position of the predetermined virtual object can be synchronized by the server in real time to the screen of the plurality of clients participating in the interaction. Here, the server is usually a remote server that implements the same set of AR interactive services for a group of clients. Clients are usually closer in physical distance, for example, all in the same room, on the same floor, or in the same public area, or clients that meet predetermined conditions.

It should be noted that, in the embodiment of the present invention, a “client” may refer to a “user” that uses a terminal device, and an application (also referred to as an APP) capable of implementing the technical solution of the present invention may be installed on the terminal device. For example, communication class APP, payment class APP, and the like. In general, "client" and "user" are interchangeable. The "client" in the embodiment of the present invention may also refer to a terminal device.

The "interaction" in which the client participates may refer to an activity initiated by a server or a client that enables multiple clients to participate and interact with predetermined virtual objects. Virtual objects may include, but are not limited to, virtual characters, virtual goods, virtual currency, virtual tools, and the like. In a preferred embodiment, the interactive activity may be a transceiving activity of a virtual electronic red envelope, which is a predetermined virtual object capable of interacting with multiple clients separately.

In addition, in the embodiment of the present invention, one or more virtual objects of the augmented reality display may be displayed on a year-on-year basis, or may be reduced or enlarged by a predetermined ratio, so that the virtual object can enhance the reality display more vividly and enhance the sense of reality.

In order to better understand the present invention, various functions performed by the server according to the present invention will be described in detail as follows.

1. Identify the clients that participate in the interaction

Multiple clients participating in the interaction can be determined by the server, the server can determine the number of clients participating in the interaction, and multiple ways to determine the multiple clients participating in the interaction. For example, the server may determine multiple clients participating in the interaction according to the preset conditions of the system, or may determine the participation interaction according to the specific conditions specified by the specific client of the multiple clients or any one or more clients. Multiple clients, or alternatively, may determine multiple clients participating in the interaction according to other suitable methods. Of course, the term "determined by the server" as referred to herein may also refer to a process that is determined or selected by the initiator of one or more AR interactions and ultimately implemented via the server, in other words, "determined by the server". It is understood as the process of "the client that initiates the interaction determines via its server (ie, the client that initiated the interaction)."

The qualifications may include, but are not limited to, identifying a plurality of clients connected to a particular wireless local area network as multiple clients participating in the interaction, or identifying multiple clients entering the same buddy group as multiple clients participating in the interaction. Or, it is to limit multiple clients entering the same virtual room to multiple clients participating in the interaction, and so on. Other suitable qualifications may also be set, for example, multiple clients belonging to the same area (eg, the same room, the same floor, the same public area, etc.) are defined as multiple clients participating in the interaction, or will belong to the same level. Multiple clients (eg, account level, membership level, etc.) are defined as multiple clients participating in the interaction. The server may also determine a plurality of clients participating in the interaction according to other suitable qualifications or limited manners, which are not enumerated here. Similarly, the above qualifications may also be determined by the client initiating the interaction or selected and implemented through the server.

The server may determine a plurality of clients that satisfy either or both of the qualifications as the plurality of clients participating in the interaction. As a result, multiple clients participating in the interaction can be screened according to specific applications and in various ways to ensure that the interactive activities are fun.

The system preset limit can be the default condition of the system, or it can be a predefined condition of the system each time the interactive activity is initiated. For example, in the transmission and reception scenario of the virtual electronic red envelope, the system may determine, by default, all or a part of the active friends in the group of friends in which the red envelope is issued by the user when the electronic red envelope is issued as the user participating in the interaction. For another example, the system may determine, by default, other users in the same area as the red envelope issuing user as the participating users. For another example, when the user uses a certain payment APP to send and receive electronic red packets, the server may be authorized to read a predetermined number of recent contacts in the predetermined communication application (eg, WeChat, QQ, Weibo, etc.) installed on the terminal device. Users who participate in the interaction. For another example, in a scenario where a merchant issues a virtual currency or a coupon, the merchant can determine a user who meets the predetermined user rating as a user participating in the interaction via the system setting, and the like.

A specific client and any client can be understood as two relative concepts. For example, it may refer to a difference in the identity, status, etc. of the client (user) that defines the qualification, or that the role played by the client in the interactive activity is different or different.

A specific client may refer to a client with a specific identity among a plurality of clients participating in the interaction, for example, one or more clients that initiate an interaction. Specific clients can specify qualifications, and specific clients can also specify specific interaction rules and implement the above qualifications and operational rules via the server. For example, in a virtual electronic red envelope transmission and reception scenario, a red envelope issuance user can be regarded as having a specific identity, which can select other clients participating in the interaction, and can also determine a part of users who can perform an interactive operation (eg, can determine the management user, so that Some of the plurality of clients participating in the interaction can interact with one or more predetermined virtual objects, and can also specify other users to interact with the electronic red envelope, for example, whether the amount of the electronic red envelope can be changed, The number, the form of reception, or the properties or status settings of the electronic red envelope, and so on.

Any client can refer to any one of the multiple clients participating in the interaction. In this case, multiple clients can have equal identities with each other. For example, when participating in the same merchant or system interaction, all participating users have equal status, and they can all obtain virtual currency and virtual coupons issued by the system. For another example, any person in the same buddy group or virtual room can issue a red envelope and a rule that stipulates participation in the interaction. The above “equal identity” can be understood as “equality” of the same participants rather than the initiator. In the specific interaction operation, the “equal identity” users can obtain different levels of virtuality according to their user level, activity level, participation time, etc. Items, for example, high-level users get more virtual currency and so on.

Thus, the server can determine a plurality of users participating in the interaction via a variety of methods or qualifications. Engaged users or participating interaction rules can make the interaction between the system and the user, between multiple users, between the user and the virtual object, and between the virtual object and the real world. Make the user's sense of participation stronger and fully enhance the user's interactive experience.

2. Identify the actual interaction area for interaction

The actual interaction area for interaction refers to the real activity range in which one or more predetermined virtual objects are superimposed to the real world. The predetermined virtual object can be displayed by the server in the actual interaction area, and the display position of the predetermined virtual object can also be real-time by the server. Sync to the screen of multiple clients participating in the interaction.

The actual interaction area can be determined by the server. In a specific application, the server can determine the actual interaction area for interaction in a variety of ways. Similarly, the above "determined by the server" can also be understood as the process of "initiating an interactive client to implement its (ie, initiating an interactive client) determination via a server.

The server can determine the actual interaction area for interaction based on the specific information of the specific client or the client or any one of the multiple clients. For example, in the transmission and reception scenario of the virtual electronic red envelope, the server may obtain the geographical location information of the red envelope issuer and determine the predetermined area near the location as the actual interactive area; or the server may also read the area photographed or uploaded by the user. The image information determines the area determined by the analyzed image information as the actual interaction area of the electronic red envelope. In the same scenario, the actual interaction area determined by the positioning information or the shooting information may be the same or different.

The server may also determine the actual interaction area for interaction according to the preset conditions of the system, or may also determine the actual interaction area for interaction according to the coverage of the same WLAN connected by multiple clients, or may also be based on the camera of the terminal device. The range that the device can capture determines the actual interaction area in which the interaction takes place, and so on. The actual interaction area in which the interaction is performed may also be determined in other ways in the embodiments of the present invention, and will not be enumerated here.

Thus, the actual interaction area of the virtual object is determined by various limited manners, and in combination with the above definition of the participating users, the solution completes the determination of the participation scope of the interactive activity (ie, the activity participant and the activity development area). The present invention limits complex AR interactions to a more specific and specific limited scope than existing schemes in which the participating regions are not limited and/or participate in the user's unlimited, thereby providing a viable subsequent interactive event. Sex. By limiting the participants and participation of interactive activities (eg, interactive games), the sense of participation in interactive activities (eg, interactive games) can be enhanced, thereby enhancing the fun of the interaction and improving the user's interactive experience.

3. Interactive operation on virtual objects

As described above, the predetermined virtual object can be displayed by the server in the actual interactive area, and the display position of the predetermined virtual object can also be synchronized by the server in real time to the screen of the plurality of clients participating in the interaction. In this way, the virtual object moving in the actual interactive area can be photographed via a camera device (eg, a camera) and augmented reality display on the display screen of the terminal device, and the user can see one or more virtual objects of the augmented reality display.

A server or multiple clients participating in an interaction can interact with one or more predetermined virtual objects.

Specifically, one or more predetermined virtual objects may be distributed to the actual interaction area by a specific one or more of the plurality of clients participating in the interaction, and the other clients may display one or more of the augmented reality display in the actual interaction area. Scheduled virtual objects to interact.

The specific "specific one or more clients" and "other clients" are two relative concepts. For example, one or more predetermined virtual objects may be issued to the actual interactive area by the current client (eg, the activity originating client), and the other clients further perform one or more predetermined virtual objects of the augmented reality in the actual interactive area. Interactive operation. One or more predetermined virtual objects may also be issued by one or more other users to the actual interactive area, the current client (eg, the active participating client), one or more predetermined virtual objects displayed by the augmented reality within the actual interactive area. Conduct further interactions.

The interaction operation refers to an interaction operation between a plurality of clients or a part of the clients and one or more predetermined virtual objects, and may include, but is not limited to, a client setting or motion setting of the virtual object, and the predetermined virtual object may respond to Interact with the corresponding commands. This can increase the fun of interaction.

Client interactions can affect the properties of virtual objects. For example, virtual electronic red envelopes can have different forms, and the client can change the shape of the virtual electronic red envelope from a rectangle to a heart shape to interact in a more interesting and interesting way. As another example, the client can also scale the size of the electronic red envelope or hide it in a corner of the actual interactive area and meet certain conditions (eg, a specific operation at the correct location or a length of attention long enough). In this way, when other users find the red envelope after much searching, they will be more excited and have a sense of accomplishment.

The one or more predetermined virtual objects may stay in a predetermined location within the actual interaction area, or may move within the actual interaction area, and their status or motion may be affected by the server or multiple clients participating in the interaction.

The interactive operation of the client may be to change the state of the virtual object, for example, to change it from a stationary state to a motion state, or to change it from a motion state to a stationary state. The interactive operation of the client may also be to cause a plurality of virtual objects to float in the actual interaction area, and to change the floating position regularly or randomly.

The interactive operation of the client can affect the movement path of the virtual object. For example, one or more predetermined virtual objects may move according to a route specified by the server, or may change a motion line in response to a virtual operation performed by a plurality of clients or some of the clients thereof; or may also respond to the client Interacting with them in the actual interactive area, or covering the participants' faces.

The interactive operation of the client may also include changing the virtual object to its own by a predetermined virtual operation. For example, you can find or capture virtual electronic red envelopes, virtual currency, virtual goods, and so on by clicking and other means.

In this way, by enabling a plurality of users to perform richer interactive operations on the virtual objects, the user can truly participate in the interactive activities, giving the user a deep impression and enhancing the user's interactive experience.

Heretofore, the technical solutions of the present invention can be clearly understood by the above respective descriptions.

The specific implementation and application of the present invention will be described in detail by way of example with the virtual electronic red envelope as the virtual object as follows.

Embodiment 1

When a company opens its annual meeting, the annual meeting organizers can create a virtual room and limit the number of rooms N, and the first N users entering the virtual room as interactive participants. The server can obtain the positioning information of the organization of the annual meeting, and by default, the area within 100 meters of the location of the annual meeting organizer is used as the actual interactive area, allowing users in the range to participate in the interaction. Multiple users participating in the interaction can see the electronic red envelope of the augmented reality display on the screen through the mobile phone camera in the interactive area. A plurality of participating users or some of them can perform virtual operations on the electronic red envelope, for example, can make it move in various ways, such as flying around, stealing dishes on the table, or covering the face of the user who participates in the interaction. Multiple users can find the electronic red envelope of the sport in the actual interactive area, and capture it by clicking or sliding on the screen to compete for the electronic red envelope.

Embodiment 2

When a friend is gathering, any one of the friends can first create a virtual room or group of friends (or a virtual room or group of friends that already existed before) and pull all the friends involved. Each person can take or upload a photo of the actual meeting place, and the server can determine the actual active area of the virtual electronic red envelope according to the characteristics of the photo, that is, the actual interaction range. When one of them issues a red envelope, friends can see that the electronic red envelope is active in the activity area just defined. Friends or a few of them can perform virtual operations on the electronic red envelope, or capture red envelopes by clicking or sliding.

So far, the augmented reality display system implementing the technical solution of the present invention has been described in detail with reference to FIG. 2 and the embodiment. In addition, the technical solution of the present invention can also be implemented as an augmented reality interaction method. FIG. 3 shows a schematic flow chart of an augmented reality interaction method in accordance with one embodiment of the present invention. The method can be implemented by the augmented reality interaction system shown in FIG. 2, and the augmented reality interaction method is briefly described as follows.

As shown in FIG. 3, in step S310, the server determines whether the current client meets the qualification condition. If the judgment result is yes, that is, the current client meets the qualification condition of the participation interaction, the current client joins the interaction to participate. If the result of the determination is no, that is, the current client does not meet the qualification condition of the participation interaction, the process returns to step S310, and the client other than the current client is used as the new current client to determine whether the new current client meets the qualified condition.

The current client refers to the client of any user, which can be a friend or contact of a user, a client of a certain merchant or system, or any player of the game, and the like.

The qualification may be defined by the current client, or by one or more other clients participating in the interaction, or by a server implementing the interaction. The qualification may satisfy one, or several items, or any one of the following: a client connected to a specific wireless local area network is identified as a client participating in the interaction; a client entering the same friend group is identified as a client participating in the interaction The client that enters the same virtual room is identified as the participating client.

The server can determine whether the current client meets the predetermined qualification condition, and join the current client to the interaction when the current client meets the predetermined qualification condition.

In step S320, the actual interactive area in which the interaction is to be performed is displayed on the screen.

After the current client joins the interaction, the actual interaction area for interaction is displayed on the current client's screen.

The actual interaction area may be determined by the server according to the positioning information or the shooting information of the current client and/or one or more other clients; or may be determined by the server according to the predefined conditions of the system; or may be performed by the server according to the participation interaction. The coverage of the same WLAN connected by the client is determined. The way in which the actual interaction area is determined is not limited.

The actual interactive area can be displayed completely on the screen or partially, and the user can display other parts of the actual interactive area on the screen by moving or rotating the mobile client.

At step S330, the augmented reality displays one or more predetermined virtual objects within the actual interactive area of the display.

The predetermined virtual object can be displayed by the server within the actual interactive area, and the display position of the predetermined virtual object can also be synchronized in real time to the screen of other clients participating in the interaction. In this way, multiple users participating in the interaction can see one or more virtual objects displayed by the augmented reality, giving the user a realistic sensory effect.

Additionally, in one embodiment, the method can further include an interactive operation step S340 of interacting with one or more predetermined virtual objects that augment the reality display within the actual interaction area. The interaction operation may be performed by multiple clients participating in the interaction or some of the clients, and the interaction operations may include, but are not limited to, virtual operations on the virtual object, for example, the shape, attributes, motion, display form, etc. of the virtual object may be changed. Wait.

The motion of one or more predetermined virtual objects can be affected by the virtual operations that the current client is making. For example, the current client issues one or more predetermined virtual objects to the actual interaction area, and the other clients perform further interaction operations on one or more predetermined virtual objects of the augmented reality display in the actual interaction area. Or one or more other clients issue one or more predetermined virtual objects to the actual interaction area, and the current client performs further interaction operations on one or more predetermined virtual objects of the augmented reality display in the actual interaction area.

For the specific implementation of the foregoing steps, refer to the detailed description in FIG. 2, and details are not described herein again. So far, the augmented reality interaction scheme of the present invention has been described in detail with reference to FIGS. 2-3 and the embodiments. The augmented reality interaction scheme of the present invention can also be implemented by an augmented reality interactive device.

4 shows a schematic block diagram of an augmented reality interactive device in accordance with one embodiment of the present invention. The functional modules of the interactive device 400 may be implemented by hardware, software or a combination of hardware and software that implements the principles of the present invention. Those skilled in the art will appreciate that the functional blocks depicted in FIG. 4 can be combined or divided into sub-modules to implement the principles of the above described invention. Accordingly, the description herein may support any possible combination, or division, or further limitation of the functional modules described herein.

The interactive device 400 shown in FIG. 4 can be used to implement the augmented reality interaction scheme shown in FIG. 2-3. In the following, only the functional modules that the interactive device 400 can have and the operations that can be performed by the functional modules are briefly described. For details of the details, refer to the description above with reference to FIG. 2-3, and details are not described herein again.

As shown in FIG. 4, the augmented reality interactive device of the present invention may include an interactive joining unit 410, an interactive area display unit 420, and a virtual object display unit 430.

The interactive joining unit 410 can be used to join the current client to participate in the interaction that meets the participation constraints. The qualification may be defined by the current client, one or more other clients participating in the interaction, and/or by the server implementing the interaction, and the qualification may include: identifying the client connected to the particular wireless local area network as participating An interactive client; and/or a client that enters the same buddy group as a participating client; and/or a client that enters the same virtual room as a participating client.

The interactive area display unit 420 can be used to display the actual interactive area on the screen where the interaction is to be performed. The interactive area display unit 420 may determine the actual interaction area for the interaction according to the positioning information or the shooting information of the current client and/or one or more other clients; or determine the actual interaction area for the interaction according to the predefined conditions of the system; It is also possible to determine the actual interaction area in which the interaction takes place based on the coverage of the same WLAN connected to the interacting client.

The virtual object display unit 430 can be configured to display one or more predetermined virtual objects in the actual interactive area of the display, wherein the predetermined virtual object is limited by the server to be displayed in the actual interactive area, and the display position of the predetermined virtual object is synchronized in real time. Go to the screen of other clients participating in the interaction.

In one example, the interactive device can also include an interactive operating unit 440.

The interactive operation unit 440 can be used to perform an interactive operation of displaying one or more predetermined virtual objects in the augmented reality in the actual interaction area. In one embodiment, the interactive operation unit 440 may issue one or more predetermined virtual objects to the actual interaction area, and the interaction operation unit 440 may also perform interaction on one or more predetermined virtual objects issued by themselves or other clients. .

The invention also discloses a graphical user interface system on a portable electronic device with a touch screen display, comprising: an interactive area display interface, in response to the current client joining the interaction to participate, displaying an interaction on the current client's screen The actual interaction area; the virtual object interaction interface, in the actual interaction area displayed, augmented reality displays one or more predetermined virtual objects for interactive operations. The participating interactive and actual interaction areas may be determined by the server or the activity initiator via the server as described above, thereby providing feasibility for enriching the interactive content.

The present invention also discloses a computing device that implements the augmented reality interactive scheme of the present invention. Figure 5 shows a schematic block diagram of a computing device in accordance with one embodiment of the present invention.

As shown in FIG. 5, computing device 500 of the present invention can include a processor 510 and a memory 520.

Executable code may be stored on the memory 520, and when the executable code is executed by the processor 510, cause the processor to perform the augmented reality interaction method of the present invention.

In one example, the computing device 500 can also include a display screen 530 or can be connected to a display screen. The actual interactive area for interaction can be displayed on the display screen 530, and the augmented reality display of one or more predetermined virtual objects in the actual interactive area can also be displayed.

The augmented reality interactive system, method and apparatus according to the present invention have been described in detail above with reference to the accompanying drawings.

Furthermore, the method according to the invention may also be embodied as a computer program or computer program product comprising computer program code instructions for performing the various steps defined above in the above method of the invention.

Alternatively, the present invention may be embodied as a non-transitory machine readable storage medium (or computer readable storage medium, or machine readable storage medium) having stored thereon executable code (or computer program, or computer instruction code) When the executable code (or computer program, or computer instruction code) is executed by a processor of an electronic device (or computing device, server, etc.), causing the processor to perform various steps of the above method in accordance with the present invention .

The various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or combinations of both.

The flowchart and block diagrams in the Figures illustrate the architecture, functionality and operation of possible implementations of systems and methods in accordance with various embodiments of the present invention. In this regard, each block of the flowchart or block diagram can represent a module, a program segment, or a portion of code that includes one or more of the Executable instructions. It should also be noted that in some alternative implementations, the functions noted in the blocks may also occur in a different order than the ones in the drawings. For example, two consecutive blocks may be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending upon the functionality involved. It is also noted that each block of the block diagrams and/or flowcharts, and combinations of blocks in the block diagrams and/or flowcharts, can be implemented in a dedicated hardware-based system that performs the specified function or operation. Or it can be implemented by a combination of dedicated hardware and computer instructions.

The embodiments of the present invention have been described above, and the foregoing description is illustrative, not limiting, and not limited to the disclosed embodiments. Numerous modifications and changes will be apparent to those skilled in the art without departing from the scope of the invention. The choice of terms used herein is intended to best explain the principles, practical applications, or improvements of the techniques in the various embodiments of the embodiments, or to enable those of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

An augmented reality interactive system comprising at least one server and a plurality of clients, wherein

The server determines a plurality of clients participating in the interaction;

The server determines the actual interaction area in which the interaction takes place;

The plurality of clients or portions of the clients interacting on their respective screens with one or more predetermined virtual objects of the augmented reality display within the actual interactive area captured via their cameras, wherein the predetermined virtual The object is displayed by the server in the actual interactive area, and the display position of the predetermined virtual object is synchronized by the server in real time to the screen of the plurality of clients participating in the interaction.
The interactive system of claim 1 wherein said server determines that a plurality of clients participating in the interaction comprises:

Determining, by the server, a plurality of clients participating in the interaction according to a qualification specified by a specific one or any one of the plurality of clients; or

The server determines a plurality of clients participating in the interaction according to a predefined condition of the system.
The interactive system of claim 1 or 2, wherein the qualifications include:

Identifying a client connected to a particular WLAN as the plurality of clients participating in the interaction; and/or

Identifying the clients entering the same buddy group as the plurality of clients participating in the interaction; and/or

Clients entering the same virtual room are identified as the plurality of clients participating in the interaction.
The interactive system of any of claims 1-3, wherein the server determines that the actual interaction area in which the interaction is to be performed comprises:

Determining, by the server, the actual interaction area for interacting according to positioning information or shooting information of a specific one or any one of the plurality of clients; and/or

Determining, by the server, the actual interaction area for interacting according to a predefined condition of the system; and/or

The server determines the actual interaction area for interaction based on the coverage of the same wireless local area network to which the plurality of clients are connected.
The interactive system of any one of claims 1 to 4, wherein the one or more predetermined virtual objects move in accordance with a route specified by the server; and/or

The one or more predetermined virtual objects change motion lines in response to virtual operations performed by the plurality of clients or a portion of the clients thereof.
The interactive system of any one of claims 1 to 5, wherein the one or more predetermined virtual objects are issued to the one or more of the plurality of clients participating in the interaction to the The actual interaction area, the other client interacts with the one or more predetermined virtual objects displayed by the augmented reality in the actual interaction area.
An augmented reality interaction method that includes:

Join the interaction to participate when the current client meets the qualifications for participation in the interaction;

Displaying the actual interaction area to interact on the current client screen;

Augmented reality displays one or more predetermined virtual objects within the actual interactive area displayed, wherein the predetermined virtual objects are displayed by the server within the actual interactive area, and the display position of the predetermined virtual objects is Real-time sync to the screen of other clients participating in the interaction.
The method of claim 7 wherein said defining condition is by said current client, and/or one or more other clients participating in said interaction, and/or by a server implementing said interaction limited.
The method of claim 7 or 8, wherein said defining conditions comprise:

Identifying clients connected to a particular WLAN as participating clients; and/or

Identifying clients that enter the same buddy group as participating clients; and/or

Clients entering the same virtual room are identified as participating clients.
The method of any of claims 7-9, wherein

Determining the actual interaction area for interaction based on positioning information or shooting information of the current client and/or one or more other clients; and/or

Determining the actual interaction area in which the interaction is performed according to system-defined constraints; and/or

The actual interaction area in which the interaction is performed is determined based on the coverage of the same WLAN connected to the interacting client.
The method of any of claims 7-10, further comprising:

An interactive operation of displaying one or more predetermined virtual objects in the augmented reality in the actual interaction area.
The method of any of claims 7-11, wherein the motion of the one or more predetermined virtual objects is affected by virtual operations performed by the current client.
The method according to any one of claims 7 to 12, wherein the current client issues the one or more predetermined virtual objects to the actual interaction area, and the other clients augment the reality in the actual interaction area Displaying the one or more predetermined virtual objects for further interaction, or

And one or more other clients issues the one or more predetermined virtual objects to the actual interaction area, and the current client further performs the one or more predetermined virtual objects of the augmented reality display in the actual interaction area Interactive operation.
An augmented reality interactive device comprising:

An interaction adding unit, configured to join the current client to the interaction to participate when the current client meets the qualification condition of the participation interaction;

An interactive area display unit for displaying an actual interactive area on the screen of the client to interact;

a virtual object display unit, configured to display one or more predetermined virtual objects in the actual interactive area, wherein the predetermined virtual object is displayed in the actual interactive area by the server, and the predetermined virtual The display position of the object is synchronized in real time to the screen of other clients participating in the interaction.
The apparatus of claim 14, wherein the qualification is defined by the current client, one or more other clients participating in the interaction, and/or by a server implementing the interaction, and

The qualifications include:

Identifying clients connected to a particular WLAN as participating clients; and/or

Identifying clients that enter the same buddy group as participating clients; and/or

Clients entering the same virtual room are identified as participating clients.
The device according to claim 14 or 15, wherein the interactive area display unit is further configured to:

Determining the actual interaction area for interaction based on positioning information or shooting information of the current client and/or one or more other clients; and/or

Determining the actual interaction area in which the interaction is performed according to system-defined constraints; and/or

The actual interaction area in which the interaction is performed is determined based on the coverage of the same WLAN connected to the interacting client.
The apparatus of any of claims 14-16, further comprising:

An interactive operation unit, configured to perform an interactive operation of displaying one or more predetermined virtual objects in the augmented reality in the actual interaction area.
The apparatus of any of claims 14-17, the interactive operating unit:

Used to issue the one or more predetermined virtual objects to the actual interactive area; and/or

Used to interact with the one or more predetermined virtual objects issued by themselves or other clients.
A graphical user interface system on a portable electronic device having a touch screen display, comprising:

The interactive area display interface displays the actual interaction area to be interactive on the screen of the current client in response to the current client joining the interaction to participate;

A virtual object interaction interface that displays one or more predetermined virtual objects for interactive operations within the actual interactive area of the display.
A computing device comprising:

Processor;

A memory having executable code stored thereon that, when executed by the processor, causes the processor to perform the method of any of claims 7-13.
A non-transitory machine readable storage medium having stored thereon executable code that, when executed by a processor of an electronic device, causes the processor to perform any of claims 7-13 Said method.