WO2022062441A1

WO2022062441A1 - Augmented reality information interaction method and apparatus, and electronic device, medium and product

Info

Publication number: WO2022062441A1
Application number: PCT/CN2021/095805
Authority: WO
Inventors: 侯欣如
Original assignee: 北京市商汤科技开发有限公司
Priority date: 2020-09-23
Filing date: 2021-05-25
Publication date: 2022-03-31
Also published as: CN112150318A; KR20220117339A

Abstract

An augmented reality information interaction method and apparatus, and an electronic device, a medium and a product. The method comprises: firstly, identifying current tour state information of an augmented reality (AR) device (S110); then, acquiring AR scene data matching the tour state information, wherein the AR scene data comprises AR effect data of a virtual tour guide (S120); when there are at least two kinds of AR effect data of the virtual tour guide, on the basis of a playing priority of each kind of AR effect data of the virtual tour guide, determining target AR effect data of the virtual tour guide (S130); and then, displaying the target AR effect data of the virtual tour guide on the AR device (S140).

Description

Augmented reality information interaction method, device, electronic device, medium and product

CROSS-REFERENCE TO RELATED APPLICATIONS

This patent application requires the priority of the Chinese patent application number 202011011479.0 filed on September 23, 2020, the applicant is Beijing Shangtang Technology Development Co., Ltd. The entirety of this disclosure is incorporated by reference into this disclosure.

technical field

The present disclosure relates to the technical field of computer vision, and in particular, to an augmented reality information interaction method, apparatus, electronic device, medium and product.

Background technique

When users visit a certain scenic spot or go to a certain destination, they usually rely on audio guides or map navigation. The above methods have defects such as poor intuition and limited guidance information, which lead to the efficiency of guidance. Low.

SUMMARY OF THE INVENTION

Embodiments of the present disclosure provide an augmented reality information interaction method, apparatus, electronic device, medium, and product.

In a first aspect, an embodiment of the present disclosure provides an augmented reality information interaction method, including:

Identify the current tour state information of the augmented reality AR device;

Acquiring AR scene data that matches the tour state information; the AR scene data includes AR effect data of the virtual tour guide;

In the case that there are at least two kinds of AR effect data of the virtual tour guide, based on the playback priority level of the AR effect data of each virtual tour guide, determine the target AR effect data of the virtual tour guide;

The target AR effect data of the virtual tour guide is displayed through the AR device.

In a possible implementation manner, the identifying the current tour state information of the augmented reality AR device includes:

Acquiring information about the real scene where the AR device is currently located;

Based on the real scene information, the current tour state information of the AR device is determined.

In this embodiment, the information of the actual scene where the AR device is located can more accurately determine the tour state information of the AR device, for example, whether the AR device is in a moving state, Whether it is in a stationary state, determine the location of the AR device or the scene feature information, etc. The above-determined motion state information, location information, and scene feature information can all be used as tour state information.

In a possible implementation manner, the tour state information includes at least one of the following:

Moving state information, stationary state information, location information of the AR device, and feature information of the scene where the AR device is located.

In this embodiment, when the tour state information includes the movement state information, the AR scene data matching the movement state information can be obtained, and then the target AR effect data of the virtual tour guide corresponding to the movement state information can be displayed, so that the user can see The target AR effect data is more realistic; when the tour state information includes the static state information, the AR scene data matching the static state information can be obtained, and then the target AR effect data of the virtual tour guide corresponding to the static state information can be displayed, so that the target AR effect data of the virtual tour guide corresponding to the static state information can be displayed. The user can learn the AR scene data in the current static scene; when the tour state information includes the location information of the AR device, the user can obtain the AR scene data that matches the location information of the AR device, and then display the location information of the AR device. The target AR effect data of the corresponding virtual tour guide, so that the user can know the detailed AR scene data corresponding to the location information of the AR device; when the tour state information includes the scene feature information of the AR device, it can make the user know the detailed AR scene data corresponding to the location information of the AR device. Seeing the AR scene data corresponding to the scene feature information where the AR device is located reduces the fact that users are likely to miss the information in the real scene, resulting in the inability to know the AR scene data corresponding to the feature information of a scene.

In response to the input display request information, the display request information is used as the tour state information where the AR device is currently located.

In this embodiment, the display request information input by the user is used as the tour status information, which can provide the user with a personalized virtual tour guide according to the user's needs, which is beneficial to improve the guidance efficiency. The presentation demand information can be matched to obtain the introduction information of the scenic spot as the tour state, and then the AR device can display the introduction information of the scenic spot on the AR device in the form of at least one of text, image, animation, etc.

In a possible implementation manner, displaying the target AR effect data of the virtual tour guide through the AR device includes:

Obtain the AR effect data currently being displayed by the AR device and the original tour status information corresponding to the AR effect data currently being displayed;

Based on the original tour state information and/or the current tour state information of the AR device, the target AR effect data of the virtual tour guide is displayed on the AR device.

In this embodiment, according to the current tour state information of the AR device and the original tour state information corresponding to the AR effect data currently displayed by the AR device, it is determined whether to immediately switch the AR effect data displayed on the AR device to the target AR effect data, which can Improve the smoothness and integrity of AR effect data switching, and reduce the situation of abruptly changing the AR effect data displayed on the AR device.

In a possible implementation manner, based on the original tour state information and the current tour state information of the AR device, displaying the target AR effect data of the virtual tour guide on the AR device, including:

In the case that the original tour state information indicates that the AR device is in a moving state, and the tour state information that the AR device is currently in indicates that the AR device is currently in a stationary state or a scene-triggered state, the currently in-progress state The displayed AR effect data is switched to the target AR effect data.

In this embodiment, the AR effect data displayed by default in the moving state is switched to the target AR effect data corresponding to the static state or the scene-triggered state, which improves the AR effect data when the AR device changes from the moving state to the static state or the scene-triggered state. The switching speed of AR effect data displayed by the device improves the guidance efficiency in this state change mode.

When the original tour state information indicates that the AR device is in a stationary state, and the tour state information that the AR device is currently in indicates that the AR device is currently in a moving state or a scene-triggered state, the currently in-progress state The displayed AR effect data is switched to the target AR effect data.

In this embodiment, the AR effect data displayed by default in the static state is switched to the target AR effect data corresponding to the moving state or the scene-triggered state, which improves the AR effect data when the AR device changes from the static state to the moving state or the scene-triggered state. The switching speed of AR effect data displayed by the device improves the guidance efficiency in this state change mode.

In a possible implementation manner, based on the original tour state information, displaying the target AR effect data of the virtual tour guide on the AR device, including:

In the case where the original tour state information indicates that the AR device is in a scene trigger state, after the current display of the AR effect data currently being displayed is completed, the target AR effect data is displayed in the AR device.

In this embodiment, when the AR device is playing the AR effect data corresponding to the scene trigger state, in the case of recognizing new tour state information, the target AR effect data can be displayed only after the current display of the currently displayed AR effect data is completed. , which can improve the smoothness and integrity of AR effect data switching, and reduce the situation of abruptly changing the AR effect data displayed on the AR device.

In a possible implementation manner, the displaying the target AR effect data of the virtual tour guide through the AR device further includes:

In the case that the AR effect data currently being displayed by the AR device is not obtained, the target AR effect data is displayed in the AR device.

In this implementation manner, in the case where the AR effect data being played by the AR device is not obtained, the target AR effect data is displayed on the AR device, which helps to improve the efficiency of virtual guidance.

In a possible implementation manner, the scene trigger state includes at least one of the following:

The AR device is in a state of a preset position; the AR device is in a state of a preset scene; and the AR device is in a state of receiving display request information.

In this embodiment, when the scene trigger state includes the state where the AR device is in the preset position, the target AR effect data of the virtual tour guide corresponding to the preset position can be displayed, so that the user can know the AR scene related to the preset position. data; when the scene triggering state includes the state of the AR device in the preset scene, the target AR effect data of the virtual tour guide corresponding to the preset scene can be displayed, so that the user can know the AR scene data related to the preset scene; In the case where the scene trigger state includes the AR device receiving the display request information, the display request information can be used as the current tour state information of the AR device, and the target AR effect data of the virtual tour guide corresponding to the display request information can be displayed, so that the target AR effect data of the virtual tour guide corresponding to the display request information can be displayed. The user learns the AR scene data related to the display request information.

In a possible implementation, the target AR effect data of the virtual tour guide is determined based on the playback priority level of the AR effect data of each virtual tour guide, including:

Respectively determine the degree of matching between the current tour state information of the AR device and the AR effect data of each of the virtual tour guides;

Based on the degree of matching corresponding to the AR effect data of each kind of virtual tour guide, respectively determine the playback priority level of the AR effect data of each kind of virtual tour guide;

The AR effect data of the virtual tour guide with the highest playback priority is used as the target AR effect data of the virtual tour guide.

In this embodiment, based on the degree of match between the tour state information and the AR effect data, the playback priority level of the AR effect data is determined, and then the target AR effect data to be played on the AR device is determined based on the playback priority level, which can play the most relevant tour state information. The AR effect data can improve the guidance efficiency.

In a second aspect, the present disclosure provides an augmented reality information interaction device, including:

The state identification part is configured to identify the current tour state information of the augmented reality AR device;

The display data acquisition part is configured to acquire AR scene data matching the tour state information; the AR scene data includes AR effect data of the virtual tour guide;

The display data screening part is configured to determine the target AR effect data of the virtual tour guide based on the playback priority level of the AR effect data of each virtual tour guide when there are at least two kinds of AR effect data of the virtual tour guide;

The data display part is configured to display the target AR effect data of the virtual tour guide through the AR device.

In a possible implementation manner, the state identification part is further configured to: acquire information about the actual scene where the AR device is currently located; and determine the current tour state where the AR device is located based on the information about the actual scene information.

In a possible implementation manner, the tour state information includes at least one of the following: moving state information, stationary state information, location information of the AR device, and scene feature information where the AR device is located.

In a possible implementation manner, the state identification part is further configured to: in response to the input display request information, use the display request information as the tour state information where the AR device is currently located.

In a possible implementation manner, the data display part is further configured to: acquire AR effect data currently being displayed by the AR device and original tour state information corresponding to the currently displayed AR effect data; based on the original tour state information and/or the current tour state information of the AR device, and display the target AR effect data of the virtual tour guide on the AR device.

In a possible implementation manner, the data presentation part is further configured to: when the original tour state information indicates that the AR device is in a moving state, and the current tour state information of the AR device indicates that the AR device is in a moving state When the AR device is currently in a static state or a scene-triggered state, the AR effect data currently being displayed is switched to the target AR effect data.

In a possible implementation manner, the data display part is further configured to: when the original tour state information indicates that the AR device is in a stationary state, and the current tour state information of the AR device indicates that the AR device is in a stationary state When the AR device is currently in a moving state or a scene triggering state, the AR effect data currently being displayed is switched to the target AR effect data.

In a possible implementation manner, the data display part is further configured to: in the case that the original tour state information indicates that the AR device is in the scene trigger state, the AR effect data currently being displayed is currently displayed. After the secondary display is completed, the target AR effect data is displayed on the AR device.

In a possible implementation manner, the data display part is further configured to: display the target AR effect data in the AR device without acquiring the AR effect data currently being displayed by the AR device .

In a possible implementation manner, the display data screening part is further configured to: respectively determine the degree of matching between the current tour state information of the AR device and the AR effect data of each type of virtual tour guide; According to the matching degree corresponding to the AR effect data of each virtual tour guide, the playback priority level of the AR effect data of each virtual tour guide is determined respectively; the AR effect data of the virtual tour guide with the highest playback priority level is used as the virtual tour guide. The target AR effect data of the tour guide.

In a third aspect, embodiments of the present disclosure provide an electronic device, including: a processor, a memory, and a bus, where the memory stores machine-readable instructions executable by the processor, and when the electronic device runs, the processing A bus communicates between the processor and the memory, and when the machine-readable instructions are executed by the processor, the steps of the above-mentioned augmented reality information interaction method are executed.

In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and the computer program executes the steps of the above augmented reality information interaction method when the computer program is run by a processor.

In a fifth aspect, an embodiment of the present disclosure provides a computer program product, including computer-readable code, and when the computer-readable code is executed in an electronic device, a processor in the electronic device executes the above augmented reality The steps of the information exchange method.

In the embodiment of the present disclosure, based on the identified tour state information of the AR device, the matching AR scene data is obtained, and in the case that the AR scene data includes multiple AR effect data, the playback according to the AR effect data is prioritized. Level, determine the target AR effect data played on the AR device, not only realizes the intuitive display of guidance information on the AR device, that is, the target AR effect data, but also provides rich guidance information such as images or animations, which improves the guidance efficiency.

In order to make the above-mentioned objects, features and advantages of the present disclosure more obvious and easy to understand, the preferred embodiments are exemplified below, and are described in detail as follows in conjunction with the accompanying drawings.

Description of drawings

In order to explain the technical solutions of the embodiments of the present disclosure more clearly, the following briefly introduces the accompanying drawings required in the embodiments, which are incorporated into the specification and constitute a part of the specification. The drawings illustrate embodiments consistent with the present disclosure, and together with the description serve to explain the technical solutions of the present disclosure. It should be understood that the following drawings only show some embodiments of the present disclosure, and therefore should not be regarded as limiting the scope. Other related figures are obtained from these figures.

FIG. 1 shows a flowchart of an augmented reality information interaction method provided by an embodiment of the present disclosure;

2 shows a schematic diagram of the switching logic of AR effect data when the AR device changes from a moving state to a stationary state or a scene-triggered state in an embodiment of the present disclosure;

3 shows a schematic diagram of the switching logic of AR effect data when the AR device changes from a static state to a moving state or a scene-triggered state in an embodiment of the present disclosure;

4 shows a schematic diagram of the switching logic of AR effect data when the AR device changes from a scene-triggered state to a static state or a moving state in an embodiment of the present disclosure;

5 shows a schematic diagram of the switching logic of AR effect data when the AR device changes from a scene-triggered state to a new scene-triggered state in an embodiment of the present disclosure;

FIG. 6 shows a schematic structural diagram of an augmented reality information interaction device provided by an embodiment of the present disclosure;

FIG. 7 shows a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure.

detailed description

In order to make the purposes, technical solutions and advantages of the embodiments of the present disclosure more clear, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only These are some, but not all, embodiments of the present disclosure. The components of the disclosed embodiments generally described and illustrated in the drawings herein may be arranged and designed in a variety of different configurations. Therefore, the following detailed description of the embodiments of the disclosure provided in the accompanying drawings is not intended to limit the scope of the disclosure as claimed, but is merely representative of selected embodiments of the disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present disclosure.

It should be noted that like numerals and letters refer to like items in the following figures, so once an item is defined in one figure, it does not require further definition and explanation in subsequent figures.

The term "and/or" in this paper only describes an association relationship, which means that there can be three kinds of relationships, for example, A and/or B, which can mean: the existence of A alone, the existence of A and B at the same time, the existence of B alone. a situation. In addition, the term "at least one" herein refers to any combination of any one of the plurality or at least two of the plurality, for example, including at least one of A, B, and C, and may mean including from A, B, and C. Any one or more elements selected from the set of B and C.

Multiple or multiple in the embodiments of the present disclosure may refer to at least two or at least two, respectively.

Aiming at the defects of poor intuitiveness of tour guides, limited guidance information provided, and low guidance efficiency, the present disclosure provides an augmented reality information interaction method, device, electronic device, computer storage medium, and computer program product. The present disclosure is based on identifying According to the tour state information of the Augmented Reality (AR) device, obtain the matching AR scene data, and in the case that the AR scene data includes multiple AR effect data, determine the playback priority of the AR effect data according to the AR effect data. The target AR effect data played on the AR device not only realizes the intuitive display of guidance information on the AR device, that is, the target AR effect data, but also provides rich guidance information such as images and animations, which improves the guidance efficiency.

The augmented reality information interaction method, device, electronic device, and storage medium in the embodiments of the present disclosure will be described below through specific embodiments.

As shown in FIG. 1 , an embodiment of the present disclosure discloses an augmented reality information interaction method, which can be applied to a server or an electronic device such as an AR device, and is used to determine that the AR device needs to play based on the current tour state information of the AR device The target AR effect data of the virtual tour guide. The augmented reality information interaction method may include the following steps:

S110. Identify the current tour state information of the augmented reality AR device.

The electronic devices in the embodiments of the present disclosure may further include one of the following: handheld devices such as mobile phones, tablet computers (Pad), computers with wireless transceiver functions, handheld computers, desktop computers, personal digital assistants, portable Media players, smart speakers, navigation devices, smart watches, smart glasses, smart necklaces and other wearable devices, pedometers, digital TVs, virtual reality (Virtual Reality, VR) terminal equipment, wireless terminals in industrial control (Industrial Control) , wireless terminals in Self Driving, wireless terminals in Remote Medical Surgery, wireless terminals in Smart Grid, wireless terminals in Transportation Safety, smart cities ( Wireless terminals in Smart City), wireless terminals in Smart Home, cars, in-vehicle devices or in-vehicle modules in the Internet of Vehicles system, etc.

Here, the tour state information may include at least one of the following: moving state information, stationary state information, location information of the AR device, scene feature information where the AR device is located, display request information input by the user, and the like. The moving state information indicates that the AR device is in a moving state, and the stationary state information indicates that the AR device is in a stationary state.

In some implementations, the identifying the tour state information where the AR device is currently located may include: in response to the input display request information, using the display request information as the tour where the AR device is currently located status information.

The above display request information is obtained by the AR device from the user. In this embodiment, the display request information input by the user is used as the tour state information, which can provide the user with a personalized virtual tour guide according to the user's needs, which is beneficial to improve the guidance efficiency. , the display demand information can be matched to obtain the introduction information of the scenic spot as the tour state, and then the AR device can display the introduction information of the scenic spot on the AR device in the form of at least one of text, image, animation, etc.

When the tour state information includes the movement state information, the AR scene data matching the movement state information can be obtained, and then the target AR effect data of the virtual tour guide corresponding to the movement state information can be displayed, so that the user can see the target AR effect data. More realistic; when the tour state information includes static state information, AR scene data matching the static state information can be obtained, and then the target AR effect data of the virtual tour guide corresponding to the static state information can be displayed, so that users can understand to the AR scene data in the current static scene; when the tour status information includes the location information of the AR device, the AR scene data that matches the location information of the AR device can be obtained, and then the virtual image corresponding to the location information of the AR device can be displayed. The target AR effect data of the tour guide, so that the user can know the detailed AR scene data corresponding to the location information of the AR device; when the tour state information includes the scene feature information of the AR device, it can enable the user to see and The AR scene data corresponding to the scene feature information of the AR device is located, thereby reducing the situation that the user easily misses the information in the real scene, resulting in the inability to know the AR scene data corresponding to the feature information of a certain scene.

At least one of the tour state information, such as moving state information, stationary state information, location information of the AR device, and scene feature information where the AR device is located, can be obtained by using the following sub-steps:

Acquire the information of the real scene where the AR device is currently located; and determine the current tour state information of the AR device based on the real scene information.

In this embodiment, the information of the actual scene where the AR device is located can be used to more accurately determine the tour state information of the AR device. For example, according to the information of the actual scene where the AR device is located, it can be more accurately determined whether the AR device is in a The moving state, whether it is in a stationary state, determining the location or scene feature information of the AR device, etc. The above-determined motion state information, location information, and scene feature information can all be used as tour state information.

The image corresponding to the real scene information can be processed to determine whether the AR device is in a moving state or whether it is in a stationary state, and then determine the location information of the AR device or determine the scene feature information where the AR device is located.

Exemplarily, when the AR device is in a moving state, default AR effect data corresponding to the moving state can be played on the AR device.

When the AR device is in a stationary state, default AR effect data corresponding to the stationary state can be played on the AR device.

If it is determined based on the location information of the AR device that the AR device is in a preset special location, such as a location related to a famous person, AR effect data corresponding to the preset special location can be played on the AR device.

If it is determined that the AR device is in a preset special scene based on the feature information of the scene where the AR device is located, such as a scene related to a famous historical event, the AR device corresponding to the preset special scene can be played on the AR device performance data.

Using the real scene information where the AR device is located, the above-mentioned tour state information of the AR device can be more accurately determined.

In addition, the above-mentioned moving state information, static state information, or location information of the AR device may also be obtained by using a positioning technology.

S120. Acquire AR scene data matching the tour state information; the AR scene data includes AR effect data of the virtual tour guide.

Before performing this step, corresponding AR effect data may be set for each kind of tour state information in advance. For example, AR effect data of characters and/or historical events related to the special location is set for a special location. For another example, for a special scene, AR effect data corresponding to the special scene is set. Here, one or more kinds of corresponding AR effect data may be set for a certain tour state information.

After the corresponding AR effect data is set for each kind of tour state information, each kind of tour state information and the corresponding AR effect data may be stored in the storage part. After acquiring the new tour state information of the AR device, the AR effect data matching the new tour state information is acquired from the storage component.

The above AR scene data includes at least one kind of AR effect data that is matched.

The above AR effect data includes one or more of audio, image, animation, and video.

In the case of storing each AR effect data, the storage component not only stores the information displayed on the AR device such as audio and/or images, but also stores the description text corresponding to the tour state information corresponding to the AR effect data, the text corresponding to the audio, and the animation. At least one of the name of the audio, whether to play it in a loop, the name of the audio, an identifier, and a playback priority. The playback priority level can be identified by an integer, 1 indicates the lowest playback priority level, and the larger the number, the higher the level.

S130. In the case that there are at least two kinds of AR effect data of the virtual tour guide, determine the target AR effect data of the virtual tour guide based on the playback priority level of the AR effect data of each virtual tour guide.

Before performing this step, it is necessary to determine the playback priority of the AR effect data of each type of virtual tour guide that is matched. After that, the AR effect data with the highest playback priority is used as the target AR effect data of the virtual guide.

If only one kind of AR effect data of the virtual tour guide is matched based on the tour state information, the matched AR effect data is used as the target AR effect data of the virtual guide.

In the case of matching the AR effect data of various virtual tour guides, the AR effect data of the various virtual tour guides can be stored in the form of a queue, and after selecting the target AR effect data, the selected target AR effect data in the queue can be stored in the queue. Empty to prevent outdated AR effect data in the queue.

S140. Display the target AR effect data of the virtual tour guide through the AR device.

After the target AR effect data is determined and before the target AR effect data is displayed, the AR effect data currently being displayed by the AR device and the original tour state information corresponding to the currently displayed AR effect data can be obtained first; The original tour state information and/or the current tour state information of the AR device is displayed, and the target AR effect data of the virtual tour guide is displayed on the AR device.

In this embodiment, according to the current tour state information of the AR device and the original tour state information corresponding to the AR effect data currently displayed by the AR device, it is determined whether to immediately switch the AR effect data displayed on the AR device to the target AR The effect data can improve the fluency and integrity of AR effect data switching, and reduce the situation of abruptly changing the AR effect data displayed on the AR device.

Exemplarily, the target AR effect data played on the AR device may include a virtual character corresponding to the AR device, and there is a preset relative positional relationship between the virtual character and the AR device.

Exemplarily, the above-mentioned AR device may include at least two types of tour state information, wherein at least one of the moving state information, the stationary state information, etc. may belong to the default type of state information, and the rest, such as the location information of the AR device, the At least one of the scene feature information and the like where the device is located may belong to the status information of the scene trigger type.

In some embodiments, as shown in FIG. 2 , in the embodiment shown in FIG. 2 , it may be determined that the AR device is in a moving state according to the original tour state information; then, according to the current tour state information of the AR device, it is determined whether the AR device is switched It is a static state or a scene-triggered state; when the AR device is switched to the static state, the AR device switches the currently playing AR effect data to the AR effect data corresponding to the static state; when the AR device is switched to the scene-triggered state, The AR device switches the currently playing AR effect data to the AR effect data corresponding to the scene trigger state.

In some embodiments, the above-mentioned displaying the target AR effect data of the virtual tour guide on the AR device based on the original tour state information and the current tour state information of the AR device can be achieved by using the following steps:

The original tour state information indicates that the AR device is in a moving state, so that the AR device is playing AR effect data corresponding to the moving state, and the current tour state information of the AR device indicates that the AR device is currently in a stationary state Or when the scene is triggered, switch the currently displayed AR effect data to the target AR effect data.

Exemplarily, if the current tour state information of the AR device indicates that the AR device is currently in a stationary state, the AR effect data currently being displayed is switched to the AR effect data corresponding to the stationary state. If the tour state information indicates that the AR device is currently in a scene-triggered state, the AR effect data currently being displayed is switched to AR effect data corresponding to the scene-triggered state.

If the AR device moves to the end point, the AR device switches from the moving state to the stationary state.

In this embodiment, the AR effect data displayed by default in the moving state is switched to the target AR effect data corresponding to the static state or the scene-triggered state, which improves the situation when the AR device changes from the moving state to the static state or the scene-triggered state The switching speed of the AR effect data displayed by the AR device improves the guidance efficiency in this state change mode.

In some embodiments, as shown in FIG. 3 , in the embodiment shown in FIG. 3 , it may be determined that the AR device is in a stationary state according to the original tour state information; then, according to the current tour state information of the AR device, it is determined whether the AR device is switched It is the mobile state or the scene-triggered state; when the AR device is switched to the mobile state, the AR device switches the currently playing AR effect data to the AR effect data corresponding to the mobile state; when the AR device is switched to the scene-triggered state, The AR device switches the currently playing AR effect data to the AR effect data corresponding to the scene trigger state.

When the original tour state information indicates that the AR device is in a stationary state, the AR device is playing AR effect data corresponding to the stationary state, and the current tour state information of the AR device indicates that the AR device is currently in a moving state Or when the scene is triggered, switch the currently displayed AR effect data to the target AR effect data.

During the implementation process, if the current tour state information of the AR device indicates that the AR device is currently in a moving state, the AR effect data currently being displayed is switched to the AR effect data corresponding to the moving state. The tour state information indicated that the AR device is currently in the scene-triggered state, and the AR effect data currently being displayed is switched to the AR effect data corresponding to the scene-triggered state.

In this embodiment, the AR effect data displayed by default in the static state is switched to the target AR effect data corresponding to the moving state or the scene-triggered state, which improves the situation when the AR device changes from the static state to the moving state or the scene-triggered state The switching speed of the AR effect data displayed by the AR device improves the guidance efficiency in this state change mode.

In some embodiments, based on the original tour state information, displaying the target AR effect data of the virtual tour guide on the AR device can be achieved by using the following steps:

In this embodiment, when the AR device is currently playing the AR effect data corresponding to the scene trigger state, in the case of recognizing new tour state information, the AR effect data currently being displayed can be displayed after the current display is completed. Display the target AR effect data in the AR device. This method can improve the fluency and integrity of AR effect data switching, and reduce the situation of abruptly changing the AR effect data displayed on the AR device.

In some embodiments, as shown in FIG. 4 , in the embodiment shown in FIG. 4 , the AR device is playing the AR effect data corresponding to the trigger state of the scene; when the AR device is finished playing, it can be determined that the AR device is currently in The moving state or the static state, in the case of the moving state, the AR device plays the AR effect data corresponding to the moving state, and in the static state, the AR device plays the AR effect data corresponding to the static state.

In some embodiments, if the new tour state information obtained by identification indicates that the AR device is in a stationary state or a moving state, the AR effect data corresponding to the stationary state or the moving state is default, and there may be one or more types. After the current display of the AR effect data currently being displayed is completed, the AR effect data corresponding to the stationary state or the moving state is displayed on the AR device.

In some embodiments, as shown in FIG. 5 , in the embodiment shown in FIG. 5 , it is recognized that the AR device is in a new scene trigger state; then it can be determined whether to obtain the corresponding trigger state of the original scene currently played by the AR device. AR effect data; in the case of no, the AR device plays the AR effect data corresponding to the trigger state of the new scene; in the case of yes, determine: (1) Whether the AR effect data corresponding to the trigger state of the original scene has a loop, In the case of a loop, the execution is completed. In the case of a loop, the current playback is completed; (2) Put the AR effect data A corresponding to the new scene trigger state into the queue; (3) Put the AR corresponding to the new scene trigger state The effect data N is put into the queue. When all of the above conditions are met, that is, when the above (1), (2) and (3) are met, select the AR effect data with the highest playback priority in the queue; then determine the AR effect data with the highest playback priority Whether there are more than one, in the case of no, the AR effect data with the highest priority is played, and in the case of yes, the AR effect data with the highest playback priority obtained first is played.

In some embodiments, if the newly identified tour state information indicates that the AR device is in a scene-triggered state, the following steps can be used to determine target AR effect data, and display the target AR effect data on the AR device:

Step 1. Identify that the AR device is in a new scene trigger state;

Step 2: Determine whether the AR effect data corresponding to the trigger state of the original scene currently played by the AR device is obtained, and if not, play the target AR effect data corresponding to the trigger state of the new scene on the AR device;

Step 3. If the AR effect data corresponding to the trigger state of the original scene is obtained, the following conditions must be met to select the target AR effect data with the highest playback priority from the AR effect data matching the new tour state information:

If the currently playing AR effect data is the cyclically played AR effect data, the currently displayed AR effect data is displayed at the time;

If the currently playing AR effect data is not the cyclically played AR effect data, the currently displayed AR effect data is displayed;

Get all AR effect data matching the new tour state information.

Step 4: After satisfying the above conditions, select the target AR effect data with the highest playback priority from the AR effect data that matches the new tour state information;

Step 5. Determine whether there are multiple AR effect data with the highest playback priority. If so, take the AR effect data with the earliest acquisition time as the target AR effect data, and display the determined target AR effect data on the AR device. If not, the AR effect data with the highest playback priority will be displayed on the AR device.

The above-mentioned playback priority may be manually set or determined according to a preset method. For example, the following sub-steps may be used to determine:

Sub-step 1, respectively determine the degree of matching between the current tour state information of the AR device and the AR effect data of each virtual tour guide.

In the implementation process, the first object described by the AR effect data and the second object corresponding to the tour state information may be determined first, and then the matching degree may be determined according to the degree of association between the first object and the second object. In some embodiments, the higher the degree of association, the higher the degree of matching; the lower the degree of association, the lower the degree of matching.

When it is determined to extract the first object described by the AR effect data, feature points in the image corresponding to the AR effect data may be extracted, and then semantic segmentation is performed on the feature points, and the first object is determined according to the semantic segmentation result. Alternatively, keywords related to the audio or text corresponding to the AR effect data and objects are extracted, and the first object is determined according to the extracted keywords.

Sub-step 2: Based on the degree of matching corresponding to the AR effect data of each virtual tour guide, respectively determine the playback priority level of the AR effect data of each virtual tour guide.

In some embodiments, the higher the matching degree, the higher the playback priority level; the lower the matching degree, the lower the playback priority level.

In some embodiments, when the playback priority level of AR effect data of each virtual tour guide is obtained, the AR effect data of the virtual tour guide with the highest playback priority level can be used as the target AR effect data of the virtual tour guide .

In this embodiment, based on the matching degree between the tour state information and the AR effect data, the playback priority level of the AR effect data is determined, and then the target AR effect data to be played on the AR device is determined based on the playback priority level. The most relevant AR effect data to improve the guidance efficiency.

The above-mentioned playback priority level may also be preset, and is not determined in real time according to the current tour state information of the AR device.

Since the tour status information changes at any time during the user's movement, in order to control the virtual tour guide to respond appropriately, each AR effect data can be configured with a playback priority. When a response operation corresponding to a certain AR effect data is currently performed, multiple AR effect data matching the current tour state information can be received at the same time, and arranged in the queue according to the playback priority, and then based on the currently played AR The effect data and the information of the AR effect data in the queue, such as the playback priority level, are used to determine the audio and animation corresponding to which AR effect data is played next.

In addition, each time you select an AR effect data with the highest playback priority from the queue, you can clear the selected AR effect data in the queue, so that there is no outdated AR effect data in the queue. The AR effect data obtained from the tour status information is added to the queue.

In some embodiments, displaying the target AR effect data of the virtual tour guide through the AR device can also be achieved by using the following steps:

In this embodiment, in the case where the AR effect data being played by the AR device is not obtained, displaying the target AR effect data on the AR device helps to improve the efficiency of virtual guidance.

In some implementation manners, any scene trigger state in the embodiments of the present disclosure may include at least one of the following:

In this embodiment, when the scene trigger state includes the state where the AR device is in the preset position, the target AR effect data of the virtual tour guide corresponding to the preset position can be displayed, so that the user can know that the AR device is related to the preset position. When the scene trigger state includes the AR device in the preset scene, the target AR effect data of the virtual tour guide corresponding to the preset scene can be displayed, so that the user can know the AR related to the preset scene. Scene data; when the scene trigger state includes the AR device receiving the display request information, the display request information can be used as the current tour state information of the AR device, and the target AR effect data of the virtual tour guide corresponding to the display request information can be displayed, Thus, the user can learn AR scene data related to the display request information.

The above-mentioned augmented reality information interaction method provided by the present disclosure is suitable for a virtual tour guide scene of a certain scenic spot, etc., and also suitable for a scene such as novice guidance and/or line guidance.

The augmented reality information interaction method provided by the embodiments of the present disclosure can identify, based on the positioning or image recognition technology, that a scene corresponding to the scene can be obtained when currently in a certain scene (corresponding to the tour state information in the above-mentioned embodiment). file (for example, AR scene data in the above embodiment), based on the information in the scene file, control the virtual tour guide to play corresponding audio and animation (for example, the target AR effect data of the virtual tour guide in the above embodiment).

The virtual tour guide provided by the embodiment of the present disclosure may have a set relative positional relationship with the user. When the user moves, the virtual tour guide may move according to where the user moves.

The animation and audio of the virtual tour guide can include two types: the first type includes animation and audio in the state of not moving in place or in the state of movement; the second type is animation and audio in the case of scene triggering, such as , the server or the electronic device may include a scene-triggered execution table of the virtual tour guide, and then determine the corresponding execution table through the scene trigger, display animation and audio corresponding to the execution table, and the execution table may include at least one of the following: The scene identifier (ID) of the scene, the scene description used to indicate the description text of the scene, the playback priority of the AR effect data, the text content corresponding to the audio, the animation name, the animation loop or non-loop indication, and the audio name. The text content corresponding to the audio may be the virtual guide text configured in the current scene; the animation name may indicate the animation file configured in the current scene; the audio name may indicate the audio file configured in the current scene.

Corresponding to the above-mentioned augmented reality information interaction method, the present disclosure also discloses an augmented reality information interaction device, each module in the device can implement each step in the augmented reality information interaction method of the above-mentioned various embodiments, and can achieve the same beneficial effect. In some embodiments, as shown in FIG. 6 , the augmented reality information interaction device includes:

The state identification part 610 is configured to identify the tour state information in which the augmented reality AR device is currently located.

The presentation data acquisition part 620 is configured to acquire AR scene data matching the tour state information; the AR scene data includes AR effect data of the virtual tour guide.

The presentation data screening part 630 is configured to determine the target AR effect data of the virtual tour guide based on the playback priority level of the AR effect data of each virtual tour guide when there are at least two kinds of AR effect data of the virtual tour guide.

The data display part 640 is configured to display the target AR effect data of the virtual tour guide through the AR device.

In some embodiments, the state identification part 610 is configured to: in the case of identifying the tour state information that the augmented reality AR device is currently in

In some embodiments, the state identification part 610 is further configured to: acquire information about a real scene where the AR device is currently located; and determine information about a current tour state where the AR device is currently located based on the information about the real scene.

In some embodiments, the tour state information includes at least one of the following: moving state information, stationary state information, location information of the AR device, and scene feature information where the AR device is located.

In some embodiments, the state identification part 610 is further configured to: in response to the input display request information, use the display request information as the tour state information where the AR device is currently located.

In some embodiments, the data display part 640 is further configured to: acquire AR effect data currently being displayed by the AR device and original tour state information corresponding to the currently displayed AR effect data; based on the original tour state information and /or the current tour state information of the AR device, displaying the target AR effect data of the virtual tour guide on the AR device.

In some embodiments, the data presentation part 640 is further configured to: the original tour state information indicates that the AR device is in a moving state, and the tour state information that the AR device is currently in indicates that the AR device is currently in a moving state In the case of a static state or a scene-triggered state, the AR effect data currently being displayed is switched to the target AR effect data.

In some embodiments, the data presentation part 640 is further configured to: the original tour state information indicates that the AR device is in a stationary state, and the tour state information that the AR device is currently in indicates that the AR device is currently in a stationary state In the case of being in a moving state or a scene-triggered state, the AR effect data currently being displayed is switched to the target AR effect data.

In some embodiments, the data display part 640 is further configured to: in the case that the original tour state information indicates that the AR device is in a scene trigger state, the current display of the currently displayed AR effect data is completed. Then, the target AR effect data is displayed in the AR device.

In some embodiments, the data display part 640 is further configured to: display the target AR effect data in the AR device if the AR effect data currently being displayed by the AR device is not obtained.

In some embodiments, the scene trigger state includes at least one of the following:

In some embodiments, the display data screening part 630 is further configured to respectively determine the degree of matching between the tour state information that the AR device is currently in and the AR effect data of each kind of virtual tour guide; According to the matching degree corresponding to the AR effect data of the virtual tour guide, the playback priority level of the AR effect data of each virtual tour guide is determined respectively; the AR effect data of the virtual tour guide with the highest playback priority level is used as the target AR effect of the virtual tour guide. data.

Corresponding to the above augmented reality information interaction method, an embodiment of the present disclosure further provides an electronic device 700. As shown in FIG. 7 , a schematic structural diagram of the electronic device 700 provided by the embodiment of the present disclosure includes:

A processor 71, a memory 72, and a bus 73; the memory stores machine-readable instructions executable by the processor, and when the electronic device operates, the processor and the memory communicate through the bus, and the When the machine-readable instructions are executed by the processor, the steps of the augmented reality information interaction method in any of the foregoing embodiments are performed.

The memory 72 is configured to store the execution instructions, including the memory 721 and the external memory 722; the memory 721 here is also called the internal memory, and is configured to temporarily store the operation data in the processor 71 and the data exchanged with the external memory 722 such as the hard disk. 71 performs data exchange with the external memory 722 through the memory 721. When the electronic device 700 is running, the processor 71 communicates with the memory 72 through the bus 73, so that the processor 71 executes the following instructions: Identify the tour that the augmented reality AR device is currently in. state information; obtain AR scene data that matches the tour state information; the AR scene data includes the AR effect data of the virtual tour guide; in the case that there are at least two kinds of AR effect data of the virtual tour guide, based on each The playback priority level of the AR effect data of the virtual tour guide is determined, and the target AR effect data of the virtual tour guide is determined; the target AR effect data of the virtual tour guide is displayed through the AR device.

Embodiments of the present disclosure further provide a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is run by a processor, the steps of the augmented reality information interaction method described in the foregoing method embodiments are executed. Wherein, the storage medium may be a volatile or non-volatile computer-readable storage medium.

A computer program product provided by an embodiment of the present disclosure includes computer-readable code. When the computer-readable code is executed in an electronic device, a processor in the electronic device executes any of the foregoing embodiments. The steps of the augmented reality information interaction method. In another embodiment, the computer program product includes a computer-readable storage medium storing program codes, and the instructions included in the program codes can be used to execute the steps of the augmented reality information interaction method described in the above method embodiments. The above method embodiment.

Embodiments of the present disclosure also provide a computer program, which implements any one of the methods in the foregoing embodiments when the computer program is executed by a processor. The computer program product can be specifically implemented by hardware, software or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK), etc. Wait.

The above-mentioned augmented reality information interaction device or processor may include the integration of any one or more of the following: an application specific integrated circuit (ASIC), a digital signal processor (Digital Signal Processor, DSP), a digital signal processing device ( Digital Signal Processing Device, DSPD), Programmable Logic Device (Programmable Logic Device, PLD), Field Programmable Gate Array (Field Programmable Gate Array, FPGA), Central Processing Unit (Central Processing Unit, CPU), Graphics Processing Unit (Graphics) Processing Unit, GPU), embedded neural-network processing units (NPU), controller, microcontroller, microprocessor. It can be understood that, the electronic device implementing the function of the above-mentioned processor may also be other, which is not limited in the embodiment of the present disclosure.

The above-mentioned computer storage medium/memory can be an integration of the following or more: Read Only Memory (Read Only Memory, ROM), Programmable Read Only Memory (Programmable Read-Only Memory, PROM), Erasable Programmable Read Only Memory ( Erasable Programmable Read-Only Memory, EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Magnetic Random Access Memory (Ferromagnetic Random Access Memory, FRAM), Flash Memory (Flash Memory), magnetic surface memory, optical disk, or CD-ROM (Compact Disc Read-Only Memory, CD-ROM), etc.; it can also be various terminals including one or any combination of the above memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc.

Those skilled in the art can clearly understand that, for the convenience and brevity of description, for the specific working process of the system and device described above, reference may be made to the corresponding process in the foregoing method embodiments. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. The apparatus embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces, indirect coupling or communication connection of devices or units, which may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.

The functions, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a processor-executable non-volatile computer-readable storage medium. Based on such understanding, the technical solutions of the present disclosure can be embodied in the form of software products in essence, or the parts that contribute to the prior art or the parts of the technical solutions. The computer software products are stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present disclosure. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program codes .

Finally, it should be noted that the above-mentioned embodiments are only specific implementations of the present disclosure, and are used to illustrate the technical solutions of the present disclosure rather than limit them. The protection scope of the present disclosure is not limited thereto, although referring to the foregoing The embodiments describe the present disclosure in detail. Those of ordinary skill in the art should understand that: any person skilled in the art can still modify the technical solutions described in the foregoing embodiments within the technical scope disclosed by the present disclosure. Changes can be easily thought of, or equivalent replacements are made to some of the technical features; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be covered in the present disclosure. within the scope of protection. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Industrial Applicability

The embodiments of the present disclosure disclose an augmented reality information interaction method, device, electronic device, medium and product, wherein the augmented reality information interaction method includes: identifying the current tour state information of the augmented reality AR device; AR scene data matching the tour state information; the AR scene data includes AR effect data of the virtual tour guide; in the case that there are at least two kinds of AR effect data of the virtual tour guide, the AR effect data of each virtual tour guide is based on The playback priority level of the virtual tour guide is determined, and the target AR effect data of the virtual tour guide is determined; the target AR effect data of the virtual tour guide is displayed through the AR device. This method not only realizes the intuitive display of guidance information, that is, target AR effect data, on the AR device, but also provides rich guidance information such as images or animations, thereby improving the guidance efficiency.

Claims

An augmented reality information interaction method, comprising:

Identify the current tour state information of the augmented reality AR device;

Acquiring AR scene data that matches the tour state information; the AR scene data includes AR effect data of the virtual tour guide;

In the case that there are at least two kinds of AR effect data of the virtual tour guide, based on the playback priority level of the AR effect data of each virtual tour guide, determine the target AR effect data of the virtual tour guide;

The target AR effect data of the virtual tour guide is displayed through the AR device.
The augmented reality information interaction method according to claim 1, wherein the identifying the current tour state information of the augmented reality AR device comprises:

Acquiring information about the real scene where the AR device is currently located;

Based on the real scene information, the current tour state information of the AR device is determined.
The augmented reality information interaction method according to claim 1 or 2, wherein the tour state information includes at least one of the following:

Mobile state information, stationary state information, location information of the AR device, and scene feature information of the AR device.
The augmented reality information interaction method according to claim 1, wherein the identifying the current tour state information of the augmented reality AR device comprises:

In response to the input display request information, the display request information is used as the tour state information where the AR device is currently located.
The augmented reality information interaction method according to any one of claims 1 to 4, wherein the displaying the target AR effect data of the virtual tour guide through the AR device comprises:

Obtain the AR effect data currently being displayed by the AR device and the original tour status information corresponding to the AR effect data currently being displayed;

Based on the original tour state information and/or the current tour state information of the AR device, the target AR effect data of the virtual tour guide is displayed on the AR device.
The augmented reality information interaction method according to claim 5, wherein the target AR effect of the virtual tour guide is displayed on the AR device based on the original tour state information and the current tour state information of the AR device data, including:

In the case that the original tour state information indicates that the AR device is in a moving state, and the tour state information that the AR device is currently in indicates that the AR device is currently in a stationary state or a scene-triggered state, the currently in-progress state The displayed AR effect data is switched to the target AR effect data.
The augmented reality information interaction method according to claim 5 or 6, wherein the target of the virtual tour guide is displayed on the AR device based on the original tour state information and the current tour state information of the AR device AR performance data, including:

When the original tour state information indicates that the AR device is in a stationary state, and the tour state information that the AR device is currently in indicates that the AR device is currently in a moving state or a scene-triggered state, the currently in-progress state The displayed AR effect data is switched to the target AR effect data.
The augmented reality information interaction method according to any one of claims 5 to 7, wherein, based on the original tour state information, displaying the target AR effect data of the virtual tour guide on the AR device, comprising:

In the case where the original tour state information indicates that the AR device is in a scene trigger state, after the current display of the AR effect data currently being displayed is completed, the target AR effect data is displayed in the AR device.
The augmented reality information interaction method according to any one of claims 1 to 8, wherein the displaying the target AR effect data of the virtual tour guide through the AR device further comprises:

In the case that the AR effect data currently being displayed by the AR device is not obtained, the target AR effect data is displayed in the AR device.
The augmented reality information interaction method according to any one of claims 6 to 8, wherein the scene trigger state comprises at least one of the following:

The AR device is in a state of a preset position; the AR device is in a state of a preset scene; and the AR device is in a state of receiving display request information.
The augmented reality information interaction method according to any one of claims 1 to 10, wherein the determining the target AR effect data of the virtual tour guide based on the playback priority level of the AR effect data of each virtual tour guide comprises:

Respectively determine the degree of matching between the current tour state information of the AR device and the AR effect data of each of the virtual tour guides;

Based on the degree of matching corresponding to the AR effect data of each kind of virtual tour guide, respectively determine the playback priority level of the AR effect data of each kind of virtual tour guide;

The AR effect data of the virtual tour guide with the highest playback priority is used as the target AR effect data of the virtual tour guide.
An augmented reality information interaction device, comprising:

The state identification part is configured to identify the current tour state information of the augmented reality AR device;

The display data acquisition part is configured to acquire AR scene data matching the tour state information; the AR scene data includes AR effect data of the virtual tour guide;

The display data screening part is configured to determine the target AR effect data of the virtual tour guide based on the playback priority level of the AR effect data of each virtual tour guide when there are at least two kinds of AR effect data of the virtual tour guide;

The data display part is configured to display the target AR effect data of the virtual tour guide through the AR device.
An electronic device, comprising: a processor, a memory and a bus, the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus , the machine-readable instructions execute the steps of the augmented reality information interaction method according to any one of claims 1 to 11 when the machine-readable instructions are executed by the processor.
A computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the augmented reality information interaction method according to any one of claims 1 to 11 are executed.
A computer program product comprising computer readable code, where the computer readable code is run in an electronic device, a processor in the electronic device executes the method of any one of claims 1 to 11 .