TW202248808A - Augmented reality data presentation method, electronic equipment and computer-readable storage medium - Google Patents

Abstract

The invention provides an augmented reality data presentation method, electronic equipment and computer-readable storage medium, the method comprises: obtaining positioning information of an augmented reality (AR) equipment; based on the positioning information, when it is determined that the preset position conditions are met between the AR equipment and the explanation area corresponding to any preset knowledge point, obtain the audio explanation data corresponding to any preset knowledge point and the AR special effect data matched with the audio explanation data; controlling the AR equipment to play audio explanation data, and displaying AR special effect data matched with the currently played audio explanation data.

Description

Augmented reality data presentation method, electronic device and computer readable storage medium

The present invention relates to the field of augmented reality technology, in particular to a method for presenting augmented reality data, electronic equipment, and a computer-readable storage medium.

Augmented Reality (Augmented Reality, AR) is a new technology that integrates real world information and virtual world information. The goal of this technology is to put the virtual world on the screen and interact with the real world. With the development of science and technology, AR technology is applied in various scenarios, such as travel scenarios, shopping scenarios, education scenarios and so on.

Therefore, it is particularly important to propose a display method for AR data.

Embodiments of the present invention at least provide an augmented reality data presentation method, an electronic device, and a computer-readable storage medium.

In the first aspect, the embodiment of the present invention provides a method for presenting augmented reality data, including: obtaining the positioning information of the AR device; based on the positioning information, determining the explanation area corresponding to the AR device and any preset knowledge point If the preset location conditions are met, obtain the audio explanation data corresponding to any preset knowledge point and the AR special effect data matching the audio explanation materials; control the AR device to play the audio explanation materials, and Display the AR special effect data that matches the currently playing audio commentary data.

In the embodiment of the present invention, by controlling the AR device to play the audio explanation material corresponding to any preset knowledge point, while playing the audio explanation material, display the AR special effect material matching the currently played audio explanation material, and use the AR special effect material to explain the audio The supplementary explanation of the information makes the explanation process of the preset knowledge points clearer and more intuitive, and improves the efficiency of the explanation of the preset knowledge points.

In some possible implementation manners, the controlling the AR device to play audio commentary data and display AR special effect data matching the currently played audio commentary data includes: determining that the AR device satisfies the requirements based on the positioning information In the case of triggering display conditions of the AR special effect data, control the AR device to play the audio explanation data and display the AR special effect data.

In some possible implementation manners, the method also includes: When it is determined based on the positioning information that the AR device does not meet the triggering display condition of the AR special effect data, the AR device is controlled to display first guidance information for guiding adjustment of the positioning information of the AR device.

In some possible implementation manners, the method further includes: when the audio explanation material corresponding to any preset knowledge point is played, and/or when the display of the AR special effect material ends, controlling the AR The device displays at least one of the following information: Second guidance information for instructing the AR device to move to the explaining area corresponding to the next preset knowledge point; List information used to indicate the default knowledge points that the AR device has not browsed; The navigation end information is used to indicate that the AR device has played the audio explanation data corresponding to each preset knowledge point.

In some possible implementation manners, the displaying the AR special effect material corresponding to the currently played audio commentary material includes: during the process of playing the audio commentary material, switching and displaying various AR effects corresponding to the currently played audio commentary material special effects data.

In some possible implementation manners, the switching display of various AR special effect materials corresponding to the currently played audio commentary material includes: responding to a target trigger operation, from among the various AR special effect materials corresponding to the currently played audio commentary material, Determine the AR special effect data to be displayed; control the AR device to display the AR special effect data to be displayed.

In some possible implementation manners, the obtaining the positioning information of the augmented reality AR device includes: obtaining the real-time scene image collected by the AR device; based on the real-time scene image and the constructed 3D scene model, determining Positioning information of the AR device.

In some possible implementations, the audio explanation materials corresponding to each preset knowledge point are predetermined according to the following steps: acquiring the audio explanation materials to be processed corresponding to the target area; Determine target keywords that match multiple preset knowledge points in the data; according to the corresponding playback time positions of each of the target keywords in the audio explanation materials to be processed, from the audio explanation materials to be processed, respectively Determine the audio explanation materials corresponding to each of the preset knowledge points.

In some possible implementation manners, the method is applied to a client application platform, and the client application platform is a web page (Web) application platform or a small program application platform.

In the second aspect, an embodiment of the present invention provides an augmented reality data presentation device, including: The first acquisition module is configured to acquire the positioning information of the AR device; The second acquisition module is configured to acquire any preset knowledge point when it is determined that the explanation area corresponding to the AR device and any preset knowledge point satisfies a preset location condition based on the positioning information. Corresponding audio explanation materials, and AR special effect materials matching the audio explanation materials; The control module is configured to control the AR device to play audio explanation materials, and display AR special effect materials matching the currently played audio explanation materials.

In some possible implementation manners, the control module is configured to: control the AR device to play the Audio explanation materials, and display the AR special effect materials.

In some possible implementation manners, the device further includes: a first display module, configured to: control The AR device displays first guidance information for guiding adjustment of the positioning information of the AR device.

In some possible implementation manners, the device further includes: a second display module configured to: When the audio explanation data corresponding to any of the preset knowledge points has been played, and/or the display of the AR special effect data is finished, the AR device is controlled to display at least one of the following information: Second guidance information for instructing the AR device to move to the explaining area corresponding to the next preset knowledge point; List information used to indicate the default knowledge points that the AR device has not browsed; The navigation end information is used to indicate that the AR device has played the audio explanation data corresponding to each preset knowledge point.

In some possible implementation manners, the control module is configured to: in the process of playing the audio commentary material, switch and display various AR special effect materials corresponding to the currently played audio commentary material.

In some possible implementation manners, the control module is configured to: in response to a target trigger operation, determine the AR special effect data to be displayed from a variety of AR special effect data corresponding to the currently played audio explanation data; control the AR The device displays the AR special effect data to be displayed.

In some possible implementation manners, the first acquiring module is configured to: acquire the real-time scene image collected by the AR device; based on the real-time scene image and the constructed 3D scene model, determine the AR device location information for .

In some possible implementation manners, the device further includes a determination module configured to predetermine the audio explanation materials corresponding to each preset knowledge point according to the following steps: obtain the audio explanation materials to be processed corresponding to the target area; A knowledge point library, determining target keywords that match a plurality of preset knowledge points from the audio explanation materials to be processed; according to the corresponding playing time positions of each of the target keywords in the audio explanation materials to be processed, From the audio explanation materials to be processed, audio explanation materials corresponding to each of the preset knowledge points are respectively determined.

In some possible implementation manners, the method is applied to a client application platform, and the client application platform is a Web application platform or an applet application platform.

In a third aspect, an embodiment of the present invention provides an electronic device, including: a processor, a memory, and a bus bar, the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the The processor communicates with the memory through a bus, and when the machine-readable instructions are executed by the processor, the steps of the method for presenting augmented reality data as described in the first aspect or any implementation mode described above are executed .

In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is run by a processor, the above-mentioned first aspect or any one of the implementation modes is executed. The steps of the augmented reality data presentation method.

In order to make the above-mentioned objects, features and advantages of the embodiments of the present invention more comprehensible, preferred embodiments will be described below in detail together with the accompanying drawings.

In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments It is only some embodiments of the present invention, but not all embodiments. The elements of the embodiments of the invention generally described and illustrated in the drawings herein may be arranged and designed in a variety of different configurations. Accordingly, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed embodiments of the invention, but represents only selected embodiments of the invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative efforts belong to the protection scope of the embodiments of the present invention.

AR is a new technology that integrates real world information and virtual world information. The goal of this technology is to put the virtual world on the screen and interact with the real world. With the development of science and technology, AR technology is applied in various scenarios, such as travel scenarios, shopping scenarios, education scenarios and so on. Based on this, the embodiments of the present invention provide an augmented reality data presentation method, device, electronic device, and computer-readable storage medium.

The defects in the above schemes are all the results obtained by the inventor after practice and careful research. Therefore, the discovery process of the above-mentioned problems and the solutions proposed by the present invention for the above-mentioned problems hereinafter should be the inventors Contributions to the invention made during the course of the invention.

It should be noted that like numerals and letters denote similar items in the following figures, therefore, once an item is defined in one figure, it does not require further definition and explanation in subsequent figures.

To facilitate the understanding of the embodiments of the present invention, a method for presenting augmented reality data disclosed in the embodiments of the present invention is firstly introduced in detail. The execution subject of the augmented reality data display method provided by the embodiment of the present invention may be an AR device, and the AR device is a smart device capable of supporting AR functions. For example, the AR device includes but is not limited to a mobile phone, a tablet, and AR glasses.

Referring to FIG. 1 , it is a schematic flowchart of a method for presenting augmented reality data provided by an embodiment of the present invention. The method includes S101-S103, wherein: S101, acquiring positioning information of the AR device; S102. Based on the positioning information, when it is determined that a preset location condition is satisfied between the AR device and an explanation area corresponding to any preset knowledge point, acquire audio explanation materials corresponding to any preset knowledge point, And the AR special effect data matched with the audio commentary data; S103. Control the AR device to play the audio explanation material, and display the AR special effect material matching the currently played audio explanation material.

In the above method, by controlling the AR device to play the audio explanation material corresponding to any preset knowledge point, while playing the audio explanation material, display the AR special effect material matching the currently played audio explanation material, and use the AR special effect data to explain the audio explanation The supplementary explanation of the information makes the explanation process of the preset knowledge points clearer and more intuitive, and improves the efficiency of the explanation of the preset knowledge points.

S101-S103 will be specifically described below.

For S101: After the AR device reaches the target area, you can scan the information code set in the target area to enter the AR navigation process, that is, you can match the corresponding audio explanation for the AR device through the obtained positioning information of the AR device data, and the AR special effect data matched with the audio explanation data, provide audio guide and AR special effect guide for AR equipment. Wherein, for example, in the cultural tourism scene, the target area may be an area corresponding to any target scenic spot, or may be various exhibition areas in the target scenic spot, etc.

In some optional embodiments, the positioning information of the AR device can be determined according to the sensors set on the AR device. For example, the sensor can be a Global Positioning System (Global Positioning System, GPS), an inertial sensor (Inertial Measurement Unit, IMU), etc.

In an optional implementation manner, in S101, obtaining the positioning information of the AR device may include: obtaining the real-time scene image collected by the AR device; based on the real-time scene image and the constructed 3D scene model, determining the The positioning information of the AR device.

After acquiring the real-time scene image collected by the AR device, you can extract the feature points in the real-time scene image, match the feature point with the feature point cloud included in the 3D scene model, and determine the location when the AR device collects the real-time scene image Information. Wherein, the positioning information may include position information and/or orientation information, for example, the position information may be the coordinate information of the AR device in the coordinate system corresponding to the 3D scene model; the orientation information may be the Euler angle corresponding to the AR device.

In some optional embodiments, the three-dimensional scene model can be constructed according to the following steps: collect multiple frames of scene images at different positions, different angles, and different times in the target area, and extract feature points from each frame of the scene image to obtain The point cloud set corresponding to each frame of the scene image; the feature point cloud corresponding to the target area is obtained by using the point cloud set corresponding to the multi-frame scene image, and the feature point cloud corresponding to the target area constitutes a three-dimensional scene model.

In some other optional embodiments, it is also possible to collect scene videos at different positions, different angles, and different times, obtain multiple frames of video frames from the collected scene videos, and extract feature points from each frame of video frames to obtain each frame. The point cloud set corresponding to the video frame; the 3D scene model corresponding to the target area is obtained by using the point cloud set corresponding to the multi-frame video frame.

Here, by utilizing the real-time scene image collected by the AR device and the constructed three-dimensional scene model, the positioning information of the AR device can be determined more accurately.

For S102: preset knowledge points can be set as any explainable knowledge points in the target area. For example, when the target area is a calligraphy and painting exhibition hall, the preset knowledge point can be the name of any painting and calligraphy in the exhibition; for example, when the target area is the Forbidden City Museum, the preset knowledge point can be the name of any building in the exhibition, etc. .

In some optional embodiments, the spatial position of each preset knowledge point and the explanation area corresponding to each preset knowledge point can be determined in the constructed three-dimensional scene model. For example, the position of the real object in the target area can be determined as the spatial position of the preset knowledge point corresponding to the real object; the surrounding area of the real object in the target area can also be set as the preset knowledge corresponding to the real object Points of explanation area.

Exemplarily, the preset location condition can be that the distance between the AR device and the explanation area corresponding to the preset knowledge point is less than the set distance threshold, or the preset location condition can also be that the AR device is located in the explanation area corresponding to the preset knowledge point area etc.

In some optional embodiments, the distance between the AR device and the explaining area corresponding to each preset knowledge point may be determined based on the location information indicated by the location information of the AR device. For example, the distance between the AR device and the central point of the explaining area may be determined. When the distance between the AR device and the explaining area corresponding to the preset knowledge point is less than the preset distance threshold, the audio explaining material corresponding to the preset knowledge point and the AR special effect material matching the audio explaining material are obtained.

Alternatively, based on the explanation area corresponding to the preset knowledge point and the location information indicated by the positioning information of the AR device, it can be determined whether the AR device is located in the explanation area of the preset knowledge point, if it is determined that the AR device is located in the explanation area of the preset knowledge point , obtain the audio commentary data corresponding to the preset knowledge point, and the AR special effect data matching the audio commentary data.

The audio explanation material can be a preset audio material explaining the preset knowledge point; the AR special effect material can be an AR material including one or more of the following contents: video content, image content, and text content. For example, when the default knowledge point is the Hall of Supreme Harmony, the audio explanation materials can be audio materials explaining the purpose, history, culture, architectural style and other information of the Hall of Supreme Harmony; Or, the AR special effect data can also be the AR data of the overall picture of the Hall of Supreme Harmony in the historical time period, etc. Wherein, the AR special effect data matched with the audio commentary data can be set as required; and the audio commentary data can correspond to one kind of AR special effect data, or can correspond to multiple AR special effect data.

Among them, the display pose of the AR special effect data can be set in the 3D scene model as required. For example, the display pose of the AR special effect data can be located in the explanation area of the preset knowledge points. Alternatively, the display pose of the AR special effect data may also be determined based on the spatial position corresponding to the preset knowledge point.

In some optional embodiments, each audio commentary material can be matched with the corresponding AR special effect material in advance, and after the audio commentary material is acquired, the AR special effect material matching the audio commentary material can be determined.

Alternatively, after acquiring the audio explanation material corresponding to any preset knowledge point, the AR special effect material matching the audio explanation material may be determined in real time. For example, after obtaining the audio explanation materials corresponding to any preset knowledge point, the target keywords corresponding to the audio explanation materials can be identified; based on the target keywords corresponding to the audio explanation materials and the keywords corresponding to each AR special effect material, Determine the AR special effect data that matches the audio commentary data.

Optionally, after obtaining the audio explanation materials corresponding to any preset knowledge point, the audio explanation materials can be identified, and the target keywords included in the audio explanation materials can be determined; or, the text content of the audio explanation materials can also be determined , identifying the text content corresponding to the audio explanation material, and determining a target keyword included in the audio explanation material; the target keyword may be a word related to a preset knowledge point. Then, based on the target keyword corresponding to the audio explanation material and the keyword corresponding to each AR special effect material, the AR special effect material matching the audio explanation material may be determined.

In this embodiment, when controlling the AR device to play the audio explanation material, the AR device can be controlled to display the AR special effect material related to the target keyword according to the target keyword of the currently played audio explanation material, so that the played AR special effect material and The content of the played audio explanation materials is consistent, and the preset knowledge points can be explained and displayed more clearly.

In an optional implementation manner, the audio explanation materials corresponding to each preset knowledge point can be pre-determined according to the following steps: Step A1, obtaining the audio explanation materials to be processed corresponding to the target area; Step A2, according to the preset knowledge point library, determine target keywords respectively matching with multiple preset knowledge points from the audio explanation materials to be processed; Step A3, according to the corresponding playing time position of each target keyword in the audio explanation materials to be processed, from the audio explanation materials to be processed, respectively determine the audio explanation materials corresponding to each of the preset knowledge points .

Exemplarily, the to-be-processed audio explanation data corresponding to the target area can be generated according to the multiple knowledge point information included in the target area, wherein the to-be-processed audio explanation data can be complete audio explanations for each knowledge point respectively material. Then, according to the preset knowledge point library, the target keywords corresponding to multiple preset knowledge points can be determined from the audio explanation materials to be processed; Set the target keywords that the knowledge points match respectively. Finally, according to the corresponding playing time position of each target keyword in the audio explanation materials to be processed, the audio explanation materials to be processed are segmented, and from the multiple audio explanation materials obtained after segmentation, it is determined that each preset knowledge Click on the corresponding audio guide.

Here, according to the preset knowledge point library, the audio explanation materials to be processed can be identified, and the target keywords in the audio materials to be processed that match the preset knowledge points can be determined; From the audio explanation materials to be processed, determine the audio explanation materials corresponding to each preset knowledge point more accurately and quickly.

For S103: It is possible to play (or output) the audio commentary material through the voice playback device of the AR device, and control the AR device to display the AR special effect material matching the currently played audio commentary material.

In an optional implementation manner, the controlling the AR device to play the audio commentary material, and displaying the AR special effect material matching the currently played audio commentary material may include: Method 1. When it is determined based on the positioning information that the AR device satisfies the trigger display condition of the AR special effect data, control the AR device to play the audio explanation data and display the AR special effect data. Method 2: When it is determined based on the location information that the AR device does not satisfy the triggering display condition of the AR special effect data, control the AR device to display the first guidance for guiding the adjustment of the location information of the AR device Information.

Here, when the AR device satisfies the triggering display conditions of the AR special effect data, the AR device is controlled to play the audio explanation data and display the AR special effect data, so as to avoid the poor playback effect of the AR special effect data when the positioning information of the AR device is inappropriate. Occurs to improve the display effect of AR special effect data.

When the AR device does not meet the trigger display conditions of the AR special effect data, the AR device can be controlled to display the first guidance information for guiding the adjustment of the positioning information of the AR device, so that the positioning information of the AR device can be adjusted according to the first guidance information , so that the adjusted AR device can display AR special effect data more clearly and intuitively, avoiding the omission of preset knowledge points, and improving the efficiency of explaining preset knowledge points.

In some optional embodiments, a corresponding trigger display condition may be set for each AR special effect material. For example, the triggering display condition may include a triggering display direction and/or a triggering display distance. When the triggering display condition includes the triggering display direction, and when the positioning information of the AR device indicates that the orientation of the AR device is within the set triggering display direction, it is determined that the AR device satisfies the triggering display condition of the AR special effect data. When the trigger display condition includes the trigger display distance, when the positioning information of the AR device indicates that the distance between the AR device and the display pose of the AR special effect data is less than the set first distance value, and/or greater than the set second distance value , to determine that the AR device satisfies the trigger display condition for the AR special effect data.

Referring to FIG. 2 , in FIG. 2 , for the AR special effect data, the trigger display direction and trigger display distance corresponding to the AR special effect data can be generated based on the display pose of the AR special effect data. Exemplarily, when the orientation of the AR device is within the direction range corresponding to the trigger display direction, it is determined that the AR device satisfies the trigger display condition, for example, for the AR device 21 in FIG. 2 , when the orientation of the AR device 21 is direction 211, If the orientation of the AR device 21 is within the direction range corresponding to the set trigger display direction, the AR device 21 satisfies the trigger display condition; when the AR device 21 is facing in direction 212, the AR device 21 does not meet the trigger display condition.

For another example, if the distance between the AR device 21 in FIG. 2 and the display location of the AR special effect material is less than the set first distance value and greater than the set second distance value, then the AR device 21 meets the set trigger display distance. The distance between the AR device 22 in FIG. 2 and the display position of the AR special effect material is greater than the set first distance value, so the AR device 22 does not meet the set trigger display distance.

When the triggering display conditions include the triggering display direction and the triggering display distance, when the orientation indicated by the positioning information of the AR device satisfies the triggering display direction condition, and the location information indicated by the AR device’s positioning information satisfies the triggering display distance condition, the AR is determined. The device meets the trigger display conditions for AR special effect data. When the orientation indicated by the positioning information of the AR device does not meet the conditions for triggering the display direction, and/or the location information indicated by the positioning information of the AR device does not meet the conditions for triggering the display distance, determine that the AR device does not meet the conditions for triggering the display of AR special effect data .

When the AR device satisfies the trigger display condition corresponding to the AR special effect data, the AR device is controlled to play the audio explanation data and display the AR special effect data. If the AR device does not meet the trigger display condition corresponding to the AR special effect data, generate the first guide information for guiding the adjustment of the positioning information of the AR device, and display the first guide information through the AR device, so as to pass the first guide information. The guidance information adjusts the pose of the AR device, so that the AR device can display the AR special effect data more clearly.

Refer to Figure 3 for a schematic interface diagram of an AR device, which shows the first guide information, for example, the first guide information can be "Please move the AR device to the right to view the AR special effect corresponding to the preset knowledge point A material".

In an optional implementation manner, the method further includes: controlling the AR device when the audio explanation material corresponding to any preset knowledge point is played, and/or when the display of the AR special effect material ends Displaying at least one of the following information: second guidance information for instructing the AR device to move to the explaining area corresponding to the next preset knowledge point; list information for indicating the default knowledge point that the AR device has not browsed; The navigation end information indicating that the AR device has played the audio explanation data corresponding to each preset knowledge point.

In this embodiment, a variety of information types to be displayed are set, so that after the audio explanation data corresponding to any preset knowledge point is played and/or the AR special effect data is displayed, the AR device can be controlled to display at least one type of information, which improves the display. variety and flexibility.

In one embodiment, after the audio explanation material corresponding to any preset knowledge point is played and/or the display of the AR special effect material is finished, when the AR device still has preset knowledge points that have not been browsed, the AR device can be controlled to display the Second guidance information that instructs the AR device to move to the explaining area corresponding to the next preset knowledge point. Referring to FIG. 4a, the second guidance information included in FIG. 4a may be "please move forward and view the default knowledge point B". Exemplarily, Fig. 4a may also include navigation information from the current position to the next preset knowledge point (ie, preset knowledge point B).

In another embodiment, the AR device can also be controlled to display list information indicating the preset knowledge points that the AR device has not browsed; see FIG. 4b.

In yet another embodiment, the AR device may also be controlled to display list information of all preset knowledge points, and mark browsed preset knowledge points and unbrowsed preset knowledge points among all preset knowledge points. See Figure 4c, where the preset knowledge points with checkmark "√" can be browsed preset knowledge points, and the preset knowledge points without checkmark "√" can be unbrowsed preset knowledge points Knowledge point; and then in response to the trigger operation, determine the preset knowledge point for the next browsing of the AR device, and control the AR device to display the first instruction area for instructing the AR device to move to the corresponding explanation area of the determined preset knowledge point for the next browsing 2. Guiding information. Wherein, the triggering operation may be voice data, touch screen operation, and the like.

In another embodiment, when the AR device has no unbrowsed preset knowledge points, the AR device can also be controlled to display navigation end information indicating that the AR device has finished playing the audio explanation materials corresponding to each preset knowledge point , as shown in FIG. 4d, the navigation end information may be "this navigation is over, welcome to use next time". It is also possible to control the AR device to display a navigation end mark, for example, the smiley face mark in FIG. 4d may be a navigation end mark.

In an optional implementation manner, the displaying the AR special effect materials corresponding to the currently played audio explanation materials includes: during the process of playing the audio explanation materials, switching and displaying various AR special effects corresponding to the currently played audio explanation materials material.

When the audio explanation material is matched with multiple AR special effect materials, in the process of playing the audio explanation material, various AR special effect materials corresponding to the currently played audio explanation material can be switched and displayed. For example, when the audio explanation data is audio data explaining historical figures, multiple AR special effect materials may include AR special effects of historical figures in different postures. For example, multiple AR special effect materials may include AR special effects of historical figures in learning postures, AR special effects of historical figures in a resting posture, etc. When the audio commentary data is audio data explaining buildings, the various AR special effect data may include AR special effects of buildings under different lighting conditions, or the AR special effect data may include AR special effects of the building in different historical periods, etc. .

For example, according to the target keyword being explained in the currently played audio explanation material, various AR special effect materials can be switched and displayed, so that the displayed AR special effect data matches the target keyword being explained. Or, during the process of playing the audio commentary data, the user can switch and display various AR special effect data corresponding to the currently played audio commentary data by adjusting the position of the AR device. For example, in the process of playing the audio explanation data, the user can switch and display various AR special effect materials corresponding to the currently played audio explanation data by moving the AR device left and right, or turning the AR device clockwise.

In this embodiment, the audio explanation materials can correspond to various AR special effect materials, so that in the process of playing the audio explanation materials, various AR special effect materials corresponding to the currently played audio explanation materials can be switched and displayed, and the display flexibility of the AR special effect materials can be improved. and diversity.

In an optional implementation manner, the switch displays a variety of AR special effect materials corresponding to the currently played audio commentary materials, including: Step B1, in response to the target trigger operation, determine the AR special effect data to be displayed from among the various AR special effect data corresponding to the currently played audio explanation data; Step B2, controlling the AR device to display the AR special effect data to be displayed.

When the AR device plays the audio explanation material, the AR device can be synchronously controlled to play the AR special effect material corresponding to the audio explanation material. For example, one of various AR special effect materials can be randomly selected to play. If the user wants to view other AR special effect materials, the target trigger operation can be executed, and in response to the target trigger operation, multiple audio explanation materials corresponding to the currently playing audio explanation data can be played. In the AR special effect data, determine the AR special effect data to be displayed; control the AR device to display the AR special effect data to be displayed.

The target trigger operation can be set according to actual needs. For example, the target trigger operation can be voice data, touch screen operation, pose change status of the AR device, etc. Exemplarily, voice data can be input into the AR device. For example, the input voice data can be "special effect one". Display AR special effect data. Alternatively, a touch screen operation may be performed on the AR device, and in response to the touch screen operation, the AR special effect data to be displayed is determined from various AR special effect data corresponding to the currently played audio explanation data. Alternatively, the AR special effect data to be displayed may be determined from various AR special effect data corresponding to the currently played audio explanation data through the pose change state of the AR device. For example, if the pose transformation state of the AR device is moving left and right, the next AR special effect data after the currently playing AR special effect data among the various special effect data can be determined as the AR special effect data to be displayed. Furthermore, the AR device can be controlled to display the determined AR special effect data to be displayed.

Here, when there are multiple AR special effect materials corresponding to the audio explanation data, in response to the target trigger operation, the AR special effect data to be displayed can be determined from the various AR special effect data corresponding to the currently played audio explanation data, so that the determined AR special effects to be displayed The data can meet the needs of users and improve the flexibility of AR special effect data display.

In an optional implementation manner, the method may be applied to a client application platform, and the client application platform is a web application platform or a small program application platform.

In this embodiment, the method can be applied to a client application platform, and the client can be a web application platform on the AR device, or a small program application platform on the AR device. Alternatively, the client application platform can also be an application for AR navigation on the AR device.

Referring to FIG. 5 , the method for presenting augmented reality data may include the following steps.

S501. Obtain positioning information of the AR device after the AR device arrives at the target area.

S502. Based on the positioning information of the AR device, determine whether a preset position condition is satisfied between the AR device and an explanation area corresponding to any preset knowledge point.

S503. If the preset position condition is met, acquire the audio commentary data corresponding to any preset knowledge point satisfying the preset position condition, and the AR special effect data matching the audio commentary data. For example, when the distance between the AR device and the explaining area corresponding to any preset knowledge point is less than 2 meters, it is confirmed that the AR device satisfies the preset location condition. If the AR device does not meet the preset location conditions between the explanation area corresponding to any preset knowledge point, you can continue to obtain the positioning information of the AR device.

S504. Based on the positioning information, determine whether the AR device satisfies the trigger display condition of the AR special effect data.

S505. If the trigger display condition of the AR special effect data is satisfied, control the AR device to play the audio explanation data and display the AR special effect data.

S506. If the triggering display condition of the AR special effect data is not met, control the AR device to display the first guidance information for guiding the adjustment of the positioning information of the AR device, so as to guide the AR device to view the AR special effect data. For example, when the orientation of the AR device is within the trigger display direction corresponding to the AR special effect data, and/or, the distance between the AR device and the display pose of the AR device and the AR special effect data is within 0.5 meters to 5 meters (including 0.5 meters) and 5 meters), determine that the AR device satisfies the trigger display conditions for AR special effect data.

S507. After the audio explanation data of any preset knowledge point is played, and/or, after the AR special effect data is displayed, the AR device may be prompted to view the next preset knowledge point until all the preset knowledge points in the target area correspond to After all the audio explanation materials have been played, it is confirmed that the navigation is finished.

Those skilled in the art can understand that in the above method of specific implementation, the writing order of each step does not mean a strict execution order and constitutes any limitation on the implementation process. The specific execution order of each step should be based on its function and possible The inner logic is OK.

Based on the same idea, the embodiment of the present invention also provides an augmented reality data presentation device, as shown in FIG. Group 601, second acquisition module 602 and control module 603, specifically: The first obtaining module 601 is configured to obtain positioning information of the AR device; The second acquiring module 602 is configured to acquire any preset knowledge point when it is determined that the explanation area corresponding to the AR device and any preset knowledge point satisfies a preset location condition based on the positioning information. Set the audio explanation materials corresponding to the knowledge points, and the AR special effect materials matching the audio explanation materials; The control module 603 is configured to control the AR device to play audio explanation materials, and display AR special effect materials matching the currently played audio explanation materials.

In a possible implementation manner, the control module 603 is configured to: control the AR device to play the Audio explanation materials, and display the AR special effect materials.

In a possible implementation manner, the device further includes: a first display module 604, configured to: control The AR device displays first guidance information for guiding adjustment of the positioning information of the AR device.

In a possible implementation manner, the device further includes: a second display module 605 configured to: When the audio explanation data corresponding to any of the preset knowledge points has been played, and/or the display of the AR special effect data is finished, the AR device is controlled to display at least one of the following information: Second guidance information for instructing the AR device to move to the explaining area corresponding to the next preset knowledge point; List information used to indicate the default knowledge points that the AR device has not browsed; The navigation end information is used to indicate that the AR device has played the audio explanation data corresponding to each preset knowledge point.

In a possible implementation manner, the control module 603 is configured to: in the process of playing the audio commentary material, switch and display various AR special effect materials corresponding to the currently played audio commentary material.

In a possible implementation manner, the control module 603 is configured to: in response to a target trigger operation, determine the AR special effect data to be displayed from a variety of AR special effect data corresponding to the currently played audio explanation data; control the AR The device displays the AR special effect data to be displayed.

In a possible implementation manner, the first acquisition module 601 is configured to: acquire the real-time scene image collected by the AR device; determine the AR device based on the real-time scene image and the constructed 3D scene model location information for .

In a possible implementation manner, the device further includes a determination module 606 configured to predetermine the audio explanation materials corresponding to each preset knowledge point according to the following steps: obtain the audio explanation materials to be processed corresponding to the target area; A knowledge point library, determining target keywords that match a plurality of preset knowledge points from the audio explanation materials to be processed; according to the corresponding playing time positions of each of the target keywords in the audio explanation materials to be processed, From the audio explanation materials to be processed, audio explanation materials corresponding to each of the preset knowledge points are respectively determined.

In a possible implementation manner, the method is applied to a client application platform, and the client application platform is a Web application platform or an applet application platform.

The functions or included templates of the device provided by the embodiment of the present invention can be used to execute the method described in the method embodiment above, and its specific implementation can refer to the description of the method embodiment above, and for the sake of brevity, details are not repeated here.

Based on the same technical idea, the embodiment of the present invention also provides an electronic device. Referring to FIG. 7 , which is a schematic structural diagram of an electronic device provided by an embodiment of the present invention, an electronic device 700 includes a processor 701 , a memory 702 and a bus 703 . Among them, the memory 702 is used to store execution instructions, including an internal memory 7021 and an external memory 7022; the internal memory 7021 is used to temporarily store calculation data in the processor 701, and exchange data with external memory 7022 such as hard disks The processor 701 exchanges data with the external memory 7022 through the memory 7021. When the electronic device 700 is running, the processor 701 communicates with the memory 702 through the bus 703, so that the processor 701 executes the The steps of the augmented reality data presentation method.

In addition, an embodiment of the present invention also provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is run by a processor, the augmented reality data presentation described in the above-mentioned method embodiments is executed. method steps. Wherein, the storage medium may be a volatile or non-volatile computer-readable storage medium.

An embodiment of the present invention also provides a computer program product, the computer program product carries a program code, and the instructions included in the program code can be used to execute the steps of the augmented reality data presentation method described in the above method embodiments, specifically, Refer to the foregoing method embodiments, and details are not repeated here.

Wherein, the above-mentioned computer program product can be realized by means of hardware, software or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium. In another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK), etc. Wait.

Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working process of the above-described system and device can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here. In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or elements can be combined or can be Integrate into another system, or some features may be ignored, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some communication interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or may also be distributed to multiple network units . Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit.

If the functions described above are realized in the form of software function units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium executable by a processor. Based on this understanding, the essence of the technical solution of the present invention or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of software products, which are stored in a storage medium, including several The instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk, etc., which can store program codes. media.

The above is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person familiar with the technical field can easily think of changes or replacements within the technical scope disclosed in the present invention, and should be included in the scope of the present invention. within the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the patent application.

21: AR equipment 211: direction 212: direction 22: AR equipment 601: The first acquisition module 602: The second acquisition module 603: Control module 604: The first display module 605: The second display module 606: Determine the module 700: Electronic equipment 701: Processor 702: memory 7021: internal memory 7022: external memory 703: busbar S101~S103, S501~S507: steps

In order to illustrate the technical solutions of the embodiments of the present invention more clearly, the accompanying drawings used in the embodiments will be briefly introduced below, and the drawings here are incorporated into the specification and constitute a part of the specification. The drawings show the embodiments consistent with the present invention, and are used together with the description to illustrate the technical solution of the present invention. It should be understood that the following drawings only show some embodiments of the present invention, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, they can also make From these drawings other related drawings are obtained. FIG. 1 shows a schematic flowchart of a method for presenting augmented reality data provided by an embodiment of the present invention; FIG. 2 shows a schematic diagram of triggering display conditions corresponding to AR special effect data in a method for presenting augmented reality data provided by an embodiment of the present invention; FIG. 3 shows a schematic interface diagram of an AR device provided by an embodiment of the present invention; Fig. 4a shows a schematic interface diagram of an AR device provided by an embodiment of the present invention; Fig. 4b shows a schematic interface diagram of an AR device provided by an embodiment of the present invention; Fig. 4c shows a schematic interface diagram of an AR device provided by an embodiment of the present invention; Fig. 4d shows a schematic interface diagram of an AR device provided by an embodiment of the present invention; Fig. 5 shows a schematic flowchart of another augmented reality data presentation method provided by an embodiment of the present invention; Fig. 6 shows a schematic structural diagram of an augmented reality data presentation device provided by an embodiment of the present invention; Fig. 7 shows a schematic structural diagram of an electronic device provided by an embodiment of the present invention.

S101~S103: Steps

Claims (10)

A method for presenting augmented reality data, comprising: Obtain the positioning information of the augmented reality AR device; Based on the positioning information, when it is determined that a preset position condition is satisfied between the AR device and an explanation area corresponding to any preset knowledge point, acquire audio explanation data corresponding to any preset knowledge point, and AR special effect data matched with the audio commentary data; Controlling the AR device to play the audio commentary material, and displaying the AR special effect material matching the currently played audio commentary material. According to the method described in claim 1, wherein the controlling the AR device to play the audio commentary material and display the AR special effect material matching the currently played audio commentary material includes: When it is determined based on the positioning information that the AR device satisfies the triggering display condition of the AR special effect data, the AR device is controlled to play the audio explanation data and display the AR special effect data. The method according to claim 1 or 2, wherein the method further comprises: When it is determined based on the positioning information that the AR device does not meet the triggering display condition of the AR special effect data, the AR device is controlled to display first guidance information for guiding adjustment of the positioning information of the AR device. The method according to claim 1 or 2, wherein the method further comprises: When the audio explanation data corresponding to any of the preset knowledge points has been played, and/or the display of the AR special effect data is finished, the AR device is controlled to display at least one of the following information: Second guidance information for instructing the AR device to move to the explaining area corresponding to the next preset knowledge point; List information used to indicate the default knowledge points that the AR device has not browsed; The navigation end information is used to indicate that the AR device has played the audio explanation data corresponding to each preset knowledge point. According to the method described in claim 1 or 2, wherein the displaying the AR special effect data corresponding to the currently played audio commentary data includes: In the process of playing the audio explanation material, various AR special effect materials corresponding to the currently played audio explanation material are switched and displayed. According to the method described in claim 5, wherein the switching display of various AR special effect materials corresponding to the currently played audio explanation materials includes: In response to the target trigger operation, determine the AR special effect data to be displayed from among the various AR special effect data corresponding to the currently played audio explanation data; Controlling the AR device to display the AR special effect data to be displayed. According to the method described in claim 1 or 2, wherein the acquiring the positioning information of the AR device includes: Obtain the instant scene image collected by the AR device; Based on the real-time scene image and the constructed 3D scene model, determine the positioning information of the AR device. According to the method described in claim 1 or 2, wherein, the audio explanation materials corresponding to each preset knowledge point are pre-determined according to the following steps: Obtain the audio commentary data to be processed corresponding to the target area; According to the preset knowledge point library, determine target keywords respectively matching with multiple preset knowledge points from the audio explanation materials to be processed; According to the playing time position corresponding to each target keyword in the audio commentary material to be processed, the audio commentary material corresponding to each preset knowledge point is respectively determined from the audio commentary material to be processed. An electronic device, including: a processor, a memory, and a bus bar, the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the connection between the processor and the memory Through bus communication, when the machine-readable instructions are executed by the processor, the steps of the method for presenting augmented reality data as described in any one of claims 1 to 8 are executed. A computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is run by a processor, the steps of the augmented reality data presentation method described in any one of claims 1 to 8 are executed.

Images