WO2022252509A1

WO2022252509A1 - Data display method and apparatus, device, storage medium, and computer program product

Info

Publication number: WO2022252509A1
Application number: PCT/CN2021/132281
Authority: WO
Inventors: 田真; 李斌; 欧华富; 王婷婷
Original assignee: 北京市商汤科技开发有限公司
Priority date: 2021-06-03
Filing date: 2021-11-23
Publication date: 2022-12-08
Also published as: CN113360805B; TW202248896A; CN113360805A

Abstract

A data display method and apparatus, a device, a storage medium, and a computer program product. The method comprises: displaying a live-view image comprising a bottle body acquired by an augmented reality (AR) device (S101); upon detection that a physical object associated with the bottle body in the live-view image changes, determining a first AR special effect displayed and triggered by the bottle body and the changed physical object (S103); and displaying the first AR special effect on a special effect display interface of the AR device (S105).

Description

Data display method and device, equipment, storage medium and computer program product

Cross References to Related Applications

This application is based on a Chinese patent application with application number 202110621248.X and a filing date of June 3, 2021, and claims the priority of this Chinese patent application. The entire content of this Chinese patent application is hereby incorporated by reference in its entirety this application.

technical field

The present application relates to the technical field of augmented reality, involving but not limited to a data presentation method, device, device, storage medium and computer program product.

Background technique

Augmented reality (Augmented Reality, referred to as AR) technology is also called augmented reality. AR augmented reality technology is a relatively new technical content that promotes the integration of real world information and virtual world information content, so that the real environment and Virtual objects are presented in the same screen or space in real time. In recent years, with the rapid development of smart terminal devices, AR devices have become more and more widely used, for example, AR cultural and creative products. In the existing AR technology, a single object can be identified, and the virtual content of the single object can be triggered to be displayed, and the virtual content of the object is usually fixed virtual content such as an object introduction of the object. Therefore, the application scenarios of the existing AR technology are relatively single, and cannot meet the increasingly rich demands of users.

Contents of the invention

Embodiments of the present application provide at least a data presentation method and device, equipment, storage medium, and computer program product.

In the first aspect, the embodiment of the present application provides a method for displaying data, including: displaying a real scene image including a bottle collected by an augmented reality AR device; In the case of a change, determine the first AR special effect triggered by the bottle body and the changed physical item; and display the first AR special effect on the special effect display interface of the AR device.

In the embodiment of the present application, when it is detected that the physical item associated with the bottle in the real scene image has changed, it can be determined that the first AR special effect triggered by the bottle and the changed physical item is displayed, and the special effect The first AR special effect is displayed on the display page. Through the above processing method, it is possible to trigger and display different AR special effects based on the change of the item combination of the bottle body and the physical item associated with the bottle body. Through the above processing method, the trigger conditions of AR special effects can be enriched, and users and bottles can also be realized. AR interaction between objects. While increasing the user's interest, it can improve the user's experience of using AR technology, meet the diverse user needs of the user, and can also widely promote the relevant information of the bottle.

In an optional implementation manner, the determining the first AR special effect triggered by the bottle body and the changed physical item includes: determining a change type of the physical item, wherein the change type includes the following At least one of: addition of physical items, deletion of physical items, change of placement status of physical items; determining the first AR special effect triggered by the bottle and the changed physical items according to the type of change.

In the above embodiments, by determining the way in which different types of AR special effects are triggered by the change type of the physical item, the triggering method of the first AR special effect can be divided in more detail. While enriching the triggering methods of AR special effects, it can also provide Users bring a more delicate user experience.

In an optional implementation manner, the determining the first AR special effect triggered by the display of the bottle body and the changed physical item according to the change type includes: determining a target AR special effect that matches the change type ; Determine the fused AR special effect of the target AR special effect and the initial AR special effect as the first AR special effect, and the initial AR special effect is displayed on the special effect display interface before the physical item in the real scene image changes AR special effects.

In the above embodiment, by combining the target AR special effect corresponding to the change type with the initial AR special effect to obtain the first AR special effect, it is possible to trigger the display of different AR special effects based on different change types of physical items. Through the above The processing method can not only increase the user's interest, but also improve the user's experience of using AR technology, meet the diverse user needs of the user, and can also widely promote the relevant information of the bottle.

In an optional implementation manner, the detecting that the physical item associated with the bottle in the real-scene image changes includes: determining a target identification code associated with the bottle in the real-scene image; When it is detected that the target identification code has changed, it is determined that the physical item associated with the bottle in the real-scene image has changed.

In the above embodiment, since a physical item corresponds to an identification code, and the association between the bottle body and the target identification code is preset, by determining the change type of the target identification code associated with the bottle body in the real scene image, the The method of determining whether the physical item in the real-scene image has changed can quickly detect the change of the physical item, thereby quickly determining the AR special effect triggered by the bottle body and the physical item after the status change, thereby improving the user experience.

In an optional implementation manner, the detecting that the physical item associated with the bottle body in the real-scene image changes includes: performing image processing on the real-scene image to obtain the bottle in the real-scene image. The image detection result of the entity item associated with the object; and determining the change information of the entity item in the real-scene image according to the image detection result.

In the above embodiments, by performing image processing on the real-scene image, various state changes of physical items associated with the bottle can be detected, thereby triggering the display of more diverse AR special effects to improve user experience.

In an optional implementation manner, the displaying the real-scene image of the bottle body collected by the augmented reality AR device includes: responding to the loading request of the H5 page, loading and displaying the image on the display interface of the AR device. H5 page, wherein, the H5 page includes prompt information prompting the user to jump to the special effect display page; in response to a trigger operation on the prompt information in the H5 page, jump to the special effect display page, and in the The real scene image is displayed on the special effect display page.

In the above embodiment, by loading the H5 page, and performing corresponding trigger operations on the prompt information in the H5 page, thereby jumping to the special effect display page to display the real scene image, the bottle can be displayed without downloading the corresponding client. The AR special effect of the body, thereby saving the device memory of the AR device, can also simplify the user operation, and improve the user experience of the user using the data display method provided by the embodiment of the present application.

In an optional implementation manner, the determination of the first AR special effect triggered by the bottle body and the changed physical item includes: acquiring user information of the user to which the AR device belongs; wherein the user information Including at least one of the following: basic attributes of the user, the device type of the AR device, the number of times the user visits the special effect display page, and the historical AR special effects pushed for the user; determine at least one second AR device that matches the user information AR special effects: in the at least one second AR special effects, determine the first AR special effects triggered by the display of the bottle body and the changed physical item.

In the above embodiments, by determining at least one second AR special effect that matches the user information, and determining the first AR special effect among the at least one second AR special effect, different users can customize different styles of AR special effects. , to improve user experience.

In an optional implementation manner, the determining at least one second AR special effect matching the user information includes: if the user information contains the historical AR special effect, in the historical AR special effect, Determining the latest target historical AR special effect pushed for the user in the special effects; determining the at least one second AR special effect among the multiple preset AR special effects except the target historical AR special effect.

In the above implementation manner, through the above processing manner, it is possible to avoid continuously pushing the same AR special effect for the user, thereby improving user experience.

In an optional implementation manner, the determining the first AR special effect triggered by the bottle body and the changed physical item includes: determining the item type of the changed physical item, wherein the item type includes : a three-dimensional object and/or a two-dimensional plane object; determining the first AR special effect triggered by the bottle body and the changed physical item according to the item type.

In the above embodiment, the method of triggering the display of the first AR special effect triggered by the bottle and the changed physical object is determined according to the item type of the physical object, which can enrich the triggering methods of the AR special effect of the bottle and improve user experience.

In an optional implementation manner, the determination of the first AR special effect triggered by the bottle body and the changed physical item includes: when a change in the state of the bottle body in the real scene image is detected , to determine the changed bottle state and the first AR special effect triggered by the changed physical item.

In the above-mentioned embodiment, through the above-mentioned processing method, it is possible to trigger and display different AR special effects based on the change of the state of the bottle itself. Through the above-mentioned processing method, the user's experience of using AR technology can be improved while increasing the interest of the user. It can meet the diversified user needs of users, and can also widely promote the relevant information of the bottle body.

In an optional implementation manner, the method is applied to a client application platform, and the client application platform is a Web application platform or an applet application platform.

In the above embodiments, the AR special effect is displayed through the applet or the wed terminal, and the download process of the corresponding application program APP can be omitted, thereby simplifying the display process of the AR special effect.

In the second aspect, the embodiment of the present application also provides a data display device, including: a first display unit configured to display a real scene image containing a bottle collected by an augmented reality AR device; a determination unit configured to detect the When the physical item associated with the bottle in the real scene image changes, determine the first AR special effect triggered by the bottle and the changed physical item; the second display unit is configured to be displayed on the AR device The first AR special effect is displayed on the special effect display interface.

In the third aspect, the embodiment of the present application also provides a computer device, including: a processor, a memory, and a bus, the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processing The processor communicates with the memory through a bus, and when the machine-readable instructions are executed by the processor, the above-mentioned first aspect, or the steps in any possible implementation manner of the first aspect are executed.

In the fourth aspect, the embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the above-mentioned first aspect, or any one of the first aspects in the first aspect, is executed. Steps in one possible implementation.

In the fifth aspect, an embodiment of the present application provides a computer program product, the computer program product includes a non-transitory computer-readable storage medium storing a computer program, and when the computer program is read and executed by a computer, the implementation of the present application can be realized. Some or all steps of the method described in the example. The computer program product may be a software installation package.

In order to make the above-mentioned purpose, features and advantages of the present application more comprehensible, preferred embodiments will be described in detail below together with the accompanying drawings.

Description of drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the accompanying drawings that need to be used in the embodiments. The accompanying drawings here are incorporated into the specification and constitute a part of the specification. The figures show the embodiments according to the application, and are used together with the description to explain the technical solution of the application. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, they can also From these drawings other related drawings are obtained.

FIG. 1 is a flow chart of a data presentation method provided by an embodiment of the present application;

Fig. 2 is a schematic diagram of the effect of a special effect display page provided by the embodiment of the present application;

FIG. 3 is a schematic diagram of the effect of the second special effect display page provided by the embodiment of the present application;

FIG. 4 is a schematic diagram of the effect of the third special effect display page provided by the embodiment of the present application;

FIG. 5 is a schematic diagram of a data display device provided by an embodiment of the present application;

FIG. 6 is a schematic structural diagram of a computer device provided by an embodiment of the present application.

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only It is a part of the embodiments of this application, not all of them. The components of the embodiments of the application generally described and illustrated in the figures herein may be arranged and designed in a variety of different configurations. Accordingly, the following detailed description of the embodiments of the application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of the application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative efforts belong to the scope of protection of the present application.

It should be noted that like numerals and letters denote similar items in the following figures, therefore, once an item is defined in one figure, it does not require further definition and explanation in subsequent figures.

The term "and/or" in this article only describes an association relationship, which means that there can be three kinds of relationships, for example, A and/or B can mean: there is A alone, A and B exist at the same time, and B exists alone. situation. In addition, the term "at least one" herein means any one of a variety or any combination of at least two of the more, for example, including at least one of A, B, and C, which may mean including from A, Any one or more elements selected from the set formed by B and C.

After research, it is found that AR augmented reality technology is a relatively new technical content that promotes the integration of real world information and virtual world information content, so that the real environment and virtual objects are presented in the same picture or space in real time. In recent years, with the rapid development of smart terminal devices, AR devices have become more and more widely used, for example, AR cultural and creative products. In the existing AR technology, a single object can be identified, and the virtual content of the single object can be triggered to be displayed, and the virtual content of the object is usually fixed virtual content such as an object introduction of the object. Therefore, the application scenarios of the existing AR technology are relatively single, and cannot meet the increasingly rich demands of users.

Based on the above research, an embodiment of the present application provides a data presentation method. In the embodiment of the present application, when it is detected that the physical item associated with the bottle in the real scene image has changed, it can be determined that the first AR special effect triggered by the bottle and the changed physical item is displayed, and the special effect The first AR special effect is displayed on the display page. Through the above processing method, it is possible to trigger and display different AR special effects based on the change of the item combination of the bottle body and the physical item associated with the bottle body. Through the above processing method, the trigger conditions of AR special effects can be enriched, and users and bottles can also be realized. AR interaction between objects. While increasing the user's interest, it can improve the user's experience of using AR technology, meet the diverse user needs of the user, and can also widely promote the relevant information of the bottle.

To facilitate the understanding of this embodiment, a data display method disclosed in the embodiment of the present application is firstly introduced in detail. The data display method provided in the embodiment of the present application is generally executed by a computer device with certain computing capabilities. In some possible implementation manners, the data presentation method may be implemented by a processor invoking computer-readable instructions stored in a memory.

Fig. 1 is a flow chart of a data display method provided by the embodiment of the present application, the method is applied to a client application platform, and the client application platform can be a small program application platform or a Web application platform, or can also be an independent application.

In the technical solution of the embodiment of the present application, based on the client application platform (for example, a small program application platform or a Web application platform), the display of the bottle body and the item combination corresponding to the physical item associated with the bottle body is triggered. AR special effects can realize the display of AR special effects without installing the corresponding application APP. For example, open a small program application platform on a social platform, and then take real-scene images in the small program, and display AR special effects; or open a web-side application platform in a browser, and then take real-scene images on the web side, and display AR special effects. Therefore, the technical solution of the embodiment of the present application displays the AR special effects through the applet or the wed terminal, and can omit the download process of the corresponding application program APP, thereby simplifying the display process of the AR special effects.

In the embodiment of the present application, the data presentation method in FIG. 1 includes steps S101-S105, wherein:

S101, displaying a real-scene image including a bottle collected by an augmented reality AR device.

In the embodiment of the present application, steps S101 to S105 may be performed by a computer device.

Here, the computer device for displaying the real-scene image and the AR device for capturing the real-scene image may be the same device, or may be different devices.

In this embodiment of the application, the AR device may be an AR wearable device, an AR handheld device, and other terminal devices supporting AR technology.

In an optional implementation manner, an AR handheld device (for example, an AR smart terminal) may collect the real-scene image, and display the real-scene image on the AR device.

In an optional implementation, the AR handheld device (for example, an AR smart terminal) can collect the real scene image; then, the AR handheld device can send the real scene image to the AR wearable device based on the communication connection with the AR wearable device. displayed on the device.

In the embodiment of the present application, the bottle body may be a container for carrying water or containing water, and the material of the bottle body may be any type of material, for example, plastic material, glass material, iron material, ceramic material and the like. For example, the bottle body can be a beverage bottle, any type of iron (or glass material, or plastic material) storage bottle body. The embodiment of the present application does not limit the material and shape of the bottle body.

S103. In a case where a change in the physical item associated with the bottle body in the real-scene image is detected, determine a first AR special effect triggered by the bottle body and the changed physical item.

After the real-scene image containing the bottle is acquired, it can be detected whether the physical item associated with the bottle in the real-scene image has changed. If a change is detected, the first AR special effect triggered by the bottle body and the changed physical item is determined.

Here, the change of the physical item can be understood as at least one of the following: addition of the physical item in the real-scene image, deletion of the physical item in the real-scene image, or change of the placement status of the physical item in the real-scene image. The change of the placement state of the physical object can be understood as at least one of the following: opening the bottle cap of the physical object, closing the bottle cap of the physical object, placing the physical object on the carrier, removing the physical object from the carrier, placing the physical object Changes in any placement state such as inclined placement.

In addition, the change of the placement status of the physical item may also be any other type of placement status change, which will not be listed one by one in the embodiment of the present application.

The physical item may be a bottle body, or may be other items other than the bottle body.

In the case that the physical item is a bottle, the physical item can be a bottle with a different number from the bottle in the real image, or a bottle with a different specification from the bottle in the real image, or a bottle with a different specification from the bottle in the real image. Bottles with different types of bottles.

In addition, the physical item can also be a combination of items. The combination can be a combination of a three-dimensional item and a two-dimensional plane object, or a combination of a three-dimensional item and a three-dimensional item, or a two-dimensional plane object and a two-dimensional plane object. The combination. For example, the combination may be a combination of a bottle and a non-bottle three-dimensional object, or a combination of multiple non-bottle three-dimensional objects.

In the embodiment of the present application, the two-dimensional planar object may be a card-like object such as a postcard, an admission ticket, or a leaflet. In addition, the physical object other than the target bottle may also be other types of objects, which is not limited in this embodiment of the present application.

S105. Display the first AR special effect on a special effect display interface of the AR device.

The above step S101 to step S105 will be described in detail below.

In an optional implementation manner, the above step S101, displaying the real scene image including the bottle body collected by the augmented reality AR device, includes the following process:

Step S1011, in response to the loading request of the H5 page, load and display the H5 page on the display interface of the AR device, wherein the H5 page includes prompt information prompting the user to jump to the special effect display page;

Step S1012, jumping to the special effect display page in response to a trigger operation on the prompt information in the H5 page, and displaying the real scene image on the special effect display page.

Figure 2 is a schematic diagram of a special effect display page provided by the embodiment of the present application. In the embodiment of the present application, the AR device starts to scan the bottle body in response to the scanning instruction initiated by the user, and when the identification code of the bottle body is scanned In this case, initiate a loading request for the H5 page. At this time, the AR device requests the server to load the H5 page, and displays the H5 page returned by the server on the display interface of the AR device. At this time, as shown in Figure 2, it will be displayed on the display interface of the AR device for prompting The prompt information for the user to jump to the target browser. Wherein, the web page of the target browser is a special effect display page for displaying corresponding special effects.

Here, the identification code may be the bottle body itself, or an identification having a relative positional relationship with the bottle body. For example, the identification code may be a logo pasted on the bottle body, may also be a logo pasted on the bottle body carrier, or may be a logo set independently from the bottle body.

When the identification code is an identification having a relative positional relationship with the bottle body, the identification code may be a template card or a two-dimensional code drawn with a certain specification and shape, for example, as shown in FIG. 2 . In addition, other identifiers may also be used, which are not limited in this embodiment of the present application.

After the user clicks the prompt information on the H5 page, it is determined that a user's trigger operation on the prompt information in the H5 page is detected. At this point, in response to the trigger operation, jump to the web page of the target browser (that is, the special effect display page), and display the real-scene image of the bottle body collected by the augmented reality AR device on the web page.

In addition to the methods for displaying real-scene images described in S1011 to S1012 above, real-scene images can also be displayed in the following manner, and the process is described as follows:

Open the AR applet in the client installed on the AR device that supports opening the applet. After opening the applet, enter the special effect display page, and display the real image of the bottle collected by the AR device on the special effect display page.

In the embodiment of the present application, after displaying the real-scene image including the bottle captured by the AR device, it can be detected whether the physical item associated with the bottle in the real-scene image has changed. If it is detected that the physical item associated with the bottle in the real-scene image has changed, it is determined that the physical item after the change of the bottle triggers the displayed first AR special effect.

In an optional implementation manner, detecting that the physical item associated with the bottle in the real-scene image changes, includes the following process:

Step S11, determining the target identification code associated with the bottle body in the real-scene image;

Step S12, if it is detected that the target identification code has changed, it is determined that the physical item associated with the bottle in the real-scene image has changed.

In the embodiment of the present application, after the real scene image is collected and displayed on the special effect display page of the AR device, the identification code of the real scene image can be detected to determine the bottle body in the real scene image according to the detection result. The associated target identifier.

Here, the identification code may be the physical item itself, or the second identification may also be an identification attached to the physical item.

Here, if the real scene image contains multiple bottles, it is necessary to determine the target identification code associated with each bottle in the real scene image, and the detection process is described as follows:

In an optional embodiment, the real-scene image can be used to detect the physical object to obtain the detection result, wherein the detection result includes: the position of each physical object in the real-scene image, and the feature information of each physical object . At this time, the feature information can be used as the identification code of each physical item, and the target identification code associated with the bottle body can be determined from the identified identification codes. For example, the target identification code can be determined by querying the data table. Wherein, the data table includes the identification codes (ie, characteristic information) of each physical item associated with each bottle.

In another optional embodiment, the identification code set on the physical item can be detected, wherein the identification code can be a mark in the form of a two-dimensional code, a bar code, etc.; The target identification code is determined in the identification code. Wherein, the data table includes the identification codes (ie, two-dimensional codes, barcodes and other information) of each physical item associated with each bottle.

In the embodiment of the present application, after the target identification code associated with the bottle body in the real-scene image is determined, it can be determined whether the target identification code has changed. If it is determined that a change has occurred, it is determined that the physical item associated with the bottle in the real-scene image has changed.

After it is determined that the physical item associated with the bottle body has changed, the change type of the physical item may also be determined according to the change type of the target identification code. For example, if it is detected that the change type of the target identification code is: new addition of the target identification code, then it is determined that the change type of the physical item is: new addition of the physical item; if the change type of the detected target identification code is: new addition of the target identification code Deletion, then determine the change type of the physical item as: deletion of the physical item.

After determining the change type of the physical item, the change type can be determined to determine the first AR special effect triggered by the bottle and the changed physical item, and display the first AR special effect on the special effect display interface of the AR device .

For example, when the change type of the physical item is the addition of the physical item, the first AR special effect that is triggered to be displayed may be a fusion effect of the AR special effect displayed at the current moment and the AR special effect corresponding to the newly added physical item. When the change type of the state of the physical item is deletion of the physical item, the first AR special effect that is triggered to be displayed may be the first AR special effect after canceling the AR special effect corresponding to the deleted physical item. When the change in the state of the physical item is a change in the placement state of the physical item, then the first AR special effect may be an AR special effect corresponding to the change process of the placement state of the physical item.

In an optional implementation, it may be determined whether the target identification code has changed in the following ways, including:

First, detect whether the number of target identification codes in the real-scene image has changed. If there is a change, image recognition is performed on the target identification code in the real-scene image, so as to determine the change type of the target identification code according to the image recognition result. If it is determined that the number of target identification codes has not changed, it can be set to detect the number of target identification codes in the real scene image every few frames, and when the number of target identification codes is detected to change, the target The identification code is subjected to image recognition, so as to determine the change type of the target identification code according to the image recognition result.

In another optional implementation manner, detecting that the physical item associated with the bottle in the real-scene image changes, further includes the following process:

Step S21, performing image processing on the real-scene image to obtain an image detection result of a physical object associated with the bottle in the real-scene image;

Step S22, determining change information of the physical item in the real-scene image according to the image detection result.

In this embodiment of the present application, in addition to the way of detecting the state of the bottle body described in the above step S11 and step S12, the state of the bottle body described in step S21 and step S22 can also be detected.

For the above step S11 and step S12, if it is detected that the target identification code has not changed, it is also possible to detect whether the physical item associated with the bottle has changed in the manner described in step S21 and step S22.

From the above description, it can be seen that the change of the physical item can be understood as at least one of the following: addition of the physical item in the real-scene image, deletion of the physical item in the real-scene image, or change of the placement status of the physical item in the real-scene image. When the placement state of the physical item changes, the target identification code may not change (for example, the quantity does not change). In order to accurately detect the change of the physical item, it is possible to detect whether the physical item associated with the bottle has changed through the steps S21 and S22. The process is described as follows:

In the embodiment of the present application, the item information of all physical items associated with the bottle body is pre-stored in the database, wherein the item information may be at least one item image of the physical item in various placement states, It may also be item feature information of the physical item in various placement states.

Here, image processing may be performed on the real-scene image through an image processing model to obtain item information of all physical items included in the real-scene image, for example, an item image of each physical item or item feature information of the physical item.

After the item information of all the physical items is determined, the item information can be compared with the item information in the database, so as to determine the physical item associated with the bottle among all the physical items. Afterwards, it may be determined whether the physical item associated with the bottle has changed in combination with the determined physical item associated with the bottle at the previous moment.

After it is determined that the physical item associated with the bottle has changed, the first AR special effect triggered by the bottle and the changed physical item can be determined according to the change type of the physical item, and the special effect in the AR device The first AR special effect is displayed on the display interface.

When it is detected that the physical item associated with the bottle in the real scene image has changed, for the step in step S103: determining the first AR special effect triggered by the bottle and the changed physical item, including The following process:

S1031. Determine a change type of the physical item, wherein the change type includes at least one of the following: addition of a physical item, deletion of a physical item, or change of a placement state of a physical item;

S1032. Determine, according to the change type, the first AR special effect triggered by the display of the bottle body and the changed physical item.

In the embodiment of the present application, after it is determined that the physical item associated with the bottle has changed, the change type of the physical item may be determined, and then the first AR special effect may be determined according to the change type.

Here, the change type includes at least one of the following: addition of physical items, deletion of physical items, and change of placement status of physical items. For each change type, one or more subtypes can be set correspondingly.

For example, if the change type is the addition of physical items, the subtype corresponding to the change type can be a subtype determined according to the following data dimensions: the number of new physical items, the type of new physical items, the number of new physical items increasing order etc.

For example, if the change type is the deletion of physical items, the subtype corresponding to the change type can be a subtype determined according to the following data dimensions: the number of deleted physical items, the type of deleted physical items, the number of deleted physical items order of deletion, etc.

One or more AR special effects can be correspondingly set for each subtype of the change type. And the AR special effects corresponding to different subtypes in any variation type are not exactly the same.

In the embodiment of the present application, after the change type of the physical item is determined, at least one subtype to which the change type belongs may be determined, and then, in the at least one subtype, the target subtype to which the change process of the physical item belongs is determined, And according to the determined target subtype, determine the first AR special effect triggered by the bottle body and the changed physical item.

In an optional implementation, if it is determined that there are multiple variation types of the physical item, a matching AR special effect may be determined for each variation type, and then, according to the determined variation types, determine the bottle The first AR special effects displayed are triggered by the body and the changed physical items.

For the above step S1032, determine the first AR special effect triggered by the bottle and the physical item after the change according to the change type, including the following process:

(1) Determine the target AR special effect matching the change type;

(2) Determine the fused AR special effect of the target AR special effect and the initial AR special effect as the first AR special effect, and the initial AR special effect is displayed on the special effect display interface before the physical item in the real-scene image changes The AR effects shown above.

In an optional implementation manner, in a case where it is determined according to the change type that the real-scene image contains a newly added physical item, a target AR special effect matching the newly added physical item is determined. At this time, the target AR special effect may also be determined according to the type of the newly added physical item.

In some embodiments, the types of physical items can be classified according to the attribute parameters of the physical items. For example, the attribute parameters of the physical items can include at least one of the following: the type of the physical item, the color of the physical item, the function of the physical item, the entity The pattern of the item, the size of the physical item, etc.

For example, if it is determined that the physical item is a children's bottle, a target AR special effect matching the "children's bottle" may be determined. If it is determined that the physical item is "the commemorative bottle of the XXX sports event", the target AR special effect matching the "XXX sports event" can be determined.

After the target AR special effect is determined, the target AR special effect and the initial AR special effect may be fused according to the target AR special effect and the special effect type of the initial AR special effect, and the fused AR special effect after fusion may be determined as the first AR special effect.

Here, fusion can be understood as sequentially connecting the target AR special effect and the initial AR special effect to form the first AR special effect. The fusion can also be understood as merging the special effect content of the target AR special effect and the initial AR special effect to obtain the first AR special effect.

In the above embodiment, by combining the target AR special effect with the initial AR special effect to obtain the first AR special effect, it is possible to trigger the display of different AR special effects based on the change of the item combination of the bottle body and its corresponding physical item , through the above-mentioned processing method, while increasing the interest of the user, the user experience of the AR technology can be improved, the diversified user needs of the user can be met, and the relevant information of the bottle body can also be widely promoted.

In an optional embodiment, when it is determined based on the change type that the placement state of the physical item in the real-scene image has changed, according to the target AR that matches the change process of the placement state of the physical item The special effect determines the first AR special effect.

Here, the fused AR special effect of the target AR special effect and the initial AR special effect may be determined as the first AR special effect.

Assuming that the physical item is another bottle, and the change process of the placement state of the physical item is: the bottle cap of the physical item is closed-the bottle cap of the physical item is opened, then determine the target AR special effect matching the change process. For example, the target AR special effect may be an AR special effect in which water in the bottle body sprays out.

Figure 3 is a schematic diagram of the effect of the second special effect display page provided by the embodiment of the present application. If the initial AR special effect is the AR special effect of "little dolphin sprays out from the bottle", then as shown in Figure 3, when the physical item is placed The state change process is: the bottle cap of the physical object is closed-the bottle cap of the physical object is opened, and the first AR special effect can be a fusion effect of the initial AR special effect and the target AR special effect: that is, water is sprayed from the physical object, and the little dolphin is released from the The inside of the bottle sprays outward.

Assuming that the change process of the placement state of the physical item is: the bottle cap of the physical item is opened-the bottle cap of the physical item is closed, then the target AR special effect matching the change process is determined. For example, the target AR special effect may be an AR special effect in which water in the physical object is retracted into the physical object.

In the above embodiment, by determining the first AR special effect and displaying the first AR special effect according to the target AR special effect matched with the change process of the placement state of the physical item, it is possible to trigger the display of different AR special effects based on the change of the state of the physical item itself, Through the above-mentioned processing method, while increasing the interest of the user, the user experience of the AR technology can be improved, the diversified user needs of the user can be met, and the relevant information of the bottle body can also be widely promoted.

When it is detected that the physical item associated with the bottle in the real scene image changes, for the step in step S103: the first AR special effect triggered by the determination of the bottle and the changed physical item is displayed , also includes the following process:

(1) Obtain the user information of the user to which the AR device belongs; wherein, the user information includes at least one of the following: basic user attributes, device type of the AR device, number of times the user visits the special effect display page, and Historical AR special effects pushed by the user;

(2) Determine at least one second AR special effect matching the user information;

(3) In the at least one second AR special effect, determine the first AR special effect triggered by the display of the bottle body and the changed physical item.

In this embodiment of the application, when the AR device is triggered to load the H5 page, the user information of the user to which the AR device belongs can be obtained.

Here, the basic attribute of the user may be at least one of the following: attribute information such as age, gender, and hobbies. The device type of the AR device may be at least one of the following: a head-mounted AR device, a handheld AR device. The number of times the user visits the special effect display page can be understood as: the number of times the user visits the special effect display page (or loads an H5 page) within a period of time. The historical AR special effects pushed for the user can be understood as: the record of pushing AR special effects for the user with the same user information in the past period of time.

After the user information is determined, at least one second AR special effect matching the user information may be determined.

For example, different styles of second AR special effects may be determined for users according to different basic user attributes, different visit times, historical AR special effects, and different device types.

After at least one second AR special effect is determined, the first AR special effect triggered by the bottle body and the changed physical item can be determined in at least one second AR special effect. The process is described as follows:

The change type of the physical item may be determined, and then according to the change type, the at least one second AR special effect is determined to be the same as the first AR special effect.

For each second AR special effect, a corresponding special effect tag can be set for it, wherein the special effect tag is used to indicate the change type of the physical item. One or more special effect tags can be set corresponding to each second AR special effect, and the special effect tags corresponding to different second AR special effects are not completely the same.

At this time, the second AR special effect matching the change type may be determined as the first AR special effect according to the special effect tag of each second AR special effect.

In the embodiment of the present application, the above step: determining at least one second AR special effect matching the user information includes the following process:

(1) In the case that the user information includes the historical AR special effects, determine the latest target historical AR special effects pushed for the user in the historical AR special effects;

(2) Determine the at least one second AR special effect among the plurality of preset AR special effects other than the target historical AR special effect.

In the embodiment of the present application, if the user information includes the historical AR special effect, the latest target historical AR special effect pushed for the user is determined according to the historical AR special effect. Then, among the plurality of preset AR special effects included in the special effect library, the AR special effect matching the user information is selected from other AR special effects except the target historical AR special effect as the second AR special effect.

In this embodiment of the application, after the user information is obtained, the third AR special effect can also be displayed on the special effect display page according to the user information. For example, the third AR special effect can be the user of another user with the same basic attributes as the user Information, for example, the third AR special effect may also be a historical AR special effect pushed by the user.

FIG. 4 is a schematic diagram of the effect of the third special effect display page provided by the embodiment of the present application. Here, the display may be performed as shown in FIG. 4 . As shown in FIG. 4 , in the upper left corner of the special effect display page, the other user may be displayed in the form of bubbles with gradually changing sizes.

(1) Determine the item type of the physical item after the change, wherein the item type includes: a three-dimensional object and/or a two-dimensional planar object;

(2) According to the item type, determine the first AR special effect triggered by the bottle body and the changed physical item.

In the embodiment of the present application, after it is determined that the physical item associated with the bottle changes, it may be determined whether the item type of the changed physical item is a three-dimensional object or a two-dimensional planar object.

Here, the item type corresponding to the physical item can be determined through the identification code corresponding to each physical item.

For example, in the case that the identification code is the physical item itself, its item type can be determined by analyzing the item attributes of the physical item itself. In the case that the identification code is a two-dimensional code attached to a physical item, the type of the item can be determined by identifying the information of the two-dimensional code.

After the item type is determined, the AR special effect triggered by the physical item can be determined, which is recorded as AR special effect 1, and the AR special effect triggered by the bottle body can be determined, which is recorded as AR special effect 2; after that, the AR special effect can be determined according to the item type 1 and AR special effect 2, and then, according to the special effect fusion method, AR special effect 1 and AR special effect 2 are fused to obtain the first AR special effect.

In a case where a change in the state of the bottle body in the real-scene image is detected, the first AR special effect triggered by the changed state of the bottle body and the physical item after the change is determined.

In the embodiment of the present application, in addition to detecting changes in physical objects related to the bottle, changes in the state of the bottle may also be detected. Wherein, the change of the bottle state can be understood as the change of the bottle placement state, for example, the change of the bottle cap opening of the bottle body is the bottle cap closure, the change of the bottle cap closure of the bottle body is the bottle cap opening, and the vertical change of the bottle body is The bottle is placed horizontally (or placed obliquely).

After the change of the bottle state is detected, the AR special effect triggered by the changed bottle state can be determined, which is recorded as AR special effect 3, and the AR special effect triggered by the changed physical object can be determined, which is recorded as AR special effect 4 , after that, the AR special effect 3 and the AR special effect 4 are combined to obtain the first AR special effect.

Those skilled in the art can understand that in the above method of actual implementation, the order of writing of each step does not mean a strict execution order and constitutes any limitation on the implementation process. The actual execution order of each step should be based on its function and possible The inner logic is OK.

Based on the same inventive concept, the embodiment of the present application also provides a data display device corresponding to the data display method. Since the problem-solving principle of the device in the embodiment of the present application is similar to the above-mentioned data display method in the embodiment of the present application, the implementation of the device See the implementation of the method.

Fig. 5 is a schematic diagram of a data display device provided by an embodiment of the present application. As shown in Fig. 5, the device includes: a first display unit 51, a determination unit 52, and a second display unit 53; wherein,

The first display unit 51 is configured to display the real scene image including the bottle collected by the augmented reality AR device;

The determining unit 52 is configured to determine the first AR special effect triggered by the bottle and the changed physical item when it is detected that the physical item associated with the bottle in the real-scene image changes;

The second display unit 53 is configured to display the first AR special effect on a special effect display interface of the AR device.

In a possible implementation manner, the determining unit is configured to: determine a change type of the physical item, wherein the change type includes at least one of the following: addition of a physical item, deletion of a physical item, change of a physical item A change in the placement state; determining the first AR special effect triggered by the bottle body and the changed physical item according to the change type.

In a possible implementation manner, the determining unit is further configured to: determine a target AR special effect matching the change type; determine a fusion AR special effect of the target AR special effect and an initial AR special effect as the first AR special effect The initial AR special effect is the AR special effect displayed on the special effect display interface before the physical item in the real-scene image changes.

In a possible implementation manner, the device is further configured to: determine a target identification code associated with the bottle body in the real-scene image; if a change in the target identification code is detected, determine the The physical item associated with the bottle in the live image has changed.

In a possible implementation manner, the device is further configured to: perform image processing on the real-scene image to obtain an image detection result of a physical item associated with the bottle body in the real-scene image; detect As a result, the change information of the entity item in the real-scene image is determined.

In a possible implementation manner, the display unit is configured to: in response to a loading request of the H5 page, load and display the H5 page on the display interface of the AR device, wherein the H5 page includes prompting the user to skip Go to the prompt information on the special effect display page; in response to the trigger operation on the prompt information in the H5 page, jump to the special effect display page, and display the real scene image on the special effect display page.

In a possible implementation manner, the determining unit is further configured to: acquire user information of a user to which the AR device belongs; wherein the user information includes at least one of the following: basic attributes of the user, device type of the AR device, the user's The number of visits to the special effect display page, the historical AR special effects pushed for the user; determining at least one second AR special effect matching the user information; determining the bottle body in the at least one second AR special effect And the first AR special effect triggered by the changed physical items.

In a possible implementation manner, the determining unit is further configured to: if the user information contains the historical AR special effects, determine the latest target historical AR special effects pushed for the user in the historical AR special effects ; Determining the at least one second AR special effect among the plurality of preset AR special effects other than the target historical AR special effect.

In a possible implementation manner, the determining unit is further configured to: determine the item type of the physical item after the change, wherein the item type includes: a three-dimensional object and/or a two-dimensional planar object; The first AR special effects displayed are triggered by the above-mentioned bottle body and the changed physical items.

In a possible implementation manner, the determining unit is further configured to: in the case of detecting that the state of the bottle body in the real-scene image changes, determine the state of the bottle body after the change and the display triggered by the physical item after the change. The first AR special effects.

In a possible implementation manner, the method is applied to a client application platform, and the client application platform is a Web application platform or an applet application platform.

For the description of the processing flow of each module in the device and the interaction flow between the modules, reference may be made to the relevant description in the above method embodiment, and details will not be described here.

Corresponding to the data presentation method in FIG. 1 , an embodiment of the present application also provides a computer device. FIG. 6 is a schematic structural diagram of a computer device provided in an embodiment of the present application. As shown in FIG. 6, the computer device 600 includes:

Processor 61, memory 62, and bus 63; memory 62 is used for storing and executing instruction, comprises memory 621 and external memory 622; memory 621 here is also called internal memory, is used for temporarily storing computing data in processor 61, and The data exchanged by the external memory 622 such as hard disk, the processor 61 exchanges data with the external memory 622 through the memory 621, when the computer device 600 is running, the processor 61 communicates with the memory 62 through the bus 63, so that The processor 61 executes the following instructions:

Display the real scene image including the bottle body collected by the augmented reality AR device;

In the case of detecting that the physical item associated with the bottle in the real scene image changes, determine the first AR special effect triggered by the bottle and the changed physical item;

The first AR special effect is displayed on the special effect display interface of the AR device.

The embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is run by a processor, the steps of the data presentation method described in the foregoing method embodiments are executed. Wherein, the storage medium may be a volatile or non-volatile computer-readable storage medium.

The embodiment of the present application also provides a computer program product, the computer program product is loaded with program code, and the instructions included in the program code can be used to execute the steps of the data display method described in the above method embodiment, please refer to the above method implementation example. That is, the computer program product provided by the embodiments of the present application includes a computer-readable storage medium that stores program codes, and the instructions included in the program codes can be used to execute the steps of the data display method described in the above method embodiments. For implementation, refer to the above method embodiments.

Wherein, the above-mentioned computer program product may be realized by hardware, software or a combination thereof. In an optional embodiment, the computer program product may be embodied as a computer storage medium, and in another optional embodiment, the computer program product may be embodied as a software product, such as a software development kit (Software Development Kit, SDK), etc. Wait.

Those skilled in the art can clearly understand that for the convenience and brevity of description, the actual working process of the system and device described above can refer to the corresponding process in the foregoing method embodiments. In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods may be implemented in other ways. The device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or May be integrated into another system, or some features may be ignored, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some communication interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit.

If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium executable by a processor. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including Several instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disc and other media that can store program codes. .

Finally, it should be noted that: the above-described embodiments are only specific implementations of the application, used to illustrate the technical solutions of the application, rather than limiting it, and the scope of protection of the application is not limited thereto, although referring to the aforementioned The embodiment has described this application in detail, and those of ordinary skill in the art should understand that any person familiar with this technical field can still modify the technical solutions described in the foregoing embodiments within the technical scope disclosed in this application Changes can be easily imagined, or equivalent replacements can be made to some of the technical features; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the application, and should be covered by this application. within the scope of protection. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Industrial Applicability

The embodiment of the present application provides a data display method and device, device, storage medium and computer program product, the method includes: displaying a real scene image including a bottle collected by an augmented reality AR device; When the physical item associated with the bottle changes, determine the first AR special effect triggered by the bottle and the changed physical item; display the first AR on the special effect display interface of the AR device special effects. According to the data display method provided in the embodiment of the present application, different AR special effects can be triggered based on the change of the item combination of the bottle and the physical item associated with the bottle, which can enrich the trigger conditions of the AR special effects, and can also realize the user and The AR interaction between the bottles; while increasing the user's interest, it can improve the user's experience of using AR technology, meet the diverse user needs of the user, and can also widely promote the relevant information of the bottle.

Claims

A method for displaying data, including:

Display the real scene image including the bottle body collected by the augmented reality AR device;

In the case of detecting that the physical item associated with the bottle in the real scene image changes, determine the first AR special effect triggered by the bottle and the changed physical item;

The first AR special effect is displayed on the special effect display interface of the AR device.
The method according to claim 1, wherein the determining the first AR special effect triggered by the bottle body and the changed physical item comprises:

Determine the type of change of the physical item, wherein the type of change includes at least one of the following: addition of a physical item, deletion of a physical item, or a change in the placement status of a physical item;

The first AR special effect triggered by the bottle body and the changed physical item is determined according to the change type.
The method according to claim 2, wherein the determining the first AR special effect triggered by the display of the bottle body and the physical item after the change according to the change type includes:

determining a target AR special effect matching the change type;

Determining the fused AR special effect of the target AR special effect and the initial AR special effect as the first AR special effect, and the initial AR special effect is displayed on the special effect display interface before the physical item in the real-scene image changes AR special effects.
The method according to any one of claims 1 to 3, wherein the detection of a change in the physical item associated with the bottle in the real-scene image includes:

Determining the target identification code associated with the bottle in the real-scene image;

If it is detected that the target identification code has changed, it is determined that the physical item associated with the bottle in the real-scene image has changed.
The method according to any one of claims 1 to 4, wherein the detection of a change in the physical item associated with the bottle in the real-scene image includes:

performing image processing on the real-scene image to obtain an image detection result of an entity item associated with the bottle in the real-scene image;

Determining change information of the entity item in the real-scene image according to the image detection result.
The method according to any one of claims 1 to 5, wherein the displaying the real scene image including the bottle body collected by the augmented reality AR device includes:

In response to the loading request of the H5 page, load and display the H5 page on the display interface of the AR device, wherein the H5 page includes prompt information prompting the user to jump to the special effect display page;

In response to a trigger operation on the prompt information in the H5 page, jump to the special effect display page, and display the real scene image on the special effect display page.
The method according to any one of claims 1 to 6, wherein the determining the first AR special effect triggered by the display of the bottle body and the changed physical item comprises:

Obtain user information of the user to which the AR device belongs; wherein, the user information includes at least one of the following: basic attributes of the user, device type of the AR device, number of times the user visits the special effect display page, and the number of visits pushed for the user. Historical AR special effects;

determining at least one second AR special effect matching the user information;

In the at least one second AR special effect, determine the first AR special effect triggered by the bottle body and the changed physical item.
The method according to claim 7, wherein said determining at least one second AR special effect matching said user information comprises:

In the case that the user information includes the historical AR special effects, determine the latest target historical AR special effects pushed for the user in the historical AR special effects;

The at least one second AR special effect is determined among the plurality of preset AR special effects other than the target historical AR special effect.
The method according to any one of claims 1 to 8, wherein the determining the first AR special effect triggered by the display of the bottle body and the changed entity item comprises:

Determine the item type of the physical item after the change, wherein the item type includes: a three-dimensional object and/or a two-dimensional plane object;

The first AR special effect triggered by the bottle body and the changed physical item is determined according to the item type.
The method according to any one of claims 1 to 9, wherein the determining the first AR special effect triggered by the display of the bottle body and the changed physical item comprises:

In a case where a change in the state of the bottle body in the real-scene image is detected, the first AR special effect triggered by the changed state of the bottle body and the physical item after the change is determined.
The method according to any one of claims 1 to 10, wherein the method is applied to a client application platform, and the client application platform is a Web application platform or an applet application platform.
A data display device, including:

The first display unit is configured to display the real image of the bottle body collected by the augmented reality AR device;

The determining unit is configured to determine the first AR special effect triggered by the bottle and the changed physical item when it is detected that the physical item associated with the bottle in the real-scene image changes;

The second display unit is configured to display the first AR special effect on a special effect display interface of the AR device.
A computer device, including: a processor, a memory, and a bus, the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor and the memory pass through Bus communication, when the machine-readable instructions are executed by the processor, the steps of the data presentation method according to any one of claims 1 to 11 are executed.
A computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and when the computer program is run by a processor, the steps of the data display method according to any one of claims 1 to 11 are executed.
A computer program product, the computer program product includes a non-transitory computer-readable storage medium storing a computer program, and when the computer program is read and executed by a computer, the method described in any one of claims 1 to 11 is realized. The steps of the data presentation method.