WO2022252518A1

WO2022252518A1 - Data presentation method and apparatus, and computer device, storage medium and computer program product

Info

Publication number: WO2022252518A1
Application number: PCT/CN2021/133452
Authority: WO
Inventors: 田真; 李斌; 欧华富; 王婷婷
Original assignee: 北京市商汤科技开发有限公司
Priority date: 2021-06-03
Filing date: 2021-11-26
Publication date: 2022-12-08
Also published as: TW202248961A; CN113359985A

Abstract

Provided in the present disclosure are a data presentation method and apparatus, and a computer device, a storage medium and a computer program product. The data presentation method comprises: acquiring a real-scene image, which is collected by an augmented reality (AR) device and includes a target bottle body; detecting, in the real-scene image, a first identifier corresponding to the target bottle body and a second identifier corresponding to an entity item other than the target bottle body; and on the basis of a first AR special effect corresponding to the first identifier and a second AR special effect corresponding to the second identifier, determining a third AR special effect for presentation.

Description

Data display method, device, computer equipment, storage medium and computer program product

Cross References to Related Applications

The embodiment of the present disclosure is based on the Chinese patent application with the application number 202110620107.6, the application date is June 3, 2021, and the application name is "data display method, device, computer equipment and storage medium", and the priority of the Chinese patent application is required Right, the entire content of this Chinese patent application is hereby incorporated into this disclosure as a reference.

technical field

Embodiments of the present disclosure relate to but are not limited to the field of augmented reality technology, and in particular, relate to a data display method, device, computer equipment, storage medium, and computer program product.

Background technique

Augmented reality (Augmented Reality, referred to as AR) technology is also called augmented reality. AR augmented reality technology is a relatively new technical content that promotes the integration of real world information and virtual world information content, so that the real environment and Virtual objects are presented in the same screen or space in real time. In recent years, with the rapid development of smart terminal devices, augmented reality AR devices have become more and more widely used, for example, AR cultural and creative products.

Contents of the invention

Embodiments of the present disclosure at least provide a data presentation method, device, computer equipment, storage medium, and computer program product.

An embodiment of the present disclosure provides a data presentation method, including: acquiring a real-scene image including a target bottle collected by an augmented reality AR device; detecting a first logo corresponding to the target bottle in the real-scene image, and A second logo corresponding to a physical item other than the target bottle; based on the first AR special effect corresponding to the first logo and the second AR special effect corresponding to the second logo, determine the first AR special effect for display. Three AR special effects.

An embodiment of the present disclosure also provides a data display device, including: an acquisition part configured to acquire a real-scene image including a target bottle captured by an augmented reality AR device; a detection part configured to detect that in the real-scene image, and A first logo corresponding to the target bottle, and a second logo corresponding to an entity item other than the target bottle; the determining part is configured to be based on the first AR special effect corresponding to the first logo, and The second AR special effect corresponding to the second identifier determines the third AR special effect for display.

An embodiment of the present disclosure also provides a computer device, including: a processor, a memory, and a bus, the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor and the The memories communicate through the bus, and the machine-readable instructions are executed by the processor to execute the steps of the data presentation method above.

An embodiment of the present disclosure further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the above-mentioned data presentation method are executed.

An embodiment of the present disclosure also provides a computer program product, including a computer-readable storage medium storing program codes, and when instructions included in the program codes are executed by a processor of a computer device, the steps of the above-mentioned data presentation method are implemented.

In order to make the above-mentioned objects, features and advantages of the present disclosure more comprehensible, the following specifically exemplifies the exemplary embodiments together with the accompanying drawings and describes them in detail as follows.

Description of drawings

In order to illustrate the technical solutions of the embodiments of the present disclosure more clearly, the following will briefly introduce the accompanying drawings used in the embodiments. The accompanying drawings here are incorporated into the specification and constitute a part of the specification. The drawings show the embodiments consistent with the present disclosure, and are used together with the description to explain the technical solution of the present disclosure. It should be understood that the following drawings only show some embodiments of the present disclosure, and therefore should not be regarded as limiting the scope. For those skilled in the art, they can also make From these drawings other related drawings are obtained.

FIG. 1 is a schematic diagram of the implementation flow of a data display method provided by an embodiment of the present disclosure;

FIG. 2 is a schematic diagram showing a real scene image provided by an embodiment of the present disclosure;

FIG. 3 is a schematic diagram showing a real scene image provided by an embodiment of the present disclosure;

FIG. 4 is a schematic diagram of an implementation process for obtaining a real-scene image including a target bottle collected by an augmented reality AR device in a data presentation method provided by an embodiment of the present disclosure;

FIG. 5 is a schematic diagram of prompt information for prompting the user to jump to the target browser displayed on the display interface of the augmented reality AR device in a data presentation method provided by an embodiment of the present disclosure;

FIG. 6 is a schematic diagram of a data display device provided by an embodiment of the present disclosure;

FIG. 7 is a schematic diagram of a computer device provided by an embodiment of the present disclosure.

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only It is a part of the embodiments of the present disclosure, but not all of them. The components of the disclosed embodiments generally described and illustrated in the figures herein may be arranged and designed in a variety of different configurations. Accordingly, the following detailed description of the embodiments of the present disclosure provided in the accompanying drawings is not intended to limit the scope of the claimed disclosure, but merely represents selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative effort shall fall within the protection scope of the present disclosure.

It should be noted that like numerals and letters denote similar items in the following figures, therefore, once an item is defined in one figure, it does not require further definition and explanation in subsequent figures.

The term "and/or" in this article only describes an association relationship, which means that there can be three kinds of relationships, for example, A and/or B can mean: there is A alone, A and B exist at the same time, and B exists alone. situation. In addition, the term "at least one" herein means any one of a variety or any combination of at least two of the more, for example, including at least one of A, B, and C, which may mean including from A, Any one or more elements selected from the set formed by B and C.

After research, it is found that in the existing AR technology, a single object can be recognized, and the virtual content of the single object can be triggered to be displayed, and the virtual content of the object is usually fixed virtual content such as the object introduction of the object. Therefore, the application scenarios of the existing AR technology are relatively single, and cannot meet the increasingly rich demands of users.

The defects in the above solutions are all the results obtained by the inventor after practice and careful research. Therefore, the discovery process of the above problems and the solutions to the above problems proposed by the embodiments of the present disclosure below should be Contributions made by the inventors to the disclosure during the course of the disclosure.

Based on the above research, the embodiments of the present disclosure provide a data display method, device, computer equipment, storage medium and computer program product. , and detect the second logo corresponding to the physical item other than the target bottle; and determine the third AR special effect for display based on the first AR special effect corresponding to the first logo and the second AR special effect corresponding to the second logo. AR special effects can trigger and display different AR special effects when the target bottle is combined with other physical objects. Through the above processing method, the trigger conditions of AR special effects can be enriched, and the AR interaction between the user and the bottle can be realized. While increasing the user's interest, it can improve the user's experience of using AR technology, meet the diverse user needs of the user, and can also widely promote the relevant information of the bottle.

In order to facilitate the understanding of this embodiment, a data display method disclosed in the embodiments of the present disclosure is firstly introduced in detail. The execution subject of the data display method provided in the embodiments of the present disclosure is generally a computer device with certain computing capabilities. In some implementations, the data presentation method may be implemented by a processor invoking computer-readable instructions stored in a memory.

Referring to FIG. 1 , it is a schematic diagram of an implementation process of a data display method provided by an embodiment of the present disclosure. The method is applied to a client application platform, and the client application platform can be a small program application platform or a Web application platform, or It can also be a standalone application.

In the embodiment of the present disclosure, the client application platform (for example, a small program application platform or a web application platform) is used to display the determination based on the first AR special effect corresponding to the first identifier and the second AR special effect corresponding to the second identifier. The third AR special effect can realize the display of AR special effects without installing the corresponding application APP. For example, open the small program application platform on the social platform, and then shoot real-scene images in the small program, and display the above-mentioned third AR special effect; or open the web-side application platform in the browser, and then take real-scene images on the web side , and display the above-mentioned third AR special effect.

The data presentation method includes steps S101-S105, and steps S101-S105 can be executed by computer equipment, wherein:

S101: Acquire a real-scene image including a target bottle collected by an augmented reality AR device.

During implementation, the real scene image including the target bottle body can be collected by the AR device, and the real scene image can be displayed on the display interface of the computer device. Wherein, the computer device used to display the real-scene image and the AR device that collects the real-scene image may be the same device, or may be different devices.

Here, the AR device may be any terminal device supporting AR technology, such as an AR wearable device, an AR handheld device, etc., which is not limited in this embodiment of the present disclosure.

AR wearable devices may include the following types of AR devices: helmet-type AR devices and eye-type AR devices. The AR handheld device may include the following types of AR devices: mobile terminal devices such as mobile phones supporting AR technology, and tablet computers supporting AR technology.

The target bottle body may be a container for carrying water or containing water, and the material of the target bottle body may be any type of material, for example, plastic material, glass material, iron material, ceramic material and the like. For example, the target bottle may be a beverage bottle, any type of iron (or glass, or plastic) liquid storage bottle. The embodiment of the present disclosure does not limit the material and shape of the target bottle.

S103: Detect a first mark corresponding to the target bottle and a second mark corresponding to a physical item other than the target bottle in the real-scene image.

Wherein, the real scene image may contain multiple bottles, and/or, contain multiple physical objects, at this time, the target bottle body may be determined in the real scene image, and the first identification corresponding to the target bottle body may be determined, and the real scene image may be detected The second identifier corresponding to each physical item in .

The first mark can be the target bottle itself, or an additional mark on the target bottle. The second identification may be the physical item itself, wherein the physical item may be a two-dimensional plane object or a three-dimensional item, or the second identification may also be an additional identification on the physical item.

The physical item may be a bottle body, or may be other items other than the bottle body.

In the case that the physical item is a bottle, the physical item may be a bottle with a different number, a different specification, or a different type from the target bottle.

In the case that the physical object is an item other than the bottle body, the physical item may also be a combination of items, wherein the combination of the items may be a combination of a three-dimensional item and a two-dimensional plane object, or a combination of a three-dimensional item and a three-dimensional item. combination, or a combination of two-dimensional planar objects and two-dimensional planar objects. For example, the combination may be a combination of a bottle and a non-bottle three-dimensional object, or a combination of multiple non-bottle three-dimensional objects. Here, the two-dimensional planar object may be card-like objects such as postcards, admission tickets, and leaflets.

The physical item may also be another type of object, which is not limited in this embodiment of the present disclosure.

As shown in FIG. 2 , it is a schematic diagram of the effect of a real-scene image 200 collected by an AR device. As can be seen from FIG. 2 , the target bottle 201 is placed on the desktop 202 , and the physical objects in the real scene image except the target bottle are cards 203 (postcards or admission tickets).

In some embodiments, object detection can be performed on the real-scene image, and the position information of the target bottle and the physical object can be obtained through detection. Afterwards, the information image located at the position is extracted from the real-scene image, and the sub-image B1 containing the target bottle and the sub-image B2 containing the physical item are obtained.

At this time, the first logo can be determined according to the sub-image B1, and the second logo can be determined according to the sub-image B2. For example, the sub-image B1 may be determined as the first identifier, and the sub-image B2 may be determined as the second identifier. For another example, image processing may be performed on the sub-image B1 and the sub-image B2 respectively, and the first identification and the second identification are sequentially obtained after the processing.

Wherein, the first identification obtained after processing can be used to characterize at least one of the following information: the basic attribute characteristics of the target bottle (for example, bottle type, bottle color, bottle size, bottle code), target bottle The bottle type of the body (ie, a three-dimensional item). The second identification obtained after processing can be used to characterize at least one of the following information: basic object attributes (type, color, size, code) of the physical object, and item type of the physical object (ie, a three-dimensional object or a two-dimensional plane object).

FIG. 3 is a schematic diagram showing a real-scene image 300 provided by an embodiment of the present disclosure. As shown in FIG. 3 , the target bottle 301 is placed on a desktop 302 , and the physical objects in the real-scene image other than the target bottle are cards 303 (postcard or admission ticket), wherein the target bottle body 301 is provided with an identification code 304, and the card 303 (postcard or admission ticket) is provided with an identification code 305.

In some embodiments, it is also possible to detect the identification code set on the target bottle body to obtain the first identification, and detect the identification codes set on the physical items other than the target bottle body to obtain the second identification. Here, the forms of the first identification and the second identification may be identifications for containing different AR special effect information, for example, they may be in the form of a two-dimensional code, a bar code, and the like.

S105: Based on the first AR special effect corresponding to the first identifier and the second AR special effect corresponding to the second identifier, determine a third AR special effect for presentation.

During implementation, after the first identifier and the second identifier are determined, the AR special effects corresponding to the first identifier and the second identifier may be respectively determined, and the first AR special effect and the second AR special effect are sequentially obtained.

Wherein, the above-mentioned first AR special effect may be a three-dimensional special effect (hereinafter referred to as 3D special effect), and the above-mentioned second AR special effect may be a 2D special effect (hereinafter referred to as 2D special effect). Here, the 3D special effect may be a video special effect, and the 2D special effect may be a sticker special effect.

In some implementations, the third AR special effect may include, but not limited to, at least one of the first AR special effect, the second AR special effect, a fusion AR special effect in which the first AR special effect matches the second AR special effect, and the like.

In the embodiment of the present disclosure, by detecting the first logo corresponding to the target bottle in the real scene image, and detecting the second logo corresponding to the entity item other than the target bottle; and based on the first logo corresponding to The first AR special effect and the second AR special effect corresponding to the second logo determine the third AR special effect for display, which can trigger the display of different AR special effects when the target bottle is combined with other physical objects, Through the above processing method, triggering conditions of AR special effects can be enriched, and AR interaction between the user and the bottle can also be realized. While increasing the user's interest, it can improve the user's experience of using AR technology, meet the diverse user needs of the user, and can also widely promote the relevant information of the bottle.

The process described in the above step S101 to step S105 will be introduced below.

In some implementations, step S101, as shown in FIG. 4, acquires the real scene image containing the target bottle captured by the augmented reality AR device, including:

Step S1011, in response to the page loading request, load and display the loading page on the display interface of the AR device, wherein the loading page includes prompt information prompting the user to jump to the special effect display page;

Step S1012, in response to the trigger operation on the prompt information in the loading page, jump to the special effect display page, acquire the real scene image, and display the real scene image on the special effect display page

During implementation, the augmented reality AR device starts to scan the target bottle in response to the scanning instruction initiated by the user, and initiates a page loading request when the first mark set on the target bottle is scanned. At this point, the augmented reality AR device requests the server to load the loading page, and displays the loading page returned by the server on the display interface of the augmented reality AR device. At this time, as shown in FIG. A prompt message 51 for prompting the user to jump to the target browser is displayed on the interface. Wherein, the web page of the target browser is a special effect display page for displaying corresponding special effects. The loading page is, for example, a page implemented based on HTML5.

After the user clicks the prompt information on the loading page, it is determined that a trigger operation by the user on the prompt information in the loading page is detected. At this point, in response to the trigger operation, jump to the web page of the target browser (that is, the special effect display page), and display the real-scene image of the target bottle collected by the augmented reality AR device on the web page.

In the above embodiment, by loading the loading page, and performing corresponding trigger operations on the prompt information in the loading page, so as to jump to the special effect display page to display the real scene image, it is possible to display the real scene image without downloading the corresponding client. The AR special effect of the target bottle not only saves the device memory of the augmented reality AR device, but also simplifies the user operation and improves the user experience of the user using the data display method provided by the embodiment of the present disclosure.

In some embodiments, the real scene image can also be acquired and displayed in the following ways, including:

Open the AR applet in a client that supports opening applets installed on the augmented reality AR device. After opening the applet, enter the special effect display page, and display the real scene image of the target bottle collected by the augmented reality AR device on the special effect display page.

After the real-scene image is acquired, step S103 is executed: detecting the first mark corresponding to the target bottle in the real-scene image, and the second mark corresponding to an entity item other than the target bottle, include:

(1) In the case that the number of bottles contained in the real-scene image is multiple, in response to a selection instruction for the target bottle in the real-scene image, determine that the label of the target bottle is the first a logo.

(2) Among the identifiers corresponding to all the physical items in the real-scene image, determine that the identifier associated with the first identifier is the second identifier.

During implementation, if the above-mentioned captured real-scene image contains multiple bottles, the user can select the target bottle that needs to be displayed with special effects among the multiple bottles by triggering a selection instruction.

Among them, the user can trigger the corresponding selection command by clicking on the real-scene image. At this time, the AR device can determine the target bottle that needs to be displayed with special effects according to the user's click position on the real-scene image. After the target bottle is determined, the first identification corresponding to the target bottle can be determined.

After the first identifier is determined, identifiers corresponding to all physical items in the real-scene image may be detected to obtain at least one identifier. Then, determining that the identity associated with the first identity is the second identity among the at least one identity includes:

An associated identifier that has an associated relationship with the first identifier may be searched in the data table, and then a target associated identifier that belongs to the associated identifier is determined from at least one identifier corresponding to the physical item, and the target associated identifier is determined as the second identifier. Wherein, the identification information of all identification codes associated with each first identification is preset in the data table.

During implementation, if the association between the first identifier and at least one identifier corresponding to the physical item is not found in the data table, the user can also manually establish the relationship between the first identifier and at least one identifier corresponding to the physical item , and store the established relationship in the data table.

After the first logo and the second logo are determined in the manner described above, the third AR special effect for display can be determined based on the first AR special effect corresponding to the first logo and the second AR special effect corresponding to the second logo .

In some implementation manners, after the display of the third AR special effect is completed, guide information may also be generated on the special effect display page, the guide information is used to guide the user to continue to select the target bottle for AR special effect display in the AR device, and Determine the first logo and the second logo according to the above described process, and after the first logo and the second logo are determined, based on the first AR special effect corresponding to the first logo and the second AR special effect corresponding to the second logo, determine the The third AR special effect on display. Through the above processing manner, the user's degree of freedom when watching the third AR special effect can be enriched, and the user experience is improved.

In the above-mentioned embodiment, in the case that the real-scene image contains multiple bottles, by responding to the user's selection instruction for the target bottle, by determining the first mark and the second mark according to the selection instruction, it can be realized in one In the real-time picture (that is, the real image), the AR special effects of multiple bottles are displayed separately, so that the user's special effect display operation can be simplified to further improve the user experience.

In some implementations, determining the identity of the target bottle as the first identity includes:

(1). Determine a trigger position of the selection instruction in the real-scene image.

(2) Determine the target bottle according to the trigger position, identify the mark corresponding to the target bottle, and determine the recognized mark as the first mark.

During implementation, when the user selects the target bottle through the selection instruction, the trigger position of the selection instruction in the real scene image can be determined.

Wherein, after the trigger position is determined, the trigger position and the real scene image are input into the deep learning model, so as to determine the bounding box of the target bottle body selected by the user through the deep learning model. Afterwards, the image located in the bounding box in the real-scene image is identified, so as to identify the first logo in the image.

In the case that multiple bottles are included in the real scene image, the user's triggering gesture to trigger the display of the AR special effect of each bottle can also be displayed on the special effect display page.

During implementation, a corresponding label can be set for each bottle in the special effect display page, for example: bottle 1, bottle 2, and bottle 3. For each bottle, a corresponding trigger gesture can be set, for example, the trigger gesture corresponding to bottle 1 can be an "OK" gesture, the trigger gesture corresponding to bottle 2 can be a "yeah" gesture, and the corresponding trigger gesture of bottle 3 can be The triggering gesture may be a gesture of "compassion". In addition, other trigger gestures may also be set, which is not limited in this embodiment of the present disclosure.

After the trigger gesture is displayed, the user's gesture information can be detected by the camera device of the AR device, and the target bottle that the user triggers to display can be determined according to the gesture information, and the identity of the target bottle can be determined, and the determined identity can be used as First logo.

When the number of bottles contained in the real-scene image exceeds the preset number, the foreground bottle in the real-scene image may be extracted, and a corresponding trigger gesture may be added for the foreground bottle.

In the above-mentioned embodiment, by determining the mark of the target bottle as the first mark according to the trigger position of the selection instruction in the real-scene image, it is possible to realize the AR special effect on multiple bottles in one real-time picture (that is, the real-scene image). Displaying them separately can simplify the user's operation of displaying special effects, so as to further improve user experience.

After the first logo and the second logo are detected, the third AR special effect for presentation may be determined based on the first AR special effect corresponding to the first logo and the second AR special effect corresponding to the second logo.

During implementation, detecting the first logo and the second logo in the real scene image includes:

The real-scene image is input into the image segmentation network for processing, and the bounding boxes of each logo in the real-scene image are obtained through processing. Then, the images located in each bounding box in the real-scene image are extracted to obtain multiple sub-images. Afterwards, image processing is performed on each sub-image through the image processing network to obtain the identification information of the identification contained in the sub-image.

In some embodiments, step S105 includes:

S1051. Search the fusion AR special effect matching the first AR special effect and the second AR special effect in the special effect database, wherein the special effect database includes a plurality of fusion AR special effect information identifying the respective corresponding AR special effects.

S1052. Determine the found fusion AR special effect as the third AR special effect.

After the first logo and the second logo are identified, the first AR special effect corresponding to the first logo can be determined, and the second AR special effect corresponding to the second logo can be determined. Here, the number of the first AR special effects may be multiple, and the number of the second AR special effects may also be multiple.

During implementation, an AR special effect matching the first identifier may be searched in the special effect library, and the determined matching AR special effect may be determined as the first AR special effect. It is also possible to search for an AR special effect matching the second identifier in the special effect library, and determine the matched AR special effect as the second AR special effect.

After determining the first AR special effect and the second AR special effect, you can search for integrated AR special effects in the special effects library, including:

First, the special effect tag corresponding to each AR special effect in the special effect library is obtained, where the special effect tag is used to indicate the type of each AR special effect (for example, fusion special effect or non-fusion special effect). For each fusion effect, its corresponding special effect tag also includes: special effect information of the AR special effect used for fusion; for non-fusion special effects, its corresponding special effect tag also includes: identification information (for example, the identification of the first identification information, or the identification information of the second identification).

At this point, the fusion AR special effect matching the first AR special effect and the second AR special effect can be searched in the special effect library according to the special effect tag. After the fusion AR special effect is found, a third AR special effect is determined according to the fusion AR special effect.

When one fusion AR special effect is determined, the fusion AR special effect may be determined as the third AR special effect. When the determined number of fused AR special effects is multiple, the fused AR special effect matching the user information may be selected from the multiple fused AR special effects in combination with the user information, and used as the third AR special effect.

In the embodiment of the present disclosure, the special effect library may include at least one AR special effect corresponding to the first identifier, and may further include at least one AR special effect corresponding to the second identifier. The AR special effects corresponding to each first logo can be 3D special effects, and the AR special effects corresponding to each second logo can be 2D special effects and/or 3D special effects. Fusion effects between 2D effects, and fusion effects between various 3D effects.

In some implementation manners, the special effect tag corresponding to each AR special effect in the special effect library may also include a fusion method between special effects. For example, in the special effect library, the 3D special effect record corresponding to the first identifier is record A ₁ , and the 2D special effect record corresponding to the second identifier is B ₁ . Then, the fusion AR special effect between the first AR special effect A ₁ and the 2D special effect B ₁ can be recorded as A ₁ B ₁ , and at this time, the special effect tag for the fusion AR special effect A ₁ B ₁ in the special effect library can be Contains the following information: the information of 3D special effect A ₁ and 2D special effect B ₁ , the fusion method of 3D special effect A ₁ and 2D special effect B ₁ .

In the above embodiment, by determining the third AR special effect according to the determined fusion effect of the first AR special effect and the second AR special effect, it can be realized that when the target bottle is combined with other non-bottle objects, triggering To display different AR special effects, the trigger conditions of AR special effects can be enriched through the above-mentioned processing methods, and the AR interaction between the user and the target bottle can also be realized, thereby increasing the fun of the user during use.

In some embodiments, step S105 further includes:

Perform special effect fusion processing on the first AR special effect and the second AR special effect according to a preset fusion manner to obtain the third AR special effect.

In some implementation manners, if the above-mentioned fused AR special effect is not found in the above-mentioned special effect library, the above-mentioned first AR special effect and the above-mentioned second AR special effect are subjected to special effect fusion processing according to a preset fusion method. Wherein, the aforementioned preset fusion method may be set to be associated with the special effect types of the first AR special effect and the second AR special effect.

For example, if both the first AR special effect and the second AR special effect are 3D special effects, the first AR special effect and the second AR special effect may be spliced according to a specified splicing sequence to obtain the third AR special effect.

For another example, if the first AR special effect is a 3D special effect, and the second AR special effect is a 2D special effect, then the display position of the second AR special effect can be determined in the first AR special effect, and in the process of displaying the first AR special effect, the display position of the second AR special effect can be determined. The second AR special effect is displayed at the position, so as to achieve the fusion of the first AR special effect and the second AR special effect to obtain the third AR special effect.

During implementation, the first AR special effect and the second AR special effect are fused according to a preset fusion method, and after the fusion obtains the third AR special effect, the third AR special effect can also be added to the special effect library, and add a special effect tag for the third AR special effect. For example, the special effect tag may be: special effect information of the first AR special effect and the second AR special effect, and a preset fusion method.

In the above embodiment, if the fused AR special effect is not found, the method of combining the first AR special effect and the second AR special effect to obtain the third AR special effect can enrich the special effect content in the special effect library, and Display richer special effects for users to improve user experience.

Wherein, the described preset fusion manner may include an embedded fusion manner and a splicing fusion manner, and these two fusion manners will be introduced below.

Method 1: Embedded Fusion

During implementation, the adding position of the second AR special effect may be determined in the first AR special effect. For example, if the first AR special effect is a video special effect, the adding position of the second AR special effect may be determined in each video frame of the video special effect. Wherein, determining the adding position of the second AR special effect includes:

The area near the second logo in the first AR special effect is used as the adding position of the second AR special effect. For example, a circular area or a rectangular area may be set with the center of the second mark as the origin, and further, the circular area or the rectangular area may be used as the adding position of the second AR special effect.

Alternatively, a special effect type of the second AR special effect is determined. If it is determined according to the special effect type that the second AR special effect is a sticker special effect acting on the target bottle in the real scene image, the target bottle can be detected in each video frame of the first AR special effect, thereby determining the second AR special effect. The position where the AR special effect is added on the target bottle.

Alternatively, the adding position of the second AR special effect is a fixed adding position, for example, it may be the lower left corner area or the lower right corner area of the first AR special effect.

Take the first AR special effect as 3D special effect and the second AR special effect as 2D special effect (for example, sticker special effect) as an example: if the 3D special effect is the promotional video of the manufacturer of the target bottle, the 2D special effect is the production of the target bottle. The manufacturer's logo, for example, when the manufacturer's logo (i.e. logotype), then can display the promotional video of the manufacturer of the target bottle at the fixed position of the promotional video (i.e. the position where the second AR special effect is added) ) to display the logo of the manufacturer, such as the lower left corner or the lower right corner.

Method 2: splicing fusion method

During implementation, the setting of the splicing position, that is, the setting of the presentation sequence, may be performed for each 3D special effect corresponding to the first AR special effect. For example, the splicing position of the above-mentioned 3D special effects can be set before or after the display, and if the splicing position of the 3D special effects is set after the display, the above-mentioned second AR special effects can be added to the above-mentioned first AR when performing special effect fusion After the special effect, the third AR special effect is obtained, that is, when the third AR special effect is displayed, the first AR special effect is displayed first, and then the second AR special effect is displayed. For example, it is possible to first confirm the display order of the 3D special effects and 2D special effects in the third AR special effects, wherein the 3D special effects can be the promotional video of the manufacturer producing the target bottle, and the 2D special effects can be the promotional video of the manufacturer of the target bottle. Logo, for example, the manufacturer's logo.

In some embodiments, the method also includes:

Step S107, identifying the type information of the target object in the real-scene image, wherein the target object includes: the target bottle, and/or the physical item.

The step S105 includes:

Determine a third AR special effect for presentation based on the type information, the first AR special effect corresponding to the first identifier, and the second AR special effect corresponding to the second identifier.

In some implementations, after the corresponding first AR special effect and second AR special effect are respectively determined according to the above-mentioned first identifier and second identifier, the type information of the target object in the recognized real-scene image, the first The AR special effect and the second AR special effect determine a third AR special effect for display.

In the case that different types of bottles are provided with the same first logo, for example, sports bottles and children's bottles are provided with the same first logo, then different types of bottles can be displayed The third AR special effect with different display effects. For example, for sports bottles, sports elements can be added to the third AR special effects, and for children's bottles, cartoon elements can be added to the third AR special effects.

In the case that different types of physical items are provided with the same second identification, for example, the physical items other than the target bottle are postcards or admission tickets. For postcards, elements corresponding to the patterns contained in the postcard can be added to the third AR special effect; for admission tickets, relevant display elements of venues corresponding to the admission ticket can be added to the third AR special effect.

In the above embodiments, by identifying the type information of the target object in the real-scene image, and then determining the third AR special effect for display according to the type information, the first AR special effect and the second AR special effect, it is possible to realize different types of target objects Bottles and/or different types of physical objects trigger the display of the third AR special effects containing different elements, thus enriching the types of special effects of AR special effects and the triggering methods of AR special effects, improving the fun of AR technology and allowing users to experience more A good understanding of the relevant content of the target bottle, in order to achieve the promotion of the target bottle.

In some embodiments, the method also includes:

Step 109. Obtain user information of the user to which the AR device belongs; wherein, the user information includes at least one of the following: basic attributes of the user, device type of the AR device, number of times the user visits the special effect display page, and Historical AR special effects pushed by users.

Step 111. Determine the fusion special effect type matching the user information.

The step S105 includes:

Determine a third AR special effect for presentation based on the fusion special effect type, the first AR special effect corresponding to the first identifier, and the second AR special effect corresponding to the second identifier.

During implementation, when the augmented reality AR device is triggered to load the loading page, the user information of the user to whom the augmented reality AR device belongs can be obtained.

Wherein, the basic attributes of the user may include at least one of the following: attribute information such as age, gender, and hobbies. The device type of the augmented reality AR device may include at least one of the following: a wearable augmented reality AR device, and a handheld terminal device supporting the augmented reality AR function. The number of times the user visits the special effect display page may be: the number of times the user visits the special effect display page (or loads the loading page) within a period of time. The historical AR special effects pushed for the user may be: a record of pushing AR special effects for the user with the same user information in the past period of time.

After the user information is determined, at least one fusion effect type matching the user information may be determined.

For example, according to different user basic attributes, different visit times, historical AR special effects, and different device types, different fusion special effect types can be determined for users.

After at least one fusion effect type is determined, a third AR special effect for display is determined based on the at least one fusion effect type, the first AR special effect, and the second AR special effect.

In the above embodiments, by determining at least one fusion effect type that matches the user information, and determining the third AR special effect in the at least one fusion effect type, different users can customize different styles of AR special effects. Improve user experience.

In some implementation manners, when determining the fusion special effect type matching the user information, it includes:

(1) In the case that the user information includes the historical AR special effect, determine the historical special effect type of the target historical AR special effect last pushed by the user in the historical AR special effect.

(2) Determining the fusion special effect type in other special effect types except the historical special effect type among the plurality of preset special effect types.

Wherein, if the user information includes a historical AR special effect, the latest target historical AR special effect pushed for the user is determined according to the historical AR special effect, and then the historical special effect type corresponding to the target historical AR special effect can be determined. Then, other special effect types except the historical special effect type are determined among the plurality of preset special effect types in the special effect library, and the fusion special effect type is determined among the other special effect types.

After the fusion special effect type is determined, a third AR special effect for display may be determined according to the fusion special effect type, the first AR special effect, and the second AR special effect.

In the above implementation manner, through the above processing manner, it is possible to avoid continuously pushing the same AR special effect for the user, thereby improving user experience.

Those skilled in the art can understand that in the above method of specific implementation, the writing order of each step does not mean a strict execution order and constitutes any limitation on the implementation process. The specific execution order of each step should be based on its function and possible The inner logic is OK.

Based on the same inventive concept, the embodiment of the present disclosure also provides a data display device corresponding to the data display method. Since the problem-solving principle of the device in the embodiment of the present disclosure is similar to the above-mentioned data display method in the embodiment of the present disclosure, the implementation of the device See the implementation of the method.

Referring to Figure 6, it is a schematic diagram of a data display device provided by an embodiment of the present disclosure, the device includes: an acquisition part 61, a detection part 62, and a determination part 63; wherein,

The acquiring part 61 is configured to acquire the real scene image including the target bottle collected by the augmented reality AR device;

The detection part 62 is configured to detect a first mark corresponding to the target bottle and a second mark corresponding to an entity item other than the target bottle in the real scene image;

The determining part 63 is configured to determine a third AR special effect for presentation based on the first AR special effect corresponding to the first identifier and the second AR special effect corresponding to the second identifier.

In the embodiment of the present disclosure, by detecting the first logo corresponding to the target bottle in the real scene image, and detecting the second logo corresponding to the physical item other than the target bottle; and based on the first AR corresponding to the first logo The special effect and the second AR special effect corresponding to the second logo determine the third AR special effect for display, which can trigger the display of different AR special effects when the target bottle is combined with physical items other than the target bottle , the triggering conditions of AR special effects can be enriched through the above processing method, and the AR interaction between the user and the target bottle can also be realized. While increasing the user's interest, it can improve the user's experience of using AR technology, meet the diverse user needs of the user, and can also widely promote the relevant information of the target bottle.

In some implementations, the determining part 63 is further configured to: search for a fusion AR special effect that matches the first AR special effect and the second AR special effect in the special effect library, wherein the special effect library contains multiple identifying the special effect information of the fusion AR special effect corresponding to the respective AR special effects; and determining the found fusion AR special effect as the third AR special effect.

In some implementations, the determining part 63 is further configured to: perform special effect fusion processing on the first AR special effect and the second AR special effect according to a preset fusion manner, to obtain the third AR special effect.

In some embodiments, the device further includes: an identification part configured to identify the type information of the target object in the real-scene image, wherein the target object includes: the target bottle body, and/or the the physical item; the determining part 63 is further configured to: based on the type information, the first AR special effect corresponding to the first identification, and the second AR special effect corresponding to the second identification, determine The third AR special effect.

In some implementations, the apparatus further includes: a first acquiring subsection configured to acquire user information of a user to whom the augmented reality AR device belongs; wherein the user information includes at least one of the following: basic user attributes , the device type of the AR device, the number of times the user visits the special effect display page, and the historical AR special effects pushed for the user; the first determining subpart is configured to determine the fusion special effect type that matches the user information; The determining part 63 is further configured to: determine a third AR special effect for presentation based on the fusion special effect type, the first AR special effect corresponding to the first identifier, and the second AR special effect corresponding to the second identifier. AR special effects.

In some implementations, the first determining subpart is further configured to: in the case that the historical AR special effects are included in the user information, determine the latest target history pushed for the user in the historical AR special effects The historical special effect type of the AR special effect; the fusion special effect type is determined in other special effect types except the historical special effect type among the plurality of preset special effect types.

In some embodiments, the determining part 63 is further configured to: in the case that the number of bottles contained in the real-scene image is multiple, in response to a selection instruction for the target bottle in the real-scene image, determine The identification of the target bottle is the first identification; among the identifications corresponding to all the physical items in the real scene image, it is determined that the identification associated with the first identification is the second identification.

In some embodiments, the determination part 63 is further configured to: determine the trigger position of the selection instruction in the real scene image; determine the target bottle according to the trigger position, and identify the target bottle corresponding , and determine the identified identity as the first identity.

For the description of the processing flow of each part in the device and the interaction flow between each part, reference may be made to the relevant descriptions in the foregoing method embodiments.

The embodiment of the present disclosure also provides a computer device 700, as shown in FIG. 7, which is a schematic structural diagram of the computer device 700 provided by the embodiment of the present disclosure, including:

Processor 71, memory 72, and bus 73; Memory 72 is used for storing execution order, comprises memory 721 and external memory 722; Memory 721 here is also called internal memory, is used for temporarily storing the operation data in processor 71, and with The data exchanged by the external memory 722 such as hard disk, the processor 71 exchanges data with the external memory 722 through the memory 721, when the computer device 700 is running, the processor 71 communicates with the memory 72 through the bus 73, so that The processor 71 executes the following instructions:

Acquiring the real scene image including the target bottle body collected by the augmented reality AR device; detecting the first logo corresponding to the target bottle body in the real scene image, and the first logo corresponding to the entity item other than the target bottle body The second identifier: based on the first AR special effect corresponding to the first identifier and the second AR special effect corresponding to the second identifier, determine the third AR special effect for presentation.

An embodiment of the present disclosure also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is run by a processor, the steps of the data display method described in the above-mentioned method embodiments are executed, Wherein, the storage medium may be a volatile or non-volatile computer-readable storage medium.

Embodiments of the present disclosure also provide a computer program product, the computer program product carries a program code, and the instructions included in the program code can be used to execute the steps of the data display method described in the above method embodiment, for details, please refer to the above method Example.

Wherein, the above-mentioned computer program product may be specifically implemented by means of hardware, software or a combination thereof. In one embodiment, the computer program product is embodied as a computer storage medium, and in another embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK) and the like.

For the effect description of the above-mentioned data display device, computer equipment, computer-readable storage medium and computer program product, please refer to the description of the above-mentioned data display method.

Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working process of the system and device described above can refer to the corresponding process in the foregoing method embodiment. In the several embodiments provided in the present disclosure, it should be understood that the disclosed systems, devices and methods may be implemented in other ways. The device embodiments described above are schematic. For example, the division of the units is a logical function division. In actual implementation, there may be another division method. For example, multiple units or components can be combined or integrated. to another system, or some features may be ignored, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some communication interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit.

If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium executable by a processor. Based on this understanding, the technical solution of the embodiments of the present disclosure is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , including several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present disclosure. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disc and other media that can store program codes. .

Finally, it should be noted that: the above-mentioned embodiments are only specific implementations of the present disclosure, and are used to illustrate the technical solutions of the present disclosure, rather than limit them, and the protection scope of the present disclosure is not limited thereto, although referring to the aforementioned The embodiments have described the present disclosure in detail, and those skilled in the art should understand that any person familiar with the technical field can still modify the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present disclosure Changes can be easily imagined, or equivalent replacements can be made to some of the technical features; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in this disclosure. within the scope of protection.

Industrial Applicability

An embodiment of the present disclosure provides a data display method, device, computer equipment, storage medium, and computer program product, wherein the data display method includes: acquiring a real scene image containing a target bottle collected by an augmented reality AR device; detecting the In the real scene image, the first logo corresponding to the target bottle, and the second logo corresponding to physical items other than the target bottle; based on the first AR special effect corresponding to the first logo, and the The second AR special effect corresponding to the second identifier is used to determine the third AR special effect for display. The above scheme can enrich the triggering conditions of AR special effects, can also improve the user experience of using AR technology, and can also widely promote the relevant information of the bottle body.

Claims

A method for displaying data, comprising:

Obtain the real image of the target bottle collected by the augmented reality AR device;

Detecting a first logo corresponding to the target bottle and a second logo corresponding to an entity item other than the target bottle in the real scene image;

Based on the first AR special effect corresponding to the first identifier and the second AR special effect corresponding to the second identifier, a third AR special effect for display is determined.
The method according to claim 1, wherein said determining a third AR special effect for display based on the first AR special effect corresponding to the first logo and the second AR special effect corresponding to the second logo includes :

Find a fusion AR special effect that matches the first AR special effect and the second AR special effect in the special effect library, wherein the special effect library includes a plurality of special effect information of fusion AR special effects that identify the corresponding AR special effects;

The found fusion AR special effect is determined as the third AR special effect.
The method according to claim 1 or 2, wherein said determining a third AR special effect based on the first AR special effect corresponding to the first identifier and the second AR special effect corresponding to the second identifier comprises:

Perform special effect fusion processing on the first AR special effect and the second AR special effect according to a preset fusion manner to obtain the third AR special effect.
The method according to any one of claims 1 to 3, wherein the method further comprises:

Identifying the type information of the target object in the real-scene image, wherein the target object includes: the target bottle, and/or, the physical item;

The determining the third AR special effect for display based on the first AR special effect corresponding to the first logo and the second AR special effect corresponding to the second logo includes

Determine a third AR special effect for presentation based on the type information, the first AR special effect corresponding to the first identifier, and the second AR special effect corresponding to the second identifier.
The method according to any one of claims 1 to 4, wherein the method further comprises:

Obtain the user information of the user to which the augmented reality AR device belongs; wherein the user information includes at least one of the following: basic user attributes, device type of the AR device, number of times the user visits the special effect display page, and Push historical AR special effects;

Determine the fusion effect type matching the user information;

The determining the third AR special effect for display based on the first AR special effect corresponding to the first logo and the second AR special effect corresponding to the second logo includes

Determine a third AR special effect for presentation based on the fusion special effect type, the first AR special effect corresponding to the first identifier, and the second AR special effect corresponding to the second identifier.
The method according to claim 5, wherein said determining the fusion effect type matching the user information comprises:

In the case that the user information includes the historical AR special effect, determine the historical special effect type of the target historical AR special effect pushed last time for the user in the historical AR special effect;

The fusion special effect type is determined in other special effect types except the historical special effect type among the plurality of preset special effect types.
The method according to any one of claims 1 to 6, wherein the detection of the first mark corresponding to the target bottle in the real scene image and the entity other than the target bottle The second identification corresponding to the item, including:

In the case that the number of bottles contained in the real-scene image is multiple, in response to a selection instruction for the target bottle in the real-scene image, determine that the mark of the target bottle is the first mark;

Among the identifiers corresponding to all the physical items in the real-scene image, it is determined that the identifier associated with the first identifier is the second identifier.
The method according to claim 7, wherein said determining that the mark of the target bottle body is the first mark comprises:

determining a trigger position of the selection instruction in the real scene image;

The target bottle body is determined according to the trigger position, the mark corresponding to the target bottle body is identified, and the recognized mark is determined as the first mark.
A data display device, comprising:

The acquiring part is configured to acquire the real scene image including the target bottle collected by the augmented reality AR device;

The detection part is configured to detect a first mark corresponding to the target bottle and a second mark corresponding to an entity item other than the target bottle in the real scene image;

The determining part is configured to determine a third AR special effect for display based on the first AR special effect corresponding to the first identifier and the second AR special effect corresponding to the second identifier;

The display part is configured to display the third AR special effect.
The device according to claim 9, wherein the determining part is further configured to: search for a fusion AR special effect matching the first AR special effect and the second AR special effect in a special effect library, wherein the The special effect library includes a plurality of special effect information of fused AR special effects that respectively identify corresponding AR special effects; and the found fused AR special effect is determined as the third AR special effect.
The device according to claim 9 or 10, wherein the determining part is further configured to: perform special effect fusion processing on the first AR special effect and the second AR special effect according to a preset fusion method to obtain the The third AR special effect.
The device according to any one of claims 9 to 11, wherein the device further comprises: an identification part configured to identify the type information of the target object in the real-scene image, wherein the target object includes: The target bottle, and/or, the physical item; the determining part is further configured to: based on the type information, the first AR special effect corresponding to the first identification, and the corresponding first AR special effect of the second identification The second AR special effect is determined to be the third AR special effect for display.
The apparatus according to any one of claims 9 to 11, wherein the apparatus further includes: a first acquiring subpart configured to acquire user information of a user to whom the augmented reality AR device belongs; wherein the The user information includes at least one of the following: basic attributes of the user, device type of the AR device, number of times the user visits the special effect display page, and historical AR special effects pushed for the user; the first determination subsection is configured to determine and The fusion special effect type that the user information matches; the determining part is further configured to: based on the fusion special effect type, the first AR special effect corresponding to the first identifier, and the second AR special effect corresponding to the second identifier AR special effects, determine the third AR special effects for display.
The device according to claim 13, wherein the first determination subpart is further configured to: if the user information contains the historical AR special effects, determine the latest AR special effects in the history The historical special effect type of the target historical AR special effect pushed by the user; the fusion special effect type is determined from other special effect types in multiple preset special effect types except the historical special effect type.
The device according to any one of claims 9 to 14, wherein the determining part is further configured to: in the case that the number of bottles contained in the real-scene image is multiple, responding to the According to the selection instruction of the target bottle in the real-scene image, it is determined that the mark of the target bottle is the first mark; among the marks corresponding to all physical items in the real-scene image, it is determined to be associated with the first mark The identifier of is the second identifier.
The device according to claim 15, wherein the determination part is further configured to: determine the trigger position of the selection instruction in the real scene image; determine the target bottle according to the trigger position, and identify the The identification corresponding to the target bottle body is determined, and the identified identification is determined as the first identification.
A computer device, comprising: a processor, a memory, and a bus, the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor communicates with the memory through the bus , when the machine-readable instructions are executed by the processor, the steps of the data presentation method according to any one of claims 1 to 8 are executed.
A computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is run by a processor, the steps of the data display method according to any one of claims 1 to 8 are executed.
A computer program product, comprising a computer-readable storage medium storing program code, when the instructions included in the program code are executed by the processor of the computer device, the data presentation method described in any one of claims 1 to 8 is realized A step of.