WO2023020622A1 - Display method and apparatus, electronic device, computer-readable storage medium, computer program, and computer program product - Google Patents

Abstract

Disclosed in embodiments of the present application are a display method and apparatus, an electronic device, a computer-readable storage medium, a computer program, and a computer program product. The method comprises: acquiring a target image of a real object in a real environment in response to a trigger operation on a target function entry; performing recognition on the target image to determine a virtual object corresponding to the target image, and displaying, in a target function page, a virtual effect image obtained by rendering the virtual object, the target function page comprising at least one interactive control, and the virtual effect image being used for displaying an augmented reality effect corresponding to the target image; and in response to an interactive operation on a target interactive control in the at least one interactive control, determining response effect data corresponding to the virtual object, and updating the displayed virtual effect image according to the response effect data. The present application allows for improvement of the richness and display effect of display content.

Description

A display method, device, electronic device, computer readable storage medium, computer program and computer program product

Cross References to Related Applications

This disclosure is based on the Chinese patent application with the application number 202110961689.4, the application date is August 20, 2021, and the application name is "a display method, device, electronic device, and computer-readable storage medium", and requires that the Chinese patent application The priority of the Chinese patent application, the entire content of the Chinese patent application is hereby incorporated into this application by reference.

technical field

The present disclosure relates to the technical field of terminals, and in particular, to a display method, device, electronic equipment, computer-readable storage medium, computer program, and computer program product.

Background technique

At present, some commodity merchants will attach virtual service products to the physical commodities they sell for users to enjoy. For example, create exclusive events or services in official apps for users to participate in. However, at present, the interaction mode between virtual service products and users is usually relatively simple, thus reducing the richness and display effect of display content in virtual service products, such as some display service applications.

Contents of the invention

Embodiments of the present disclosure expect to provide a display method, device, electronic device, computer-readable storage medium, computer program, and computer program product, which can improve the richness and display effect of displayed content.

The disclosed technical solution is achieved in this way:

An embodiment of the present disclosure provides a display method, including:

Collecting target images of real objects in the real scene in response to a trigger operation on the target function entry;

Identifying the target image, determining a virtual object corresponding to the target image, and displaying a virtual effect image obtained by rendering the virtual object on a target function page; the target function page includes at least one interactive control; The virtual effect image is used to display the augmented reality effect corresponding to the target image;

In response to an interactive operation on a target interactive control in the at least one interactive control, determine response effect data corresponding to the virtual object, and update the displayed virtual effect image based on the response effect data.

The display method is realized through the small program page or the web page, which improves the convenience of using the display method and is more conducive to the promotion of virtual service products.

An embodiment of the present disclosure provides a display device, including:

The collection part is configured to collect a target image of a real object in a real scene in response to a trigger operation on a target function entry;

The rendering part is configured to determine the virtual object corresponding to the target image by identifying the target image, and display the virtual effect image obtained by rendering the virtual object on the target function page; the target function page includes At least one interactive control; the virtual effect image is used to display the augmented reality effect corresponding to the target image;

The operation part is configured to determine response effect data corresponding to the virtual object in response to an interactive operation on a target interactive control in the at least one interactive control, and update the displayed virtual effect image through the response effect data.

An embodiment of the present disclosure provides an electronic device, including:

display screen; memory configured to store an executable computer program;

The processor is configured to, when executing the executable computer program stored in the memory, implement any one of the above display methods in combination with the display screen.

An embodiment of the present disclosure provides a computer-readable storage medium, on which a computer program is stored, for causing a processor to execute any one of the above display methods.

An embodiment of the present disclosure provides a computer program, including computer readable code. When the computer readable code is run in an electronic device, the processor in the electronic device implements the steps of the above-mentioned display method when executed. .

An embodiment of the present disclosure provides a computer program product, including computer program instructions, where the computer program instructions cause a computer to execute the steps of the above-mentioned display method.

Embodiments of the present disclosure provide a display method, device, electronic equipment, computer-readable storage medium, computer program, and computer program product; by collecting target images of real objects in real scenes, virtual objects are determined, and virtual objects are rendered The virtual effect image is obtained, so that the augmented reality effect of the virtual effect image is used to improve and enrich the display effect of the display application. In addition, the electronic device can receive the user's interactive operation through at least one interactive control, and perform corresponding operation and update display on the virtual object, thereby increasing the richness of the displayed content.

Description of drawings

In order to illustrate the technical solutions of the embodiments of the present disclosure more clearly, the following will briefly introduce the accompanying drawings used in the embodiments. The accompanying drawings here are incorporated into the specification and constitute a part of the specification. The drawings show the embodiments consistent with the present disclosure, and are used together with the description to explain the technical solutions of the present disclosure. It should be understood that the following drawings only show some embodiments of the present disclosure, and therefore should not be regarded as limiting the scope. For those skilled in the art, they can also make From these drawings other related drawings are obtained.

FIG. 1 is a schematic structural diagram of a display system provided by an embodiment of the present disclosure;

FIG. 2 is an optional schematic flowchart of a display method provided by an embodiment of the present disclosure;

FIG. 3 is an optional schematic flowchart of a display method provided by an embodiment of the present disclosure;

FIG. 4 is an optional schematic flowchart of a display method provided by an embodiment of the present disclosure

FIG. 5 is a schematic diagram of an optional effect of an image acquisition interface provided by an embodiment of the present disclosure;

FIG. 6 is an optional schematic flowchart of a display method provided by an embodiment of the present disclosure;

FIG. 7 is an optional schematic flowchart of a display method provided by an embodiment of the present disclosure;

FIG. 8 is an optional schematic flowchart of a display method provided by an embodiment of the present disclosure;

FIG. 9 is an optional schematic flowchart of a display method provided by an embodiment of the present disclosure;

FIG. 10 is an optional schematic flowchart of a display method provided by an embodiment of the present disclosure;

FIG. 11 is an optional schematic flowchart of a display method provided by an embodiment of the present disclosure;

FIG. 12 is a schematic diagram of an optional interaction effect of a synthesized photographing function provided by an embodiment of the present disclosure;

FIG. 13 is an optional structural schematic diagram of a display device provided by an embodiment of the present disclosure;

FIG. 14 is a schematic structural diagram of an optional electronic device provided by an embodiment of the present disclosure.

Detailed ways

The following will clearly and completely describe the technical solutions in the embodiments of the present disclosure with reference to the drawings in the embodiments of the present disclosure.

In order to make the purpose, technical solutions and advantages of the present disclosure clearer, the present disclosure will be further described in detail below in conjunction with the accompanying drawings. All other embodiments obtained under the premise of creative labor belong to the protection scope of the present disclosure.

In the following description, references to "some embodiments" describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or a different subset of all possible embodiments, and Can be combined with each other without conflict.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. The terms used herein are only for the purpose of describing the embodiments of the present disclosure, and are not intended to limit the present disclosure.

Before the embodiments of the present disclosure are further described in detail, the nouns and terms involved in the embodiments of the present disclosure will be described, and the nouns and terms involved in the embodiments of the present disclosure are applicable to the following explanations.

1) The target client can be a small program client or a web client running on electronic devices such as mobile phones and tablet computers. A small program client is also called a small program; a webpage client (Client) is also called a webpage client, which refers to a program corresponding to a server and providing local services for clients. A Mini Program (Mini Program), also known as a Web Program (Web Program), is a program developed based on a front-end-oriented language (such as JavaScript) and implemented in a Hyper Text Markup Language (HTML, Hyper Text Markup Language) page. The program is software downloaded by the client via the network and interpreted and executed in the browser environment of the client, saving the steps of installation on the client.

2) Application (Application, APP) refers to a computer program for completing one or more specific tasks, which can interact with users and has a visual user interface.

3) The identity of the user, that is, unionID, short for union Identity document; if the developer has multiple mobile applications, website applications, and public accounts (including applets), the uniqueness of the user can be distinguished by unionID, because if For mobile applications, website applications and public accounts (including applets) under the same WeChat open platform account, the user's unionID can be unique; or, the user's identity can also refer to the uniqueness of the electronic device used by the user. The equipment identity of the device, for example, the International Mobile Equipment Identity (IMEI) of the electronic device, etc.

4) User Interface (UI), including graphics, icons (Icon), color, text design, etc., usually used for human-computer interaction.

Embodiments of the present disclosure provide a display method, which can improve the convenience and richness of display applications. The display method provided by the embodiments of the present disclosure is applied to electronic equipment.

The exemplary application of the electronic device provided by the embodiment of the present disclosure is described below. The electronic device provided by the embodiment of the present disclosure can be implemented as augmented reality (Augmented Reality, AR) glasses, a notebook computer, a tablet computer, a desktop computer, a set-top box, a mobile device ( For example, various types of user terminals (hereinafter referred to as terminals) such as mobile phones, portable music players, smart watches, personal digital assistants, dedicated information devices, and portable game devices).

Referring to FIG. 1 , FIG. 1 is a schematic diagram of an optional architecture of a display system provided by an embodiment of the present disclosure. In the display system 100, a terminal (electronic device) 400 is connected to a server 200 through a network 300. The network 300 may be a wide area network or a local area network. Or a combination of both. The terminal 400 is configured to obtain the display applet from the server 200 and load the display applet to run on itself when receiving the user's operation of starting the display applet, for example, receiving the user's trigger operation on the display applet entry; Through the built-in or external display screen 401, the home page of the applet is displayed, for example, a function menu page is displayed. Wherein, at least one function entry may be displayed on the display function menu page, so as to receive user operations through at least one function entry.

The terminal 400 is also configured to enter the image collection page corresponding to the target function in response to the trigger operation of the target function entry from the user in response to the trigger operation of the target function entry in at least one function entry, and use the built-in or external image collection The device 402 collects a target image 501 of a real object 500 in a real scene. Exemplarily, the real object may be a book or a calendar, and the target image may be an image on the book or calendar, such as an exhibit image and the like.

The terminal 400 is also used to identify the target image, determine the virtual object corresponding to the target image, and display the virtual effect image 600 obtained by rendering the virtual object on the target function page through the display screen 401; the target function page contains at least one Interactive control; the virtual effect image is used to display the augmented reality effect corresponding to the target image; and in response to the interactive operation of the target interactive control in at least one interactive control, determine the response effect data corresponding to the virtual object, and update the virtual object through the response effect data. The effect image shows the response effect corresponding to the interactive operation. Exemplarily, the virtual object may be an AR model corresponding to an image on a book or a calendar.

In some embodiments of the present disclosure, the terminal 400 can pre-store the virtual object locally in the terminal, and the terminal 400 can also generate or determine the virtual object corresponding to the target image by means of a server, such as the above-mentioned server 200 or other servers with image processing functions. object, and then load the virtual object from the server to the local terminal 400, render the virtual object and perform subsequent interactive operations. The server here can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or it can provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud Cloud servers for basic cloud computing services such as communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal and the server may be connected directly or indirectly through wired or wireless communication, which is not limited in this embodiment of the present disclosure.

Hereinafter, an exemplary application in a case where an electronic device is implemented as a terminal will be explained.

FIG. 2 is a schematic flowchart of an optional display method provided by an embodiment of the present disclosure, which will be described in conjunction with the steps shown in FIG. 2 .

S101. Collect a target image of a real object in a real scene in response to a trigger operation on a target function entry.

The display method in the embodiments of the present disclosure is applicable to a scenario where a user experiences a virtual service product attached to a real item through a small program or a web client. Such as AR display experience for exhibit images in the paper calendar, or multimedia interactive experience for exhibit pictures on event exhibition boards, etc. The display method in the embodiment of the present disclosure can rely on a small program or a webpage client to provide a user experience of combining physical objects and virtual objects and rich interaction modes in a lightweight manner.

In the embodiment of the present disclosure, the target function entry can be displayed on the applet page or the webpage client, and is used to receive a user's trigger operation to activate the corresponding target function. Here, the target function is used to collect images of real scenes and determine virtual objects corresponding to real scene images, such as AR models or special effect images, so as to provide augmented reality effects or image synthesis effects combining real scenes and virtual objects.

In some embodiments, the electronic device can display the applet entry displayed on the preset page, for example, the applet icon entry displayed in the applet list on the application software pull-down page, or the display applet displayed on the web page. For the applet link entry, etc., in the case of receiving the user's trigger operation for displaying the applet entry, the electronic device displays a display function menu page in response to the trigger operation for displaying the applet entry. At least one function entry corresponding to at least one function contained in the display applet is displayed by displaying a function menu page. In this way, the electronic device can enter the target function corresponding to the target function entry in response to a trigger operation on the target function entry in the at least one function entry.

In some embodiments, the electronic device can also directly display the target function entry on a preset page, such as a small program page or a web client, or design a menu hierarchy according to actual needs to lay out the target function entry, and specifically select according to the actual situation. The embodiments of the present disclosure are not limited.

In the embodiment of the present disclosure, the electronic device may respond to a trigger operation for the target function entry, enter the image collection page corresponding to the target function, and call the image collection device to collect the image of the real object in the real scene to obtain the target image.

In the embodiments of the present disclosure, the real scene is a real real scene, and real objects exist in the real scene; in some embodiments, the real scene may contain at least one real item, and the real object may be a real object in the at least one real item. Items; in some embodiments, real objects may also include real people. Exemplarily, the real scene may contain physical items, such as a paper calendar, and the paper calendar contains at least one exhibit image, wherein the real object may be a target exhibit image in the at least one exhibit image, such as incense burner pottery images, or artistic portraits, etc. The electronic device can collect the image of the target exhibit to obtain the target image.

In some embodiments, real objects in real scenes can also be exhibits or exhibition boards displayed in science and technology museums or art museums, or commodities or commodity catalogs displayed in shopping malls, etc. Examples are not limited.

S102. Determine the virtual object corresponding to the target image by identifying the target image, and display the virtual effect image obtained by rendering the virtual object on the target function page; the target function page includes at least one interactive control; the virtual effect image is used for display The augmented reality effect corresponding to the target image.

In the embodiment of the present disclosure, when the electronic device obtains the target image, it may perform feature extraction and image analysis on the target image, so as to identify the target image. For the case where the real object is the target real item, the electronic device can identify the preset real item corresponding to the target image, and then determine the virtual object corresponding to the preset real item, such as the virtual AR model corresponding to the preset real item. For the case where the real object is a real person, the electronic device can perform human body recognition on the target image, recognize the human body parts contained in the target image, such as facial features or head, hands, etc., and then determine the virtual object corresponding to the human body part, For example, special effect stickers corresponding to specific human body parts.

In some embodiments, the virtual object can be a pre-modeled image model, including specific image parameters or action logic. The electronic device can pre-generate and save a preset virtual model corresponding to each preset real item according to the type of the preset real item, and acquire the preset real item when the corresponding preset real item is identified from the target image The corresponding preset virtual model is used as a virtual object. In some embodiments, the electronic device can also generate a virtual object corresponding to the target object in real time by recognizing the target image. For example, the electronic device can perform real-time AR modeling based on the display images collected from the calendar. , get the AR model of the exhibit as a virtual object. The specific selection is made according to the actual situation, and the embodiment of the present disclosure makes no limitation.

In the embodiment of the present disclosure, when the virtual object is determined, the electronic device can enter the target function page, and based on the target image, use the computer vision method to estimate the real position information of the real object in the real scene, and obtain the target image in the real scene. The screen position information in the display screen of the electronic device is used to establish a mapping relationship between the real world and the screen. In the target function page, the electronic device can use the preset rendering data corresponding to the virtual object to render the virtual object according to the mapping relationship between the real world and the screen, and attach the rendering result of the virtual object to the item image in the real scene , thus showing the enhanced display effect of the combination of virtual and reality.

In some embodiments, the real object may be a planar image in a real scene, such as an exhibit image in a calendar. The electronic device can use the real object as a marked plane in the real scene, and perform recognition and pose estimation (Pose Estimation) on the marked plane through the camera, and determine the position information of the marked plane in the real scene. The electronic device maps the marked plane in the three-dimensional real scene to the two-dimensional screen page of the electronic device, and renders and draws a virtual effect image corresponding to the virtual object on the mapped two-dimensional screen page.

S103. In response to an interactive operation on a target interactive control in at least one interactive control, determine response effect data corresponding to the virtual object, and update a displayed virtual effect image based on the response effect data.

In the embodiment of the present disclosure, the target function page includes at least one interactive control, and the electronic device can receive the user's interactive operation through the target interactive control in the at least one interactive control, and then respond to the interactive operation, according to the control function of the target interactive control. Operate, so that the user's interactive operation is applied to the virtual object, so as to realize the interaction between the user and the virtual object.

In the embodiment of the present disclosure, the electronic device can determine the operation mode of the virtual object through the interactive operation of the target interactive control, and then determine the response effect data corresponding to the operation mode, and the electronic device uses the response effect data to update the current real virtual effect image , so that the virtual object can reflect the interactive effect that responds to the interactive operation.

In some embodiments, based on the interactive function corresponding to each interactive control in at least one interactive control, the response effect data may be rendering data for re-rendering the virtual object, or text or image effect data displayed in combination with the virtual object , the specific selection is made according to the actual situation, which is not limited by the embodiments of the present disclosure.

It can be understood that, in the embodiments of the present disclosure, by collecting the target image of the real object in the real scene, determining the virtual object, and rendering the virtual object to obtain a virtual effect image, the augmented reality effect of the virtual effect image is used to improve and enrich In order to display the display effect of the application. In addition, the electronic device can receive the user's interactive operation through at least one interactive control, and perform corresponding operation and update display on the virtual object, thereby increasing the richness of the displayed content.

In some embodiments, based on FIG. 2, as shown in FIG. 3, S101 can be implemented through S1011-S1012, as follows:

S1011. In response to the trigger operation on the target function entry, jump from the display function menu page to the target function loading interface, and synchronously load the background data of the target function during the display process of the target function loading interface.

In the embodiment of the present disclosure, in response to the trigger operation on the target function entry, the electronic device can jump from the display function menu page to the target function loading interface before entering the image acquisition interface, and display the background data of the target function through the target function loading interface The loading progress of , and the introduction information of the target function.

In the embodiment of the present disclosure, the electronic device may synchronously load the background data of the target function during the display process of the target function loading interface, so as to render the virtual object in a subsequent step. Here, the background data of the target function includes at least one preset rendering data; the background data of the target function can be stored on the server and loaded locally by the electronic device; it can also be stored locally on the electronic device and loaded into its own memory during operation, specifically The selection is made according to the actual situation, and is not limited by the embodiments of the present disclosure.

S1012. When the background data loading of the target function is completed, enter the image collection interface, and collect the target image of the real object in the real scene.

In some embodiments, for example, the target function may be the "Take Home Exhibit" function in the virtual service product attached to the calendar, which may provide user interaction with the AR model of the exhibit, and its corresponding target function The loading interface can display information such as the name of the target function, the loading progress of the background data of the target function, and the introduction information of the target function.

In some embodiments, based on FIG. 3 , when the electronic device enters the image acquisition interface, the process of acquiring a target image of a real object in a real scene may be implemented as shown in FIG. 4 by executing S201-S202, as follows:

S201. When entering the image collection interface, display collection range guide information on the image collection interface; the collection range guide information includes a preset collection range boundary.

In the embodiment of the present disclosure, when the electronic device enters the image collection interface, it can display the collection range guide information on the image collection interface, so as to focus the image collection range of the real scene by the electronic device on the real object in a targeted manner, Reduce the interference of irrelevant information in real scenes, and improve the accuracy of target image recognition in subsequent steps.

In some embodiments, the acquisition range guide information may include a preset acquisition range boundary. Exemplarily, the preset acquisition range boundary may be a graphic boundary such as a scanning frame or an aiming frame. In some embodiments, the acquisition range guide information may also include text guide information for providing text guidance for image acquisition.

Exemplarily, the image acquisition interface may be as shown in FIG. 5 , which shows a preset acquisition range boundary 21 .

S202. Collect images of real objects in the real scene through the image collection interface, and use images within the boundaries of the preset collection range as target images.

In the embodiments of the present disclosure, the electronic device may collect images of real objects in real scenes through the image collection interface, discard image information outside the boundaries of the preset collection range, and use images within the boundaries of the preset collection range as target images.

In some embodiments, the manner in which the electronic device collects the image of the real object may include scanning or taking pictures, or other image collection manners, which are selected according to actual conditions, and are not limited by the embodiments of the present disclosure.

In some embodiments, the real object includes: a target real item, and the virtual object is a target virtual model corresponding to the target real item. Exemplarily, the target real item can be an exhibit image in a calendar, such as an incense burner image; the virtual object can be It is the virtual model corresponding to the exhibit image, such as the AR model of the incense burner. The display method in the embodiment of the present disclosure will be introduced below by taking a real object as a target real item as an example.

Based on Figure 2 and Figure 3, as shown in Figure 6, by identifying the target image in S102, the process of determining the virtual object corresponding to the target image can be realized by executing the process of S1021-S1022; in S102, it is displayed on the target function page The process of rendering a virtual object to obtain a virtual effect image can be realized through the process of S1023-S1025, which will be described in conjunction with each step.

S1021. Perform target recognition of at least one preset real item on the target image to obtain a detection result of the target real item.

In the embodiment of the present disclosure, the electronic device may obtain a neural network for multi-objective classification through machine learning, which is used to identify the classification probability of the input image corresponding to at least one preset real item. The electronic device can perform feature extraction and classification prediction on the target image through a multi-target classification neural network, and obtain the classification probability that the target image belongs to each preset real item in at least one preset real item; based on the target image belonging to each preset The classification probability of the real item is used to obtain the detection result of the target real item corresponding to the target image.

In some embodiments, based on the image collection interface shown in FIG. 5 , the electronic device can scan the target image within the boundary of the preset collection range in a scanning manner, and recognize the target image obtained by real-time scanning in real time, and predict the target image. The preset real item corresponding to the image is used as the detection result of the target real item.

S1022. Based on the detection result, acquire a target virtual model corresponding to the target real item as a virtual object.

In the embodiment of the present disclosure, the electronic device can use the preset modeling data to generate the target virtual model corresponding to the target real item as a virtual object in real time based on the detection result of the target real item, and the electronic device can also display the target function of the applet At least one preset virtual model corresponding to at least one preset real item is pre-stored in the background data, so that the target virtual model corresponding to the target real item can be obtained from the at least one pre-stored preset virtual model based on the detection result of the target real item as a virtual object. The specific selection is made according to the actual situation, and the embodiment of the present disclosure makes no limitation.

Exemplarily, the electronic device recognizes the target image, and obtains a detection result that the target real item is an incense burner image. The electronic device can perform three-dimensional modeling in a virtual scene for the image of the incense burner, and obtain the three-dimensional virtual model of the incense burner as a virtual object corresponding to the image of the incense burner.

S1023. Obtain the corresponding position information of the target image in the preset screen coordinate system as the screen position information of the target real item displayed on the target function page.

In the embodiment of the present disclosure, since the target image is the image displayed by the target real item on the target function page currently displayed on the display screen, the electronic device can obtain each pixel in the target image in the preset screen coordinate system of the display screen The corresponding location information is used as the screen location information of the target real item displayed on the target function page.

S1024. Perform attitude assessment based on the target image to obtain real position information of the real target object in the real scene.

In the embodiment of the present disclosure, since the target image also reflects the position and attitude information of the target real object in the real scene, the electronic device can perform the target image based on the target image, combined with the calibration parameters of the image acquisition device, such as the internal reference matrix of the camera. Pose evaluation, to obtain the real position information of the target real object in the real scene.

In some embodiments, the electronic device can use the coordinate system with the center of the target real object as the origin as the real coordinate system, and use the transformation relationship of 3D projective geometry to rotate and translate the real coordinate system to the camera coordinate system corresponding to the image acquisition device, Then map from the camera coordinate system to the screen coordinate system, so as to obtain the mapping relationship between the real position information and the screen position information.

S1025. Obtain target preset rendering data corresponding to the virtual object from the background data of the target function, and use the target preset rendering data to render the virtual object based on the mapping relationship between real position information and screen position information to obtain a virtual effect image.

In the embodiment of the present disclosure, when the real coordinate system and the screen coordinate system have established a mapping relationship, the electronic device can render the virtual object on the display screen according to the mapping relationship between the real position information and the screen position information, and obtain A virtual effect image in which the rendered virtual object image is attached to the item image in the real scene.

In the embodiment of the present disclosure, when the electronic device acquires the background data of the target function through a loading process, it may acquire the target preset rendering data corresponding to the virtual object from the background data of the target function. Here, the at least one preset rendering data included in the background data of the target function may be preset rendering data corresponding to at least one preset virtual model. The electronic device may acquire preset rendering data corresponding to the target virtual model in at least one preset rendering data as the target preset rendering data.

In some embodiments, when the electronic device recognizes the detection result of the target real object from the target image, it can jump from the image acquisition interface to the target function page, and perform virtual object rendering and virtual effects on the target function page Display of images. For example, when the electronic device recognizes the incense burner image from the target image, it can acquire the incense burner AR model corresponding to the incense burner image, and render the virtual object image of the incense burner AR model on the target function page through preset rendering data. Here, the virtual object image can be a censer model image with a three-dimensional effect. The electronic device superimposes the virtual object image on the calendar item image in the real scene to obtain a virtual effect image, and displays the combination of virtual and reality through the target function page. augmented reality effect.

It can be understood that, in the embodiment of the present disclosure, by rendering the virtual object to obtain the virtual effect image of the augmented reality effect, the real item can be seamlessly combined with the display content in the virtual world, which enhances the diversity and variety of the display content. The sense of experience improves the richness of the displayed content.

In some embodiments, based on FIG. 6, as shown in FIG. 7, after S1021, S1026 may also be executed, as follows:

S1026. In the case that the preset object is not recognized in the target image, display guidance information for prompting to adjust the position of the boundary of the preset collection range on the image collection interface.

In the embodiment of the present disclosure, the electronic device performs target recognition of at least one preset real item on the target image, and if the preset real item is not recognized, the electronic device determines that any preset Assuming a real object, the electronic device displays guidance information on the image collection interface for prompting to adjust the position of the boundary of the preset collection range, so as to prompt the user to align the boundary of the preset collection range with the real object for image collection.

Exemplarily, based on Figure 5, in the case where the boundary of the preset acquisition range, that is, the scanning frame, is not aligned with the exhibits in the calendar, such as the image of the incense burner, the electronic device cannot normally recognize the preset object image according to the target image in the scanning frame. Showcase object. The electronic device can display guidance information on the image collection interface, such as a text prompt "Please align the exhibits on the calendar" to prompt the user to align the scanning frame with the exhibits on the calendar.

In some embodiments, based on FIG. 2 and FIG. 6 , at least one interactive control displayed by the electronic device on the target function page may include: a rotation control. In S103, in response to an interactive operation on the target interactive control in the at least one interactive control, determine The process of responding to the effect data corresponding to the virtual object can be shown in Figure 8, through S1031-S1033, as follows:

S1031. In the case that the target interactive control is a rotation control, display a rotation guide icon.

In the embodiment of the present disclosure, the rotation control is used to rotate the virtual object, so as to fully display the target real item through the rotation of the virtual object. In the case that the target interactive control corresponding to the interactive operation triggered by the user is a rotation control, the electronic device may display a rotation guide icon to guide the user to perform an operation indicated by the icon to rotate the virtual object.

In some embodiments, the rotation control can be displayed on the target function page. When the rotation control receives an interaction operation, such as when the user clicks the rotation control, in response to the interaction operation on the rotation control, the electronic device can be near the rotation control Displays the spinner icon. The rotation guide icon is used to instruct the user to rotate the virtual object by sliding left or right.

S1032. Receive a rotation operation on the rotation guide icon, and determine a rotation direction and a rotation angle of the virtual object according to the rotation operation.

In the embodiment of the present disclosure, the electronic device receives the rotation operation through the rotation guide icon, and determines the rotation direction and rotation angle of the virtual object according to the operation information of the rotation operation, such as the operation direction or the operation distance.

In some embodiments, the rotation operation may be a left-right sliding operation, and the electronic device may determine the clockwise or counterclockwise rotation direction of the virtual object according to the left-right sliding direction, and determine the virtual object's rotation angle according to the left-right sliding distance.

S1033. Rotate the virtual object according to the rotation direction and the rotation angle, and acquire preset rendering data corresponding to the rotated virtual object as response effect data.

In the embodiment of the present disclosure, the electronic device may rotate the virtual object according to the rotation direction and the rotation angle indicated by the rotation operation to obtain the rotated virtual object. The electronic device acquires corresponding preset rendering data as response effect data according to the rotated virtual object, so as to update the virtual effect image through the response effect data, and display the virtual effect image presented after the virtual object is rotated.

It should be noted that, in some embodiments, when the electronic device receives an interactive operation on the rotation control, such as clicking, it can set the rotation control to an on state; thus, when the rotation control is on, Display the rotation guide icon, and the user can rotate the virtual object through the rotation operation. When the rotation of the virtual object is completed, the user can click the rotation control again to switch the rotation control to the off state, thereby exiting the rotation state of the virtual object and closing the rotation guide icon. Responds to rotation operations.

In some embodiments, as shown in FIG. 9, at least one interactive control on the target function page may include: an information display control, and S103 may be implemented through S1034, as follows:

S1034. If the target interactive control is an information display control, obtain preset introduction data corresponding to the virtual object as response effect data; the preset introduction data is used to display the introduction information of the target real item at a preset display position.

In the embodiment of the present disclosure, the information display control is used to display the preset introduction data of the virtual object, and the preset introduction data is used to display the introduction information of the target real item at the preset display position. Here, the introduction information may include text introduction information and At least one of image introduction information. When the electronic device receives the interactive operation for the information display control, it obtains the preset introduction data corresponding to the virtual object as the response effect data, so as to update the virtual effect image through the response effect data, and display the preset introduction data on the on the current target feature page.

In some embodiments, the electronic device may display preset introduction data through a pop-up window or a pop-up interface at a preset display position to update the virtual effect image. Exemplarily, an information display control may be displayed on the target function page. When the information display control receives an interactive operation, such as when the user clicks on the information display control, the electronic device may acquire the virtual object in response to the interactive operation on the information display control. For example, the introduction information of the incense burner corresponding to the AR model of the incense burner is used as the preset introduction data, and the introduction window of the incense burner pops up at the preset display position of the current target function page, and the preset introduction data is displayed through the introduction window.

In some embodiments, at least one interactive control on the target function page may include: an image acquisition control, as shown in FIG. 10 , S103 may be implemented through S1035-S1036, as follows:

S1035. In the case that the target interactive control is an image collection control, collect the current virtual effect image corresponding to the virtual object to obtain the current object image.

In the embodiment of the present disclosure, the image collection control is used to collect the virtual effect image currently presented on the target function page. In the case that the target interaction control is an image acquisition control, the electronic device acquires the current virtual effect image corresponding to the virtual object, for example, by capturing a screen or screenshot to obtain the current object image.

S1036. Acquire a first preset template, combine the first preset template and the current object image to generate a template object image, and use the template object image as response effect data; the first preset template is used to provide preset information for displaying the current object image Image template.

In the embodiment of the present disclosure, the electronic device acquires the first preset template, and combines the current object image with the first preset template to generate the template object image as the response effect data.

In the embodiment of the present disclosure, the first preset template provides a preset image template for displaying the current object image. In some embodiments, the first preset template may be a preset photo frame, and the electronic device combines the current object image with the preset photo frame to obtain the template object image. Exemplarily, an image capture control is displayed on the target function page. When the image capture control receives an interactive operation, such as when the user clicks the image capture control, in response to the interactive operation on the image capture control, the electronic device can The virtual effect image currently presented on the page is collected to obtain the current object image. The electronic device acquires a first preset template, and combines the first preset template with the current object image to generate a template object image. Exemplarily, the template object image may be a current object image displayed in a photo frame of a preset style. The electronic device takes the template object image as response effect data.

In some embodiments of the present disclosure, the target function page can display at least one interactive control at the same time. Exemplarily, the target function page displays a rotation control, an information display control, and an image acquisition control at the same time, so as to realize the interactive control on the target function page. Multiple interaction modes corresponding to rotation control, information display control and image acquisition control. In some embodiments, when at least two interactive controls are simultaneously displayed on the target function page, the electronic device only responds to an interactive operation on one interactive control each time. Exemplarily, when the user clicks on the rotation control, the rotation control enters the open state, and the rotation guide icon is displayed; at this time, if an interactive operation for the information display control is received, the rotation control is converted to the off state, and the rotation guide icon is turned off , exit the closed state of the virtual object, and pop up the preset introduction data interface; at this time, if an interactive operation for the image acquisition control is received, the display of the preset introduction data interface will be turned off, and the current virtual effect image corresponding to the virtual object Collect to obtain the current object image; combine the first preset template with the current object image to generate a template object image.

It can be understood that in the embodiments of the present disclosure, at least one of image acquisition, rotation, and information introduction may be used to enrich the display effect of the display application and improve the interaction between the display application and the user.

In some embodiments, the first preset template may further include: a preset identification code; the preset identification code is used to provide a link to the preset function.

In the embodiment of the present disclosure, the electronic device may generate and display a preset identification code on the first preset template, so as to attach the preset identification code to the template object image generated according to the first preset template, so that the target device can pass To identify the preset identification code, enter the link of the preset function. In some embodiments, the preset identification code is an applet identification code, and the applet identification code is used to provide an entry for displaying an applet. When the target device successfully scans and recognizes the applet identification code, the target function can be started on the target device.

In some embodiments, when the template object image is generated, the electronic device can jump from the current target function page to the sharing page, and display the template object image on the sharing page, and the sharing page includes the first sharing control. In case of receiving the sharing operation for the first sharing control, the electronic device may generate a first applet link corresponding to the template object image, and share the first applet link to the target device. In this way, the target device can click the link of the first applet to open the template object image on the target device, or enter a page displaying the applet, so as to realize sharing to the target device.

In some embodiments, the above-mentioned sharing page may further include a first save control, and when a save operation for the first save control is received, the template object image is saved to a local storage space.

In some embodiments, the shared page may also include a return control, such as a "retake" function control. The return control is used to return to the target function page to collect the virtual effect image when the corresponding trigger operation is received.

In some embodiments, the sharing page may also be in the form of text information, prompting the user to perform a preset trigger sharing operation, such as long pressing the image of the template object to share. The specific selection is made according to the actual situation, and the embodiment of the present disclosure makes no limitation.

In some embodiments, the template object image generated by the electronic device may be a page in the form of webview. Since the page in the form of webview is not convenient for direct sharing, the electronic device may respond to the sharing operation of the first sharing control; or, respond to the The preset trigger sharing operation, such as long pressing on the image of the template object, jumps from the sharing page to the sharing applet, and enters the page of the sharing applet, so as to save the image of the template object by using the page of the sharing applet Action or share action.

In some embodiments, the electronic device may use the page of the sharing applet as a secondary sharing page, and display the second saving control and the second sharing control on the secondary sharing page. When the second saving control is triggered, save the template object image locally; when the second sharing control is triggered, generate the first applet link corresponding to the template object image, and share the first applet link to the target device.

It can be understood that in the embodiment of the present disclosure, by sharing the virtual effect image to the target device, the interactivity of the displayed applet is further enriched, and it is beneficial to the promotion of the virtual service product corresponding to the displayed applet.

In some embodiments, the real object may be an artistic portrait in the calendar, and the target function may be the "art painting moving" function in the virtual service product attached to the calendar. Based on the above descriptions in Fig. 2, Fig. 3 and Fig. 6, the electronic device can respond to the trigger operation of the target function entry "artistic drawing", enter the target function loading interface, and synchronize during the display process of the target function loading interface Load target function background data. Here, the target function background data may include preset dynamic effect rendering data and preset multimedia audio data. During the loading process of loading the background data of the target function, the electronic device may also pop up a prompt window for turning on the sound on the target function loading interface to remind the user to turn on the sound to obtain a better experience. When the background data loading of the target function is completed, enter the image collection interface from the target function loading interface, and collect the artistic portrait on the calendar through the image collection interface. In the case that the target image obtained by collecting the art portrait is successfully recognized through the image acquisition interface, the electronic device determines the target virtual model corresponding to the art portrait as a virtual object, and obtains the target preset rendering data corresponding to the virtual object, and displays it on the target function page Render the virtual object to obtain a virtual effect image. In some embodiments, the virtual effect image may be a dynamic artistic portrait. When the electronic device displays a virtual effect image, that is, a dynamic artistic portrait, it can also play multimedia audio data corresponding to the virtual effect image.

In some embodiments, before the electronic device enters the target function page from the image collection interface, it can detect whether its own sound is turned on. If the sound is not turned on, a prompt window pops up on the image collection interface to remind the user to turn on the sound.

In some embodiments, the target function page may include an image acquisition control and an information display control. When the information display control receives an interactive operation, the electronic device pops up a window interface corresponding to the preset introduction data on the current target function page. It is used to introduce the currently recognized art portraits. When the image collection control receives the interactive operation, the electronic device collects the current virtual effect image, and obtains the first preset template to generate a template object image. When the template object image is generated, the electronic device jumps to the sharing interface, and displays the template object image and the first sharing control on the sharing interface.

In some embodiments, when receiving a sharing operation for the first sharing control, or responding to a preset triggering sharing operation, the electronic device can jump to the sharing applet and enter the second-level sharing program corresponding to the sharing applet. Interface; generate the first applet link corresponding to the template object image through the second sharing control on the secondary sharing page, share the first applet link to the target device, and complete the sharing of the template object image. Alternatively, the template object image can be saved locally through the second save control on the secondary sharing page.

It can be understood that, in the embodiment of the present disclosure, the display effect of the display function can be further enriched and the interactivity with the user can be improved by combining the dynamic virtual effect image with the multimedia audio data.

In some embodiments, the real object includes: a real person, and at least one function entry on the display function menu page includes: a composite photo entry corresponding to the composite photo function, and the electronic device can realize the real person in the preset theme through the composite photo function. Under special effects shooting. The display method in the embodiment of the present disclosure will be described below by taking a real object as a real person as an example.

Based on FIG. 2, as shown in FIG. 11, the process of S101 can be realized through S301-S302, and the process of S102 can be realized through S303, which will be described in combination with each step.

S301. In response to a trigger operation on the composite photo entry, jump from a display function menu page to a composite photo interface, and display a photo control on the composite photo interface.

In the embodiment of the present disclosure, when the target function entry is a composite photo entry, the electronic device responds to the trigger operation on the composite photo entry, and jumps from the display function menu page to the composite photo interface, so that real people can be photographed through the composite photo interface. Perform image acquisition.

In some embodiments, the electronic device may also jump to the target function loading interface as shown in FIG. 12 in response to the trigger operation on the composite photo entry, and load the background data of the target function on the target function loading interface. Wherein, the background data of the target function may include preset special effect stickers. When the background data loading of the target function is completed, jump to the composite photo interface.

In the embodiment of the present disclosure, the electronic device displays a photographing control on a synthesized photographing interface. This is shown as control 140 in FIG. 12 .

In some embodiments, after jumping to the synthesized photographing interface, the electronic device may load preset special effect stickers from the background data of the target function, and display the preset special effect stickers on the synthesized photographing interface. As shown in the sticker 141 of FIG. 12 .

S302. In the case of receiving an interactive operation on the camera control, image acquisition is performed on a real person in a real scene, and an image of the real person is obtained as a target image.

In the implementation of the present disclosure, when receiving an interactive operation on the camera control, the electronic device collects images of real people in a real scene, and obtains images of real people as target images.

S303. When the target image is a real person image, identify the real person image, determine the preset special effect sticker corresponding to the real person image, superimpose the real person image and the preset special effect sticker, generate and display the superimposed current The composite image serves as a virtual effect image.

In the embodiment of the present disclosure, when the target image is a real person image, the electronic device can identify the real person image, obtain the human body part information contained in the real person image, and then determine the preset special effect corresponding to the human body part information Stickers, such as the preset special effect stickers corresponding to the head or the preset special effect stickers corresponding to the eyes, and then superimpose the corresponding parts of the real person image and the preset special effect stickers, generate and display the superimposed current composite image as a virtual effect image.

In some embodiments, after the electronic device generates the current composite image, it may jump to the photographing completion interface as shown in FIG. 12 to display the current composite image. The current composite image is shown as image 151 in FIG. 12 . The electronic device may use the photographing completion interface as the target function page, and display the third sharing control among at least one interactive control on the photographing completion interface, as shown by the control 150 in FIG. 12 . When the electronic device receives a sharing operation for the third sharing control 150, it can acquire a second preset template, and generate a template composite image 160 as shown in FIG. 12 based on the second preset template and the current composite image; The synthesized image 160 is used as the response effect data to update the current synthesized image, that is, the virtual effect image.

In some embodiments, FIG. 12 shows a third save control 161 and a fourth share control 162 . When the third saving control 161 receives a click operation, the template composite image is saved locally. When the fourth sharing control 162 receives a click operation, it generates a second applet link corresponding to the composite image of the template, and shares the second applet link to the target device. The sharing process of the template composite image by the electronic device is consistent with the above description of the sharing process of the template object image.

In some embodiments, the target device can scan the preset identification code on the composite image of the template to enter the loading page, and after the loading is completed, the target device presents a sharing effect page containing the image of the template object. The user of the target device can experience the composite photo taking function through the photo taking experience control on the sharing effect page.

It can be understood that, through the composite photo function, users can be provided with special effect photos of preset themes in the virtual service product, which further enriches the display function of the virtual service product and the user's interactive experience.

An embodiment of the disclosure provides a display device. As shown in FIG. 13 , the display device 1 includes: an acquisition part 10, a rendering part 20 and an operation part 30, wherein

The collection part 10 is configured to collect a target image of a real object in a real scene in response to a trigger operation on a target function entry;

The rendering part 20 is configured to determine the virtual object corresponding to the target image by identifying the target image, and display a virtual effect image obtained by rendering the virtual object on the target function page; the target The function page includes at least one interactive control; the virtual effect image is used to display the augmented reality effect corresponding to the target image;

The operation part 30 is configured to determine response effect data corresponding to the virtual object in response to an interactive operation on a target interactive control in the at least one interactive control, and update the displayed virtual object through the response effect data. effect image.

In some embodiments, the display device 1 further includes: a display part, the display part is configured to display a display function menu page, and the display function menu page is used to display at least one function entry;

The collection part 10 is further configured to, in response to a trigger operation on the target function entry in the at least one function entry, enter an image collection page corresponding to the target function, and collect a target image of a real object in a real scene.

In some embodiments, the display device 1 further includes: a loading part, the loading part is configured to jump from the display function menu page to the target function loading interface, during the display process of the target function loading interface Synchronously load the background data of the target function; the background data of the target function includes at least one preset rendering data; when the loading of the background data of the target function is completed, enter the image acquisition interface.

In some embodiments, the real object includes: a real person; the target function entry includes: a composite photo entry; the collection part 10 is also configured to jump from the display function menu page to a composite photo interface, and displaying a photographing control on the composite photographing interface; in the case of receiving an interactive operation for the photographing control, performing image acquisition on the real person in the real scene, and obtaining an image of the real person as the target image .

In some embodiments, the loading part is further configured to load preset special effect stickers from the background data of the target function after jumping from the display function menu page to the synthetic photographing interface, and display them on the synthetic photographing interface The preset special effect stickers.

In some embodiments, the at least one interactive control includes: a rotation control; the operation part 30 is further configured to display a rotation guide icon when the target interactive control is the rotation control; The rotation operation of the rotation guide icon determines the rotation direction and rotation angle of the virtual object according to the rotation operation; rotates the virtual object according to the rotation direction and the rotation angle, and obtains the rotated The preset rendering data corresponding to the virtual object is used as the response effect data.

In some embodiments, the at least one interactive control includes: an information display control; and the operation part 30 is further configured to acquire the corresponding information of the virtual object when the target interactive control is the information display control. The preset introduction data is used as the response effect data; the preset introduction data is used to display the introduction information of the target real item at a preset display position.

In some embodiments, the at least one interactive control includes: an image capture control; and the operating part 30 is further configured to, when the target interactive control is the image capture control, corresponding to the virtual object Collect the current virtual effect image to obtain the current object image; obtain the first preset template, combine the first preset template and the current object image to generate a template object image, and use the template object image as the Response effect data; the first preset template is used to provide a preset image template for displaying the current object image.

In some embodiments, the first preset template further includes: a preset identification code; the preset identification code is used to provide a link to a preset function.

In some embodiments, the preset identification code is an applet identification code, and the applet identification code is used to provide a link for displaying an entry of the applet.

In some embodiments, the operation part 30 is further configured to jump to a sharing page when the template object image is generated, and display the template object image on the sharing page, the sharing page includes A first sharing control; when receiving a sharing operation for the first sharing control, generating a first applet link corresponding to the template object image, and sharing the first applet link to a target device.

In some embodiments, the sharing page further includes a first save control; and the operation part 30 is further configured to save the template object image when receiving a save operation for the first save control to local storage.

In the above device, the acquisition part 10 is further configured to display acquisition range guide information on the image acquisition interface when entering the image acquisition interface; the acquisition range guide information includes a preset acquisition range boundary; through the The image acquisition interface performs image acquisition on the real object, and takes the image within the boundary of the preset acquisition range as the target image.

In some embodiments, the real object includes: a target real item, and the virtual object is a target virtual model corresponding to the target real item; the rendering part 20 also includes an identification subsection, the identification subsection, It is configured to perform target recognition of at least one preset real item on the target image to obtain a detection result of the target real item; based on the detection result, obtain a target virtual model corresponding to the target real item as the virtual object.

In some embodiments, the image acquisition part 10 is further configured to display on the image acquisition interface a prompt to adjust the preset acquisition if no preset real item is recognized in the target image. Bootstrap information for the location of the bounds of the range.

In some embodiments, the display device 1 further includes a playing part; the playing part is further configured to play multimedia audio data corresponding to the virtual effect image when the virtual effect image is displayed.

In some embodiments, the rendering part 20 is further configured to acquire the corresponding screen position information of the target image in the preset screen coordinate system; perform pose estimation based on the target image, and obtain the position of the target real item in the The real position information in the real scene; from the background data of the target function, obtain the target preset rendering data corresponding to the virtual object, and use the target preset based on the mapping relationship between the real position information and the screen position information The rendering data renders the virtual object to obtain the virtual effect image.

In some embodiments, the target image includes: a real person image, and the virtual object includes: a preset special effect sticker; superimposing, generating and displaying the superimposed current composite image as the virtual effect image.

In some embodiments, the display device 1 is implemented through a small program or a web client.

It should be noted that the description of the above device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the device embodiments of the present disclosure, please refer to the description of the method embodiments of the present disclosure for understanding.

The embodiment of the present disclosure also provides an electronic device. FIG. 14 is a schematic structural diagram of the electronic device 2 provided by the embodiment of the present disclosure. As shown in FIG. 201. The memory 202 and the processor 203 are connected through a communication bus 204; the memory 202 is configured to store an executable computer program; the processor 203 is configured to execute the executable computer program stored in the memory 202, combined with the display screen 201, to realize the present invention The method provided by the disclosed embodiment, for example, the display method provided by the disclosed embodiment.

The embodiment of the present disclosure provides a computer-readable storage medium, on which a computer program is stored, configured to cause the processor 203 to implement the method provided in the embodiment of the present disclosure, for example, the display method provided in the embodiment of the present disclosure.

In some embodiments of the present disclosure, a computer-readable storage medium may be a tangible device that holds and stores instructions for use by an instruction execution device, and may be a volatile storage medium or a non-volatile storage medium. The computer-readable storage medium may be memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; it may also be various devices including one or any combination of the above memories.

In some embodiments of the present disclosure, executable instructions may take the form of programs, software, software modules, scripts, or code written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages) , and it may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

As an example, executable instructions may, but do not necessarily correspond to files in a file system, may be stored as part of a file that holds other programs or data, for example, in a Hyper Text Markup Language (HTML) document in one or more scripts, in a single file dedicated to the program in question, or in multiple cooperating files (for example, files that store one or more modules, subroutines, or sections of code).

As an example, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices located at one site, or alternatively, on multiple computing devices distributed across multiple sites and interconnected by a communication network. to execute.

Those skilled in the art should understand that the embodiments of the present disclosure may be provided as methods, systems, or computer program products. Accordingly, the present disclosure may take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present disclosure may take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, optical storage, etc.) having computer-usable program code embodied therein.

The present disclosure is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each procedure and/or block in the flowchart and/or block diagram, and a combination of procedures and/or blocks in the flowchart and/or block diagram can be realized by computer program instructions. These computer program instructions may be provided to a general purpose computer, special purpose computer, embedded processor, or processor of other programmable data processing equipment to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing equipment produce a An apparatus for realizing the functions specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to operate in a specific manner, such that the instructions stored in the computer-readable memory produce an article of manufacture comprising instruction means, the instructions The device realizes the function specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions can also be loaded onto a computer or other programmable data processing device, causing a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby The instructions provide steps for implementing the functions specified in the flow diagram procedure or procedures and/or block diagram procedures or blocks.

This disclosure relates to the field of augmented reality. By acquiring the image information of the target object in the real environment, and then using various visual correlation algorithms to detect or identify the relevant features, states and attributes of the target object, and thus obtain the image information that matches the specific application. AR effect combining virtual and reality. Exemplarily, the target object may involve faces, limbs, gestures, actions, etc. related to the human body, or markers and markers related to objects, or sand tables, display areas or display items related to venues or places. Vision-related algorithms can involve visual positioning, SLAM, 3D reconstruction, image registration, background segmentation, object key point extraction and tracking, object pose or depth detection, etc. Specific applications can not only involve interactive scenes such as guided tours, navigation, explanations, reconstructions, virtual effect overlays and display related to real scenes or objects, but also special effects processing related to people, such as makeup beautification, body beautification, special effect display, virtual Interactive scenarios such as model display. The relevant features, states and attributes of the target object can be detected or identified through the convolutional neural network. The above-mentioned convolutional neural network is a network model obtained by performing model training based on a deep learning framework.

In summary, by collecting the target image of the real object in the real scene, determining the virtual object, and rendering the virtual object to obtain a virtual effect image, the augmented reality effect of the virtual effect image is used to improve and enrich the display effect of the display application . In addition, the electronic device can receive the user's interactive operation through at least one interactive control, and perform corresponding operation and update display on the virtual object, thereby increasing the richness of the displayed content. In some embodiments, by sharing the virtual effect image to the target device, the interactivity of the displayed applet is further enriched, and it is beneficial to the promotion of the virtual service product corresponding to the displayed applet. In some embodiments, by synthesizing the photographing function, users can be provided with special effects photographing of preset themes in the virtual service product, which further enriches the display function of the virtual service product and the user's interactive experience.

The above descriptions are only preferred embodiments of the present disclosure, and are not intended to limit the protection scope of the present disclosure.

Industrial Applicability

In the embodiment of the present disclosure, by collecting the target image of the real object in the real scene, determining the virtual object, and rendering the virtual object to obtain a virtual effect image, the augmented reality effect of the virtual effect image is used to improve and enrich the display of the display application Effect. In addition, the electronic device can receive the user's interactive operation through at least one interactive control, and perform corresponding operation and update display on the virtual object, thereby increasing the richness of the displayed content. In some embodiments, by sharing the virtual effect image to the target device, the interactivity of the displayed applet is further enriched, and it is beneficial to the promotion of the virtual service product corresponding to the displayed applet. In some embodiments, by synthesizing the photographing function, the user can be provided with a special effect photographing of a preset theme in the virtual service product, which further enriches the display function of the virtual service product and the interactive experience of the user.

Claims (16)

1. A display method comprising:

   Collecting target images of real objects in the real scene in response to a trigger operation on the target function entry;

   Identifying the target image, determining a virtual object corresponding to the target image, and displaying a virtual effect image obtained by rendering the virtual object on a target function page; the target function page includes at least one interactive control; The virtual effect image is used to display the augmented reality effect corresponding to the target image;

   In response to an interactive operation on a target interactive control in the at least one interactive control, determine response effect data corresponding to the virtual object, and update the displayed virtual effect image based on the response effect data.
2. The method according to claim 1, said method further comprising:

   Displaying a display function menu page, the display function menu page is used to display at least one function entry;

   The step of collecting target images of real objects in a real scene in response to a trigger operation on a target function entry includes:

   In response to a trigger operation directed at the target function entry in the at least one function entry, the image collection page corresponding to the target function is entered, and a target image of a real object in a real scene is collected.
3. According to the method according to claim 1 or 2, the real object comprises: a real person; the target function entry comprises: a synthetic photo entry; target image, including:

   Jumping from the display function menu page to a synthetic photographing interface, and displaying photographing controls on the synthetic photographing interface;

   In the case of receiving an interactive operation on the photographing control, image acquisition is performed on the real person in the real scene, and an image of the real person is obtained as the target image.
4. The method according to claim 3, after jumping from the display function menu page to the synthetic camera interface, the method further comprises:

   Load preset special effect stickers from the background data of the target function, and display the preset special effect stickers on the composite photo interface.
5. The method according to any one of claims 1-3, wherein the at least one interactive control includes: a rotation control; and determining the virtual object in response to an interactive operation on a target interactive control in the at least one interactive control Corresponding response effect data, including:

   If the target interactive control is the rotation control, displaying a rotation guide icon;

   receiving a rotation operation on the rotation guide icon, and determining a rotation direction and a rotation angle of the virtual object according to the rotation operation;

   The virtual object is rotated according to the rotation direction and the rotation angle, and preset rendering data corresponding to the rotated virtual object is acquired as the response effect data.
6. The method according to any one of claims 1-3, wherein the at least one interactive control includes: an information display control; and determining the virtual The response effect data corresponding to the object, including:

   In the case where the target interactive control is the information display control, the preset introduction data corresponding to the virtual object is obtained as the response effect data; the preset introduction data is used for displaying at a preset display position Introduction information of the target real item.
7. According to the method according to any one of claims 1-3, the at least one interactive control includes: an image acquisition control; and determining the virtual The response effect data corresponding to the object, including:

   When the target interactive control is the image acquisition control, the current virtual effect image corresponding to the virtual object is collected to obtain the current object image;

   Obtain a first preset template, combine the first preset template and the current object image to generate a template object image, and use the template object image as the response effect data; the first preset template is used for Displaying the current object image provides a preset image template.
8. According to the method according to claim 7, the first preset template further comprises: a preset identification code; the preset identification code is used to provide a link of a preset function.
9. The method according to any one of claims 1-8, said collecting target images of real objects in a real scene, comprising:

   In the case of entering the image acquisition interface, display acquisition range guide information on the image acquisition interface; the acquisition range guide information includes a preset acquisition range boundary;

   The image acquisition of the real object is performed through the image acquisition interface, and the image within the boundary of the preset acquisition range is used as the target image.
10. According to the method according to any one of claims 1-9, the real object comprises: a target real item, and the virtual object is a target virtual model corresponding to the target real item;

    The determining the virtual object corresponding to the target image by identifying the target image includes:

    Perform target recognition of at least one preset real item on the target image to obtain a detection result of the target real item;

    Based on the detection result, a target virtual model corresponding to the target real item is acquired as the virtual object.
11. According to the method according to any one of claims 1-10, the displaying the virtual effect image obtained by rendering the virtual object on the target function page includes:

    Acquiring screen position information corresponding to the target image in a preset screen coordinate system;

    performing attitude assessment based on the target image to obtain real position information of the target real item in the real scene;

    Obtain target preset rendering data corresponding to the virtual object from the target function background data, and use the target preset rendering data to process the virtual object based on the mapping relationship between the real position information and the screen position information rendering to obtain the virtual effect image.
12. A display device comprising:

    The collection part is configured to collect a target image of a real object in a real scene in response to a trigger operation on a target function entry;

    The rendering part is configured to determine the virtual object corresponding to the target image by identifying the target image, and display the virtual effect image obtained by rendering the virtual object on the target function page; the target function page includes At least one interactive control; the virtual effect image is used to display the augmented reality effect corresponding to the target image;

    The operation part is configured to determine response effect data corresponding to the virtual object in response to an interactive operation on a target interactive control in the at least one interactive control, and update the displayed virtual effect image through the response effect data.
13. An electronic device comprising:

    display screen; memory configured to store an executable computer program;

    A processor configured to implement the method according to any one of claims 1 to 11 in combination with the display screen when executing the executable computer program stored in the memory.
14. A computer-readable storage medium, on which a computer program is stored for causing a processor to implement the method according to any one of claims 1 to 11.
15. A computer program, comprising computer-readable codes, when the computer-readable codes run in an electronic device, the processor in the electronic device executes the display method according to any one of claims 1 to 11 A step of.
16. A computer program product, comprising computer program instructions, which cause a computer to execute the steps of the display method according to any one of claims 1 to 11.

Images