WO2022095467A1 - 增强现实场景下的展示方法、装置、设备、介质及程序 - Google Patents

增强现实场景下的展示方法、装置、设备、介质及程序 Download PDF

Info

Publication number
WO2022095467A1
WO2022095467A1 PCT/CN2021/102191 CN2021102191W WO2022095467A1 WO 2022095467 A1 WO2022095467 A1 WO 2022095467A1 CN 2021102191 W CN2021102191 W CN 2021102191W WO 2022095467 A1 WO2022095467 A1 WO 2022095467A1
Authority
WO
WIPO (PCT)
Prior art keywords
target object
position information
image
special effect
effect data
Prior art date
Application number
PCT/CN2021/102191
Other languages
English (en)
French (fr)
Inventor
刘旭
栾青
李斌
Original Assignee
北京市商汤科技开发有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京市商汤科技开发有限公司 filed Critical 北京市商汤科技开发有限公司
Publication of WO2022095467A1 publication Critical patent/WO2022095467A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G06T19/006—Mixed reality
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G06T2207/10021—Stereoscopic video; Stereoscopic image sequence

Definitions

  • the present disclosure relates to the technical field of augmented reality, and in particular, to a display method, apparatus, device, medium and program in an augmented reality scene.
  • Augmented Reality (AR) technology superimposes physical information (visual information, sound, touch, etc.) into the real world after simulation, so that the real environment and virtual objects can be displayed on the same screen or in real time. spatial presentation.
  • physical information visual information, sound, touch, etc.
  • the positioning method is to identify the current position of the AR device, map it to a certain position in the three-dimensional map model, and then display the preset virtual special effect data within the range of the position.
  • This method not only needs to collect a large number of images to reconstruct the 3D map model corresponding to the real environment, but also the effect displayed by the preset virtual data is single, which is not rich and vivid enough.
  • Embodiments of the present disclosure provide a display method, apparatus, device, medium, and program in an augmented reality scene.
  • An embodiment of the present disclosure provides a display method in an augmented reality scene, the method is executed by an electronic device, and the method includes:
  • the AR device is controlled to play the special effect data;
  • the special effect data includes at least one of a virtual image and audio, and there is a preset between the display position of the virtual image and the target object Positional relationship.
  • a comprehensive display of virtual images and audio can be realized, that is, not only virtual images such as AR images, videos, and holograms that match the target object can be displayed, but also there is no need to rebuild the three-dimensional map model.
  • the matching special effect data can be triggered for display directly through the recognition result of the target object, and there is a preset positional relationship between the display position of the virtual image in the special effect data and the target object, so that the display effect can be closely related to the target object, It can display the special effect data in a more targeted manner.
  • the determining the display position information of the special effect data based on the recognition result of the target object in the current scene image includes: recognizing the target object in the current scene image
  • the display position information of the special effect data is determined based on the image position information of the target object in the current scene image.
  • switching the corresponding positioning method to determine the display position information of the special effect data can effectively reduce the probability of interrupting the display of the special effect data due to the failure of one of the positioning methods, and improve the special effect data. Stability of data presentation.
  • the determining the display position information of the special effect data based on the recognition result of the target object in the current scene image includes: the target object is not recognized in the current scene image
  • obtain the relative position information between the target object and the AR device in the world coordinate system and determine the display position information of the special effect data based on the relative position information.
  • switching the corresponding positioning method to determine the display position information of the special effect data can effectively reduce the probability of interrupting the display of the special effect data due to the failure of one of the positioning methods, and improve the special effect data. Stability of data presentation.
  • the controlling the AR device to play the special effect data based on the display position information includes: after determining that at least part of the target object is within an image display range of the AR device In this case, based on the display position information, the AR device is controlled to play at least part of the special effect data in the special effect data; wherein, the at least part of the special effect data is the virtual image and the virtual image corresponding to at least part of the target object. At least one of audios; in the case that it is determined that the target object is not in the image display range of the AR device, based on the display position information, the AR device is controlled to continue to play the audio according to the playback progress of the audio. audio.
  • the image display range of the AR device includes at least part of the target object, at least one of at least part of the corresponding virtual image and audio is displayed, and when the image display range of the AR device does not include the target object, it is not displayed.
  • the virtual image only displays audio, which makes the effect of special effect data display more reasonable, and also makes the display effect of special effect data more coherent.
  • the virtual image includes a hologram
  • the display method further includes: acquiring a to-be-processed video matching the target object, where the to-be-processed video includes a video associated with the target object target associated object; set a transparent channel for each pixel in the to-be-processed video to obtain a first video; remove background pixels from the first video based on the transparent channel to obtain a second video; The second video generates a hologram including the target associated object.
  • the virtual image also includes a holographic image, displaying the holographic image corresponding to the target-related object associated with the target object, and can also display the holographic image superimposed on the current scene image, so that the display effect of the AR content is more abundant.
  • the step of removing background pixels from the first video based on the transparency channel to obtain the second video includes: making transparent pixels corresponding to the background pixels in the first video The channel is set to white to obtain a third video; the first video includes the target pixels of the target associated object and the background pixels except the target pixels; the first type of pixels in the first video is The transparent channel corresponding to the point is set to black, the transparent channel corresponding to the second type of pixel in the first video is set to white, and the transparent channel corresponding to the third type of pixel in the first video is set to preset gray value to obtain the fourth video; the third type of pixels includes target pixels adjacent to the background pixels and background pixels adjacent to the target pixels; the first type of pixels includes The background pixels other than the third type of pixels, the second type of pixels includes target pixels other than the third type of pixels; based on the third video and the fourth video, generate the first Two videos. In this way, by processing different types of pixels of the first video, a display effect of adjusting the original video into a first video.
  • the virtual image includes images of a plurality of virtual objects, and at least one of a presentation sequence and interaction data among the plurality of virtual objects;
  • the control based on the presentation position information, controls Playing the special effect data by the AR device includes: displaying the virtual object on a display position corresponding to the display position information based on at least one of a display sequence and interaction data among the plurality of virtual objects image.
  • the images including the multiple virtual objects and the interaction data between the multiple virtual objects are displayed according to the display sequence of the multiple virtual objects, which can enrich the content displayed by the AR and improve the display effect of the AR content.
  • the determining the display position information of the special effect data based on the image position information of the target object in the current scene image includes: based on the target object in the current scene The image position information in the image, to determine the position information of the target object in the world coordinate system; based on the position information of the target object in the world coordinate system and the position of the AR device in the world coordinate system information, and determine the display position information of the special effect data.
  • the image position information of the target object in the current scene image can be more accurately determined, and the display position information of the special effect data can be obtained more accurately based on the image position information of the target object, thereby providing support for the accurate display of the special effect data.
  • the acquiring relative position information between the target object and the AR device in the world coordinate system includes: based on the current scene image, the historical scene image, and the AR device
  • the relative position information between the AR device and the target object when capturing the image of the historical scene and the target object in the world coordinate system is determined when the image of the historical scene is captured. In this way, using the current scene image, the historical scene image, and the relative position information of the AR device and the target object in the world coordinate system when shooting the historical scene image, it can be more accurately determined that the AR device is shooting the current scene When creating an image, the relative position information with the target object, so as to provide support for the accurate display of special effect data.
  • whether the target object is included in the current scene image is identified as follows: Feature point extraction is performed on the current scene image to obtain a plurality of feature points included in the current scene image, respectively. Corresponding feature information; the multiple feature points are located in the target detection area in the current scene image; based on the feature information corresponding to the multiple feature points and the pre-stored multiple feature points included in the target object The corresponding feature information is compared to determine whether the target object is included in the current scene image. In this way, the extraction and comparison of the above-mentioned feature points can more accurately determine whether there is a target object in the current scene image.
  • Embodiments of the present disclosure provide a display device in an augmented reality scene, including:
  • an image acquisition module configured to acquire the current scene image captured by the augmented reality AR device
  • a position determination module configured to determine the special effect data matched by the target object and the display position information of the special effect data based on the recognition result of the target object by the current scene image
  • a special effect playing module configured to control the AR device to play the special effect data based on the display position information; the special effect data includes at least one of a virtual image and audio, and the display position of the virtual image is related to the target Objects have preset positional relationships.
  • the position determination module is configured to, when the target object is identified in the current scene image, based on image position information of the target object in the current scene image , and determine the display position information of the special effect data.
  • the position determination module is configured to obtain a relationship between the target object and the AR device in the world coordinate system when the target object is not recognized in the current scene image relative position information between the two, and based on the relative position information, determine the display position information of the special effect data.
  • the trickplay module is configured to control the AR based on the display position information under the condition that at least part of the target object is determined to be in the image display range of the AR device
  • the device plays at least part of the special effect data in the special effect data; wherein, the at least part of the special effect data is at least one of the virtual image and audio corresponding to at least part of the target object; after determining that the target object is not In the case of the image display range of the AR device, based on the display position information, the AR device is controlled to continue playing the audio according to the played progress of the audio.
  • the virtual image includes a hologram
  • the display device further includes a hologram generation module configured to acquire a to-be-processed video matching the target object, the to-be-processed video includes a A target associated object associated with the target object; a transparent channel is set for each pixel in the video to be processed to obtain a first video; based on the transparent channel, background pixels are removed from the first video to obtain a second video; and generating a holographic image including the target associated object based on the second video.
  • the holographic image generation module is configured to set the transparent channel corresponding to the background pixel in the first video to white to obtain a third video; the first video includes the The target pixels of the target associated object and the background pixels other than the target pixels; the transparent channel corresponding to the first type of pixels in the first video is set to black, and the first type of pixels in the first video is set to black.
  • the transparent channel corresponding to the second type of pixels is set to white, and the transparent channel corresponding to the third type of pixels in the first video is set to a preset gray value to obtain a fourth video;
  • the third type of pixels includes The target pixels adjacent to the background pixels and the background pixels adjacent to the target pixels;
  • the first type of pixels includes background pixels other than the third type of pixels, and the first type of pixels includes background pixels.
  • the second type of pixels includes target pixels other than the third type of pixels; the second video is generated based on the third video and the fourth video.
  • the virtual image includes images of multiple virtual objects, and at least one of display sequence and interaction data between the multiple virtual objects; the special effect playback module is configured to On the display position corresponding to the display position information, the image of the virtual object is displayed based on at least one of the display sequence and interaction data among the plurality of virtual objects.
  • the position determination module is configured to determine the position information of the target object in the world coordinate system based on the image position information of the target object in the current scene image;
  • the position information of the target object under the world coordinate system and the position information of the AR device under the world coordinate system are used to determine the display position information of the special effect data.
  • the location determination module is configured to be based on the current scene image, the historical scene image, and the AR device and the target object in the world when the historical scene image is captured.
  • the relative position information in the coordinate system determines the relative position information between the AR device and the target object when the AR device captures the current scene image.
  • the location determination module is configured to identify whether the target object is included in the current scene image in the following manner: extract feature points from the current scene image to obtain the current scene Feature information corresponding to multiple feature points included in the image; the multiple feature points are located in the target detection area in the current scene image; based on the feature information corresponding to the multiple feature points and the pre-stored The feature information corresponding to the multiple feature points included in the target object is compared to determine whether the target object is included in the current scene image.
  • Embodiments of the present disclosure further provide an electronic device, including: a processor, a memory, and a bus, where the memory stores machine-readable instructions executable by the processor, and when the electronic device runs, the processor and the The memories communicate with each other through a bus, and when the machine-readable instructions are executed by the processor, the display method in the augmented reality scenario described in any of the foregoing embodiments is executed.
  • Embodiments of the present disclosure further provide a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is run by a processor, the display method in an augmented reality scenario described in any of the foregoing embodiments is executed .
  • Embodiments of the present disclosure further provide a computer program, where the computer program includes computer-readable codes, and when the computer-readable codes are executed in an electronic device, the processor of the electronic device executes any of the foregoing implementations
  • the display method in the augmented reality scene described in the example
  • the embodiments of the present disclosure provide at least one display method, device, device, medium and program in an augmented reality scene, which can realize the comprehensive display of virtual images and audio based on the recognition result of the target object, that is, not only can display matching the target object AR images, videos, holograms and other virtual images, in addition, without rebuilding the 3D map model, the matching special effect data can be directly triggered by the recognition result of the target object for display, and the display position of the virtual image in the special effect data There is a preset positional relationship with the target object, so that the display effect can be closely related to the target object, and the special effect data can be displayed in a more targeted manner.
  • FIG. 1 shows a schematic flowchart of a display method in an augmented reality scenario provided by an embodiment of the present disclosure
  • FIG. 2 shows a schematic diagram of a system architecture to which the display method in an augmented reality scenario according to an embodiment of the present disclosure can be applied;
  • FIG. 3 shows a schematic flowchart of controlling an AR device to play special effect data based on a target object within an image display range of the AR device provided by an embodiment of the present disclosure
  • FIG. 4A shows a schematic flowchart of generating a holographic image provided by an embodiment of the present disclosure
  • FIG. 4B shows a schematic flowchart of removing background pixels in a first video to obtain a second video provided by an embodiment of the present disclosure
  • FIG. 5A shows one of the schematic diagrams of special effect data presented in the present disclosure
  • Figure 5B shows an image in the video to be processed in the present disclosure
  • Figure 5C shows an image in a fourth video of the present disclosure
  • FIG. 6A shows the second schematic diagram of the special effect data presented in the present disclosure
  • FIG. 6B shows the third schematic diagram of the special effect data presented in the present disclosure
  • FIG. 6C shows a fourth schematic diagram of the special effect data presented in the present disclosure.
  • FIG. 7 shows a schematic flowchart of identifying whether a target object is included in the current scene image provided by an embodiment of the present disclosure
  • FIG. 8 shows a schematic structural diagram of a display device in an augmented reality scene provided by an embodiment of the present disclosure
  • FIG. 9 shows a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure.
  • Multiple or multiple in the embodiments of the present disclosure may refer to at least two or at least two, respectively.
  • AR technology is gradually applied in various fields. For example, AR content can be superimposed on physical objects, and the physical objects can be vividly introduced to users through AR content.
  • AR content when AR content is displayed on an AR device, it is necessary to identify the current position of the AR device, and then map it to a certain position in the 3D map model, and then display the preset virtual special effect data within the range of the position. .
  • This method not only needs to collect a large number of images to reconstruct the 3D map model corresponding to the real environment, but also the effect displayed by the preset virtual data is single, which is not rich and vivid enough.
  • the embodiments of the present disclosure provide a display method in an augmented reality scene, Apparatus, equipment, medium, and program, the embodiments of the present disclosure can realize the comprehensive display of virtual images and audio based on the recognition result of the target object, that is, not only can display virtual images such as AR images, videos, and holographic images that match the target object,
  • the matching special effect data can be directly triggered by the recognition result of the target object for display, and the display position of the virtual image in the special effect data has a preset positional relationship with the target object, so that The display effect can be closely related to the target object, and the special effect data can be displayed in a more targeted manner.
  • an embodiment of the present disclosure discloses a presentation method in an augmented reality scene, and the method can be applied to a device with computing capability.
  • the device may be a server or an AR device.
  • the display method in the augmented reality scene may include the following steps:
  • AR devices may include, but are not limited to, AR glasses, tablet computers, smart phones, smart wearable devices, and other devices with display functions and data processing capabilities, and applications for displaying AR scene content can be installed in these AR devices. , users can experience AR scene content in the app.
  • the AR device may also include an image acquisition component for capturing images, such as a three-primary color (Red Green Blue, RGB) camera, after acquiring the current scene image captured by the AR device, the current scene image can be identified, Identify whether it contains the target object that triggers the effect data to be displayed.
  • an image acquisition component for capturing images such as a three-primary color (Red Green Blue, RGB) camera
  • the target object can be an object with a specific shape, such as a book, calligraphy, painting, building and other physical objects. learn.
  • the target object may be a calendar with a preset shape
  • the special effect data may be virtual display content pre-designed based on the content of the calendar
  • the content of the calendar may be introduced to the user to Get users to check the calendar.
  • the captured current scene image may or may not include the target object, so before step S120 is performed, the current scene image may be identified to determine whether the current scene image includes the target object.
  • special effect data matching the target object may be acquired based on the identifier of the target object and the like.
  • the special effect data here may include virtual images, videos, and audios.
  • the virtual images may include videos, holograms, AR images, etc. matched with the target object.
  • the display position information of the special effect data matching the target object may be determined based on the current scene image by using a marker for positioning .
  • utilizing the marker to locate may be, utilizing the image of the target object as the marker, determining the image position information of the target object in the current scene image, and then determining the display position information of the special effect data based on the image position information.
  • SLAM Simultaneous Localization And Mapping
  • the above-mentioned display position information may include coordinate information of the special effect data in the world coordinate system.
  • the world coordinate system is a three-dimensional coordinate system constructed in real space, which is an absolute coordinate system. Among them, the world coordinate system does not change with the position of the AR device, the target object, and the special effect data.
  • the AR device Based on the display position information, control the AR device to play the special effect data; the special effect data includes at least one of a virtual image and audio, and there is a relationship between the display position of the virtual image and the target object. Preset positional relationship.
  • the special effect data will be controlled to be played from the beginning based on the display position information; The current playback progress, continue to play the special effect data. After the special effect data is played, you can also replay the special effect data by clicking the displayed button on the AR device.
  • the above embodiment can directly trigger the display of the matching special effect data through the recognition result of the target object, without rebuilding the three-dimensional map model.
  • the objects are closely related, and the special effect data can be displayed in a more targeted manner.
  • the embodiment of the present disclosure does not need to rebuild the three-dimensional map model, and can directly trigger the display of matching special effect data through the recognition result of the target object, and there is a preset between the display position of the virtual image in the special effect data and the target object. The positional relationship enables the display effect to be closely related to the target object, and can display the special effect data in a more targeted manner.
  • FIG. 2 shows a schematic diagram of a system architecture to which the display method in an augmented reality scene according to an embodiment of the present disclosure can be applied; as shown in FIG. 2 , the system architecture includes: a current scene image acquisition terminal 201 , a network 202 and a control terminal 203 .
  • the current scene image acquisition terminal 201 and the control terminal 203 establish a communication connection through the network 202
  • the current scene image acquisition terminal 201 reports the current scene image to the control terminal 203 through the network 202
  • the control terminal 203 responds to the current scene.
  • the control terminal 203 uploads the display position information and the special effect data to the network 202 , and sends the information to the current scene image acquisition terminal 201 through the network 202 .
  • the current scene image acquisition terminal 201 may include an image acquisition device, and the control terminal 203 may include a visual processing device or a remote server with visual information processing capability.
  • Network 202 may employ wired or wireless connections.
  • the control terminal 203 is a visual processing device
  • the current scene image acquisition terminal 201 can communicate with the visual processing device through a wired connection, such as data communication through a bus;
  • the control terminal 203 is a remote server, the current scene image
  • the acquisition terminal 201 can perform data interaction with a remote server through a wireless network.
  • the current scene image acquisition terminal 201 may be a vision processing device with a video capture module, or a host with a camera.
  • the display method in the augmented reality scene according to the embodiment of the present disclosure may be executed by the current scene image acquisition terminal 201 , and the above-mentioned system architecture may not include the network 202 and the control terminal 203 .
  • the AR device has a limited image display range and cannot display special effect data at all positions. Therefore, when controlling the AR device to play the special effect data based on the display position information, first determine whether the special effect data is located in within the image display range of the AR device.
  • the special effect data is matched with the target object, and the display position of the virtual image in the special effect data has a preset positional relationship with the target object.
  • the display position of the virtual image corresponding to the special effect data may be perpendicular to the cover of the calendar.
  • the AR device When it is determined that at least part of the target object is in the image display range of the AR device, based on the display position information, the AR device is controlled to play at least part of the special effect data in the special effect data; wherein, the At least part of the special effect data is at least one of the virtual image and audio corresponding to at least part of the target object; in the case that it is determined that the target object is not in the image display range of the AR device, based on the display position information , and control the AR device to continue playing the audio according to the playback progress of the audio.
  • the AR device when controlling the AR device to play at least part of the special effect data corresponding to the target object based on the display position information, the AR device may be controlled to play at least part of the corresponding special effect data of the target object. at least one of the virtual image and audio.
  • the target object if the target object is not included in the current scene image, it is determined that the target object is not within the image display range of the AR device, and at this time, the virtual image in the special effect data matching the target object is not in the AR device. Within the image display range, at this time, when controlling the AR device to play the special effect data based on the display position information, the AR device may be controlled to continue playing the audio according to the playback progress of the audio.
  • the target object if at least part of the target object is included in the image display range of the AR device, at least one of the corresponding virtual image and audio is displayed, and the target object is not included in the image display range of the AR device
  • the virtual image is displayed, only the audio is displayed, which makes the display effect of the special effect data more reasonable, and also makes the display effect of the special effect data more coherent.
  • the above-mentioned determining whether the target object is included in the current scene image, or whether at least part of the target object is included, for example, can be implemented according to the following steps:
  • feature point extraction is performed on the current scene image to obtain feature information corresponding to multiple feature points contained in the current scene image; the multiple feature points are located in the target detection area in the current scene image middle.
  • the second step based on the feature information corresponding to the multiple feature points and the feature information corresponding to the multiple feature points contained in the target object stored in advance, it is determined whether the current scene image contains the feature information.
  • the target object or whether to include part of the target object.
  • the current scene image includes a complete target object; If the ratio of successful matching of stored feature points is higher than the preset ratio, it is determined that the current scene image includes some target objects; if the ratio of successful matching between the feature points extracted from the current scene image and the pre-stored feature points is lower than or equal to the preset ratio If the scale is set, it is determined that the target object is not included in the current scene image. In this way, the extraction and comparison of the above-mentioned feature points can more accurately determine whether there is a target object in the current scene image.
  • the matching process can be seen in the following steps S510 to S520.
  • the image display range of the AR device includes the complete target object, and the AR device displays at least one of the complete virtual image and audio at this time; in the current scene image
  • some target objects are included in the image display range of the AR device
  • some target objects are included in the image display range of the AR device, and the AR device displays at least one of some virtual images and audio
  • the image display range of the AR device does not include the target object.
  • the AR device does not display virtual images, only audio. In this way, the rationality and coherence of the special effect data display can be improved.
  • a holographic image matching the target object may also be displayed, and the holographic image includes a target-related object associated with the target object.
  • the following steps S210 to S240 may be used to generate a holographic image:
  • the target associated object is associated with the target object.
  • the target associated object may be a tour guide 502 who introduces the place.
  • FIG. 5B it is an image in the video to be processed, and the image has a background, which includes a target associated object, that is, a tour guide 502 who introduces the place.
  • a transparent channel is set for each pixel in each image in the video to be processed.
  • the transparent channel can be used to control the degree of transparency of the corresponding pixel.
  • Transparent pixels do not contribute to the image, that is, the pixels are not displayed; opaque
  • the pixels of contribute to the image, that is, the pixels are displayed.
  • the transparent channel of a pixel is set to 0
  • the transparent channel of the pixel is set to black, and the pixel does not contribute to the image;
  • the value of the transparent channel is set to 1
  • the transparent channel of the pixel is set to white, and the pixel contributes to the image.
  • the following steps S2301 to S2303 can be used to remove background pixels in the first video to obtain the second video:
  • the transparent channel corresponding to the background pixel is set to 1.
  • the transparent channel corresponding to the third type of pixels is set to a preset gray value to obtain a fourth video;
  • the third type of pixels includes the target pixels adjacent to the background pixels and the target pixels. Adjacent background pixels; the first type of pixels includes background pixels other than the third type of pixels, and the second type of pixels includes target pixels other than the third type of pixels .
  • the transparent channel corresponding to the first type of pixels is set to 0, the transparent channel corresponding to the second type of pixels is set to 1, and the transparent channel corresponding to the third type of pixels is set to be between 0 and 1. value, that is, the preset gray value.
  • the purpose of setting the third type of pixel point as the preset gray value is to make the color of the pixel on the edge of the target related object close to the transparent color of the background, so that the displayed color of the edge of the target related object can be smooth and excessive.
  • Figure 5C shows an image in the fourth video.
  • the second video with the background removed and only the target associated object retained can be obtained.
  • the background of the guide 302 is transparent.
  • a display effect of adjusting the original video into a holographic image can be achieved.
  • the to-be-processed video that matches the target object may be processed into a video material with a transparent channel (corresponding to the first video in the embodiments of the present disclosure) through a mask method; wherein, the mask is The outside of the marquee (the inside of the marquee is the selection). Then, the transparent channel corresponding to the background pixels in the video material is set to white to obtain a third video. Set the transparent channel corresponding to the background pixel in the video material to black, set the transparent channel corresponding to the target pixel of the target associated object in the video material to white, and set the pixel point of the gradient part in the video material to white.
  • the corresponding transparent channel is set to a preset gray value, and a fourth video is obtained; wherein, the pixels of the gradient part are the target pixels adjacent to the background pixels and the background pixels adjacent to the target pixels. Finally, the third video and the fourth video are horizontally integrated to obtain the second video. In this way, the size of the video can be reduced, thereby improving the efficiency of producing the holographic effect and special effect data associated with the video.
  • the virtual image further includes a holographic image, which displays a holographic image corresponding to the target associated object associated with the target object, and can also display the holographic image superimposed on the current scene image, so that the display effect of the AR content is more abundant.
  • the virtual image may also include videos, AR images, text, etc.
  • FIG. 6A when the target object cake is recognized, the text “Happy Birthday” is displayed .
  • FIG. 6B when the calendar of the target object is recognized, an AR screen related to the calendar object is displayed, and the AR screen includes virtual objects such as dragons and squirrels.
  • images of multiple virtual objects may be set in the virtual image, and the display order among the multiple virtual objects and the interaction data among the multiple virtual objects are preset. at least one.
  • controlling the AR device to play the special effect data can be achieved through the following process:
  • the image of the virtual object is displayed based on at least one of the display sequence and interaction data among the plurality of virtual objects.
  • the images of virtual object warrior one 601 and virtual object warrior two 602 are set in the virtual image.
  • the virtual image showing warrior one 601 first appears in the screen displayed by the AR device, and warrior two
  • the combat status of the two is displayed.
  • images including multiple virtual objects and interaction data between multiple virtual objects are displayed according to the display sequence among the multiple virtual objects, which can enrich the content displayed by the AR and improve the display effect of the AR content.
  • the target object or AR device may move.
  • the target object or AR device may move.
  • the position of the target object changes, how can we continue to determine the display position information of the special effect data, so as to carry out the special effect data?
  • Coherent display to provide a more realistic display effect is a problem worthy of study.
  • the special effect data may be any content such as a three-dimensional (3-Dimension, 3D) model, video, audio, or transparent video, or may be any combination of the foregoing multiple content.
  • the embodiment of the present disclosure uses two positioning methods to determine the display position information of the special effect data, that is, the following steps can be used to determine the display position information of the special effect data:
  • the display position information of the special effect data is determined based on the image position information of the target object in the current scene image.
  • the target object In the case where the target object is not recognized in the current scene image, obtain relative position information between the target object and the AR device in the world coordinate system, and determine the relative position information based on the relative position information The placement information for the effect data.
  • a first positioning method may be used to determine the location of the special effect data based on the image position information of the target object in the current scene image. Placement information.
  • the second positioning method can be used to obtain the world coordinates
  • the relative position information between the target object and the AR device is determined, and based on the relative position information, the display position information of the special effect data is determined.
  • the display position information of the special effect data can be determined according to the second positioning method, so that the AR device can be controlled based on the determined display position information to continue to perform the special effect data that has not been displayed.
  • Display can improve the continuity of special effects data in the display process, making the display of special effects data more realistic.
  • switching the corresponding positioning method to determine the display position information of the special effect data can effectively reduce the probability of interrupting the display of the special effect data due to the failure of one of the positioning methods. , which improves the stability of special effects data display.
  • the target object is a calendar
  • the special effect data includes a video with a total duration of 30s displayed dynamically. If the target object cannot be recognized in the current scene image captured by the AR device when the video is displayed for the 10s Based on the display position information of the special effect data determined by the second positioning method, continue to control the AR device to continue to display from the tenth s according to the video. If during the continuous display process, it is determined based on the scene image that the calendar completely leaves the image display range of the AR device, for example, the shooting angle of the AR device completely leaves the calendar, and the video is naturally outside the image display range. However, the user cannot watch the video corresponding to the special effect data through the AR device.
  • the shooting angle of the AR device can also capture part of the calendar, and the user can watch the video through the AR device. part of the video.
  • the target object is a calendar
  • the special effect data includes a video with a total duration of 30s displayed dynamically. If all target objects are identified in the current scene image captured by the AR device when the video is displayed at the 10s Using the display position information determined by the image position information of the calendar in the current scene image, continue to control the AR device to continue to display from the 10s according to the video.
  • the AR device plays the special effect data that can move with the movement of the target object, that is, change according to the appearance of the target object in the current scene image. For example, when the recognition of the target object is interrupted, corresponding to the case where the target object cannot be recognized in the current scene image captured in the embodiment of the present disclosure, for example, the target object may be blocked, the scanning area may be removed, or the like,
  • the AR device can continue to be controlled to play the special effect data according to the display position information of the special effect data determined based on the SLAM positioning method. After that, when the target object is re-identified in the current scene image captured, continue to determine the display position information of the special effect data in a real-time positioning method, and control the AR device to play the special effect data.
  • the current scene image includes a complete calendar. If the ratio of successful matching between the feature points extracted from the current scene image and the feature points of the pre-stored calendar is higher than the preset ratio, it is determined that the current scene image includes part of the calendar. If the ratio of successful matching between the feature points extracted from the current scene image and the feature points of the pre-stored calendar is lower than or equal to the preset ratio, it is determined that the current scene image does not include the calendar.
  • the above-mentioned first positioning method based on image recognition technology, can more accurately determine the image position information of the target object in the current scene image. Therefore, here based on the image position information of the target object, the display position of the special effect data can be obtained more accurately. information to support the accurate presentation of special effects data.
  • the first positioning method is based on the image position information of the target object in the current scene image to determine the display position information of the special effect data. Therefore, in the process of locating the target object based on the first positioning method, the AR device can be determined at the same time.
  • the relative position information with the target object is stored, and the relative position information is saved.
  • the display position information of the special effect data may be determined based on the relative position information and the relative positional relationship between the special effect data and the target object, and the process will be described in detail later.
  • whether a target object is included in the current scene image can be identified in the following manner, as shown in FIG. 7 :
  • S510 Extract feature points on the current scene image to obtain feature information respectively corresponding to multiple feature points included in the current scene image; the multiple feature points are located in the target detection area in the current scene image.
  • an image detection algorithm may be used to locate a target detection area that includes a solid object in the current scene image. Then, feature point extraction is performed in the target detection area. For example, the feature points located on the outline of the solid object, the feature points located in the identification pattern area, and the feature points located in the text area can be extracted in the target detection area.
  • the feature points can be uniformly extracted based on the corresponding location area of the target object in the current scene image. Perform uniform extraction in the corresponding rectangular area in the current scene image.
  • the feature information included in the feature point extracted here may include information that can represent the feature of the feature point, such as texture feature value, RGB feature value, gray value, etc. corresponding to the feature point.
  • the target object may be photographed in advance in the same manner, and feature information corresponding to multiple feature points included in the target object may be obtained and saved.
  • the multiple feature points extracted based on the current scene image can be firstly compared.
  • the respectively corresponding feature information determines the first feature vector corresponding to the target detection area in the current scene image
  • the second feature vector corresponding to the target object is determined based on the feature information corresponding to the multiple feature points included in the target object.
  • the similarity between the target detection area and the target object can be determined through the first feature vector and the second feature vector, for example, it can be determined through the cosine formula.
  • the similarity between the first feature vector and the second feature vector is greater than or equal to a preset similarity threshold, it is determined that the current scene image contains the target object. Conversely, in the case that the similarity between the first feature vector and the second feature vector is determined to be less than the preset similarity threshold, it is determined that the current scene image does not contain the target object.
  • the extraction and comparison of the above-mentioned feature points can more accurately determine whether there is a target object in the current scene image.
  • the following steps can be used to determine the display position information of the special effect data:
  • an image coordinate system can be established based on the current scene image, and image coordinate values of multiple feature points included in the target object in the image coordinate system can be obtained to obtain image position information of the target object in the current scene image.
  • the above-mentioned determination of the position information of the target object in the world coordinate system based on the image position information of the target object in the current scene image may be based on the conversion relationship between the above-mentioned image position information, the image coordinate system and the camera coordinate system corresponding to the AR device , and the conversion relationship between the camera coordinate system corresponding to the AR device and the world coordinate system, to determine the position information of the target object in the world coordinate system.
  • the camera coordinate system corresponding to the AR device may be a three-dimensional rectangular coordinate system established with the focal center of the image acquisition component included in the AR device as the origin and the optical axis as the Z axis.
  • the position information of the target object in the camera coordinate system can be determined based on the conversion relationship between the image coordinate system and the camera coordinate system.
  • the world coordinate system can be established with the center point of the target object as the origin.
  • the center of the calendar can be used as the origin, and the long side passing through the center of the calendar can be used as the origin.
  • the conversion between the camera coordinate system and the world coordinate system is a rigid body conversion, that is, a conversion method in which the camera coordinate system can be rotated and translated to coincide with the world coordinate system.
  • the conversion relationship between the camera coordinate system and the world coordinate system can be determined by the position coordinates of multiple position points in the target object in the world coordinate system and the corresponding position coordinates in the camera coordinate system.
  • the position information of the target object in the world coordinate system can be determined based on the conversion relationship between the camera coordinate system corresponding to the AR device and the world coordinate system.
  • the above-mentioned determination of the display position information of the special effect data based on the position information of the target object in the world coordinate system and the position information of the AR device in the world coordinate system may be the AR device in the world coordinate system.
  • the location information below is determined by the current scene image captured by the AR device. For example, by selecting feature points in the current scene image, by determining the position coordinates of the selected feature points in the world coordinate system established by the target object, and the position coordinates of the selected feature points in the camera coordinate system corresponding to the AR device , the position information of the AR device in the world coordinate system when the current scene image is captured can be determined.
  • the display position information of the special effect data can be determined.
  • the position information of the special effect data in the world coordinate system is determined; based on the position information of the special effect data in the world coordinate system and the position information of the AR device in the world coordinate system, the special effect data is determined. 's placement information.
  • the position of the special effect data in the world coordinate system can be determined according to the position information of the target object in the world coordinate system and the preset positional relationship between the display position of the preset special effect data and the target object in the same coordinate system. Placement information.
  • the image position information of the target object in the current scene image can be determined more accurately, and the display position information of the special effect data can be obtained more accurately based on the image position information of the target object, so as to accurately display the special effect data provide support.
  • the following steps can be used to determine the display position information of the special effect data:
  • the AR device Based on the current scene image, the historical scene image, and the relative position information of the AR device and the target object in the world coordinate system when shooting the historical scene image, it is determined that the AR device is shooting the current scene
  • the relative position information between the target object and the target object when creating an image and based on the determined relative position information, the display position information of the special effect data is determined.
  • the current scene image is taken as an example of the third frame of scene image captured by the AR device, and the SLAM technology is used to briefly describe how to determine the relative position information between the AR device and the target object when the AR device captures the current scene image.
  • the world coordinate system established with the center point of the target object as the origin and the selected feature points in the first frame of the scene image captured by the AR device can be located in the world respectively.
  • the coordinate system and the position coordinates in the camera coordinate system corresponding to the AR device determine the position information of the AR device in the world coordinate system when the first frame of scene image is captured, and also include the target object when the AR device captures the first frame of the scene image.
  • the AR device Based on the position information of the AR device in the world coordinate system when the first frame of scene image is captured, and the position information of the target object in the world coordinate system when the AR device captures the first frame of scene image, it can be determined that the AR device captures the first frame of scene image.
  • the relative position information of a frame of scene image and the target object in the world coordinate system is based on the position information of the AR device in the world coordinate system when the first frame of scene image is captured, and the position information of the target object in the world coordinate system when the AR device captures the first frame of scene image.
  • the target feature points included in the first frame of scene images may be found in the second frame of scene images, and the AR device captures the two frames based on the target feature points.
  • the position information of the scene image in the camera coordinate system determines the position offset of the AR device when shooting the second frame of scene image relative to when the first frame of scene image is shot. Then, based on the position offset and the relative position information of the AR device and the target object in the established world coordinate system when the first frame of scene image is captured, it is determined that the AR device is in the same position as the target object when the second frame of scene image is captured. Relative position information in the world coordinate system.
  • the position offset of the AR device when the current scene image is captured relative to the second frame of the scene image can be determined in the same way, so that the AR device can be used in combination with the AR device to capture the current scene image compared to the current scene image.
  • the position offset of the second frame of scene image, and the relative position information of the AR device and the target object in the world coordinate system when the second frame of scene image is captured it is determined that the AR device is in the same position as the target object when the current scene image is captured. Relative position information in the world coordinate system.
  • determining the display position information of the special effect data in the world coordinate system it can also determine the display attitude information of the special effect data in the world coordinate system, and the processing logic is basically the same.
  • displaying the special effect data it can be displayed in combination with the determined display position information and the display posture information at the same time.
  • the display method in the augmented reality scene provided by the embodiment of the present disclosure can realize the comprehensive display of virtual influence and audio based on the recognition result of the target object obtained by the current scene image.
  • the display position information of the special effect data provided by the embodiment of the present disclosure may be determined according to the recognition result of the target object in the current scene image, that is, if the AR device is controlled to play the special effect data according to the display position information of the special effect data, it is The recognition result of the target object in the current scene image controls the AR device to play the special effect data, and there is preset position information between the display position of the special effect data and the target object.
  • the current scene image is a wedding invitation made
  • the target object is a newcomer.
  • special effects data matching the newcomer is determined, for example, showing the newcomer in love , the video of love, and the corresponding display location information of the video; based on the fact that the new person is not recognized in some areas of the wedding invitation, determine the audio information that matches the new person, such as the audio information about the wedding that the new person sings. , and the placement information corresponding to the audio.
  • the AR device is controlled to play the video or audio based on the two different display position information, so that the visibility and viewing of the wedding invitation can be increased.
  • the embodiment of the present disclosure further discloses a display device in an augmented reality scenario, and each module in the device can implement the above-mentioned various embodiments executed on the server or on the AR device.
  • the display method in augmented reality scene and can achieve the same beneficial effect.
  • the display device in the augmented reality scene includes:
  • the image acquisition module 810 is configured to acquire the current scene image captured by the augmented reality AR device.
  • the position determination module 820 is configured to determine the special effect data matched with the target object and the display position information of the special effect data based on the recognition result of the target object in the current scene image.
  • the special effect playing module 830 is configured to control the AR device to play the special effect data based on the display position information; the special effect data includes at least one of a virtual image and audio, and the display position of the virtual image is the same as that of the virtual image. There is a preset positional relationship between the target objects.
  • the position determination module 820 is configured to, when the target object is identified in the current scene image, based on the image position of the target object in the current scene image information, and determine the display position information of the special effect data.
  • the location determination module 820 is configured to obtain the target object and the AR device in the world coordinate system when the target object is not recognized in the current scene image relative position information between them, and based on the relative position information, determine the display position information of the special effect data.
  • the special effect playing module 830 is configured to, in the case of determining that at least part of the target object is in the image display range of the AR device, based on the display position information, control the The AR device plays at least part of the special effect data in the special effect data; wherein, the at least part of the special effect data is at least one of the virtual image and audio corresponding to at least part of the target object; after determining the target object Controlling the AR device to continue playing the audio according to the playback progress of the audio based on the display position information when the image display range of the AR device is not present.
  • the virtual image includes a hologram
  • the display device further includes a hologram generation module configured to acquire a to-be-processed video matching the target object, the to-be-processed video includes a A target associated object associated with the target object; a transparent channel is set for each pixel in the video to be processed to obtain a first video; based on the transparent channel, background pixels are removed from the first video to obtain a second video; and generating a holographic image including the target associated object based on the second video.
  • the holographic image generation module is configured to set the transparent channel corresponding to the background pixel in the first video to white to obtain a third video; the first video includes the The target pixels of the target associated object and the background pixels other than the target pixels; the transparent channel corresponding to the first type of pixels in the first video is set to black, and the first type of pixels in the first video is set to black.
  • the transparent channel corresponding to the second type of pixels is set to white, and the transparent channel corresponding to the third type of pixels in the first video is set to a preset gray value to obtain a fourth video;
  • the third type of pixels includes The target pixels adjacent to the background pixels and the background pixels adjacent to the target pixels;
  • the first type of pixels includes background pixels other than the third type of pixels, and the first type of pixels includes background pixels.
  • the second type of pixels includes target pixels other than the third type of pixels; the second video is generated based on the third video and the fourth video.
  • the virtual image includes images of multiple virtual objects, and at least one of display sequence and interaction data among the multiple virtual objects; the special effect playback module 830, configured to In order to display the image of the virtual object on the display position corresponding to the display position information, based on at least one of the display sequence and interaction data among the plurality of virtual objects.
  • the position determination module 820 is configured to determine the position information of the target object in the world coordinate system based on the image position information of the target object in the current scene image; The position information of the target object under the world coordinate system and the position information of the AR device under the world coordinate system determine the display position information of the special effect data.
  • the location determination module 820 is configured to be based on the current scene image, the historical scene image, and the location of the target object in the AR device when shooting the historical scene image.
  • the relative position information in the world coordinate system determines the relative position information between the AR device and the target object when the AR device captures the current scene image.
  • the location determination module 820 is configured to identify whether the target object is included in the current scene image in the following manner: extract feature points from the current scene image to obtain the current scene image. The feature information corresponding to the multiple feature points contained in the scene image; the multiple feature points are located in the target detection area in the current scene image; based on the feature information corresponding to the multiple feature points and the pre-stored The feature information corresponding to a plurality of feature points included in the target object is compared to determine whether the target object is included in the current scene image.
  • an embodiment of the present disclosure further provides an electronic device 900.
  • the schematic structural diagram of the electronic device 900 provided by the embodiment of the present disclosure includes:
  • the memory 92 is configured to store execution instructions, including the memory 921 and the external memory 922; the memory 921 here is also called internal memory, and is configured to temporarily store the operation data in the processor 91 and the data exchanged with the external memory 922 such as the hard disk. 91 exchanges data with the external memory 922 through the internal memory 921.
  • the processor 91 and the memory 92 communicate through the bus 93, so that the processor 91 executes the following instructions:
  • the AR device determines the special effect data matched by the target object and the display position information of the special effect data based on the recognition result of the target object by the current scene image; based on the display position information,
  • the AR device is controlled to play the special effect data;
  • the special effect data includes at least one of a virtual image and audio, and a display position of the virtual image has a preset positional relationship with the target object.
  • Embodiments of the present disclosure further provide a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is run by a processor, the display method in the augmented reality scenario described in the foregoing method embodiments is executed.
  • the storage medium may be a volatile or non-volatile computer-readable storage medium.
  • Embodiments of the present disclosure further provide a computer program, where the computer program includes computer-readable codes, and when the computer-readable codes are executed in an electronic device, the processor of the electronic device executes any of the foregoing implementations Example of the display method in the augmented reality scene.
  • Embodiments of the present disclosure further provide another computer program product, including a computer-readable storage medium storing program codes, where the instructions included in the program codes can be configured to execute the display method in the augmented reality scenario described in the foregoing method embodiments , see the above method examples.
  • the computer program product can be realized by means of hardware, software or a combination thereof.
  • the computer program product may be embodied as a computer storage medium, and in other embodiments, the computer program product may be embodied as a software product, such as a software development kit (Software Development Kit, SDK) and the like.
  • the apparatus involved in the embodiments of the present disclosure may be at least one of a system, a method, and a computer program product.
  • the computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of the present disclosure.
  • a computer-readable storage medium may be a tangible device that can hold and store instructions for use by the instruction execution device.
  • the computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing.
  • Examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable Programmable Read Only Memory (Electrical Programmable Read Only Memory, EPROM) or flash memory, Static Random Access Memory (Static Random-Access Memory, SRAM), Portable Compact Disc Read-Only Memory (CD-ROM), Digital Video Discs (DVDs), memory sticks, floppy disks, mechanical coding devices, such as punch cards or raised structures in grooves on which instructions are stored, and any suitable combination of the above.
  • RAM random access memory
  • ROM read-only memory
  • EPROM erasable Programmable Read Only Memory
  • flash memory Static Random Access Memory
  • SRAM Static Random Access Memory
  • CD-ROM Portable Compact Disc Read-Only Memory
  • DVDs Digital Video Discs
  • memory sticks floppy disks
  • mechanical coding devices such as punch cards or raised structures in grooves on which instructions are stored, and any suitable combination of the above.
  • Computer-readable storage media are not to be construed as transient signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (eg, light pulses through fiber optic cables), or through electrical wires transmitted electrical signals.
  • the computer readable program instructions described herein may be downloaded from a computer readable storage medium to various computing/processing devices, or to an external computer or external storage device over a network such as at least one of the Internet, a local area network, a wide area network, and a wireless network .
  • the network may include at least one of copper transmission cables, fiber optic transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and edge servers.
  • a network adapter card or network interface in each computing/processing device receives computer-readable program instructions from a network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing/processing device .
  • the computer program instructions for carrying out the operations of the present disclosure may be assembly instructions, Industry Standard Architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or in one or more source or object code written in any combination of programming languages, including object-oriented programming languages—such as Smalltalk, C++, etc., and conventional procedural programming languages, such as the “C” language or similar programming languages.
  • the computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server implement.
  • the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or Wide Area Network (WAN), or may be connected to an external computer (eg, using Internet service provider to connect via the Internet).
  • LAN Local Area Network
  • WAN Wide Area Network
  • electronic circuits such as programmable logic circuits, FPGAs, or Programmable Logic Arrays (PLAs), that can execute computer-readable Program instructions are read to implement various aspects of the present disclosure.
  • PDAs Programmable Logic Arrays
  • the units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.
  • each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.
  • the functions, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a processor-executable non-volatile computer-readable storage medium.
  • the computer software products are stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present disclosure.
  • the aforementioned storage medium includes: a U disk, a removable hard disk, a ROM, a RAM, a magnetic disk, or an optical disk and other mediums that can store program codes.
  • Embodiments of the present disclosure provide a display method, device, device, medium, and program in an augmented reality scene.
  • the method includes: acquiring a current scene image captured by an augmented reality AR device; and identifying a target object based on the current scene image. , determine the special effect data matched by the target object and the display position information of the special effect data; control the AR device to play the special effect data based on the display position information; the special effect data includes at least one of the virtual image and the audio One, there is a preset positional relationship between the display position of the virtual image and the target object.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computer Graphics (AREA)
  • Computer Hardware Design (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Processing Or Creating Images (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

一种增强现实场景下的展示方法、装置、设备、介质及程序，其中，首先获取增强现实AR设备拍摄的当前场景图像（S110）；其次，基于所述当前场景图像对目标对象的识别结果，确定所述目标对象匹配的特效数据以及所述特效数据的展示位置信息（S120）；最后，基于所述展示位置信息，控制所述AR设备播放所述特效数据；其中，所述特效数据包括虚拟影像和音频中的至少之一，所述虚拟影像的展示位置与所述目标对象之间具有预设位置关系（S130）。

Description

增强现实场景下的展示方法、装置、设备、介质及程序
相关申请的交叉引用
本专利申请要求2020年11月06日提交的中国专利申请号为202011233879.6、申请人为北京市商汤科技开发有限公司，申请名称为“增强现实场景下的展示方法、装置、电子设备及存储介质”的优先权，该申请的全文以引用的方式并入本申请中。
技术领域
本公开涉及增强现实技术领域，尤其涉及一种增强现实场景下的展示方法、装置、设备、介质及程序。
背景技术
增强现实(Augmented Reality，AR)技术，通过将实体信息(视觉信息、声音、触觉等)通过模拟仿真后，叠加到真实世界中，从而将真实的环境和虚拟的物体实时地在同一个画面或空间呈现。
相关技术中，定位方式是通过识别AR设备当前所处的位置，映射到三维地图模型中的某一位置，进而展示该位置所在范围内的预设好的虚拟特效数据。这种方式不仅需要采集大批量图像来重构真实环境对应的三维地图模型，而且预设好的虚拟数据展示出来的效果单一，不够丰富和生动。
发明内容
本公开实施例提供一种增强现实场景下的展示方法、装置、设备、介质及程序。
本公开实施例提供了一种增强现实场景下的展示方法，所述方法由电子设备执行，所述方法包括：
获取增强现实AR设备拍摄的当前场景图像；
基于所述当前场景图像对目标对象的识别结果，确定所述目标对象匹配的特效数据以及所述特效数据的展示位置信息；
基于所述展示位置信息，控制所述AR设备播放所述特效数据；所述特效数据包括虚拟影像和音频中的至少之一，所述虚拟影像的展示位置与所述目标对象之间具有预设位置关系。
如此，能够基于对目标对象的识别结果，实现虚拟影像和音频的综合展示，即不仅能够展示与目标对象匹配的AR画面、视频、全息影像等虚拟影像，另外，无需重新构建三维地图模型，能够直接通过目标对象的识别结果即可触发匹配的特效数据进行展示，并且，特效数据中的虚拟影像的展示位置与目标对象之间具有预设位置关系，使其展示效果能够与目标对象紧密关联，能够更有针对性的去展示特效数据。
在本公开的一些实施例中，所述基于所述当前场景图像对目标对象的识别结果，确定所述特效数据的展示位置信息，包括：在所述当前场景图像中识别到所述目标对象的情况下，基于所述目标对象在所述当前场景图像中的图像位置信息，确定所述特效数据的展示位置信息。如此，基于当前场景图像中目标对象的识别结果，切换对应的定位方式来确定特效数据的展示位置信息，可以有效降低由于其中一种定位方式定位失败而中断特效数据的展示的概率，提高了特效数据展示的稳定性。
在本公开的一些实施例中，所述基于所述当前场景图像对目标对象的识别结果，确定所述特效数据的展示位置信息，包括：在所述当前场景图像中未识别到所述目标对象的情况下，获取世界坐标系下所述目标对象与所述AR设备之间的相对位置信息，并基于所述相对位置信息，确定所述特效数据的展示位置信息。如此，基于当前场景图像中目标对象的识别结果，切换对应的定位方式来确定特效数据的展示位置信息，可以有效降低由于其中一种定位方式定位失败而中断特效数据的展示的概率，提高了特效数据展示的稳定性。
在本公开的一些实施例中，所述基于所述展示位置信息，控制所述AR设备播放所述特效数据，包括：在确定所述目标对象的至少部分在所述AR设备的图像展示范围的情况下，基于所述展示位置信息，控制所述AR设备播放所述特效数据中的至少部分特效数据；其中，所述至少部分特效数据为所述目标对象的至少部分对应的所述虚拟影像和音频中的至少之一；在确定所述目标对象未在所述AR设备的图像展示范围的情况下，基于所述展示位置信息，控制所述AR设备按照所述音频的已播 放进度继续播放所述音频。如此，在AR设备的图像展示范围内包括至少部分目标对象的情况下，展示至少部分对应的虚拟影像和音频中的至少之一，在AR设备的图像展示范围内不包括目标对象时，不展示虚拟影像，只展示音频，使得特效数据展示的效果更为合理，也使得特效数据的展示效果更为连贯。
在本公开的一些实施例中，所述虚拟影像包括全息影像；所述展示方法还包括：获取与所述目标对象匹配的待处理视频，所述待处理视频中包括与所述目标对象关联的目标关联对象；为所述待处理视频中的每个像素点设置透明通道，得到第一视频；基于所述透明通道，从所述第一视频中去除背景像素点，得到第二视频；基于所述第二视频生成包括所述目标关联对象的全息影像。如此，虚拟影像还包括全息影像，展示与目标对象关联的目标关联对象对应的全息影像，还可以在当前场景图像中叠加显示全息影像，使得AR内容的展示效果更为丰富。
在本公开的一些实施例中，所述基于所述透明通道，从所述第一视频中去除背景像素点，得到第二视频，包括：将所述第一视频中的背景像素点对应的透明通道设置为白色，得到第三视频；所述第一视频包括所述目标关联对象的目标像素点和除所述目标像素点以外的背景像素点；将所述第一视频中的第一类像素点对应的透明通道设置为黑色，将第一视频中的第二类像素点对应的透明通道设置为白色，将所述第一视频中的第三类像素点对应的透明通道设置为预设灰色值，得到第四视频；所述第三类像素点包括与所述背景像素点相邻的目标像素点和与所述目标像素点相邻的背景像素点；所述第一类像素点包括除所述第三类像素点以外的所述背景像素点，所述第二类像素点包括除所述第三类像素点以外的目标像素点；基于第三视频和第四视频，生成所述第二视频。如此，通过对第一视频的不同类型像素点进行处理，可以实现将原视频调整为全息影像的展示效果。
在本公开的一些实施例中，所述虚拟影像包括多个虚拟对象的影像，以及多个虚拟对象之间的展示顺序和交互数据中的至少之一；所述基于所述展示位置信息，控制所述AR设备播放所述特效数据，包括：在所述展示位置信息对应的展示位置上，基于所述多个虚拟对象之间的展示顺序和交互数据中的至少之一，展示所述虚拟对象的影像。如此，按照多个虚拟对象之间的展示顺序，展示包括多个虚拟对象的影像以及多个虚拟对象之间的交互数据，能够丰富AR展示的内容，提高AR内容的展示效果。
在本公开的一些实施例中，所述基于所述目标对象在所述当前场景图像中的图像位置信息，确定所述特效数据的展示位置信息，包括：基于所述目标对象在所述当前场景图像中的图像位置信息，确定所述目标对象在世界坐标系下的位置信息；基于所述目标对象在所述世界坐标系下的位置信息和所述AR设备在所述世界坐标系下的位置信息，确定所述特效数据的展示位置信息。如此，可以较为准确地确定出目标对象在当前场景图像中的图像位置信息，基于目标对象的图像位置信息可以较为准确地得到特效数据的展示位置信息，从而为特效数据的准确展示提供支持。
在本公开的一些实施例中，所述获取世界坐标系下所述目标对象与所述AR设备之间的相对位置信息，包括：基于所述当前场景图像、历史场景图像、以及所述AR设备在拍摄所述历史场景图像时与所述目标对象在所述世界坐标系下的相对位置信息，确定所述AR设备在拍摄当前场景图像时，与所述目标对象之间的相对位置信息。如此，利用当前场景图像、历史场景图像、以及所述AR设备在拍摄所述历史场景图像时与所述目标对象在世界坐标系下的相对位置信息，能够较为准确的确定AR设备在拍摄当前场景图像时，与所述目标对象之间的相对位置信息，从而为特效数据的准确展示提供支持。
在本公开的一些实施例中，按照以下方式识别所述当前场景图像中是否包含所述目标对象：对所述当前场景图像进行特征点提取，得到所述当前场景图像包含的多个特征点分别对应的特征信息；所述多个特征点位于所述当前场景图像中的目标检测区域中；基于所述多个特征点分别对应的特征信息与预先存储的所述目标对象包含的多个特征点分别对应的特征信息进行比对，确定所述当前场景图像中是否包含所述目标对象。如此，利用上述特征点的提取和比对能够较为准确的确定当前场景图像中是否存在目标对象。
以下装置、电子设备等的效果描述参见上述增强现实场景下的展示方法的说明。
本公开实施例提供了一种增强现实场景下的展示装置，包括：
图像获取模块，配置为获取增强现实AR设备拍摄的当前场景图像；
位置确定模块，配置为基于所述当前场景图像对目标对象的识别结果，确定所述目标对象匹配的特效数据以及所述特效数据的展示位置信息；
特效播放模块，配置为基于所述展示位置信息，控制所述AR设备播放所述特效数据；所述特效数据包括虚拟影像和音频中的至少之一，所述虚拟影像的展示位置与所述目标对象之间具有预设位置关系。
在本公开的一些实施例中，所述位置确定模块，配置为在所述当前场景图像中识别到所述目标对象的情况下，基于所述目标对象在所述当前场景图像中的图像位置信息，确定所述特效数据的展示位置信息。
在本公开的一些实施例中，所述位置确定模块，配置为在所述当前场景图像中未识别到所述目标对象的情况下，获取世界坐标系下所述目标对象与所述AR设备之间的相对位置信息，并基于所述相对位置信息，确定所述特效数据的展示位置信息。
在本公开的一些实施例中，所述特效播放模块，配置为在确定所述目标对象的至少部分在所述AR设备的图像展示范围的情况下，基于所述展示位置信息，控制所述AR设备播放所述特效数据中的至少部分特效数据；其中，所述至少部分特效数据为所述目标对象的至少部分对应的所述虚拟影像和音频中的至少之一；在确定所述目标对象未在所述AR设备的图像展示范围的情况下，基于所述展示位置信息，控制所述AR设备按照所述音频的已播放进度继续播放所述音频。
在本公开的一些实施例中，所述虚拟影像包括全息影像；所述展示装置还包括全息影像生成模块，配置为获取与所述目标对象匹配的待处理视频，所述待处理视频中包括与所述目标对象关联的目标关联对象；为所述待处理视频中的每个像素点设置透明通道，得到第一视频；基于所述透明通道，从所述第一视频中去除背景像素点，得到第二视频；基于所述第二视频生成包括所述目标关联对象的全息影像。
在本公开的一些实施例中，所述全息影像生成模块，配置为将所述第一视频中的背景像素点对应的透明通道设置为白色，得到第三视频；所述第一视频包括所述目标关联对象的目标像素点和除所述目标像素点以外的背景像素点；将所述第一视频中的第一类像素点对应的透明通道设置为黑色，将所述第一视频中的第二类像素点对应的透明通道设置为白色，将所述第一视频中的第三类像素点对应的透明通道设置为预设灰色值，得到第四视频；所述第三类像素点包括与所述背景像素点相邻的目标像素点和与所述目标像素点相邻的背景像素点；所述第一类像素点包括除所述第三类像素点以外的背景像素点，所述第二类像素点包括除所述第三类像素点以外的目标像素点；基于第三视频和第四视频，生成所述第二视频。
在本公开的一些实施例中，所述虚拟影像包括多个虚拟对象的影像，以及所述多个虚拟对象之间的展示顺序和交互数据中的至少之一；所述特效播放模块，配置为在所述展示位置信息对应的展示位置上，基于所述多个虚拟对象之间的展示顺序和交互数据中的至少之一，展示所述虚拟对象的影像。
在本公开的一些实施例中，所述位置确定模块，配置为基于所述目标对象在所述当前场景图像中的图像位置信息，确定所述目标对象在世界坐标系下的位置信息；基于所述目标对象在所述世界坐标系下的位置信息和所述AR设备在所述世界坐标系下的位置信息，确定所述特效数据的展示位置信息。
在本公开的一些实施例中，所述位置确定模块，配置为基于所述当前场景图像、历史场景图像、以及所述AR设备在拍摄所述历史场景图像时与所述目标对象在所述世界坐标系下的相对位置信息，确定所述AR设备在拍摄当前场景图像时，与所述目标对象之间的相对位置信息。
在本公开的一些实施例中，所述位置确定模块，配置为按照以下方式识别所述当前场景图像中是否包含所述目标对象：对所述当前场景图像进行特征点提取，得到所述当前场景图像包含的多个特征点分别对应的特征信息；所述多个特征点位于所述当前场景图像中的目标检测区域中；基于所述多个特征点分别对应的特征信息与预先存储的所述目标对象包含的多个特征点分别对应的特征信息进行比对，确定所述当前场景图像中是否包含所述目标对象。
本公开实施例还提供一种电子设备，包括：处理器、存储器和总线，所述存储器存储有所述处理器可执行的机器可读指令，当电子设备运行时，所述处理器与所述存储器之间通过总线通信，所述机器可读指令被所述处理器执行时执行上述任一实施例所述的增强现实场景下的展示方法。
本公开实施例还提供一种计算机可读存储介质，该计算机可读存储介质上存储有计算机程序，该计算机程序被处理器运行时执行上述任一实施例所述的增强现实场景下的展示方法。
本公开实施例还提供一种计算机程序，所述计算机程序包括计算机可读代码，在所述计算机可读代码在电子设备中运行的情况下，所述电子设备的处理器执行如上述任一实施例所述的增强现实场景下的展示方法。
本公开实施例至少提供一种增强现实场景下的展示方法、装置、设备、介质及程序，能够基于对目标对象的识别结果，实现虚拟影像和音频的综合展示，即不仅能够展示与目标对象匹配的AR画面、视频、全息影像等虚拟影像，另外，无需重新构建三维地图模型，能够直接通过目标对象的识别结果即可触发匹配的特效数据进行展示，并且，特效数据中的虚拟影像的展示位置与目标对象之间具有预设位置关系，使其展示效果能够与目标对象紧密关联，能够更有针对性的去展示特效数据。
为使本公开的上述目的、特征和优点能更明显易懂，下文特举较佳实施例，并配合所附附图，作详细说明如下。
附图说明
为了更清楚地说明本公开实施例的技术方案，下面将对实施例中所需要使用的附图作简单地介绍，此处的附图被并入说明书中并构成本说明书中的一部分，这些附图示出了符合本公开的实施例，并与说明书一起用于说明本公开实施例的技术方案。应当理解，以下附图仅示出了本公开的某些实施例，因此不应被看作是对范围的限定，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他相关的附图。
图1示出了本公开实施例所提供的一种增强现实场景下的展示方法的流程示意图；
图2示出可以应用本公开实施例的增强现实场景下的展示方法的一种系统架构示意图；
图3示出了本公开实施例所提供的基于目标对象在AR设备的图像展示范围内，控制AR设备播放特效数据的流程示意图；
图4A示出了本公开实施例所提供的生成全息影像的流程示意图；
图4B示出了本公开实施例所提供的去除第一视频中的背景像素点，得到第二视频的流程示意图；
图5A示出了本公开中展示的特效数据的示意图之一；
图5B示出了本公开中待处理视频中的一张图像；
图5C示出了本公开中第四视频中的一张图像；
图6A示出了本公开中展示的特效数据的示意图之二；
图6B示出了本公开中展示的特效数据的示意图之三；
图6C示出了本公开中展示的特效数据的示意图之四；
图7示出了本公开实施例所提供的识别当前场景图像中是否包含目标对象的流程示意图；
图8示出了本公开实施例所提供的一种增强现实场景下的展示装置的结构示意图；
图9示出了本公开实施例所提供的一种电子设备的结构示意图。
具体实施方式
为使本公开实施例的目的、技术方案和优点更加清楚，下面将结合本公开实施例中附图，对本公开实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本公开一部分实施例，而不是全部的实施例。通常在此处附图中描述和示出的本公开实施例的组件可以以各种不同的配置来布置和设计。因此，以下对在附图中提供的本公开的实施例的详细描述并非旨在限制要求保护的本公开的范围，而是仅仅表示本公开的选定实施例。基于本公开的实施例，本领域技术人员在没有做出创造性劳动的前提下所获得的所有其他实施例，都属于本公开保护的范围。
应注意到：相似的标号和字母在下面的附图中表示类似项，因此，一旦某一项在一个附图中被定 义，则在随后的附图中不需要对其进行进一步定义和解释。
本文中术语“和/或”，仅仅是描述一种关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。另外，本文中术语“至少一种”表示多种中的任意一种或多种中的至少两种的任意组合，例如，包括A、B、C中的至少一种，可以表示包括从A、B和C构成的集合中选择的任意一个或多个元素。
本公开实施例中的多个或者多种可以分别指的是至少两个或者至少两种。
随着AR技术的发展，逐渐将AR技术应用于多种领域中，比如可以在实体对象上叠加AR内容，通过AR内容向用户形象生动地介绍实体对象。但是相关技术中，AR设备上展示AR内容时，需要识别AR设备当前所处的位置，之后映射到三维地图模型中的某一位置，进而展示该位置所在范围内的预设好的虚拟特效数据。这种方式不仅需要采集大批量图像来重构真实环境对应的三维地图模型，而且预设好的虚拟数据展示出来的效果单一，不够丰富和生动。
针对相关技术中，存在的AR内容展示时需采集大批量图像来重构真实环境对应的三维地图模型、虚拟数据效果单一的缺陷，本公开实施例提供了一种增强现实场景下的展示方法、装置、设备、介质及程序，本公开实施例能够基于对目标对象的识别结果，实现虚拟影像和音频的综合展示，即不仅能够展示与目标对象匹配的AR画面、视频、全息影像等虚拟影像，另外，无需重新构建三维地图模型，能够直接通过目标对象的识别结果即可触发匹配的特效数据进行展示，并且，特效数据中的虚拟影像的展示位置与目标对象之间具有预设位置关系，使其展示效果能够与目标对象紧密关联，能够更有针对性的去展示特效数据。
通过以下实施例，对本公开实施例公开的增强现实场景下的展示方法、装置、设备、介质及程序进行说明。
如图1所示，本公开实施例公开了一种增强现实场景下的展示方法，该方法可以应用于具有计算能力的设备。其中，设备可以是服务器，也可以是AR设备。该增强现实场景下的展示方法可以包括如下步骤：
S110、获取增强现实AR设备拍摄的当前场景图像。
示例性地，AR设备可以包括但不限于AR眼镜、平板电脑、智能手机、智能穿戴式设备等具有显示功能和数据处理能力的设备，这些AR设备中可以安装用于展示AR场景内容的应用程序，用户可以在该应用程序中体验AR场景内容。
示例性地，AR设备还可以包含用于拍摄图像的图像采集部件，比如三原色(Red Green Blue，RGB)摄像头，在获取到AR设备拍摄的当前场景图像后，可以对该当前场景图像进行识别，识别是否包含触发特效数据进行展示的目标对象。
S120、基于所述当前场景图像对目标对象的识别结果，确定所述目标对象匹配的特效数据以及所述特效数据的展示位置信息。
示例性地，针对不同的应用场景，目标对象可以为具有特定形态的物体，比如可以为书本、字画、建筑物等实体物体，通过特效数据可以对该实体物体进行介绍，增加用户对实体物体的了解。
示例性地，针对进行日历特效数据展示的场景，目标对象可以为具有预设形态的日历，特效数据可以为基于该日历的内容预先设计好的虚拟展示内容，可以向用户介绍日历的内容，以吸引用户查阅日历。
拍摄的当前场景图像中可能包括目标对象，也可能不包括目标对象，因此在执行步骤S120之前，可以对当前场景图像进行识别，以确定当前场景图像中是否包括目标对象。
在识别到目标对象之后，可以基于目标对象的标识符等获取与目标对象相匹配的特效数据。这里的特效数据可以包括虚拟影像、视频以及音频等。虚拟影像可以包括与目标对象匹配的视频、全息影像、AR画面等。
在一些实施例中，在当前场景图像中包括目标对象的情况下，可以基于当前场景图像，利用标识物(marker)进行定位的方式，确定与所述目标对象相匹配的特效数据的展示位置信息。其中，利用marker进行定位可以是，利用目标对象的图像作为marker，确定目标对象在当前场景图像中的图像位 置信息，之后基于图像位置信息确定特效数据的展示位置信息。
在一些实施例中，在当前场景图像中不包括目标对象的情况下，利用其他的定位方式，例如实时定位或地图构建(Simultaneous Localization And Mapping，SLAM)定位方式，确定目标对象对应的位置信息或相对于AR设备的相对位置信息，之后利用确定的上述位置信息或相对位置信息来确定特效数据的展示位置信息。其中，利用SLAM确定相对位置信息的步骤见下面实施例的描述。
上述展示位置信息可以包括特效数据在世界坐标系下的坐标信息。该世界坐标系为在真实空间中构建的三维坐标系，是一个绝对坐标系。其中，世界坐标系不随AR设备、目标对象、特效数据的位置而改变。
S130、基于所述展示位置信息，控制所述AR设备播放所述特效数据；所述特效数据包括虚拟影像和音频中的至少之一，所述虚拟影像的展示位置与所述目标对象之间具有预设位置关系。
若上述当前场景图像为识别到目标对象的首帧图像，则基于展示位置信息，控制特效数据从开始进行播放；若在AR设备之前拍摄的历史场景图像中已经识别到目标对象，则基于特效数据当前的播放进度，继续播放特效数据。特效数据播放完之后，还可以通过点击AR设备上的显示的按钮，重新播放该特效数据。
上述实施例能够直接通过目标对象的识别结果即可触发匹配的特效数据进行展示，无需重新构建三维地图模型，相比基于当前AR设备的定位结果去触发展示特效数据的方式，展示效果能够与目标对象紧密关联，能够更有针对性的去展示特效数据。另外，本公开实施例无需重新构建三维地图模型，能够直接通过目标对象的识别结果即可触发匹配的特效数据进行展示，并且，特效数据中的虚拟影像的展示位置与目标对象之间具有预设位置关系，使其展示效果能够与目标对象紧密关联，能够更有针对性的去展示特效数据。
图2示出可以应用本公开实施例的增强现实场景下的展示方法的一种系统架构示意图；如图2所示，该系统架构中包括：当前场景图像获取终端201、网络202和控制终端203。为实现支撑一个示例性应用，当前场景图像获取终端201和控制终端203通过网络202建立通信连接，当前场景图像获取终端201通过网络202向控制终端203上报当前场景图像，控制终端203响应于当前场景图像，并基于当前场景图像对目标对象的识别结果，确定目标对象匹配的特效数据以及特效数据的展示位置信息，其次，基于展示位置信息，控制AR设备播放特效数据；特效数据包括虚拟影像和音频中的至少之一，虚拟影像的展示位置与目标对象之间具有预设位置关系。最后，控制终端203将展示位置信息和特效数据上传至网络202，并通过网络202发送给当前场景图像获取终端201。
作为示例，当前场景图像获取终端201可以包括图像采集设备，控制终端203可以包括具有视觉信息处理能力的视觉处理设备或远程服务器。网络202可以采用有线或无线连接方式。其中，当控制终端203为视觉处理设备时，当前场景图像获取终端201可以通过有线连接的方式与视觉处理设备通信连接，例如通过总线进行数据通信；当控制终端203为远程服务器时，当前场景图像获取终端201可以通过无线网络与远程服务器进行数据交互。
或者，在一些场景中，当前场景图像获取终端201可以是带有视频采集模组的视觉处理设备，可以是带有摄像头的主机。这时，本公开实施例的增强现实场景下的展示方法可以由当前场景图像获取终端201执行，上述系统架构可以不包含网络202和控制终端203。
在一些实施例中，AR设备图像展示范围有限，无法展示所有位置上的特效数据，因此，在基于所述展示位置信息，控制所述AR设备播放所述特效数据时，首先判断特效数据是否位于所述AR设备的图像展示范围内。
特效数据是与目标对象相匹配的，特效数据中的虚拟影像的展示位置与目标对象之间具有预设位置关系。在一些实施例中，在目标对象为日历时，特效数据对应的虚拟影像的展示位置可以与日历的封面相垂直。
在一些实施例中，通过当前场景图像可以确定目标对象是否在AR设备的图像展示范围内，以确定展示的特效数据，进而使得特效数据展示的效果更为合理。即可以利用以下步骤，如图3所示，控制所述AR设备播放所述特效数据：
在确定所述目标对象的至少部分在所述AR设备的图像展示范围的情况下，基于所述展示位置信息，控制所述AR设备播放所述特效数据中的至少部分特效数据；其中，所述至少部分特效数据为所述目标对象的至少部分对应的虚拟影像和音频中的至少之一；在确定所述目标对象未在所述AR设备的图像展示范围的情况下，基于所述展示位置信息，控制所述AR设备按照所述音频的已播放进度继续播放所述音频。
在一些实施例中，若当前场景图像中包括至少部分目标对象，则确定所述目标对象的至少部分在所述AR设备的图像展示范围内，此时目标对象的至少部分对应的的特效数据位于AR设备的图像展示范围内，此时在基于所述展示位置信息，控制所述AR设备播放目标对象的至少部分对应的特效数据时，可以是控制所述AR设备播放目标对象的至少部分对应的所述虚拟影像和音频中的至少之一。
在一些实施例中，若当前场景图像中不包括目标对象，则确定所述目标对象不在所述AR设备的图像展示范围内，此时与目标对象匹配的特效数据中的虚拟影像不在AR设备的图像展示范围内，此时在基于所述展示位置信息，控制所述AR设备播放所述特效数据时，可以是控制所述AR设备按照音频的已播放进度继续播放所述音频。
在一些实施例中，在AR设备的图像展示范围内包括至少部分目标对象的情况下，展示至少部分对应的虚拟影像和音频中的至少之一，在AR设备的图像展示范围内不包括目标对象时，不展示虚拟影像，只展示音频，使得特效数据展示的效果更为合理，也使得特效数据的展示效果更为连贯。
在一些实施例中，上述确定当前场景图像中是否包括目标对象，或者是否包括至少部分目标对象，例如可以按照如下步骤实现：
第一步，对所述当前场景图像进行特征点提取，得到所述当前场景图像包含的多个特征点分别对应的特征信息；所述多个特征点位于所述当前场景图像中的目标检测区域中。
第二步，基于所述多个特征点分别对应的特征信息与预先存储的所述目标对象包含的多个特征点分别对应的特征信息进行比对，确定所述当前场景图像中是否包含所述目标对象，或者是否包括部分目标对象。
在一些实施例中，若从当前场景图像中提取的特征点与预先存储的特征点全部匹配成功，则确定当前场景图像中包括完整的目标对象；若从当前场景图像中提取的特征点与预先存储的特征点匹配成功的比例高于预设比例，则确定当前场景图像中包括部分目标对象；若从当前场景图像中提取的特征点与预先存储的特征点匹配成功的比例低于或等于预设比例，则确定当前场景图像中不包括目标对象。如此，利用上述特征点的提取和比对能够较为准确的确定当前场景图像中是否存在目标对象。匹配过程可见下述步骤S510至S520。
在当前场景图像中包括完整的目标对象的情况下，AR设备的图像展示范围内包括完整的目标对象，此时AR设备展示完整的所述虚拟影像和音频中的至少之一；在当前场景图像中包括部分目标对象的情况下，AR设备的图像展示范围内包括部分目标对象，此时AR设备展示部分虚拟影像和音频中的至少之一；在当前场景图像中不包括目标对象的情况下，AR设备的图像展示范围内不包括目标对象，此时AR设备不展示虚拟影像，只展示音频。如此，能够提高特效数据展示的合理性以及连贯性。
在一些实施例中，为了提高展示的AR内容的丰富性，提高AR内容展示的效果，还可以展示与目标对象匹配的全息影像，全息影像中包括与目标对象关联的目标关联对象。如图4A所示，在一些实施例中可以利用如下步骤S210至S240，生成全息影像：
S210、获取与所述目标对象匹配的待处理视频，所述待处理视频中包括与所述目标对象关联的目标关联对象。
目标关联对象与目标对象相关联，例如，如图5A所示，在目标对象为某一场所501时，目标关联对象可以是对该场所进行介绍的导游502。
如图5B所示，为待处理视频中的一张图像，该图像有背景，其中包括目标关联对象，即对该场所进行介绍的导游502。
S220、为所述待处理视频中的每个像素点设置透明通道，得到第一视频。
这里是为待处理视频中每张图像中的每个像素点设置透明通道，利用透明通道可以控制对应的像素点的透明程度，透明的像素点不对图像提供贡献，即该像素点不显示；不透明的像素点对图像提供贡献，即该像素点显示。
S230、基于所述透明通道，从所述第一视频中去除背景像素点，得到第二视频。
如果将某一像素点的透明通道的值设置为0，则表示该像素点为透明，此时该像素点的透明通道设置为了黑色，该像素点不对图像提供贡献；如果将某一像素点的透明通道的值设置为1，则表示该像素点为不透明，此时该像素点的透明通道设置为了白色，该像素点对图像提供贡献。利用对透明通道透明度的设置，可以从所述第一视频中去除背景像素点。
在一些实施例中，如图4B所示，可以利用如下步骤S2301至S2303，去除第一视频中的背景像素点，得到第二视频：
S2301、将第一视频中的背景像素点对应的透明通道设置为白色，得到第三视频；所述第一视频包括所述目标关联对象的目标像素点和除所述目标像素点以外的背景像素点。
示例性的，将背景像素点对应的透明通道设置为1。
S2302、将所述第一视频中的第一类像素点对应的透明通道设置为黑色，将所述第一视频中的第二类像素点对应的透明通道设置为白色，将所述第一视频中的第三类像素点对应的透明通道设置为预设灰色值，得到第四视频；所述第三类像素点包括与所述背景像素点相邻的目标像素点和与所述目标像素点相邻的背景像素点；所述第一类像素点包括除所述第三类像素点以外的背景像素点，所述第二类像素点包括除所述第三类像素点以外的目标像素点。
示例性的，将第一类像素点对应的透明通道设置为0，将第二类像素点对应的透明通道设置为1，将第三类像素点对应的透明通道设置为0到1之间的值，即预设灰度值。将第三类像素点设置为预设灰度值是为了将目标关联对象边缘的像素的颜色与背景的透明色相接近，能够使得显示的目标关联对象边缘颜色平滑过度。如图5C所示为第四视频中一张图像。
S2303、基于第三视频和第四视频，生成所述第二视频。
将第三视频和第四视频进行整合，即能够得到去除背景，只保留目标关联对象的第二视频。如图3A所示，导游302的背景是透明的。
在一些实施例中，通过对第一视频的不同类型像素点进行处理，可以实现将原视频调整为全息影像的展示效果。
S240、基于所述第二视频生成包括所述目标关联对象的全息影像。
生成的全息影像如图5A所示。
在一些实施例中，首先，可以通过蒙版方式把与目标对象匹配的待处理视频处理为带透明通道的视频素材(对应于本公开实施例中的第一视频)；其中，蒙版即为选框的外部(选框的内部就是选区)。然后，将该视频素材中的背景像素点对应的透明通道设置为白色，得到第三视频。并将该视频素材中的背景像素点对应的透明通道设置为黑色，将该视频素材中的目标关联对象的目标像素点对应的透明通道设置为白色，将该视频素材中的渐变部分的像素点对应的透明通道设置为预设灰色值，得到第四视频；其中，渐变部分的像素点为与背景像素点相邻的目标像素点和与目标像素点相邻的背景像素点。最后，将第三视频和第四视频进行横向整合，得到第二视频。如此，能够以减少视频大小，进而提高制作视频相关联的全息效果和特效数据的效率。在一些实施例中，虚拟影像还包括全息影像，展示与目标对象关联的目标关联对象对应的全息影像，还可以在当前场景图像中叠加显示全息影像，使得AR内容的展示效果更为丰富。
在一些实施例中，为了提高AR内容显示的丰富性，虚拟影像中还可以包括视频、AR画面、文字等，如图6A所示，在识别到目标对象蛋糕时，显示“生日快乐”的文字。同时如图6B所示，在识别到目标对象日历时，显示与日历对象的AR画面，AR画面中包括龙、松鼠等虚拟对象。
在一些实施例中，为了丰富AR展示的内容，提高AR内容的展示效果，可以在虚拟影像中设置多个虚拟对象的影像，并预先设置多个虚拟对象之间的展示顺序和交互数据中的至少之一。基于所述展示位置信息，控制所述AR设备播放所述特效数据，可以是通过以下过程来实现：
在所述展示位置信息对应的展示位置上，基于多个虚拟对象之间的展示顺序和交互数据中的至少之一，展示所述虚拟对象的影像。
如图6C所示，虚拟影像中设置虚拟对象战士一601和虚拟对象战士二602的影像，在扫描到游戏字样时，显示战士一601的虚拟影像先出现在AR设备展示的画面中，战士二602战斗的虚拟影像后出现在AR设备展示的画面中。之后，根据预设的战士一601和战士二602之间的交互数据，展示两者的战斗状态。
在一些实施例中，按照多个虚拟对象之间的展示顺序，展示包括多个虚拟对象的影像以及多个虚拟对象之间的交互数据，能够丰富AR展示的内容，提高AR内容的展示效果。
特效数据展示的过程中，在一些情况下，目标对象或者AR设备可能发生移动，在移动过程中，若目标对象的位置发生变化，如何能够继续确定特效数据的展示位置信息，从而对特效数据进行连贯展示，以提供更加逼真的展示效果，是值得研究的问题。
在一些实施例中，特效数据可以是三维(3-Dimension，3D)模型、视频、音频或透明视频等任一内容，也可以是上述多种内容任意组合。
针对上述问题，本公开实施例利用两种定位方式确定特效数据的展示位置信息，即可以利用以下步骤来确定特效数据的展示位置信息：
所述当前场景图像中识别到所述目标对象的情况下，基于所述目标对象在所述当前场景图像中的图像位置信息，确定所述特效数据的展示位置信息。
在所述当前场景图像中未识别到所述目标对象的情况下，获取世界坐标系下所述目标对象与所述AR设备之间的相对位置信息，并基于所述相对位置信息，确定所述特效数据的展示位置信息。
在一些实施例中，在识别到当前场景图像中包含目标对象的情况下，可以利用第一定位方式，基于所述目标对象在所述当前场景图像中的图像位置信息，确定所述特效数据的展示位置信息。
在一些实施例中，在识别到当前场景图像中不包含目标对象的情况下，即在所述当前场景图像中未识别到所述目标对象的情况下，可以利用第二定位方式，获取世界坐标系下所述目标对象与所述AR设备之间的相对位置信息，并基于所述相对位置信息，确定所述特效数据的展示位置信息。
上述在未识别到当前场景图像中包含目标对象的情况下，可以根据第二定位方式确定出特效数据的展示位置信息，这样可以基于确定的展示位置信息控制AR设备继续对未展示的特效数据进行展示，能够提高特效数据在展示过程中的连贯性，使得特效数据的展示更加逼真。
在一些实施例中，基于当前场景图像中目标对象的识别结果，切换对应的定位方式来确定特效数据的展示位置信息，可以有效降低由于其中一种定位方式定位失败而中断特效数据的展示的概率，提高了特效数据展示的稳定性。
示例性地，目标对象为日历，特效数据包括动态展示的总时长为30s的视频，若在该视频展示到第10s时，AR设备拍摄的当前场景图像中识别不到目标对象，此时可以根据基于第二定位方式确定的特效数据的展示位置信息，继续控制AR设备按照视频继续从第10s处进行展示。若在继续展示过程中，基于场景图像确定出日历完全离开AR设备的图像展示范围，比如AR设备的拍摄角度完全离开日历，视频自然位于图像展示范围之外，此时尽管视频还在继续展示，但是用户无法通过AR设备观看到特效数据对应的视频。若在继续展示过程中，基于场景图像确定出日历偏离但是未全部离开AR设备的图像展示范围，比如AR设备的拍摄角度还可以拍摄到日历的部分区域，此时用户可以通过AR设备观看到视频的部分视频。
示例性地，目标对象为日历，特效数据包括动态展示的总时长为30s的视频，若在该视频展示到第10s时，AR设备拍摄的当前场景图像中识别全部目标对象，此时可以根据基于利用日历在当前场景图像中的图像位置信息确定的展示位置信息，继续控制AR设备按照视频继续从第10s处进行展示。
在一些实施例中，AR设备播放特效数据可以跟随目标对象的移动而移动，即根据目标对象在当前场景图像中的出现情况而变化。比如，当对目标对象的识别中断时，对应本公开实施例中在拍摄的当前场景图像中识别不到目标对象的情况下，比如，可以是目标对象被遮挡、移除扫描区域等情况下，可以根据基于SLAM定位方式确定的特效数据的展示位置信息，继续控制AR设备进行特效数据的播 放。之后在拍摄的当前场景图像中重新识别到目标对象的情况下，继续以实时定位方式确定特效数据的展示位置信息，并控制AR设备进行特效数据的播放。
示例性的，可以利用如下步骤识别日历是否在当前场景图像中：
对所述当前场景图像进行特征点提取，得到所述当前场景图像包含的多个特征点分别对应的特征信息；所述多个特征点位于所述当前场景图像中的目标检测区域中；基于所述多个特征点分别对应的特征信息与预先存储的所述日历包含的多个特征点分别对应的特征信息进行比对，确定所述当前场景图像中是否包含所述日历。
若从当前场景图像中提取的特征点与预先存储的日历特征点全部匹配成功，则确定当前场景图像中包括完整的日历。若从当前场景图像中提取的特征点与预先存储的日历的特征点匹配成功的比例高于预设比例，则确定当前场景图像中包括部分日历。若从当前场景图像中提取的特征点与预先存储的日历的特征点匹配成功的比例低于或等于预设比例，则确定当前场景图像中不包括日历。
上述利用第一定位方式，基于图像识别技术，可以较为准确地确定出目标对象在当前场景图像中的图像位置信息，因此，这里基于目标对象的图像位置信息可以较为准确地得到特效数据的展示位置信息，从而为特效数据的准确展示提供支持。
第一定位方式是基于目标对象在当前场景图像中的图像位置信息，来确定的特效数据的展示位置信息，因此在基于第一定位方式对目标对象进行定位的过程中，可以同时确定出AR设备在拍摄每张场景图像时，与目标对象之间的相对位置信息，并保存该相对位置信息。这样在当前场景图像为未识别到目标对象的情况下，可以结合保存的AR设备与目标对象之间的相对位置信息，以及实时定位与SLAM技术，确定出AR设备在拍摄当前场景图像时，与目标对象之间的相对位置信息。在一些实施例中可以基于该相对位置信息以及特效数据与目标对象的相对位置关系，确定出特效数据的展示位置信息，该过程将在后文进行详细阐述。
在一些实施例中，可以按照以下方式识别当前场景图像中是否包含目标对象，如图7所示：
S510，对当前场景图像进行特征点提取，得到当前场景图像包含的多个特征点分别对应的特征信息；多个特征点位于当前场景图像中的目标检测区域中。
示例性地，在对当前场景图像进行识别过程中，可以通过图像检测算法，定位出当前场景图像中包含实体对象的目标检测区域。然后在目标检测区域中进行特征点提取，比如可以提取目标检测区域中位于实体对象轮廓上的特征点、位于标识图案区域的特征点以及位于文字区域的特征点等。示例性地，为了使得提取到的特征点能够完整的表示目标对象，特征点可以基于目标对象在当前场景图像中对应的位置区域进行均匀提取，比如目标对象为日历的情况下，可以在日历封面在当前场景图像中对应的矩形区域中进行均匀提取。
示例性地，这里提取到的特征点包含的特征信息可以包含特征点对应的纹理特征值、RGB特征值、灰度值等能够表示该特征点特征的信息。
S520，基于多个特征点分别对应的特征信息与预先存储的目标对象包含的多个特征点分别对应的特征信息进行比对，确定当前场景图像中是否包含目标对象。
示例性地，可以按照相同的方式预先对目标对象进行拍摄，得到并保存目标对象包含的多个特征点分别对应的特征信息。
示例性地，在基于多个特征点分别对应的特征信息与预先存储的目标对象包含的多个特征点分别对应的特征信息进行比对时，可以先基于当前场景图像提取到的多个特征点分别对应的特征信息确定当前场景图像中目标检测区域对应的第一特征向量，以及基于目标对象包含的多个特征点分别对应的特征信息确定目标对象对应的第二特征向量。然后可以通过第一特征向量和第二特征向量确定目标检测区域和目标对象之间的相似度，比如可以通过余弦公式进行确定。
示例性地，在确定第一特征向量和第二特征向量之间的相似度大于或等于预设相似度阈值的情况下，确定当前场景图像中包含目标对象。反之，在确定第一特征向量和第二特征向量之间的相似度小于预设相似度阈值的情况下，确定当前场景图像中不包含目标对象。
在一些实施例中，利用上述特征点的提取和比对能够较为准确的确定当前场景图像中是否存在目 标对象。
在一些实施例中，在识别到当前场景图像包括目标对象的情况下，可以利用如下步骤确定特效数据的展示位置信息：
基于所述目标对象在所述当前场景图像中的图像位置信息，确定所述目标对象在世界坐标系下的位置信息；基于所述目标对象在所述世界坐标系下的位置信息和所述AR设备在所述世界坐标系下的位置信息，确定所述特效数据的展示位置信息。
在执行上述步骤之前，首先获取目标对象在当前场景图像中的图像位置信息。示例性地，可以以当前场景图像建立图像坐标系，获取目标对象包含的多个特征点在图像坐标系中的图像坐标值，得到目标对象在当前场景图像中的图像位置信息。
上述基于目标对象在当前场景图像中的图像位置信息，确定目标对象在世界坐标系下的位置信息，可以是基于上述图像位置信息、图像坐标系和AR设备对应的相机坐标系之间的转换关系、以及AR设备对应的相机坐标系与世界坐标系之间的转换关系，确定目标对象在世界坐标系下的位置信息。
示例性地，AR设备对应的相机坐标系可以以AR设备包含的图像采集部件的聚焦中心为原点，以光轴为Z轴建立的三维直角坐标系。在AR设备拍摄到当前场景图像后，可以基于图像坐标系和相机坐标系之间的转换关系，确定出目标对象在相机坐标系下的位置信息。
示例性地，世界坐标系可以以目标对象的中心点为原点进行建立，比如上文提到的在目标对象为日历的情况下，可以以日历的中心为原点，以通过日历中心的长边为X轴、以通过日历中心的短边为Y轴、以通过日历中心且垂直于日历封面的直线为Z轴进行建立的。
其中，相机坐标系和世界坐标系之间的转换为刚体转换，即相机坐标系经过旋转、平移可以与世界坐标系重合的一种转换方式。相机坐标系和世界坐标系之间的转换关系可以通过目标对象中的多个位置点在世界坐标系下的位置坐标，以及在相机坐标系下对应的位置坐标进行确定。这里在得到目标对象在相机坐标系下的位置信息后，可以基于AR设备对应的相机坐标系与世界坐标系之间的转换关系，确定出目标对象在世界坐标系下的位置信息。
上述基于所述目标对象在所述世界坐标系下的位置信息和所述AR设备在所述世界坐标系下的位置信息，确定所述特效数据的展示位置信息，可以是AR设备在世界坐标系下的位置信息通过AR设备拍摄的当前场景图像来确定。比如在当前场景图像中选定特征点，通过确定选定的特征点在以目标对象建立的世界坐标系下的位置坐标，以及选定的特征点在AR设备对应的相机坐标系下的位置坐标，可以确定出AR设备在拍摄当前场景图像时在世界坐标系下的位置信息。
考虑到特效数据的展示位置与目标对象在相同坐标系下具有预设位置关系，因此这里基于目标对象和AR设备在相同的世界坐标系下的位置信息，可以确定出特效数据的展示位置信息。
上述在基于目标对象在世界坐标系下的位置信息和AR设备在世界坐标系下的位置信息，确定特效数据的展示位置信息时，可以是：
基于目标对象在世界坐标系下的位置信息，确定特效数据在世界坐标系下的位置信息；基于特效数据在世界坐标系下的位置信息和AR设备在世界坐标系下的位置信息，确定特效数据的展示位置信息。
示例性地，可以按照目标对象在世界坐标系下的位置信息，以及预先设置的特效数据的展示位置与目标对象在相同坐标系下的预设位置关系，确定出特效数据在世界坐标系下的展示位置信息。
在一些实施例中，可以较为准确地确定出目标对象在当前场景图像中的图像位置信息，基于目标对象的图像位置信息可以较为准确地得到特效数据的展示位置信息，从而为特效数据的准确展示提供支持。
在一些实施例中，在当前场景图像中未识别到目标对象的情况下，可以利用如下步骤确定特效数据的展示位置信息：
基于所述当前场景图像、历史场景图像、以及所述AR设备在拍摄所述历史场景图像时与所述目标对象在所述世界坐标系下的相对位置信息，确定所述AR设备在拍摄当前场景图像时，与所述目标对象之间的相对位置信息；基于确定的相对位置信息，确定特效数据的展示位置信息。
示例性地，下面以当前场景图像为AR设备拍摄的第三帧场景图像为例，结合SLAM技术简要说明如何确定AR设备在拍摄当前场景图像时，AR设备与目标对象之间的相对位置信息。
从AR设备拍摄第一帧包含目标对象的场景图像开始，可以基于以目标对象的中心点为原点建立的世界坐标系，以及AR设备拍摄的第一帧场景图像中选定的特征点分别在世界坐标系和AR设备对应的相机坐标系下的位置坐标，确定出AR设备在拍摄第一帧场景图像时在世界坐标系下的位置信息，同时确定的还包含目标对象在AR设备在拍摄第一帧场景图像时在世界坐标系下的位置信息。基于AR设备在拍摄第一帧场景图像时在世界坐标系下的位置信息，以及目标对象在AR设备在拍摄第一帧场景图像时在世界坐标系下的位置信息，可以确定出AR设备拍摄第一帧场景图像时与目标对象在世界坐标系下的相对位置信息。
在一些实施例中，当AR设备拍摄第二帧场景图像时，可以在第二帧场景图像中找到第一帧场景图像中包含的目标特征点，基于目标特征点分别在AR设备拍摄这两帧场景图像时在相机坐标系下的位置信息，确定出AR设备在拍摄第二帧场景图像时相对于拍摄第一帧场景图像时的位置偏移量。然后基于该位置偏移量，以及AR设备在拍摄第一帧场景图像时与目标对象在建立的世界坐标系下的相对位置信息，确定出AR设备在拍摄第二帧场景图像时与目标对象在世界坐标系下的相对位置信息。
在一些实施例中，可以通过相同的方式，确定出AR设备在当前场景图像时，相对于拍摄第二帧场景图像时的位置偏移量，这样可以结合AR设备拍摄当前场景图像时相比拍摄第二帧场景图像时的位置偏移量，以及AR设备在拍摄第二帧场景图像时与目标对象在世界坐标系下的相对位置信息，确定出AR设备在拍摄当前场景图像时与目标对象在世界坐标系下的相对位置信息。
利用当前场景图像、历史场景图像、以及所述AR设备在拍摄所述历史场景图像时与所述目标对象在世界坐标系下的相对位置信息，能够较为准确的确定AR设备在拍摄当前场景图像时，与所述目标对象之间的相对位置信息，从而为特效数据的准确展示提供支持。
在确定特效数据在世界坐标系下的展示位置信息时，还可以确定特效数据在世界坐标系下的展示姿态信息，处理逻辑基本相同。在展示特效数据的时候，可以同时结合确定的展示位置信息和展示姿态信息进行展示。
本公开实施例提供的增强现实场景下的展示方法，可以基于获取的当前场景图像对目标对象的识别结果，实现虚拟影响和音频的综合展示。同时本公开实施例提供的特效数据的展示位置信息，可以是根据对当前场景图像中目标对象的识别结果而确定，即若根据特效数据的展示位置信息，控制AR设备播放特效数据，即是根据当前场景图像中目标对象的识别结果，控制AR设备播放特效数据，且特效数据的展示位置与目标对象之间具有预设位置信息。
在一种应用场景中，当前场景图像为制作的婚礼请帖，目标对象为新人，基于婚礼请帖中部分区域识别到新人的情况下，确定与新人匹配的特效数据，比如：展示新人相恋、相爱的点点滴滴的视频，以及该视频对应的展示位置信息；基于婚礼请帖中部分区域未识别到新人的情况下，确定与新人匹配的音频信息，比如：新人合唱的有关婚礼的音频，以及与该音频对应的展示位置信息。之后，分别基于两个不同的展示位置信息，控制AR设备播放该视频或音频等，如此，可增加婚礼请帖的可看性和观赏性。
对应于上述增强现实场景下的展示方法，本公开实施例还公开了一种增强现实场景下的展示装置，该装置中的各个模块能够实现上述在服务端或AR设备上执行的各个实施例的增强现实场景下的展示方法，并且能够取得相同的有益效果。如图8所示，增强现实场景下的展示装置包括：
图像获取模块810，配置为获取增强现实AR设备拍摄的当前场景图像。
位置确定模块820，配置为基于所述当前场景图像对目标对象的识别结果，确定所述目标对象匹配的特效数据以及所述特效数据的展示位置信息。
特效播放模块830，配置为基于所述展示位置信息，控制所述AR设备播放所述特效数据；所述特效数据包括虚拟影像和音频中的至少之一，所述虚拟影像的展示位置与所述目标对象之间具有预设位置关系。
在本公开的一些实施例中，所述位置确定模块820，配置为在所述当前场景图像中识别到所述目 标对象的情况下，基于所述目标对象在所述当前场景图像中的图像位置信息，确定所述特效数据的展示位置信息。
在本公开的一些实施例中，所述位置确定模块820，配置为在所述当前场景图像中未识别到所述目标对象的情况下，获取世界坐标系下所述目标对象与所述AR设备之间的相对位置信息，并基于所述相对位置信息，确定所述特效数据的展示位置信息。
在本公开的一些实施例中，所述特效播放模块830，配置为在确定所述目标对象的至少部分在所述AR设备的图像展示范围的情况下，基于所述展示位置信息，控制所述AR设备播放所述特效数据中的至少部分特效数据；其中，所述至少部分特效数据为所述目标对象的至少部分对应的所述虚拟影像和音频中的至少之一；在确定所述目标对象未在所述AR设备的图像展示范围的情况下，基于所述展示位置信息，控制所述AR设备按照所述音频的已播放进度继续播放所述音频。
在本公开的一些实施例中，所述虚拟影像包括全息影像；所述展示装置还包括全息影像生成模块，配置为获取与所述目标对象匹配的待处理视频，所述待处理视频中包括与所述目标对象关联的目标关联对象；为所述待处理视频中的每个像素点设置透明通道，得到第一视频；基于所述透明通道，从所述第一视频中去除背景像素点，得到第二视频；基于所述第二视频生成包括所述目标关联对象的全息影像。
在本公开的一些实施例中，所述全息影像生成模块，配置为将所述第一视频中的背景像素点对应的透明通道设置为白色，得到第三视频；所述第一视频包括所述目标关联对象的目标像素点和除所述目标像素点以外的背景像素点；将所述第一视频中的第一类像素点对应的透明通道设置为黑色，将所述第一视频中的第二类像素点对应的透明通道设置为白色，将所述第一视频中的第三类像素点对应的透明通道设置为预设灰色值，得到第四视频；所述第三类像素点包括与所述背景像素点相邻的目标像素点和与所述目标像素点相邻的背景像素点；所述第一类像素点包括除所述第三类像素点以外的背景像素点，所述第二类像素点包括除所述第三类像素点以外的目标像素点；基于第三视频和第四视频，生成所述第二视频。
在本公开的一些实施例中，所述虚拟影像包括多个虚拟对象的影像，以及所述多个虚拟对象之间的展示顺序和交互数据中的至少之一；所述特效播放模块830，配置为在所述展示位置信息对应的展示位置上，基于所述多个虚拟对象之间的展示顺序和交互数据中的至少之一，展示所述虚拟对象的影像。
在本公开的一些实施例中，所述位置确定模块820，配置为基于所述目标对象在所述当前场景图像中的图像位置信息，确定所述目标对象在世界坐标系下的位置信息；基于所述目标对象在所述世界坐标系下的位置信息和所述AR设备在所述世界坐标系下的位置信息，确定所述特效数据的展示位置信息。
在本公开的一些实施例中，所述位置确定模块820，配置为基于所述当前场景图像、历史场景图像、以及所述AR设备在拍摄所述历史场景图像时与所述目标对象在所述世界坐标系下的相对位置信息，确定所述AR设备在拍摄当前场景图像时，与所述目标对象之间的相对位置信息。
在本公开的一些实施例中，所述位置确定模块820，配置为按照以下方式识别所述当前场景图像中是否包含所述目标对象：对所述当前场景图像进行特征点提取，得到所述当前场景图像包含的多个特征点分别对应的特征信息；所述多个特征点位于所述当前场景图像中的目标检测区域中；基于所述多个特征点分别对应的特征信息与预先存储的所述目标对象包含的多个特征点分别对应的特征信息进行比对，确定所述当前场景图像中是否包含所述目标对象。
对应于上述增强现实场景下的展示方法，本公开实施例还提供了一种电子设备900，如图9所示，为本公开实施例提供的电子设备900结构示意图，包括：
处理器91、存储器92、和总线93；所述存储器存储有所述处理器可执行的机器可读指令，当电子设备运行时，所述处理器与所述存储器之间通过总线通信，所述机器可读指令被所述处理器执行时执行上述任一实施例中的增强现实场景下的展示方法。
存储器92配置为存储执行指令，包括内存921和外部存储器922；这里的内存921也称内存储 器，配置为暂时存放处理器91中的运算数据，以及与硬盘等外部存储器922交换的数据，处理器91通过内存921与外部存储器922进行数据交换，当电子设备900运行时，处理器91与存储器92之间通过总线93通信，使得处理器91执行以下指令：
获取增强现实AR设备拍摄的当前场景图像；基于所述当前场景图像对目标对象的识别结果，确定所述目标对象匹配的特效数据以及所述特效数据的展示位置信息；基于所述展示位置信息，控制所述AR设备播放所述特效数据；所述特效数据包括虚拟影像和音频中的至少之一，所述虚拟影像的展示位置与所述目标对象之间具有预设位置关系。
本公开实施例还提供一种计算机可读存储介质，该计算机可读存储介质上存储有计算机程序，该计算机程序被处理器运行时执行上述方法实施例中所述增强现实场景下的展示方法。其中，该存储介质可以是易失性或非易失的计算机可读取存储介质。
本公开实施例还提供一种计算机程序，所述计算机程序包括计算机可读代码，在所述计算机可读代码在电子设备中运行的情况下，所述电子设备的处理器执行如上述任一实施例所述增强现实场景下的展示方法。
本公开实施例还提供另一种计算机程序产品，包括存储了程序代码的计算机可读存储介质，所述程序代码包括的指令可配置为执行上述方法实施例中所述增强现实场景下的展示方法，可参见上述方法实施例。
其中，该计算机程序产品可以通过硬件、软件或其结合的方式实现。在一些实施例中，所述计算机程序产品可以体现为计算机存储介质，在另一些实施例中，计算机程序产品可以体现为软件产品，例如软件开发包(Software Development Kit，SDK)等等。
本公开实施例中涉及的设备可以是系统、方法和计算机程序产品中的至少之一。计算机程序产品可以包括计算机可读存储介质，其上载有用于使处理器实现本公开的各个方面的计算机可读程序指令。
计算机可读存储介质可以是可以保持和存储由指令执行设备使用的指令的有形设备。计算机可读存储介质例如可以是但不限于电存储设备、磁存储设备、光存储设备、电磁存储设备、半导体存储设备或者上述的任意合适的组合。计算机可读存储介质的例子(非穷举的列表)包括：便携式计算机盘、硬盘、随机存取存储器(Random Access Memory，RAM)、只读存储器(Read-Only Memory，ROM)、可擦除可编程只读存储器(Electrical Programmable Read Only Memory，EPROM)或闪存、静态随机存取存储器(Static Random-Access Memory，SRAM)、便携式压缩盘只读存储器(Compact Disc Read-Only Memory，CD-ROM)、数字多功能盘(Digital Video Disc，DVD)、记忆棒、软盘、机械编码设备、例如其上存储有指令的打孔卡或凹槽内凸起结构、以及上述的任意合适的组合。这里所使用的计算机可读存储介质不被解释为瞬时信号本身，诸如无线电波或者其他自由传播的电磁波、通过波导或其他传输媒介传播的电磁波(例如，通过光纤电缆的光脉冲)、或者通过电线传输的电信号。
这里所描述的计算机可读程序指令可以从计算机可读存储介质下载到各个计算/处理设备，或者通过网络、例如因特网、局域网、广域网和无线网中的至少之一下载到外部计算机或外部存储设备。网络可以包括铜传输电缆、光纤传输、无线传输、路由器、防火墙、交换机、网关计算机和边缘服务器中的至少之一。每个计算/处理设备中的网络适配卡或者网络接口从网络接收计算机可读程序指令，并转发该计算机可读程序指令，以供存储在各个计算/处理设备中的计算机可读存储介质中。
用于执行本公开操作的计算机程序指令可以是汇编指令、指令集架构(Industry Standard Architecture，ISA)指令、机器指令、机器相关指令、微代码、固件指令、状态设置数据、或者以一种或多种编程语言的任意组合编写的源代码或目标代码，所述编程语言包括面向对象的编程语言—诸如Smalltalk、C++等，以及常规的过程式编程语言，诸如“C”语言或类似的编程语言。计算机可读程序指令可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络，包括局域网(Local Area Network，LAN)或广域网(Wide Area Network，WAN)连接到用户计算机，或者，可以连接到外部计算机(例如利用 因特网服务提供商来通过因特网连接)。在一些实施例中，通过利用计算机可读程序指令的状态信息来个性化定制电子电路，例如可编程逻辑电路、FPGA或可编程逻辑阵列(Programmable Logic Arrays，PLA)，该电子电路可以执行计算机可读程序指令，从而实现本公开的各个方面。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统和装置的工作过程，可以参考前述方法实施例中的对应过程。在本公开所提供的几个实施例中，应该理解到，所揭露的系统、装置和方法，可以通过其它的方式实现。以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，又例如，多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些通信接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本公开各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。
所述功能如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个处理器可执行的非易失的计算机可读取存储介质中。基于这样的理解，本公开的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本公开各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、ROM、RAM、磁碟或者光盘等各种可以存储程序代码的介质。
最后应说明的是：以上所述实施例，仅为本公开的具体实施方式，用以说明本公开的技术方案，而非对其限制，本公开的保护范围并不局限于此，尽管参照前述实施例对本公开进行了详细的说明，本领域的普通技术人员应当理解：任何熟悉本技术领域的技术人员在本公开揭露的技术范围内，其依然可以对前述实施例所记载的技术方案进行修改或可轻易想到变化，或者对其中部分技术特征进行等同替换；而这些修改、变化或者替换，并不使相应技术方案的本质脱离本公开实施例技术方案的精神和范围，都应涵盖在本公开的保护范围之内。因此，本公开的保护范围应所述以权利要求的保护范围为准。
工业实用性
本公开实施例提供一种增强现实场景下的展示方法、装置、设备、介质及程序，该方法包括：获取增强现实AR设备拍摄的当前场景图像；基于所述当前场景图像对目标对象的识别结果，确定所述目标对象匹配的特效数据以及所述特效数据的展示位置信息；基于所述展示位置信息，控制所述AR设备播放所述特效数据；所述特效数据包括虚拟影像和音频中的至少之一，所述虚拟影像的展示位置与所述目标对象之间具有预设位置关系。

Claims (14)

  1. 一种增强现实场景下的展示方法，所述方法由电子设备执行，所述方法包括：
    获取增强现实AR设备拍摄的当前场景图像；
    基于所述当前场景图像对目标对象的识别结果，确定所述目标对象匹配的特效数据以及所述特效数据的展示位置信息；
    基于所述展示位置信息，控制所述AR设备播放所述特效数据；所述特效数据包括虚拟影像和音频中的至少之一，所述虚拟影像的展示位置与所述目标对象之间具有预设位置关系。
  2. 根据权利要求1所述的方法，其中，所述基于所述当前场景图像对目标对象的识别结果，确定所述特效数据的展示位置信息，包括：
    在所述当前场景图像中识别到所述目标对象的情况下，基于所述目标对象在所述当前场景图像中的图像位置信息，确定所述特效数据的展示位置信息。
  3. 根据权利要求1或2所述的方法，其中，所述基于所述当前场景图像对目标对象的识别结果，确定所述目标对象匹配的特效数据以及所述特效数据的展示位置信息，包括：
    在所述当前场景图像中未识别到所述目标对象的情况下，获取世界坐标系下所述目标对象与所述AR设备之间的相对位置信息，并基于所述相对位置信息，确定所述特效数据的展示位置信息。
  4. 根据权利要求1至3任一项所述的方法，其中，所述基于所述展示位置信息，控制所述AR设备播放所述特效数据，包括：
    在确定所述目标对象的至少部分在所述AR设备的图像展示范围的情况下，基于所述展示位置信息，控制所述AR设备播放所述特效数据中的至少部分特效数据；其中，所述至少部分特效数据为所述目标对象的至少部分对应的虚拟影像和音频中的至少之一；
    在确定所述目标对象未在所述AR设备的图像展示范围的情况下，基于所述展示位置信息，控制所述AR设备按照所述音频的已播放进度继续播放所述音频。
  5. 根据权利要求1至4任一项所述的方法，其中，所述虚拟影像包括全息影像；所述方法还包括：
    获取与所述目标对象匹配的待处理视频，所述待处理视频中包括与所述目标对象关联的目标关联对象；
    为所述待处理视频中的每个像素点设置透明通道，得到第一视频；
    基于所述透明通道，从所述第一视频中去除背景像素点，得到第二视频；
    基于所述第二视频生成包括所述目标关联对象的全息影像。
  6. 根据权利要求5所述的方法，其中，所述基于所述透明通道，从所述第一视频中去除背景像素点，得到第二视频，包括：
    将所述第一视频中的背景像素点对应的透明通道设置为白色，得到第三视频；所述第一视频包括所述目标关联对象的目标像素点和除所述目标像素点以外的背景像素点；
    将所述第一视频中的第一类像素点对应的透明通道设置为黑色，将所述第一视频中的第二类像素点对应的透明通道设置为白色，将所述第一视频中的第三类像素点对应的透明通道设置为预设灰色值，得到第四视频；所述第三类像素点包括与所述背景像素点相邻的目标像素点和与所述目标像素点相邻的背景像素点；所述第一类像素点包括除所述第三类像素点以外的背景像素点，所述第二类像素点包括除所述第三类像素点以外的目标像素点；
    基于第三视频和第四视频，生成所述第二视频。
  7. 根据权利要求1至6任一项所述的方法，其中，所述虚拟影像包括多个虚拟对象的影像，以及所述多个虚拟对象之间的展示顺序和交互数据中的至少之一；所述基于所述展示位置信息，控制所述AR设备播放所述特效数据，包括：
    在所述展示位置信息对应的展示位置上，基于所述多个虚拟对象之间的展示顺序和交互数据中的 至少之一，展示所述虚拟对象的影像。
  8. 根据权利要求2所述的方法，其中，所述基于所述目标对象在所述当前场景图像中的图像位置信息，确定所述特效数据的展示位置信息，包括：
    基于所述目标对象在所述当前场景图像中的图像位置信息，确定所述目标对象在世界坐标系下的位置信息；
    基于所述目标对象在所述世界坐标系下的位置信息和所述AR设备在所述世界坐标系下的位置信息，确定所述特效数据的展示位置信息。
  9. 根据权利要求3所述的展示方法，其中，所述获取世界坐标系下所述目标对象与所述AR设备之间的相对位置信息，包括：
    基于所述当前场景图像、历史场景图像、以及所述AR设备在拍摄所述历史场景图像时与所述目标对象在所述世界坐标系下的相对位置信息，确定所述AR设备在拍摄当前场景图像时，与所述目标对象之间的相对位置信息。
  10. 根据权利要求1至9任一项所述的方法，其中，按照以下方式识别所述当前场景图像中是否包含所述目标对象：
    对所述当前场景图像进行特征点提取，得到所述当前场景图像包含的多个特征点分别对应的特征信息；所述多个特征点位于所述当前场景图像中的目标检测区域中；
    基于所述多个特征点分别对应的特征信息与预先存储的所述目标对象包含的多个特征点分别对应的特征信息进行比对，确定所述当前场景图像中是否包含所述目标对象。
  11. 一种增强现实场景下的展示装置，包括：
    图像获取模块，配置为获取增强现实AR设备拍摄的当前场景图像；
    位置确定模块，配置为基于所述当前场景图像对目标对象的识别结果，确定所述目标对象匹配的特效数据以及所述特效数据的展示位置信息；
    特效播放模块，配置为基于所述展示位置信息，控制所述AR设备播放所述特效数据；所述特效数据包括虚拟影像和音频中的至少之一，所述虚拟影像的展示位置与所述目标对象之间具有预设位置关系。
  12. 一种电子设备，包括：处理器、存储器和总线，所述存储器存储有所述处理器可执行的机器可读指令，当电子设备运行时，所述处理器与所述存储器之间通过总线通信，所述机器可读指令被所述处理器执行时执行如权利要求1至10任一项所述的增强现实场景下的展示方法。
  13. 一种计算机可读存储介质，该计算机可读存储介质上存储有计算机程序，该计算机程序被处理器运行时执行如权利要求1至10任一项所述的增强现实场景下的展示方法。
  14. 一种计算机程序，所述计算机程序包括计算机可读代码，在所述计算机可读代码在电子设备中运行的情况下，所述电子设备的处理器执行用于实现如权利要求1至10任一项所述的增强现实场景下的展示方法。
PCT/CN2021/102191 2020-11-06 2021-06-24 增强现实场景下的展示方法、装置、设备、介质及程序 WO2022095467A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011233879.6A CN112348969B (zh) 2020-11-06 2020-11-06 增强现实场景下的展示方法、装置、电子设备及存储介质
CN202011233879.6 2020-11-06

Publications (1)

Publication Number Publication Date
WO2022095467A1 true WO2022095467A1 (zh) 2022-05-12

Family

ID=74428557

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/102191 WO2022095467A1 (zh) 2020-11-06 2021-06-24 增强现实场景下的展示方法、装置、设备、介质及程序

Country Status (3)

Country Link
CN (1) CN112348969B (zh)
TW (1) TW202220438A (zh)
WO (1) WO2022095467A1 (zh)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115002442A (zh) * 2022-05-24 2022-09-02 北京字节跳动网络技术有限公司 一种图像展示方法、装置、电子设备及存储介质
CN115242980A (zh) * 2022-07-22 2022-10-25 中国平安人寿保险股份有限公司 视频生成方法和装置、视频播放方法和装置及存储介质
CN116095293A (zh) * 2023-01-13 2023-05-09 北京达佳互联信息技术有限公司 虚拟道具的显示方法、装置、设备以及存储介质
CN116860114A (zh) * 2023-09-04 2023-10-10 腾讯科技（深圳）有限公司 基于人工智能的扩展现实交互方法及相关装置
WO2024088141A1 (zh) * 2022-10-28 2024-05-02 北京字跳网络技术有限公司 特效处理方法、装置、电子设备及存储介质
CN118151806A (zh) * 2024-05-11 2024-06-07 广州欧科信息技术股份有限公司 基于增强现实的历史数据展示方法及相关设备
CN118331430A (zh) * 2024-06-13 2024-07-12 深圳市中新云创科技有限公司 一种基于增强现实的跑步机全景体感运动系统及方法

Families Citing this family (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112348969B (zh) * 2020-11-06 2023-04-25 北京市商汤科技开发有限公司 增强现实场景下的展示方法、装置、电子设备及存储介质
CN112905014A (zh) * 2021-02-26 2021-06-04 北京市商汤科技开发有限公司 Ar场景下的交互方法、装置、电子设备及存储介质
CN112991553B (zh) * 2021-03-11 2022-08-26 深圳市慧鲤科技有限公司 信息展示方法及装置、电子设备和存储介质
CN112991555B (zh) * 2021-03-30 2023-04-07 北京市商汤科技开发有限公司 数据展示方法、装置、设备以及存储介质
CN113269782B (zh) * 2021-04-21 2023-01-03 青岛小鸟看看科技有限公司 数据生成方法、装置及电子设备
CN113220123A (zh) * 2021-05-10 2021-08-06 深圳市慧鲤科技有限公司 一种音效控制的方法、装置、电子设备及存储介质
CN113115099B (zh) * 2021-05-14 2022-07-05 北京市商汤科技开发有限公司 一种视频录制方法、装置、电子设备以及存储介质
CN113240819A (zh) * 2021-05-24 2021-08-10 中国农业银行股份有限公司 穿戴效果的确定方法、装置和电子设备
CN113329218A (zh) * 2021-05-28 2021-08-31 青岛鳍源创新科技有限公司 水下拍摄的增强现实结合方法、装置、设备及存储介质
CN113359983A (zh) * 2021-06-03 2021-09-07 北京市商汤科技开发有限公司 增强现实数据呈现方法、装置、电子设备及存储介质
CN113359986B (zh) * 2021-06-03 2023-06-20 北京市商汤科技开发有限公司 增强现实数据展示方法、装置、电子设备及存储介质
CN113542891B (zh) * 2021-06-22 2023-04-21 海信视像科技股份有限公司 一种视频特效显示方法及设备
CN113345108B (zh) * 2021-06-25 2023-10-20 北京市商汤科技开发有限公司 增强现实数据展示方法、装置、电子设备及存储介质
CN113470186A (zh) * 2021-06-30 2021-10-01 北京市商汤科技开发有限公司 Ar交互方法、装置、电子设备及存储介质
CN113542620B (zh) * 2021-07-06 2022-02-25 北京百度网讯科技有限公司 一种特效处理方法、装置及电子设备
CN113487744A (zh) * 2021-07-09 2021-10-08 浙江商汤科技开发有限公司 场景显示方法及装置、电子设备、车辆、可读存储介质
CN114153548A (zh) * 2021-12-15 2022-03-08 北京绵白糖智能科技有限公司 一种展示方法、装置、计算机设备及存储介质
CN114298897A (zh) * 2021-12-29 2022-04-08 新瑞鹏宠物医疗集团有限公司 宠物形象处理方法、装置、存储介质及电子设备
CN114299262A (zh) * 2021-12-31 2022-04-08 北京绵白糖智能科技有限公司 增强现实ar场景的展示方法及装置
CN114661398B (zh) * 2022-03-22 2024-05-17 上海商汤智能科技有限公司 一种信息展示方法、装置、计算机设备和存储介质
CN116486052A (zh) * 2023-04-11 2023-07-25 富泰华工业（深圳）有限公司 远程协作方法、远程设备及存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170092001A1 (en) * 2015-09-25 2017-03-30 Intel Corporation Augmented reality with off-screen motion sensing
CN110083238A (zh) * 2019-04-18 2019-08-02 深圳市博乐信息技术有限公司 基于增强现实技术的人机互动方法与系统
CN111510701A (zh) * 2020-04-22 2020-08-07 Oppo广东移动通信有限公司 虚拟内容的显示方法、装置、电子设备及计算机可读介质
CN111696215A (zh) * 2020-06-12 2020-09-22 上海商汤智能科技有限公司 一种图像处理方法、装置及设备
CN112348969A (zh) * 2020-11-06 2021-02-09 北京市商汤科技开发有限公司 增强现实场景下的展示方法、装置、电子设备及存储介质

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9754416B2 (en) * 2014-12-23 2017-09-05 Intel Corporation Systems and methods for contextually augmented video creation and sharing
CN110180167B (zh) * 2019-06-13 2022-08-09 张洋 增强现实中智能玩具追踪移动终端的方法
CN110213640B (zh) * 2019-06-28 2021-05-14 香港乐蜜有限公司 虚拟物品的生成方法、装置及设备
CN111640169A (zh) * 2020-06-08 2020-09-08 上海商汤智能科技有限公司 历史事件呈现方法、装置、电子设备及存储介质
CN111667588A (zh) * 2020-06-12 2020-09-15 上海商汤智能科技有限公司 人物图像处理方法、装置、ar设备以及存储介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170092001A1 (en) * 2015-09-25 2017-03-30 Intel Corporation Augmented reality with off-screen motion sensing
CN110083238A (zh) * 2019-04-18 2019-08-02 深圳市博乐信息技术有限公司 基于增强现实技术的人机互动方法与系统
CN111510701A (zh) * 2020-04-22 2020-08-07 Oppo广东移动通信有限公司 虚拟内容的显示方法、装置、电子设备及计算机可读介质
CN111696215A (zh) * 2020-06-12 2020-09-22 上海商汤智能科技有限公司 一种图像处理方法、装置及设备
CN112348969A (zh) * 2020-11-06 2021-02-09 北京市商汤科技开发有限公司 增强现实场景下的展示方法、装置、电子设备及存储介质

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115002442A (zh) * 2022-05-24 2022-09-02 北京字节跳动网络技术有限公司 一种图像展示方法、装置、电子设备及存储介质
CN115002442B (zh) * 2022-05-24 2024-05-10 北京字节跳动网络技术有限公司 一种图像展示方法、装置、电子设备及存储介质
CN115242980A (zh) * 2022-07-22 2022-10-25 中国平安人寿保险股份有限公司 视频生成方法和装置、视频播放方法和装置及存储介质
CN115242980B (zh) * 2022-07-22 2024-02-20 中国平安人寿保险股份有限公司 视频生成方法和装置、视频播放方法和装置及存储介质
WO2024088141A1 (zh) * 2022-10-28 2024-05-02 北京字跳网络技术有限公司 特效处理方法、装置、电子设备及存储介质
CN116095293A (zh) * 2023-01-13 2023-05-09 北京达佳互联信息技术有限公司 虚拟道具的显示方法、装置、设备以及存储介质
CN116860114A (zh) * 2023-09-04 2023-10-10 腾讯科技（深圳）有限公司 基于人工智能的扩展现实交互方法及相关装置
CN116860114B (zh) * 2023-09-04 2024-04-05 腾讯科技（深圳）有限公司 基于人工智能的扩展现实交互方法及相关装置
CN118151806A (zh) * 2024-05-11 2024-06-07 广州欧科信息技术股份有限公司 基于增强现实的历史数据展示方法及相关设备
CN118331430A (zh) * 2024-06-13 2024-07-12 深圳市中新云创科技有限公司 一种基于增强现实的跑步机全景体感运动系统及方法

Also Published As

Publication number Publication date
CN112348969B (zh) 2023-04-25
TW202220438A (zh) 2022-05-16
CN112348969A (zh) 2021-02-09

Similar Documents

Publication Publication Date Title
WO2022095467A1 (zh) 增强现实场景下的展示方法、装置、设备、介质及程序
US10609332B1 (en) Video conferencing supporting a composite video stream
US11620780B2 (en) Multiple device sensor input based avatar
WO2022095468A1 (zh) 增强现实场景下的展示方法、装置、设备、介质及程序
US10055888B2 (en) Producing and consuming metadata within multi-dimensional data
CN111080759B (zh) 一种分镜效果的实现方法、装置及相关产品
US9070194B2 (en) Planar surface detection
TW202117673A (zh) 一種ar場景圖像處理方法、電子設備及儲存介質
CN111638793B (zh) 飞行器的展示方法、装置、电子设备及存储介质
US20090202114A1 (en) Live-Action Image Capture
CN111694430A (zh) 一种ar场景画面呈现方法、装置、电子设备和存储介质
CN112927349B (zh) 三维虚拟特效的生成方法、装置、计算机设备及存储介质
CN107995481B (zh) 一种混合现实的显示方法及装置
CN112637665B (zh) 增强现实场景下的展示方法、装置、电子设备及存储介质
JP7150894B2 (ja) Ａｒシーン画像処理方法及び装置、電子機器並びに記憶媒体
CN105389090A (zh) 游戏交互界面显示的方法及装置、移动终端和电脑终端
CN113178017A (zh) Ar数据展示方法、装置、电子设备及存储介质
CN111651058A (zh) 历史场景的控制展示方法、装置、电子设备及存储介质
US20230386147A1 (en) Systems and Methods for Providing Real-Time Composite Video from Multiple Source Devices Featuring Augmented Reality Elements
CN111383313B (zh) 一种虚拟模型渲染方法、装置、设备及可读存储介质
CN114612637B (zh) 一种场景画面显示方法、装置、计算机设备及存储介质
CN113031846B (zh) 用于展示任务的描述信息的方法、装置及电子设备
CN114625468A (zh) 增强现实画面的展示方法、装置、计算机设备及存储介质
JP2023542598A (ja) 文字の表示方法、装置、電子機器及び記憶媒体
CN111651048B (zh) 多虚拟对象排列展示方法、装置、电子设备及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21888170

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21888170

Country of ref document: EP

Kind code of ref document: A1