WO2023124698A1 - 增强现实场景的展示 - Google Patents

增强现实场景的展示 Download PDF

Info

Publication number
WO2023124698A1
WO2023124698A1 PCT/CN2022/134853 CN2022134853W WO2023124698A1 WO 2023124698 A1 WO2023124698 A1 WO 2023124698A1 CN 2022134853 W CN2022134853 W CN 2022134853W WO 2023124698 A1 WO2023124698 A1 WO 2023124698A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
display
virtual object
virtual
objects
Prior art date
Application number
PCT/CN2022/134853
Other languages
English (en)
French (fr)
Inventor
李斌
欧华富
李颖楠
Original Assignee
上海商汤智能科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 上海商汤智能科技有限公司 filed Critical 上海商汤智能科技有限公司
Publication of WO2023124698A1 publication Critical patent/WO2023124698A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics

Definitions

  • the present disclosure relates to the technical field of augmented reality, in particular to a display of augmented reality scenes.
  • Augmented Reality (AR) technology superimposes a virtual model on the real world, thereby presenting the real environment and virtual objects in the same picture or space in real time.
  • AR Augmented Reality
  • Embodiments of the present disclosure provide a display method, device, AR device, and storage medium of an augmented reality AR scene.
  • an embodiment of the present disclosure provides a method for displaying an augmented reality AR scene, the method including: in response to identifying multiple target identification objects from the current scene image, according to the multiple target identification objects respectively Corresponding to the virtual object, determine the target joint display data for multiple virtual objects; according to the initial relative poses of each of the virtual objects and the AR device in the world coordinate system, determine each of the virtual objects and the AR device The first relative pose data in the world coordinate system; according to the first relative pose data of each of the virtual objects and the AR device and the joint display data of the target, display multiple objects on the AR device the virtual object.
  • an embodiment of the present disclosure provides a device for displaying an augmented reality AR scene, the device comprising: a joint data determination module configured to, in response to identifying multiple target identification objects from the current scene image, according to the The virtual objects respectively corresponding to the plurality of target identification objects determine the target joint display data for the plurality of virtual objects; the first pose determining module is configured to, according to each of the virtual objects and the AR device in the world coordinate system In the initial relative pose, determine the first relative pose data of each of the virtual objects and the AR device in the world coordinate system; the first presentation module is configured to, according to each of the virtual objects and the AR device The first relative pose data and the target joint display data, displaying multiple virtual objects on the AR device.
  • the embodiment of the present disclosure provides an augmented reality AR device, including: a processor; and a memory, communicably connected to the processor, the memory stores computer instructions that can be read by the processor , the computer instructions are used to cause the processor to execute the method according to any implementation manner of the first aspect.
  • an embodiment of the present disclosure provides a storage medium storing computer instructions for causing a computer to execute the method according to any embodiment of the first aspect.
  • the display method of the embodiment of the present disclosure can make the multiple virtual objects interact when multiple target identification objects are identified, presenting a joint display effect and improving the display effect of the virtual image.
  • each virtual object is not limited to being displayed at a fixed display position, which makes the joint display effect between multiple virtual objects more realistic and improves user experience.
  • Fig. 1 is a schematic structural diagram of an AR device according to some implementations of the present disclosure.
  • FIGS. 2a and 2b are schematic structural diagrams of an AR device according to some implementations of the present disclosure.
  • Fig. 3 is a structural block diagram of an AR device according to some implementations of the present disclosure.
  • FIG. 4 is a flowchart illustrating a method in some embodiments according to the present disclosure.
  • Figure 5 is a schematic diagram illustrating a method in some embodiments according to the present disclosure.
  • FIG. 6 is a flowchart illustrating a method in some embodiments according to the present disclosure.
  • Figure 7 is a flow chart illustrating a method in some embodiments according to the present disclosure.
  • FIG. 8 is a flowchart illustrating a method in some embodiments according to the present disclosure.
  • FIG. 9 is a flowchart illustrating a method in some embodiments according to the present disclosure.
  • Figure 10 is a flowchart illustrating a method in some embodiments according to the present disclosure.
  • Fig. 11 is a structural block diagram of a display device according to some embodiments of the present disclosure.
  • Displaying AR effects based on markers is a way to display virtual objects in augmented reality scenes.
  • This method requires pre-made markers, such as preset graphics drawn on the surface of items based on Hamming Code technology. etc., the preset graphics may be, for example, cartoon images, two-dimensional codes, barcodes, etc.
  • the AR device recognizes the Marker in the current scene through the camera, it can superimpose the corresponding virtual object on the current scene image to realize the superimposed display effect of the virtual object and the real scene.
  • the virtual object corresponding to the identified object is preset.
  • the AR device recognizes the identified object, it can be identified on the display screen of the AR device based on the preset display effect
  • the virtual object corresponding to the object is displayed.
  • the identification object can be a preset graphic drawn on the surface of a calendar or a postcard.
  • a corresponding virtual object can be presented on the display screen of the AR device and superimposed on the surface of the identification object to make a preset.
  • Set the visual effect of the action For a multi-identified object scene, when the AR device recognizes multiple identified objects, each corresponding virtual object can only be displayed on the surface of the corresponding identified object, and there is no interaction between the virtual objects in this display mode. Sex, the display effect is not good.
  • Embodiments of the present disclosure provide a display method, device, AR device, and storage medium of an augmented reality scene, aiming at improving the display effect of virtual objects in the AR scene and improving user experience.
  • the embodiment of the present disclosure provides a method for displaying an augmented reality scene, and the method can be applied to an AR device.
  • the AR device described in the embodiments of the present disclosure can be any terminal device with AR functions, such as head-mounted devices such as AR glasses and AR helmets, or handheld devices such as smart phones and tablet computers, or smart phones. Wrist-worn devices such as watches and smart bracelets, etc., are not limited in this disclosure.
  • the AR device described in the present disclosure is described by taking AR glasses as an example.
  • the AR device 100 includes a main body 101, and the main body 101 includes a bracket structure for wearing and supporting.
  • the display screen 110 is arranged on the main body 101 , and after the user wears the AR device 100 , the display screen 110 can be located directly in front of the user's eyes, so that the user can easily observe the images displayed on the display screen 110 .
  • the AR device 100 also includes a camera 120, which is used to collect images of the current scene.
  • the AR device 100 also includes a processor disposed inside the main body 101 , and the processor is used to superimpose the current scene image captured by the camera 120 with the virtual object, and display it on the display screen 110 .
  • the AR device described in the present disclosure is described by taking a smart phone as an example.
  • the AR device 100 includes a housing 102, which is a shell structure for supporting the main body of the mobile phone.
  • the display screen 110 is arranged on the front of the casing 102, and the camera 120 is arranged on the front and/or the back of the casing 102.
  • the camera 120 is arranged on the back of the casing.
  • the camera 120 is used to collect the current scene image, so that the processor of the smart phone superimposes the current scene image collected by the camera 120 with the virtual object, and displays it on the display screen 110 .
  • FIG. 3 shows a structural block diagram of an AR device in the present disclosure.
  • the AR device in an embodiment of the present disclosure will be further described below in conjunction with FIG. 3 .
  • the AR device 100 includes a processor 130 , a memory 140 , a camera 120 , a display screen 110 and an IMU (Inertial Measurement Unit, inertial measurement unit) sensor 160 .
  • IMU Inertial Measurement Unit, inertial measurement unit
  • the processor 130 , the memory 140 , the camera 120 , the display screen 110 and the IMU sensor 160 establish a communicable connection between any of them through the bus 150 .
  • the processor 130 can be any type of processor with one or more processing cores. It can perform single-threaded or multi-threaded operations, and is used to parse instructions to perform operations such as obtaining data, performing logical operation functions, and delivering operation processing results.
  • the memory 140 may include a non-volatile computer-readable storage medium, such as at least one magnetic disk storage device, a flash memory device, a distributed storage device located remotely with respect to the processor 130, or other non-volatile solid-state storage devices.
  • the memory may have a program storage area for storing non-volatile software programs, non-volatile computer-executable programs and modules, which are invoked by the processor 130 to make the processor 130 execute one or more method steps below.
  • the memory 140 may also include a storage part such as a volatile random storage medium or a hard disk, as a data storage area for storing calculation processing results and data output by the processor 130 .
  • the camera 120 is used to collect the current scene image, so that the processor 130 can perform fusion processing for the virtual object according to the collected current scene image, so as to realize the AR display effect.
  • the camera 120 can be any type of camera suitable for implementation, such as a black and white camera, an RGB camera, etc.
  • the camera 120 can be divided into a monocular camera and a binocular camera according to the number of cameras, which is not limited in the present disclosure. .
  • the display screen 110 is configured to receive the display information issued by the processor 130 , so as to present a corresponding display image on the display screen 110 according to the display information, so that the user can observe the AR display effect through the display screen 110 .
  • the display screen 110 may be any display screen type suitable for implementation, such as an LCD (Liquid Crystal Display, liquid crystal display) display screen, an OLED (Organic Light-Emitting Diode, organic light-emitting semiconductor) display screen, etc., This disclosure does not limit this.
  • the IMU sensor 160 is mainly used to detect and measure the acceleration and rotational motion of the AR device itself. Its basic principle is to use the law of inertia to measure the acceleration and rotational motion, and can calculate the pose of the AR device itself in real time based on the sensor signal.
  • the IMU sensor 160 may include, for example, a three-axis or six-axis gyroscope, an acceleration sensor, and the like.
  • the display method of the augmented reality AR scene of the example of the present disclosure includes:
  • each target identification object corresponds to a respective virtual object.
  • the current scene image refers to a scene image collected in real time by a camera of an AR device.
  • the camera 120 collects scene images within the viewing range, and displays the collected scene images on the display screen 110 through data processing.
  • the images observed by the user in real time on the display screen 110 are The current scene image. It can be understood that as the pose of the AR device changes, the collected current scene image changes accordingly, that is, the current scene image is a real-time image.
  • the target identification object refers to the marker used to trigger the display of the AR effect, that is, the above-mentioned Marker.
  • the specific presentation method of the target identification object may be, for example, a two-dimensional code, a preset graphic, etc., which is not limited in the present disclosure.
  • the current scene image includes at least two target identification objects, so that the AR device can recognize multiple target identification objects from the current scene image.
  • the current scene image is as shown in FIG. 5, which includes two postcards 200
  • the target identification object includes cartoon graphics drawn on the surface of each postcard 200, that is, cartoon graphics 210 and cartoon graphics 220.
  • the cartoon graphic 210 and the cartoon graphic 220 as target identification objects can be drawn based on, for example, Hamming code technology, so that the AR device can recognize and analyze the cartoon graphic when it collects an image containing the cartoon graphic.
  • the process of drawing and analyzing Hamming codes can be understood and fully implemented by those skilled in the art by referring to related technologies, which is not limited in the present disclosure.
  • the target identification object is a preset graphic drawn on the surface of the object.
  • the AR device captures the scene image including the preset graphic, it can extract and identify the feature points of the scene image through image detection technology, so that This preset graphic can be recognized from the scene image.
  • image detection and recognition of preset images can be understood and fully implemented by those skilled in the art with reference to related technologies, and the present disclosure does not limit this.
  • the target identification object is associated with the corresponding virtual object.
  • a model database including at least one virtual object may be pre-established, and the model database includes the virtual object and the target identification object associated with the virtual object.
  • the virtual object corresponding to the target identification object can be determined by searching the model database.
  • a virtual object is represented as a virtual image rendered and presented on the display screen of an AR device.
  • the display effect of the interaction between the virtual object and the real scene can be presented.
  • the AR device detects and recognizes the collected current scene image through the image detection technology, and if the target identification object is recognized from the current scene image, the virtual object corresponding to the target identification object can be determined.
  • a virtual object corresponding to each target identification object is generated, wherein each virtual object is not placed in the corresponding target identification object Positions are displayed separately, but a joint display effect is produced between multiple virtual objects.
  • the joint display effect refers to any rendering effect that can cause multiple virtual objects to be linked, for example, superposition, surround, interaction, etc. between multiple virtual objects.
  • corresponding joint display effects may be preset based on different combinations of virtual objects.
  • a joint display relationship is introduced.
  • the joint display relationship represents the corresponding relationship between the virtual object and the joint display data.
  • the joint display data refers to the data that makes the virtual object produce a corresponding joint display effect.
  • the joint display data may include, for example, the virtual object Position data and attitude data. Therefore, when the corresponding combination of virtual objects is identified from the current scene image, the corresponding target joint presentation data can be determined according to the preset joint presentation relationship, and the corresponding joint presentation effect can be presented on the display screen of the AR device.
  • the AR device can recognize two target identification objects from the current scene image, that is, a cartoon graphic 210 and a cartoon graphic 220 . Therefore, the AR device can determine the target joint presentation data corresponding to the two virtual objects in this example by searching the preset joint presentation relationship according to the virtual objects corresponding to the cartoon graphic 210 and the cartoon graphic 220 .
  • the target joint presentation data corresponding to the two virtual objects of the cartoon graphic 210 and the cartoon graphic 220 indicates "two virtual objects chasing each other".
  • the joint display data can also be any other data suitable for realizing the display effect, as long as the linkage effect between the two virtual objects is guaranteed, and the present disclosure does not limit this.
  • a corresponding object category may be set for each virtual object in advance, so as to pre-establish a correspondence relationship between the object category and joint presentation data, that is, a joint presentation relationship.
  • the AR device recognizes a virtual object belonging to one or more object categories, it can find the corresponding target joint display data according to the joint display relationship.
  • the AR device when it recognizes the target identification object from the current scene image, it can display the joint display of the virtual object on the AR device based on the simultaneous localization and mapping (Simultaneous Localization And Mapping, SLAM) technology Effect.
  • the SLAM technology can realize the spatial positioning of the AR device and the map construction for the current scene, so that the superimposed display effect of the virtual object and the real scene can be rendered on the current scene image displayed by the AR device, making the display effect of the virtual object more realistic.
  • S420 Determine first relative pose data of each virtual object and the AR device in the world coordinate system according to the initial relative pose data of each virtual object and the AR device in the world coordinate system.
  • the AR device when displaying a virtual object based on SLAM, it is necessary to map the virtual object and the AR device into the same coordinate system, so that when the two move relative to each other, the AR device can use the IMU sensor 130 to calculate and determine the distance between the two in real time.
  • the coordinate system is the world coordinate system described in this disclosure.
  • feature extraction may be performed on the target identification object in advance based on image detection technology to obtain at least one preset key point of the target identification object, and a world coordinate system is established based on the at least one preset key point.
  • the first initial pose may include the initial position and initial pose of the virtual object.
  • the initial pose of the AR device in the world coordinate system that is, the second initial pose
  • the second initial pose can also include the initial position and initial position of the AR device. attitude.
  • the pose of the IMU sensor 160 may be determined as the initial pose of the AR device in the world coordinate system when the AR device 100 captures the first frame of the scene image.
  • the initial pose of the AR device and the initial pose of the virtual object can be Get the initial relative pose of the virtual object and the AR device.
  • the relative position of the virtual object and the AR device is determined according to the position difference between the two, and the relative posture of the virtual object and the AR device is determined according to the initial posture difference between the virtual object and the AR device.
  • the IMU sensor 160 may detect that the pose (position and posture) of the AR device 100 changes.
  • the gyroscope can determine the posture of the AR device after the pose change according to the angular deviation between the AR device in the three-axis direction and the initial pose, and the acceleration sensor can detect the movement acceleration during the pose change of the AR device, and solve the problem through inertial navigation.
  • the position after the movement is obtained by the calculation method, so as to determine the pose data of the AR device after movement.
  • the aforementioned initial relative pose is updated accordingly to obtain the relative pose between the updated pose of the AR device and the pose of the virtual object data, that is, the first relative pose data.
  • the world coordinate system can be mapped to the camera coordinate system of the AR device through rotation and translation, and then mapped from the camera coordinate system to the screen coordinate system of the AR device.
  • the AR device After the AR device determines the first relative pose data between itself and each virtual object in real time, it can map the first relative pose data to the screen coordinate system of the AR device according to the above mapping relationship, and determine the position of each virtual object. Display pose data in the screen coordinate system. Then the AR device renders and displays each virtual object on the display screen of the AR device according to the display pose data of each virtual object and the previously determined joint display data, so that the user can watch the real-time joint display effect of multiple virtual objects and Pose changes.
  • a user wears AR glasses such as shown in FIG. 1 to watch a scene such as shown in FIG. 5 .
  • the target joint display data indicates "two virtual objects chasing each other", so that the AR device can render two virtual objects on the display screen according to the target joint display data, so that the two virtual objects present a display effect of chasing each other.
  • the AR device can determine the change of the relative distance between itself and each virtual object in the world coordinate system based on SLAM technology.
  • the The size of the virtual object rendered on the display screen can gradually decrease; conversely, when the relative distance between the AR device and the virtual object decreases, the size of the virtual object rendered on the display screen can gradually increase. Therefore, the rendering effect of the virtual object conforms to the "far smaller and near larger" when viewed by human eyes in a real scene, so the display of the virtual object is more realistic.
  • the AR device when the AR effect is displayed, can recognize one or more target identification objects from the collected current scene image.
  • the process of identifying the target identification object from the current scene image includes:
  • S610 Perform feature point extraction on the current scene image to obtain feature information corresponding to at least one feature point included in the current scene image.
  • the AR device 100 can collect the current scene image through the camera 120. It can be understood that the current scene image can be a single frame image collected by the camera, or a multi-frame image in the video stream collected by the camera. There is no limit to this.
  • the detection area including the object to be detected in the current scene image can be located by an image detection algorithm, and then one or more feature points of the object to be detected can be extracted in the detection area.
  • the image detection algorithm can be used to first determine the areas of the two postcards 200 as the area to be tested, and then perform feature points on the cartoon graphics 210 and cartoon graphics 220 included on the two postcards 200 to be tested. Extract to obtain the feature information of each object to be tested.
  • the feature information of the object to be tested can be compared with the feature point information of the pre-stored target identification object. If the similarity between the two is greater than the preset threshold, it means that the object to be tested is The target identification object; otherwise, if the similarity is not greater than the preset threshold, it means that the object to be tested is not the target identification object.
  • Fig. 5 it is assumed that the pre-stored feature point information of the target identification object corresponds to the cartoon figure 210 and the cartoon figure 220, so that the feature information of the cartoon figure extracted from the current scene image and the pre-stored target identification object feature Point information is compared, and the similarity between the two is greater than the preset threshold, so as to determine that the target identification object is recognized in the current scene image.
  • the feature point information described in the embodiments of the present disclosure may include texture feature value, RGB feature value, gray value and other information that can identify the feature point feature corresponding to the feature point, which is not limited in the present disclosure.
  • the virtual object corresponding to each target identification object can be further obtained, and the corresponding target joint display can be determined according to the combination of multiple virtual objects data.
  • a corresponding object category may be set for each virtual object, and the corresponding target joint presentation data is determined according to the object category of the virtual object, which will be described below with reference to the embodiment of FIG. 7 .
  • the display method of the example of the present disclosure includes:
  • the object categories described in the embodiments of the present disclosure may include, for example, "animal”, "plant”, “decoration” and so on. Therefore, when the AR device determines a virtual object through a corresponding target identification object, it can determine its corresponding object category according to the virtual object.
  • the object category of the virtual object corresponding to the cartoon graphic 210 and the cartoon graphic 220 in the example shown in FIG. 5 may be "animal".
  • the corresponding relationship between object categories and joint presentation data may be established in advance according to combinations of different object categories.
  • the pre-established correspondence is shown in Table 1 below:
  • the joint presentation effect presented by the corresponding target joint presentation data 1 is "chasing each other".
  • the presented joint display effect may be that the virtual object “Little Dragon” and the virtual object “Squirrel” chase each other.
  • the joint display effect presented by the corresponding target joint display data 2 is "the plant is superimposed as the background of the animal", for example, the virtual The object A is a lion, and the virtual object B is a bush.
  • the joint display effect presented may be that the virtual object B "grass” is presented as the background of the virtual object A lion.
  • the joint display effect presented by the corresponding target joint display data 3 is "decoration surrounds the animal", for example, The virtual object A is a little dragon, and the virtual object B is auspicious clouds.
  • the joint display effect presented can be that the virtual object B "auspicious clouds” surrounds the virtual object A "little dragon”.
  • the joint display effect presented by the corresponding target joint display data 4 is "plants superimposed on each other", for example, the virtual object A is a grass, and the virtual object B is a flower, and the joint display effect presented may be that the virtual object B "flower” is superimposed on the virtual object A "grass”.
  • the corresponding target joint display relationship can be determined according to the object categories of the multiple virtual objects and the above joint display relationship Show data.
  • the AR device recognizes that the current scene image contains two virtual objects, and the object category of the two virtual objects is "animal”.
  • the corresponding target joint display can be determined.
  • the data is "data 1”
  • the joint display effect presented on the display screen according to data 1 is "chasing each other”.
  • the presentation method of the example of the present disclosure includes:
  • S810. Perform image detection on the current scene image to obtain at least one preset key point of at least one of the multiple target identification objects.
  • the world coordinate system can be established based on at least one preset key point of any target identification object.
  • the preset key point can be the center point of the cartoon figure 210
  • the center point of the cartoon figure 210 is used as the origin O of the coordinate system
  • the direction parallel to the short side of the postcard 200 and passing through the origin O is used as the X axis, so that The direction parallel to the long side of the postcard 200 and passing through the origin O is taken as the Y axis
  • the direction perpendicular to the X and Y axes and passing through the origin O is taken as the Z axis, thereby establishing a world coordinate system O-XYZ.
  • the process of establishing the world coordinate system in FIG. 5 is only an example of the implementation of the present disclosure.
  • the world coordinate system can also be established according to other preset key points, for example, the vertex in the lower left corner of the postcard 200 can be used as The coordinate origin O establishes a coordinate system, and for example, the center point of the cartoon figure 220 can be used as the coordinate origin O to establish a coordinate system, etc., and the disclosure does not limit this.
  • the first initial pose may include the initial position and initial pose of each virtual object.
  • the initial pose of the AR device in the world coordinate system that is, the second initial pose
  • the second initial pose can also include The initial position and initial pose of the AR device.
  • the pose of the IMU sensor 160 may be determined as the initial pose of the AR device in the world coordinate system when the AR device 100 captures the first frame of the scene image.
  • the initial pose of the AR device and the virtual object can be obtained.
  • the initial relative pose of the virtual object and the AR device For example, the relative position data of the virtual object and the AR device is determined according to the initial position difference between the virtual object and the AR device, and the relative posture data of the virtual object and the AR device are determined according to the initial posture difference between the virtual object and the AR device.
  • the initial relative pose of the virtual object and the AR device is determined according to the initial position difference between the virtual object and the AR device.
  • the relative pose change between the AR device and the virtual object is detected in real time, but also the relative pose change between the target identification object and the virtual object is detected, thereby producing different auxiliary display effects, It will be described below in conjunction with FIG. 9 .
  • the presentation method of the example of the present disclosure further includes:
  • the target joint display data, and the target auxiliary display data of each of the virtual objects display multiple virtual objects on the AR device. object.
  • the linkage effect among the above-mentioned multiple virtual objects is defined as "joint display effect”.
  • Auxiliary display effects refer to auxiliary effects that can be superimposed on joint display effects, such as visual effects brought about by adjusting rendering parameters such as brightness, exposure, and transparency of joint display effects, which can be used as auxiliary display effects in this disclosure .
  • Auxiliary display data refers to data that enables a virtual object to produce a corresponding auxiliary display effect, and the auxiliary display data may include, for example, data for adjusting parameters such as brightness, transparency, exposure, and angle of the virtual object.
  • the current scene image can be as shown in FIG. 5.
  • the AR device recognizes the target identification object (cartoon graphics 210 and cartoon graphics 220) in the current scene image (FIG. 5), it can realize virtual The joint display effect of the object.
  • the pose of each target identification object can be detected simultaneously based on image detection technology. It can be understood that during the display process of virtual objects, the pose of the target identification object may change, such as user Moving the postcard 200 makes the pose of the target identification object in the current scene image detected by the AR device change.
  • the process of determining the target auxiliary display data of the virtual object includes:
  • the AR device can obtain the first pose data of the virtual object in the world coordinate system, and at the same time, the AR device can detect and obtain the second pose data of the target identification object in the world coordinate system, for example, based on the image
  • the detection technology determines the detection frame of the postcard 200, and determines the second pose data of the target identification object in the current scene image according to the difference of the detection frame before and after the pose change of the postcard 200.
  • the relative pose between the virtual object and the target identification object can be determined according to the first pose data and the second pose data, that is, the The second relative pose data.
  • the corresponding relationship between the second relative pose data and the auxiliary display data may be established in advance based on the second relative pose data.
  • the second relative pose data includes the relative distance between the virtual object and the target identification object in the world coordinate system
  • the corresponding auxiliary display data includes the transparency of the virtual object, so that the pre-established auxiliary display relationship can be shown in Table 2 below Show:
  • the target auxiliary display data of the virtual object is "transparency 0%"
  • the target auxiliary display data of the virtual object is "transparency 25%”
  • the target auxiliary display data of the virtual object is "transparency 50%. %”. That is, the farther the relative distance between the virtual object and the target identification object is, the more transparent the auxiliary display effect of the virtual object is.
  • the second relative pose data includes the relative posture (such as relative angle) of the virtual object and the target identification object in the world coordinate system
  • the corresponding auxiliary display data includes the brightness of the virtual object, so that the pre-established auxiliary display
  • the target auxiliary display data of the virtual object is "brightness 100%"
  • the target auxiliary display data of the virtual object is "brightness 75%”
  • the target auxiliary display data of the virtual object is "brightness 50%”. %" That is, the larger the relative angle between the virtual object and the target identification object, the lower the brightness of the virtual object.
  • the second relative pose data also includes the relative distance and relative posture of the virtual object and the target identification object corresponding to the virtual object in the world coordinate system
  • the corresponding auxiliary display data also includes the transparency and brightness of the virtual object
  • auxiliary display data may also include color, brightness, transparency, exposure One or more data in degrees or display angles, which is not limited in the present disclosure.
  • each virtual object can be rendered and displayed based on the target auxiliary display data and the aforementioned target joint display data, so that the user can watch the virtual objects present the joint display effect on the display screen of the AR device at the same time It also has an auxiliary display effect.
  • the corresponding multiple virtual objects can interact with each other to present a joint display effect and increase the interaction of the AR scene. and improve the display effect of virtual images.
  • virtual objects are displayed based on SLAM, and each virtual object is not limited to displaying at a fixed display location, so that the combined display effect among multiple virtual objects is more realistic and user experience is improved.
  • different auxiliary display effects can be displayed in real time according to the relative pose changes between the target identification object and the virtual object, thereby increasing the interactivity of the AR scene and improving the display effect of the virtual image.
  • the embodiment of the present disclosure provides an augmented reality AR scene display device, and the device can be applied to an AR device.
  • the AR device may be, for example, the AR glasses shown in FIG. 1 .
  • the AR device may be, for example, a smart phone as shown in Fig. 2 and Fig. 2b. This disclosure does not limit this.
  • the display device of the augmented reality AR scene of the example of the present disclosure includes:
  • the joint data determination module 10 is configured to, in response to identifying multiple target identification objects from the current scene image, determine target joint display data for multiple virtual objects according to the virtual objects corresponding to the multiple target identification objects;
  • the first pose determining module 20 is configured to determine the first relative pose data of each virtual object and the AR device in the world coordinate system according to the initial relative pose data of each virtual object and the AR device in the world coordinate system ;
  • the world coordinate system is determined according to the characteristics of any target identification object extracted in advance;
  • the first display module 30 is configured to display multiple virtual objects on the AR device according to the first relative pose data of each virtual object and the AR device and the target joint display data.
  • the display device of the present disclosure further includes:
  • the auxiliary data determination module is configured to, for each virtual object, during the display process of the virtual object, identify the second relative pose data of the object in the world coordinate system according to the virtual object and the target corresponding to the virtual object, determining targeted assistive display data for the virtual object;
  • the first display module 30 is configured to display multiple virtual objects on the AR device according to the first relative pose data of each virtual object and the AR device, target joint display data, and target auxiliary display data of each virtual object.
  • the auxiliary data determination module is configured to:
  • the first pose data of the virtual object in the world coordinate system and the second pose data of the target identification object corresponding to the virtual object in the world coordinate system are acquired;
  • the target auxiliary display data corresponding to the second relative pose data is determined.
  • the second relative pose data includes a relative position and/or a relative pose between the virtual object and a target identification object corresponding to the virtual object.
  • the preset auxiliary display data includes one or more data of the virtual object's color, brightness, transparency, or display angle.
  • the joint data determination module 10 is configured to:
  • Multiple target identification objects in the current scene image are determined by comparing the feature information corresponding to at least one feature point with the pre-stored feature point information of the identification objects.
  • the joint data determination module 10 is configured to:
  • the target joint display data is determined.
  • the first pose determination module 20 is configured to:
  • the first initial pose of each virtual object in the world coordinate system and the second initial pose of the AR device in the world coordinate system determine the initial relative pose of each virtual object and the AR device in the world coordinate system.
  • An embodiment of the present disclosure provides an AR device, including:
  • the memory is communicably connected to the processor, and the memory stores computer instructions that can be read by the processor, and the computer instructions are used to make the processor execute the method according to any implementation manner of the first aspect.
  • Embodiments of the present disclosure provide a storage medium storing computer instructions, which are used to make a computer execute the method according to any embodiment of the first aspect.
  • This disclosure relates to the field of augmented reality.
  • acquiring the image information of the target object in the real environment and then using various visual correlation algorithms to detect or identify the relevant features, states and attributes of the target object, and thus obtain the image information that matches the specific application.
  • AR effect combining virtual and reality.
  • the target object may involve faces, limbs, gestures, actions, etc. related to the human body, or markers and markers related to objects, or sand tables, display areas or display items related to venues or places.
  • Vision-related algorithms can involve visual positioning, SLAM, 3D reconstruction, image registration, background segmentation, object key point extraction and tracking, object pose or depth detection, etc.
  • Specific applications can not only involve interactive scenes such as guided tours, navigation, explanations, reconstructions, virtual effect overlays and display related to real scenes or objects, but also special effects processing related to people, such as makeup beautification, body beautification, special effect display, virtual Interactive scenarios such as model display.
  • the relevant features, states and attributes of the target object can be detected or identified through the convolutional neural network.
  • the above-mentioned convolutional neural network is a neural network model obtained by performing model training based on a deep learning framework.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Graphics (AREA)
  • Computer Hardware Design (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Processing Or Creating Images (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

本公开涉及一种增强现实AR场景的展示方法，包括响应于从当前场景图像中识别到多个目标标识对象，根据多个目标标识对象所对应的虚拟对象，确定针对多个虚拟对象的目标联合展示数据；根据各个虚拟对象与AR设备的初始相对位姿，确定各个虚拟对象与AR设备的在世界坐标系中的第一相对位姿数据；根据第一相对位姿数据和目标联合展示数据，在AR设备展示多个虚拟对象。

Description

增强现实场景的展示
相关申请的交叉引用
本申请要求在2021年12月31日提交至中国专利局、申请号为CN2021116671519的中国专利申请的优先权，其全部内容通过引用结合在本公开中。
技术领域
本公开涉及增强现实技术领域，具体涉及一种增强现实场景的展示。
背景技术
增强现实(Augmented Reality，AR)技术通过将虚拟模型叠加到真实世界中，从而将真实的环境和虚拟的对象实时地在同一个画面或者空间呈现。对于AR场景，如何丰富虚拟对象与真实场景的交互，优化展示效果是主要的研究方向之一。
发明内容
本公开实施方式提供了一种增强现实AR场景的展示方法、装置、AR设备以及存储介质。
第一方面，本公开实施方式提供了一种增强现实AR场景的展示方法，所述方法包括：响应于从当前场景图像中识别到多个目标标识对象，根据所述多个目标标识对象所分别对应的虚拟对象，确定针对多个所述虚拟对象的目标联合展示数据；根据各个所述虚拟对象与AR设备在世界坐标系中的初始相对位姿，确定各个所述虚拟对象与所述AR设备在世界坐标系中的第一相对位姿数据；根据各个所述虚拟对象与所述AR设备的所述第一相对位姿数据和所述目标联合展示数据，在所述AR设备展示多个所述虚拟对象。
第二方面，本公开实施方式提供了一种增强现实AR场景的展示装置，所述装置包括：联合数据确定模块，被配置为响应于从当前场景图像中识别到多个目标标识对象，根据所述多个目标标识对象所分别对应的虚拟对象，确定针对多个所述虚拟对象的目标联合展示数据；第一位姿确定模块，被配置为根据各个所述虚拟对象与AR设备在世界坐标系中的初始相对位姿，确定各个所述虚拟对象与所述AR设备在世界坐标系中的第一相对位姿数据；第一展示模块，被配置为根据各个所述虚拟对象与所述AR设备的所述第一相对位姿数据和所述目标联合展示数据，在所述AR设备展示多个所述虚拟对象。
第三方面，本公开实施方式提供了一种增强现实AR设备，包括：处理器；和存储器，与所述处理器可通信连接，所述存储器存储有能够被所述处理器读取的计算机指令，所述计算机指令用于使所述处理器执行根据第一方面任一实施方式所述的方法。
第四方面，本公开实施方式提供了一种存储介质，存储有计算机指令，所述计算机指令用于使计算机执行根据第一方面任一实施方式所述的方法。
本公开实施方式的展示方法，在识别到多个目标标识对象时，可以使多个虚拟对象互动，呈现联合展示效果，提高虚拟形象的展示效果。并且，基于SLAM方式展示虚拟对象，每个虚拟对象不局限于在固定的展示位置展示，使得多个虚拟对象之间的联合展示效果更加逼真，提高用户体验。
附图说明
为了更清楚地说明本公开具体实施方式或相关技术中的技术方案，下面将对具体实施方式或相关技术的描述中所需要使用的附图作简单地介绍，下面描述中的附图是本公开的一些实施方式，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1是根据本公开一些实施方式中AR设备的结构示意图。
图2a和2b是根据本公开一些实施方式中AR设备的结构示意图。
图3是根据本公开一些实施方式中AR设备的结构框图。
图4是根据本公开一些实施方式中展示方法的流程图。
图5是根据本公开一些实施方式中展示方法的原理图。
图6是根据本公开一些实施方式中展示方法的流程图。
图7是根据本公开一些实施方式中展示方法的流程图。
图8是根据本公开一些实施方式中展示方法的流程图。
图9是根据本公开一些实施方式中展示方法的流程图。
图10是根据本公开一些实施方式中展示方法的流程图。
图11是根据本公开一些实施方式中展示装置的结构框图。
具体实施方式
下面将结合附图对本公开的技术方案进行清楚、完整地描述，所描述的实施方式是本公开一部分实施方式，而不是全部的实施方式。基于本公开中的实施方式，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施方式，都属于本公开保护的范围。此外，下面所描述的本公开不同实施方式中所涉及的技术特征在彼此不冲突的情况下可以相互结合。
基于标识对象(Marker)进行AR效果展示是增强现实场景中对虚拟对象的一种展示方式，这种方式需要预先制作标识对象，例如基于汉明码(Hamming Code)技术绘制在物品表面的预设图形等，预设图形可以是例如卡通形象、二维码、条形码等。AR设备通过摄像头识别到当前场景中出现该Marker时，可在当前场景图像中叠加对应的虚拟对象，实现虚拟对象与现实场景叠加的展示效果。
在虚拟对象的展示过程中，针对每个标识对象，预先设置该标识对象对应的虚拟对象，当AR设备识别到该标识对象时，可在AR设备的显示屏基于预设的展示效果对该标识对象对应的虚拟对象进行展示。例如，标识对象可以是绘制在日历或明信片表面的预设图形，当AR设备识别到该预设图像时，在AR设备的显示屏上可以呈现出对应的虚拟对象叠加在该标识对象表面作出预设动作的视觉效果。对于多标识对象场景，AR设备在识别到多个标识对象的情况下，其对应的每个虚拟对象仅能在各自对应的标识对象表面进行展示，这种展示方式下的虚拟对象之间缺乏互动性，展示效果不佳。
本公开实施方式提供了一种增强现实场景的展示方法、装置、AR设备以及存储介质，旨在提高AR场景中虚拟对象的展示效果，提高用户体验。
本公开实施方式提供了一种增强现实场景的展示方法，该方法可以应用于AR设备。可以理解，本公开实施方式所述的AR设备，可以是任何具有AR功能的终端设备，例 如AR眼镜、AR头盔等头戴式设备，又例如智能手机、平板电脑等手持式设备，再例如智能手表、智能手环等腕戴式设备等等，本公开对此不作限制。
如图1所示，在一个示例中，本公开所述的AR设备以AR眼镜为例进行说明。AR设备100包括主体101，主体101包括实现佩戴及支撑的支架结构。显示屏110设置在主体101上，用户在佩戴AR设备100之后，显示屏110可以位于用户眼睛的正前方，从而便于用户观察到显示屏110上显示的画面。AR设备100还包括摄像头120，摄像头120用于采集当前场景图像。AR设备100还包括设于主体101内部的处理器，处理器用于将摄像头120采集的当前场景图像与虚拟对象进行叠加之后，在显示屏110上展示。
如图2a所示，在一个示例中，本公开所述的AR设备以智能手机为例进行说明。AR设备100包括壳体102，壳体102为实现手机主体支撑的外壳结构。显示屏110设置在壳体102正面，摄像头120设置在壳体102的正面和/或背面，例如图2b所示，摄像头120设于壳体的背面。摄像头120用于采集当前场景图像，从而智能手机的处理器将摄像头120采集的当前场景图像与虚拟对象进行叠加之后，在显示屏110上展示。
当然可以理解，AR设备的具体设备类型不局限于上述示例的AR眼镜和智能手机，本公开对此不再赘述。图3中示出了本公开AR设备的结构框图，下面结合图3对本公开实施方式的AR设备进一步说明。
如图3所示，在一些实施方式中，AR设备100包括处理器130、存储器140、摄像头120、显示屏110以及IMU(Inertial Measurement Unit，惯性测量单元)传感器160。
处理器130、存储器140、摄像头120、显示屏110以及IMU传感器160通过总线150建立任意两者之间的可通信连接。
处理器130可以为任何类型，具备一个或者多个处理核心的处理器。其可以执行单线程或者多线程的操作，用于解析指令以执行获取数据、执行逻辑运算功能以及下发运算处理结果等操作。
存储器140可包括非易失性计算机可读存储介质，例如至少一个磁盘存储器件、闪存器件、相对于处理器130远程设置的分布式存储设备或者其他非易失性固态存储器件。存储器可以具有程序存储区，用于存储非易失性软件程序、非易失性计算机可执行程序以及模块，供处理器130调用以使处理器130执行下文一个或者多个方法步骤。存储器140还可以包括易失性随机存储介质、或者硬盘等存储部分，作为数据存储区，用以存储处理器130下发输出的运算处理结果及数据。
摄像头120用于采集当前场景图像，从而处理器130可以根据采集到的当前场景图像进行针对虚拟对象的融合处理，实现AR展示效果。在本公开实施方式中，摄像头120可以是任何适于实施的摄像头类型，例如黑白摄像头、RGB摄像头等，摄像头120按照摄像头的数量可以分为单目摄像头和双目摄像头，本公开对此不作限制。
显示屏110用于接收处理器130下发的显示信息，从而根据显示信息在显示屏110上呈现出对应的显示图像，以使用户可以通过显示屏110观察到AR展示效果。在本公开实施方式中，显示屏110可以是任何适于实施的显示屏类型，例如LCD(Liquid Crystal Display，液晶显示)显示屏、OLED(Organic Light-Emitting Diode，有机发光半导体)显示屏等，本公开对此不作限制。
IMU传感器160主要用来检测和测量AR设备自身的加速度与旋转运动，其基本原理是利用惯性定律实现加速度与旋转运动的测量，可以基于传感器信号实时对AR设备自身位姿进行解算。在一些实施方式中，IMU传感器160可以包括例如三轴或六轴陀螺 仪、加速度传感器等。
基于上述AR设备结构，下面对本公开实施方式的增强现实AR场景的展示方法进行说明。
如图4所示，在一些实施方式中，本公开示例的增强现实AR场景的展示方法包括：
S410、响应于从当前场景图像中识别到多个目标标识对象，根据多个目标标识对象所分别对应的虚拟对象，确定针对多个虚拟对象的目标联合展示数据。其中，每个目标标识对象对应各自的一个虚拟对象。
具体而言，当前场景图像是指利用AR设备的摄像头实时采集到的场景图像。例如图1所示的AR眼镜中，通过摄像头120采集取景范围内的场景图像，通过数据处理将采集到的场景图像显示在显示屏110上，用户在显示屏110上实时观察到的图像即为当前场景图像。可以理解，随着AR设备的位姿发生改变，采集到的当前场景图像随之改变，也即当前场景图像为实时图像。
目标标识对象指用于触发AR效果展示的标识物，也即上文所述的Marker，目标标识对象的具体呈现方式可以是例如二维码、预设图形等，本公开对此不作限制。
本公开实施方式中当前场景图像中包括至少两个目标标识对象，从而AR设备可以从当前场景图像中识别到多个目标标识对象。
例如一个示例中，当前场景图像如图5所示，图5中包括两张明信片200，目标标识对象包括分别绘制在每张明信片200表面的卡通图形，也即卡通图形210和卡通图形220。可以理解，作为目标标识对象的卡通图形210和卡通图形220，可以基于例如汉明码技术进行绘制，从而AR设备在采集到包含该卡通图形的图像时，可以对卡通图形进行识别解析。对于汉明码绘制和解析的过程，本领域技术人员参照相关技术即可理解并充分实施，本公开对此不作限制。
例如另一个示例中，目标标识对象为绘制于物体表面的预设图形，AR设备在采集到包括该预设图形的场景图像时，可以通过图像检测技术对场景图像进行特征点提取和识别，从而可以从场景图像中识别到该预设图形。对于图像检测和预设图像的识别过程，本领域技术人员参照相关技术即可理解并充分实施，本公开对此不作限制。
可以理解，目标标识对象关联对应的虚拟对象，例如一个示例中，可以预先建立包括至少一个虚拟对象的模型数据库，模型数据库包括虚拟对象以及与虚拟对象关联的目标标识对象，从而在AR设备由当前场景图像中识别到目标标识对象时，可以通过查找模型数据库，确定与该目标标识对象对应的虚拟对象。
虚拟对象表示为在AR设备的显示屏上经渲染呈现的虚拟形象，通过将虚拟对象与显示屏上显示的真实场景图像进行融合展示，可以呈现出虚拟对象与真实场景产生互动的展示效果。
本公开实施方式中，AR设备通过图像检测技术对采集到的当前场景图像进行检测识别，在从当前场景图像中识别到目标标识对象的情况下，即可确定与目标标识对象对应的虚拟对象。
本公开实施方式中，在AR设备从当前场景图像中识别到多个目标标识对象的情况下，生成每个目标标识对象对应的虚拟对象，其中，并非将每个虚拟对象在对应的目标标识对象位置分别进行展示，而是使得多个虚拟对象之间产生联合展示效果。联合展示效果是指任何可以使多个虚拟对象产生联动的渲染效果，例如，多个虚拟对象之间的叠加、环绕、互动等。
具体而言，可以基于不同虚拟对象组合预先设置对应的联合展示效果。为了方便描述，引入联合展示关系，联合展示关系表示虚拟对象与联合展示数据之间的对应关系，联合展示数据是指使得虚拟对象产生相应联合展示效果的数据，联合展示数据可包括例如虚拟对象的位置数据和姿态数据。从而，在从当前场景图像中识别到对应的虚拟对象组合时，可以根据预先设置的联合展示关系确定相应的目标联合展示数据，在AR设备的显示屏上呈现对应的联合展示效果。
例如图5所示，AR设备从当前场景图像中可以识别到两个目标标识对象，即卡通图形210和卡通图形220。从而，AR设备可以根据卡通图形210和卡通图形220对应的虚拟对象，通过查找预先设置的联合展示关系，确定本示例中的两个虚拟对象所对应的目标联合展示数据。例如，针对卡通图形210和卡通图形220的两个虚拟对象所对应的目标联合展示数据表示“两个虚拟对象互相追逐”。可以理解，联合展示数据还可以是其他任何适于实现的展示效果的数据，只要保证两个虚拟对象之间具有联动效果即可，本公开对此不作限制。
在一些实施方式中，可以预先针对每个虚拟对象设置对应的对象类别，从而预先建立对象类别与联合展示数据的对应关系，也即联合展示关系。当AR设备识别到属于某个或多个对象类别的虚拟对象时，即可根据该联合展示关系找到对应的目标联合展示数据。本公开下述实施方式进行说明，在此暂不详述。
在本公开实施方式中，AR设备在从当前场景图像中识别到目标标识对象的情况下，可以基于同步定位与建图(Simultaneous Localization And Mapping，SLAM)技术在AR设备上展示虚拟对象的联合展示效果。SLAM技术能够实现AR设备的空间定位和针对当前场景的地图构建，从而在AR设备显示的当前场景图像上渲染虚拟对象与现实场景叠加的展示效果，使虚拟对象的展示效果更加逼真。下面结合S420～S430进行具体说明。
S420、根据各个虚拟对象与AR设备的在世界坐标系中的初始相对位姿，确定各个虚拟对象与AR设备的在世界坐标系中的第一相对位姿数据。
S430、根据各个虚拟对象的与AR设备第一相对位姿数据和目标联合展示数据，在AR设备展示多个虚拟对象。
具体而言，在基于SLAM进行虚拟对象展示时，需要将虚拟对象与AR设备对应到同一坐标系中，从而在两者发生相对运动时，AR设备可以利用IMU传感器130实时解算确定两者的实时相对位姿，该坐标系即本公开所述的世界坐标系。
在一些实施方式中，可以基于图像检测技术预先对目标标识对象进行特征提取，得到目标标识对象的至少一个预设关键点，基于至少一个预设关键点建立世界坐标系。本公开下文中对此进行说明，在此暂不详述。
在对虚拟对象进行展示时，首先确定每个虚拟对象在世界坐标系中的初始位姿，也即第一初始位姿，第一初始位姿可以包括虚拟对象的初始位置和初始姿态。同时，可以在AR设备拍摄第一帧场景图像时，获取AR设备在世界坐标系中的初始位姿，也即第二初始位姿，第二初始位姿同样可以包括AR设备的初始位置和初始姿态。在一些实施方式中，可以将，AR设备100在拍摄第一帧场景图像时，IMU传感器160的位姿确定为AR设备在世界坐标系中的初始位姿。
在同一世界坐标系下，针对每个虚拟对象，确定该虚拟对象的第一初始位姿和AR设备的第二初始位姿之后，即可根据AR设备的初始位姿与该虚拟对象的初始位姿得到虚拟对象与AR设备的初始相对位姿。例如，根据该虚拟对象与AR设备的位置差确定两者的相对位置，根据该虚拟对象与AR设备的初始姿态差确定两者的相对姿态。
在基于SLAM进行虚拟对象的AR效果展示时，随着AR设备的移动和/或转动，IMU传感器160可以检测到AR设备100的位姿(位置和姿态)发生变化。
例如，陀螺仪可以根据AR设备在三轴方向上与初始姿态的角度偏差，确定AR设备位姿变化后的姿态，加速度传感器可以检测到AR设备位姿变化过程中的移动加速度，通过惯性导航解算方法得到移动后的位置，从而确定AR设备移动后的位姿数据。
在确定AR设备移动和/或转动后的位姿数据后，据此对前述的初始相对位姿进行更新，即可得到AR设备更新后的位姿与虚拟对象的位姿之间的相对位姿数据，也即第一相对位姿数据。
对于利用SLAM算法确定虚拟对象与AR设备相对位姿数据的过程，本领域技术人员根据相关技术可以理解并充分实施，本公开对此不再赘述。
在虚拟对象的AR效果展示时，为了将虚拟对象在AR设备100的显示屏110上显示，需要预先建立虚拟对象所在的世界坐标系与AR设备的屏幕坐标系之间的映射关系。例如一些实施方式中，可以将世界坐标系通过旋转和平移处理对应到AR设备的相机坐标系，然后从相机坐标系映射到AR设备的屏幕坐标系中。
对于世界坐标系与屏幕坐标系的对齐处理过程，本领域技术人员参照相关技术可以理解并充分实施，本公开对此不再赘述。
AR设备在实时确定自身与每个虚拟对象的第一相对位姿数据之后，即可将第一相对位姿数据根据上述映射关系，映射至AR设备的屏幕坐标系中，确定每个虚拟对象在屏幕坐标系中的展示位姿数据。然后AR设备根据每个虚拟对象的展示位姿数据以及前述确定的目标联合展示数据，在AR设备的显示屏上渲染展示各个虚拟对象，使得用户可以观看到实时的多虚拟对象的联合展示效果及位姿变化。
在一个例子中，用户佩戴例如图1所示的AR眼镜观看例如图5所示的场景。目标联合展示数据表示“两个虚拟对象互相追逐”，从而AR设备可根据目标联合展示数据在显示屏上渲染两个虚拟对象，使两个虚拟对象呈现互相追逐的展示效果。同时，AR设备随用户远离初始位置时，AR设备基于SLAM技术可以确定自身与每个虚拟对象在世界坐标系中的相对距离的变化，当AR设备与虚拟对象的相对距离增大，在显示屏上渲染的虚拟对象的尺寸可逐渐缩小；反之，当AR设备与虚拟对象的相对距离减小，在显示屏上渲染的虚拟对象的尺寸可逐渐增大。从而，使得虚拟对象的渲染效果符合人眼在真实场景中观看时的“远小近大”，因此虚拟对象的展示更加逼真。
通过上述可知，本公开实施方式中，在AR效果展示过程中，在识别到多个目标标识对象时，可以使多个虚拟对象之间产生互动，呈现联合展示效果，增加AR场景的互动性，提高虚拟对象的展示效果。并且，本公开实施方式中基于SLAM方式展示虚拟对象，每个虚拟对象不局限在固定的展示位置展示，使得多个虚拟对象之间的联合展示效果更加逼真，提高用户体验。
本公开实施方式中，在AR效果展示时，AR设备可以从采集到的当前场景图像中识别出一个或多个目标标识对象。如图6所示，在一些实施方式中，从当前场景图像中识别目标标识对象的过程包括：
S610、对当前场景图像进行特征点提取，得到当前场景图像包括的至少一个特征点对应的特征信息。
S620、将至少一个特征点对应的特征信息与预先存储的目标标识对象的特征点信息进行对比，确定所述当前场景图像中的所述目标标识对象。
具体而言，AR设备100可以通过摄像头120采集到当前场景图像，可以理解，当 前场景图像可以是摄像头采集到的单帧图像，也可以是摄像头采集到的视频流中的多帧图像，本公开对此不作限制。
在对当前场景图像进行处理的过程中，可以通过图像检测算法，定位出当前场景图像中包括待测对象的检测区域，然后在检测区域中提取得到待测对象的一个或多个特征点。
以当前场景图像为图5为例，可以通过图像检测算法，首先确定两张明信片200区域为待测区域，然后对两张明信片200上包括的卡通图形210和卡通图形220等待测对象进行特征点提取，得到每个待测对象的特征信息。
在得到各个待测对象的特征信息之后，可以将待测对象的特征信息与预先存储的目标标识对象的特征点信息进行对比，若两者相似度大于预设阈值，则表示待测对象即为目标标识对象；反之，若相似度不大于预设阈值，表示待测对象不是目标标识对象。
仍以图5为例，假设预先存储的目标标识对象的特征点信息对应卡通图形210和卡通图形220，从而将当前场景图像中提取到的卡通图形的特征信息与预先存储的目标标识对象的特征点信息进行对比，两者相似度大于预设阈值，从而确定当前场景图像中识别到目标标识对象。
示例性地，本公开实施方式所述的特征点的信息可以包括特征点对应的纹理特征值、RGB特征值、灰度值等能够标识该特征点特征的信息，本公开对此不作限制。
本公开实施方式中，在从当前场景图像中识别到多个目标标识对象时，即可进一步获取每个目标标识对象所对应的虚拟对象，并且根据多个虚拟对象的组合确定对应的目标联合展示数据。
在一些实施方式中，可以针对每个虚拟对象设置对应的对象类别，根据虚拟对象的对象类别确定对应的目标联合展示数据，下面结合图7实施方式进行说明。
如图7所示，在一些实施方式中，本公开示例的展示方法，确定目标联合展示数据的过程，包括：
S710、获取每个虚拟对象的对象类别。
S720、基于各虚拟对象的对象类别和预先设置的联合展示数据之间的对应关系，确定目标联合展示数据。
本公开实施方式所述的对象类别，可以包括例如“动物”、“植物”、“装饰物”等等。从而，AR设备在通过对应的目标标识对象确定虚拟对象时，可以根据该虚拟对象确定其对应的对象类别。例如，图5示例中卡通图形210和卡通图形220对应的虚拟对象的对象类别即可为“动物”。
本公开实施方式中，可以预先根据不同对象类别的组合，建立对象类别与联合展示数据的对应关系。例如一个示例中，预先建立的对应关系如下表一中所示：
表一
对象类别 联合展示数据 联合展示效果
动物+动物 数据1 互相追逐
动物+植物 数据2 植物叠加为动物的背景
动物+装饰物 数据3 装饰物围绕在动物周围
植物+植物 数据4 植物互相叠加在一起
…… …… …….
在表一示例的对应关系中，在根据当前场景图像确定的多个虚拟对象的对象类别为 “动物+动物”的情况下，对应的目标联合展示数据1所呈现的联合展示效果为“互相追逐”，例如图5示例中，所呈现的联合展示效果可以是虚拟对象“小龙人”与虚拟对象“松鼠”互相追逐。
在根据当前场景图像确定的多个虚拟对象的对象类别为“动物+植物”的情况下，对应的目标联合展示数据2所呈现的联合展示效果为“植物叠加为动物的背景”，例如，虚拟对象A为狮子，虚拟对象B为草丛，所呈现的联合展示效果可以是虚拟对象B“草丛”作为虚拟对象A狮子的背景呈现。
在根据当前场景图像确定的多个虚拟对象的对象类别为“动物+装饰物”的情况下，对应的目标联合展示数据3所呈现的联合展示效果为“装饰物围绕在动物周围”，例如，虚拟对象A为小龙人，虚拟对象B为祥云，所呈现的联合展示效果可以是虚拟对象B“祥云”围绕在虚拟对象A“小龙人”的周围。
在根据当前场景图像确定的多个虚拟对象的对象类别为“植物+植物”的情况下，对应的目标联合展示数据4所呈现的联合展示效果为“植物互相叠加在一起”，例如，虚拟对象A为草丛，虚拟对象B为花朵，所呈现的联合展示效果可以是虚拟对象B“花朵”叠加在虚拟对象A“草丛”中。
可以理解，上述对应关系仅作为本公开实施方式的一种示例，本领域技术人员还可以实现其他任何对应关系，本公开对此不再枚举。
在预先建立例如上述表一所示的联合展示关系之后，在识别到当前场景图像中包括多个虚拟对象时，即可根据多个虚拟对象的对象类别及上述联合展示关系，确定对应的目标联合展示数据。
例如图5示例中，AR设备识别到当前场景图像中包括两个虚拟对象，且两个虚拟对象的对象类别均为“动物”，通过上述表一联合展示关系，即可确定对应的目标联合展示数据为“数据1”，根据数据1在显示屏上呈现的联合展示效果即为“互相追逐”。
如图8所示，在一些实施方式中，本公开示例的展示方法，确定虚拟对象与AR设备的初始相对位姿的过程，包括：
S810、对当前场景图像进行图像检测，得到多个目标标识对象中至少之一的至少一个预设关键点。
S820、基于至少一个目标标识对象的至少一个预设关键点，建立世界坐标系。
S830、根据各个虚拟对象在世界坐标系中的第一初始位姿，和AR设备在世界坐标系中的第二初始位姿，确定各个虚拟对象与AR设备的初始相对位姿。
具体而言，以当前场景图像为图5为例，目标标识对象为绘制于明信片200表面的卡通图形，当AR设备通过前述实施方式，对当前场景图像检测识别到该场景图像包括目标标识对象时，可以基于任一目标标识对象的至少一个预设关键点，建立世界坐标系。
例如图5示例中，预设关键点可以是卡通图形210的中心点，将卡通图形210中心点作为坐标系原点O，以平行于明信片200短边且穿过原点O的方向作为X轴，以平行于明信片200长边且穿过原点O的方向作为Y轴，以垂直于X轴和Y轴且穿过原点O的方向作为Z轴，从而建立世界坐标系O-XYZ。
可以理解，图5中建立世界坐标系的过程仅作为本公开实施方式的一个示例，在其他实施方式中，还可以依据其他预设关键点建立世界坐标系，例如可以明信片200左下角的顶点作为坐标原点O建立坐标系，又例如可以卡通图形220的中心点作为坐标原点O建立坐标系等，本公开对此不作限制。
在基于SLAM进行虚拟对象展示时，首先确定每个虚拟对象在世界坐标系中的初始位姿，也即第一初始位姿，第一初始位姿可以包括各个虚拟对象的初始位置和初始姿态。同时，在建立世界坐标系之后，可以在AR设备拍摄第一帧场景图像时，获取AR设备在世界坐标系中的初始位姿，也即第二初始位姿，第二初始位姿同样可以包括AR设备的初始位置和初始姿态。在一些实施方式中，可以将，AR设备100在拍摄第一帧场景图像时，IMU传感器160的位姿确定为AR设备在世界坐标系中的初始位姿。
在同一世界坐标系下，针对每个虚拟对象，在确定该虚拟对象的第一初始位姿和AR设备的第二初始位姿之后，即可根据AR设备与该虚拟对象的初始位姿得到该虚拟对象与AR设备的初始相对位姿。例如，根据该虚拟对象与AR设备的初始位置差确定两者的相对位置数据，根据该虚拟对象与AR设备的初始姿态差确定两者的相对姿态数据，相对位置数据和相对姿态数据即组成该虚拟对象与AR设备的初始相对位姿。
在一些实施方式中，在虚拟对象的展示过程中，不仅实时检测AR设备与虚拟对象的相对位姿变化，同时检测目标标识对象与虚拟对象的相对位姿变化，从而产生不同的辅助展示效果，下面结合图9进行说明。
如图9所示，在一些实施方式中，本公开示例的展示方法，还包括：
S910、针对每个虚拟对象，在该虚拟对象的展示过程中，根据该虚拟对象与该虚拟对象对应的目标标识对象在世界坐标系中的第二相对位姿数据，确定该虚拟对象的目标辅助展示数据。
S920、根据各个所述虚拟对象与AR设备的所述第一相对位姿数据、所述目标联合展示数据以及各个所述虚拟对象的所述目标辅助展示数据，在所述AR设备展示多个虚拟对象。
本公开实施方式中，将上文所述的多个虚拟对象之间的联动效果定义为“联合展示效果”。辅助展示效果是指可以叠加在联合展示效果之上的辅助效果，例如对联合展示效果的亮度、曝光度、透明度等渲染参数进行调整所带来的视觉效果，均可以作为本公开的辅助展示效果。辅助展示数据是指使得虚拟对象产生相应辅助展示效果的数据，辅助展示数据可包括例如对虚拟对象的亮度、透明度、曝光度、角度等参数进行调整的数据。
在一个示例中，当前场景图像可如图5所示，AR设备在识别到当前场景图像(图5)中的目标标识对象(卡通图形210和卡通图形220)时，可以通过前述实施方式实现虚拟对象的联合展示效果。
在虚拟对象的联合展示过程中，可以基于图像检测技术，同时检测每个目标标识对象的位姿，可以理解，在虚拟对象的展示过程中，目标标识对象的位姿可能会发生变化，例如用户移动明信片200，使得AR设备检测到的当前场景图像中目标标识对象的位姿发生变化。
下面以上述的该虚拟对象的辅助展示效果为例进行说明。如图10所示，在一些实施方式中，本公开示例的展示方法，确定该虚拟对象的目标辅助展示数据的过程包括：
S911、在该虚拟对象的展示过程中，获取该虚拟对象在世界坐标系中的第一位姿数据，以及该虚拟对象对应的目标标识对象在世界坐标系中的第二位姿数据。
S912、根据第一位姿数据和第二位姿数据，确定第二相对位姿数据。
S913、根据第二相对位姿数据与预先设置的辅助展示数据之间的对应关系，确定与第二相对位姿数据对应的目标辅助展示数据。
本公开实施方式中，AR设备可以获取虚拟对象在世界坐标系中的第一位姿数据，同时，AR设备可以检测获取目标标识对象在世界坐标系中的第二位姿数据，例如可以基于图像检测技术确定明信片200的检测框，根据明信片200位姿变化前后的检测框差异确定当前场景图像中目标标识对象的第二位姿数据。
在得到第一位姿数据和第二位姿数据之后，即可根据第一位姿数据和第二位姿数据确定虚拟对象与目标标识对象之间的相对位姿，也即本公开所述的第二相对位姿数据。
在一些实施方式中，可以预先基于第二相对位姿数据建立第二相对位姿数据与辅助展示数据之间的对应关系，也即辅助展示关系。
在一个例子中，第二相对位姿数据包括虚拟对象与目标标识对象在世界坐标系中的相对距离，对应的辅助展示数据包括虚拟对象的透明度，从而预先建立的辅助展示关系可以如下表二所示：
表二
相对距离 透明度
L0～L1 0％
L1～L2 25％
L2～L3 50％
在表二所示的辅助展示关系中，在虚拟对象与目标标识对象的相对距离为L0～L1时，虚拟对象的目标辅助展示数据为“透明度0％”；而在虚拟对象与目标标识对象的相对距离为L1～L2时，虚拟对象的目标辅助展示数据为“透明度25％”；而在虚拟对象与目标标识对象的相对距离为L2～L3时，虚拟对象的目标辅助展示数据为“透明度50％”。也即，虚拟对象与目标标识对象的相对距离越远，虚拟对象的辅助展示效果越透明。
在另一个例子中，第二相对位姿数据包括虚拟对象与目标标识对象在世界坐标系中的相对姿态(例如相对角度)，对应的辅助展示数据包括虚拟对象的亮度，从而预先建立的辅助展示关系可以如下表三所示：
表三
相对角度 亮度
α0～α1 100％
α1～α2 75％
α2～α3 50％
在表三所示的辅助展示关系中，在虚拟对象与目标标识对象的相对角度为α0～α1时，虚拟对象的目标辅助展示数据为“亮度100％”；而在虚拟对象与目标标识对象的相对角度为α1～α2时，虚拟对象的目标辅助展示数据为“亮度75％”；而在虚拟对象与目标标识对象的相对角度为α2～α3时，虚拟对象的目标辅助展示数据为“亮度50％”。也即，虚拟对象与目标标识对象的相对角度越大，虚拟对象的亮度越低。
在又一个例子中，第二相对位姿数据同时包括虚拟对象与虚拟对象对应的目标标识对象在世界坐标系中的相对距离和相对姿态，对应的辅助展示数据同时包括虚拟对象的透明度和亮度，从而预先建立的辅助展示关系可如下表四所示：
表四
Figure PCTCN2022134853-appb-000001
Figure PCTCN2022134853-appb-000002
也即，在表四所示的辅助展示关系中，虚拟对象的透明度和亮度展示效果，同时随第二相对位姿数据发生变化。
可以理解，上述仅示出了本公开实施方式中的一些示例，在其他实施方式中，预先建立的辅助展示关系还可以包括其他的方式，例如辅助展示数据还可以包括颜色、亮度、透明度、曝光度或展示角度中的一种或者多种数据，本公开对此不作限制。
在预先建立上述辅助展示关系之后，在上述图9实施方式中，即可根据虚拟对象与目标标识对象的第二相对位姿数据，通过查找上述辅助展示关系，找到当前第二相对位姿数据所对应的目标辅助展示数据。在确定目标辅助展示数据之后，即可基于目标辅助展示数据和前述的目标联合展示数据渲染展示各个虚拟对象，以使用户可以在AR设备的显示屏上观看到虚拟对象呈现出联合展示效果的同时还具有辅助展示效果。
通过上述可知，本公开实施方式中，在AR效果展示过程中，在识别到多个目标标识对象时，可以使对应的多个虚拟对象之间产生互动，呈现联合展示效果，增加AR场景的互动性，提高虚拟形象的展示效果。并且，本公开实施方式中基于SLAM方式展示虚拟对象，每个虚拟对象不局限于在固定的展示位置展示，使得多个虚拟对象之间的联合展示效果更加逼真，提高用户体验。另外，可以根据目标标识对象与虚拟对象的相对位姿变化，实时展示不同的辅助展示效果，从而增加AR场景的互动性，提高虚拟形象的展示效果。
本公开实施方式提供了一种增强现实AR场景的展示装置，该装置可应用于AR设备。在一些实施方式中，AR设备可以是例如图1所示的AR眼镜。在另一些实施方式中，AR设备可以是例如图2和图2b所示的智能手机。本公开对此不作限制。
如图11所示，在一些实施方式中，本公开示例的增强现实AR场景的展示装置，包括：
联合数据确定模块10，被配置为响应于从当前场景图像中识别到多个目标标识对象，根据多目标标识对象所对应的虚拟对象，确定针对多个虚拟对象的目标联合展示数据；
第一位姿确定模块20，被配置为根据各个虚拟对象与AR设备的在世界坐标系中的初始相对位姿，确定各个虚拟对象与AR设备的在世界坐标系中的第一相对位姿数据；世界坐标系根据预先提取的任一目标标识对象的特征确定的；
第一展示模块30，被配置为根据各个虚拟对象与AR设备的第一相对位姿数据和目标联合展示数据，在AR设备展示多个虚拟对象。
通过上述可知，本公开实施方式中，在AR效果展示过程中，在识别到多个目标标识对象时，可以使多个虚拟对象互动，呈现联合展示效果，增加AR场景的互动性，提高虚拟形象的展示效果。并且，本公开实施方式中基于SLAM方式展示虚拟对象，每个虚拟对象不局限在固定的展示位置展示，使得多个虚拟对象之间的联合展示效果更加逼真，提高用户体验。
在一些实施方式中，本公开的展示装置，还包括：
辅助数据确定模块，被配置为，针对每个虚拟对象，在该虚拟对象的展示过程中， 根据该虚拟对象与该虚拟对象对应的目标标识对象在世界坐标系中的第二相对位姿数据，确定该虚拟对象的目标辅助展示数据；
第一展示模块30被配置为根据各个虚拟对象与AR设备的第一相对位姿数据、目标联合展示数据以及各个虚拟对象的目标辅助展示数据，在AR设备展示多个虚拟对象。
在一些实施方式中，辅助数据确定模块，被配置为：
在该虚拟对象的展示过程中，获取该虚拟对象在世界坐标系中的第一位姿数据，以及该虚拟对象对应的目标标识对象在世界坐标系中的第二位姿数据；
根据第一位姿数据和第二位姿数据，确定第二相对位姿数据；
根据第二相对位姿数据与预先设置的辅助展示数据之间的对应关系，确定与第二相对位姿数据对应的目标辅助展示数据。
在一些实施方式中，第二相对位姿数据包括该虚拟对象与该虚拟对象对应的目标标识对象之间的相对位置和/或相对姿态。
在一些实施方式中，预先设置的辅助展示数据包括该虚拟对象的颜色、亮度、透明度或展示角度中的一种或多种数据。
在一些实施方式中，联合数据确定模块10，被配置为：
对当前场景图像进行特征点提取，得到当前场景图像包括的至少一个特征点对应的特征信息；
通过将至少一个特征点对应的特征信息与预先存储的标识对象的特征点信息进行对比，确定当前场景图像中的多个目标标识对象。
在一些实施方式中，联合数据确定模块10，被配置为：
获取每个虚拟对象的对象类别；
基于各虚拟对象的对象类别与预先设置的联合展示数据之间的对应关系，确定目标联合展示数据。
在一些实施方式中，第一位姿确定模块20，被配置为：
对当前场景图像进行图像检测，得到多个目标标识对象中至少之一的至少一个的预设关键点；
基于至少一个预设关键点，建立世界坐标系；
根据各个虚拟对象在世界坐标系中的第一初始位姿，和AR设备在世界坐标系中的第二初始位姿，确定各个虚拟对象与AR设备的在世界坐标系中的初始相对位姿。
通过上述可知，本公开实施方式中，在AR效果展示过程中，在识别到多个目标标识对象时，可以使多个虚拟对象互动，呈现联合展示效果，增加AR场景的互动性，提高虚拟形象的展示效果。并且，本公开实施方式中基于SLAM方式展示虚拟对象，每个虚拟对象不局限在固定的展示位置展示，使得多个虚拟对象之间的联合展示效果更加逼真，提高用户体验。另外，可以根据目标标识对象与虚拟对象的相对位姿变化，实时展示不同的辅助展示效果，从而增加AR场景的互动性，提高虚拟形象的展示效果。
本公开实施方式提供了一种AR设备，包括：
处理器；和
存储器，与处理器可通信连接，存储器存储有能够被处理器读取的计算机指令，计 算机指令用于使处理器执行根据第一方面任一实施方式的方法。
本公开实施方式提供了一种存储介质，存储有计算机指令，计算机指令用于使计算机执行根据第一方面任一实施方式的方法。
具体而言，对于本公开示例的AR设备和存储介质相关实现方式，本领域技术人员参照前述实施方式可以理解并充分实施，本公开对此不再赘述。
本公开涉及增强现实领域，通过获取现实环境中的目标对象的图像信息，进而借助各类视觉相关算法实现对目标对象的相关特征、状态及属性进行检测或识别处理，从而得到与具体应用匹配的虚拟与现实相结合的AR效果。示例性的，目标对象可涉及与人体相关的脸部、肢体、手势、动作等，或者与物体相关的标识物、标志物，或者与场馆或场所相关的沙盘、展示区域或展示物品等。视觉相关算法可涉及视觉定位、SLAM、三维重建、图像注册、背景分割、对象的关键点提取及跟踪、对象的位姿或深度检测等。具体应用不仅可以涉及跟真实场景或物品相关的导览、导航、讲解、重建、虚拟效果叠加展示等交互场景，还可以涉及与人相关的特效处理，比如妆容美化、肢体美化、特效展示、虚拟模型展示等交互场景。可通过卷积神经网络，实现对目标对象的相关特征、状态及属性进行检测或识别处理。上述卷积神经网络是基于深度学习框架进行模型训练而得到的神经网络模型。
上述实施方式仅仅是为清楚地说明所作的举例，而并非对实施方式的限定。对于所属领域的普通技术人员来说，在上述说明的基础上还可以做出其它不同形式的变化或变动。这里无需也无法对所有的实施方式予以穷举。而由此所引伸出的显而易见的变化或变动仍处于本公开创造的保护范围之中。

Claims (11)

  1. 一种增强现实AR场景的展示方法，其特征在于，所述方法包括：
    响应于从当前场景图像中识别到多个目标标识对象，根据所述多个目标标识对象所分别对应的虚拟对象，确定针对多个所述虚拟对象的目标联合展示数据；
    根据各个所述虚拟对象与AR设备的在世界坐标系中的初始相对位姿，确定各个所述虚拟对象与所述AR设备的在所述世界坐标系中的第一相对位姿数据；
    根据各个所述虚拟对象与所述AR设备的所述第一相对位姿数据和所述目标联合展示数据，在所述AR设备展示多个所述虚拟对象。
  2. 根据权利要求1所述的展示方法，其特征在于，还包括：
    针对每个所述虚拟对象，在该虚拟对象的展示过程中，根据该虚拟对象与该虚拟对象对应的所述目标标识对象在所述世界坐标系中的第二相对位姿数据，确定该虚拟对象的目标辅助展示数据；
    所述根据各个所述虚拟对象与所述AR设备的所述第一相对位姿数据和所述目标联合展示数据，在所述AR设备展示多个所述虚拟对象，包括：
    根据各个所述虚拟对象与所述AR设备的所述第一相对位姿数据、所述目标联合展示数据以及各个所述虚拟对象的所述目标辅助展示数据，在所述AR设备展示多个所述虚拟对象。
  3. 根据权利要求2所述的展示方法，其特征在于，根据该虚拟对象与该虚拟对象对应的所述目标标识对象在所述世界坐标系中的第二相对位姿数据，确定该虚拟对象的所述目标辅助展示数据，包括：
    在该虚拟对象的展示过程中，获取该虚拟对象在所述世界坐标系中的第一位姿数据，以及该虚拟对象对应的所述目标标识对象在所述世界坐标系中的第二位姿数据；
    根据所述第一位姿数据和所述第二位姿数据，确定所述第二相对位姿数据；
    根据所述第二相对位姿数据与预先设置的辅助展示数据之间的对应关系，确定与所述第二相对位姿数据对应的所述目标辅助展示数据。
  4. 根据权利要求3所述的展示方法，其特征在于，
    所述第二相对位姿数据包括该虚拟对象与该虚拟对象对应的所述目标标识对象之间的相对位置和/或相对姿态。
  5. 根据权利要求3所述的展示方法，其特征在于，
    所述预先设置的辅助展示数据包括该虚拟对象的颜色、亮度、透明度或展示角度中的一种或多种数据。
  6. 根据权利要求1至5任一项所述的展示方法，其特征在于，从所述当前场景图像中识别所述多个目标标识对象，包括：
    对所述当前场景图像进行特征点提取，得到所述当前场景图像包括的至少一个特征点对应的特征信息；
    通过将所述至少一个特征点对应的特征信息与预先存储的标识对象的特征点信息进行对比，确定所述当前场景图像中的所述多个目标标识对象。
  7. 根据权利要求1至6任一项所述的展示方法，其特征在于，根据所述多个目标标识对象所分别对应的虚拟对象，确定针对多个所述虚拟对象的所述目标联合展示数据，包括：
    获取每个所述虚拟对象的对象类别；
    基于各所述虚拟对象的对象类别与预先设置的联合展示数据之间的对应关系，确定所述目标联合展示数据。
  8. 根据权利要求1至7任一项所述的展示方法，其特征在于，还包括：
    对所述当前场景图像进行图像检测，得到所述多个目标标识对象中至少之一的至少一个预设关键点；
    基于所述至少一个预设关键点，建立所述世界坐标系；
    根据各个所述虚拟对象在所述世界坐标系中的第一初始位姿，和所述AR设备在所述世界坐标系中的第二初始位姿，确定各个所述虚拟对象与所述AR设备在所述世界坐标系中的所述初始相对位姿。
  9. 一种增强现实AR场景的展示装置，其特征在于，所述装置包括：
    展示数据确定模块，被配置为响应于从当前场景图像中识别到多个目标标识对象，根据所述多个目标标识对象所分别对应的虚拟对象，确定针对多个所述虚拟对象的目标联合展示数据；
    第一位姿确定模块，被配置为根据各个所述虚拟对象与AR设备的在世界坐标系中的初始相对位姿，确定各个所述虚拟对象与所述AR设备的在所述世界坐标系中的第一相对位姿数据；
    第一展示模块，被配置为根据各个所述虚拟对象与所述AR设备的所述第一相对位姿数据和所述目标联合展示数据，在所述AR设备展示多个所述虚拟对象。
  10. 一种AR设备，其特征在于，包括：
    处理器；和
    存储器，与所述处理器可通信连接，所述存储器存储有能够被所述处理器读取的计算机指令，所述计算机指令用于使所述处理器执行根据权利要求1至8任一项所述的方法。
  11. 一种存储介质，其特征在于，存储有计算机指令，所述计算机指令用于使计算机执行根据权利要求1至8任一项所述的方法。
PCT/CN2022/134853 2021-12-31 2022-11-29 增强现实场景的展示 WO2023124698A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111667151.9 2021-12-31
CN202111667151.9A CN114299262A (zh) 2021-12-31 2021-12-31 增强现实ar场景的展示方法及装置

Publications (1)

Publication Number Publication Date
WO2023124698A1 true WO2023124698A1 (zh) 2023-07-06

Family

ID=80972813

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/134853 WO2023124698A1 (zh) 2021-12-31 2022-11-29 增强现实场景的展示

Country Status (2)

Country Link
CN (1) CN114299262A (zh)
WO (1) WO2023124698A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114299262A (zh) * 2021-12-31 2022-04-08 北京绵白糖智能科技有限公司 增强现实ar场景的展示方法及装置

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10380803B1 (en) * 2018-03-26 2019-08-13 Verizon Patent And Licensing Inc. Methods and systems for virtualizing a target object within a mixed reality presentation
CN111610998A (zh) * 2020-05-26 2020-09-01 北京市商汤科技开发有限公司 Ar场景内容的生成方法、展示方法、装置及存储介质
CN112348969A (zh) * 2020-11-06 2021-02-09 北京市商汤科技开发有限公司 增强现实场景下的展示方法、装置、电子设备及存储介质
CN112991551A (zh) * 2021-02-10 2021-06-18 深圳市慧鲤科技有限公司 图像处理方法、装置、电子设备和存储介质
CN114299262A (zh) * 2021-12-31 2022-04-08 北京绵白糖智能科技有限公司 增强现实ar场景的展示方法及装置
CN114332429A (zh) * 2021-12-31 2022-04-12 北京绵白糖智能科技有限公司 增强现实ar场景的展示方法及装置

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10380803B1 (en) * 2018-03-26 2019-08-13 Verizon Patent And Licensing Inc. Methods and systems for virtualizing a target object within a mixed reality presentation
CN111610998A (zh) * 2020-05-26 2020-09-01 北京市商汤科技开发有限公司 Ar场景内容的生成方法、展示方法、装置及存储介质
CN112348969A (zh) * 2020-11-06 2021-02-09 北京市商汤科技开发有限公司 增强现实场景下的展示方法、装置、电子设备及存储介质
CN112991551A (zh) * 2021-02-10 2021-06-18 深圳市慧鲤科技有限公司 图像处理方法、装置、电子设备和存储介质
CN114299262A (zh) * 2021-12-31 2022-04-08 北京绵白糖智能科技有限公司 增强现实ar场景的展示方法及装置
CN114332429A (zh) * 2021-12-31 2022-04-12 北京绵白糖智能科技有限公司 增强现实ar场景的展示方法及装置

Also Published As

Publication number Publication date
CN114299262A (zh) 2022-04-08

Similar Documents

Publication Publication Date Title
US9779512B2 (en) Automatic generation of virtual materials from real-world materials
WO2023124693A1 (zh) 增强现实场景的展示
US10295826B2 (en) Shape recognition device, shape recognition program, and shape recognition method
US11127210B2 (en) Touch and social cues as inputs into a computer
US10176636B1 (en) Augmented reality fashion
CN105027033B (zh) 用于选择扩增现实对象的方法、装置和计算机可读媒体
US9224237B2 (en) Simulating three-dimensional views using planes of content
US9041734B2 (en) Simulating three-dimensional features
KR20180108709A (ko) 사용자의 현실적인 신체 모델에 가상으로 옷을 입혀보는 방법
US9979946B2 (en) I/O device, I/O program, and I/O method
EP3695381B1 (en) Floor detection in virtual and augmented reality devices using stereo images
US9906778B2 (en) Calibration device, calibration program, and calibration method
US20150378159A1 (en) Display control device, display control program, and display control method
US20210312646A1 (en) Machine learning inference on gravity aligned imagery
US20190244431A1 (en) Methods, devices, and systems for producing augmented reality
WO2014128751A1 (ja) ヘッドマウントディスプレイ装置、ヘッドマウントディスプレイ用プログラム、およびヘッドマウントディスプレイ方法
WO2019204372A1 (en) R-snap for production of augmented realities
US11670157B2 (en) Augmented reality system
JP6656382B2 (ja) マルチメディア情報を処理する方法及び装置
WO2023124698A1 (zh) 增强现实场景的展示
US10171800B2 (en) Input/output device, input/output program, and input/output method that provide visual recognition of object to add a sense of distance
US9449427B1 (en) Intensity modeling for rendering realistic images
WO2023124691A1 (zh) 增强现实场景的展示
CN105894581B (zh) 一种呈现多媒体信息的方法及装置
US11763558B1 (en) Visualization of existing photo or video content

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22913956

Country of ref document: EP

Kind code of ref document: A1