WO2022151686A1 - 场景图像展示方法、装置、设备、存储介质、程序及产品 - Google Patents

场景图像展示方法、装置、设备、存储介质、程序及产品 Download PDF

Info

Publication number
WO2022151686A1
WO2022151686A1 PCT/CN2021/106934 CN2021106934W WO2022151686A1 WO 2022151686 A1 WO2022151686 A1 WO 2022151686A1 CN 2021106934 W CN2021106934 W CN 2021106934W WO 2022151686 A1 WO2022151686 A1 WO 2022151686A1
Authority
WO
WIPO (PCT)
Prior art keywords
scene
image frame
target object
overlap
image
Prior art date
Application number
PCT/CN2021/106934
Other languages
English (en)
French (fr)
Inventor
薛地
郭玉京
Original Assignee
深圳市慧鲤科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市慧鲤科技有限公司 filed Critical 深圳市慧鲤科技有限公司
Publication of WO2022151686A1 publication Critical patent/WO2022151686A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G06T19/006—Mixed reality

Definitions

  • the present application relates to the field of computer technology, and in particular, to a scene image display method, apparatus, device, storage medium, program, and product.
  • Augmented Reality is a human-computer interaction technology that simulates real scenes. Since in the AR scene, the position of the real person will change, there may be a situation where the real person overlaps with the virtual object in the AR scene. In this case, the realism of the AR scene may be reduced.
  • the embodiment of the present application proposes a technical solution for displaying a scene image.
  • a method for displaying a scene image including: acquiring a live video stream; identifying a first image frame of the live video stream, and determining a target object in the first image frame and the target object a first position in the augmented reality AR scene; determining whether there is an overlap between the target object and the AR object based on the first position and the second position of the AR object in the AR scene; In the case where there is overlap between the target object and the AR object, determine a third position of the AR object in the AR scene, and the third position does not overlap with the first position; based on the third position position, showing the first AR scene image corresponding to the first image frame.
  • the AR object and the target object do not overlap; and when the first AR scene image is multiple frames, it can also be presented
  • the AR object moves from the second position to the third position, so as to improve the overlapping and staggered phenomenon between the target object and the AR object in the AR scene, and enhance the sense of reality, immersion and interaction of the AR scene.
  • the first image frame of the live video stream is identified, and the target object in the first image frame and the first image frame of the target object in the augmented reality AR scene are determined.
  • the location includes: performing human body recognition on the first image frame, and determining the human body area where the target object is located in the first image frame; determining the depth value of the target object in the AR scene based on the human body area , the first position includes the depth value and the human body area.
  • the human body region and depth value can be accurately determined, so that whether the target object and the AR object overlap can be accurately determined based on the accurate human body region and depth value.
  • the second position of the AR object includes a depth interval of the AR object in the AR scene and an AR object area of the AR object in the AR scene. Determining whether there is overlap between the target object and the AR object between the first position and the second position of the AR object in the AR scene includes: determining whether the depth value is within a depth interval, and determining whether the depth value is within a depth interval. whether there is an overlapping area between the human body area and the AR object area; if the depth value is within the depth interval and there is an overlapping area between the human body area and the AR object area, determine the There is an overlap between the first position and the second position, and it is determined that there is an overlap between the target object and the AR object. According to the embodiment of the present application, based on the first position and the second position, it can be accurately and effectively determined whether there is an overlap between the target object and the AR object.
  • the method further includes: identifying a second image frame, determining a fourth position of the target object in the second image frame in the AR scene, and the second image
  • the frame includes one or more image frames in the live video stream before the first image frame; the target is determined based on the first position and the second position of the AR object in the AR scene Whether there is overlap between the object and the AR object, further including: there is overlap between the first position and the second position, and there is an overlap between the fourth position and the second position Next, it is determined that there is an overlap between the target object and the AR object.
  • the target object and the AR object there is overlap between the target object and the AR object in multiple image frames, that is, when there is overlap between the target object and the AR object for a certain period of time, it is possible to determine the distance between the target object and the AR object.
  • There is an effective overlap thereby reducing the frequency of moving AR objects and reducing the computational and operational stress of AR devices.
  • the method further includes: identifying a third image frame, determining a fifth position of the target object in the third image frame in the AR scene, and the third image
  • the frame includes an image frame after the first image frame in the live video stream; in the case of an overlap between the fifth position and the second position, based on the third position and a preset A movement track is used to determine a sixth position of the AR object in the AR scene; and based on the sixth position, a second AR scene image corresponding to the third image frame is displayed.
  • the interactive effect of the AR object moving along the movement track can be presented, thereby improving the interactive feeling of the AR scene.
  • the movement track includes a circle with the second position as a center and a distance between the third position and the second position as a radius. According to the embodiment of the present application, the circular movement trajectory can better realize the movement effect of the AR object.
  • the method further includes: identifying a fourth image frame, determining a seventh position of the target object in the fourth image frame in the AR scene, and the fourth image frames include image frames in the live video stream that follow the first image frame; in the case that the seventh position and the second position do not overlap, based on the seventh position and the second position position, showing the third AR scene image corresponding to the fourth image frame.
  • the AR object can be moved to the second position under the condition that the target object does not overlap the AR object in the second position, that is, the AR object can be moved back to the initial position; and in the third AR object
  • a dynamic moving effect can be presented, thereby enhancing the realism and interaction of the AR scene.
  • the method further includes: in the case where there is no overlap between the target object and the AR object, based on the first position and the second position, displaying the the fourth AR scene image corresponding to the first image frame.
  • the AR scene image can be displayed based on the relative positions of the target object and the AR object without overlapping between the target object and the AR object, thereby highlighting the difference between the target object and the AR object in the AR scene. Relative positional relationship to achieve occlusion effect and enhance realism and immersion.
  • the displaying the first AR scene image corresponding to the first image frame based on the third position includes: based on the relative relationship between the third position and the first position positional relationship, the AR object is rendered in the first image frame, and the first AR scene image is generated and displayed.
  • the relative position between the target object and the AR object can be presented, thereby improving the realism and immersion of the generated AR scene image.
  • a scene image display device including: an acquisition part, configured to acquire a live video stream; a first determination part, configured to identify a first image frame of the live video stream, and determine the first image frame of the live video stream.
  • the judgment part is configured to determine the target object based on the first position and the second position of the AR object in the AR scene whether there is overlap between the target object and the AR object;
  • the second determination part is configured to determine that the AR object is in the AR scene when there is overlap between the target object and the AR object.
  • the third position of the , the third position does not overlap with the first position;
  • the first display part is configured to display the first AR scene image corresponding to the first image frame based on the third position.
  • the first determination part includes: a human body region determination sub-part, configured to perform human body recognition on the first image frame, and determine the human body where the target object is located in the first image frame an area; a depth value determination subsection, configured to determine a depth value of the target object in the AR scene based on the human body area, and the first position includes the depth value and the human body area.
  • the second position of the AR object includes a depth interval of the AR object in the AR scene and an AR object area of the AR object in the AR scene, and the determining part, including: a judgment subsection, configured to determine whether the depth value is within a depth interval, and whether there is an overlapping area between the human body area and the AR object area; a first determination subsection, configured to When the depth value is within the depth interval, and there is an overlapping area between the human body area and the AR object area, it is determined that there is an overlap between the first position and the second position; the second determiner The part is configured to determine that there is an overlap between the target object and the AR object if there is an overlap between the first position and the second position.
  • the apparatus further includes: a third determination part, configured to identify the second image frame, and determine the fourth position of the target object in the second image frame in the AR scene
  • the second image frame includes one or more image frames in the live video stream before the first image frame
  • the judging part further includes: a third determining sub-part, configured to When there is overlap between the first position and the second position, and there is overlap between the fourth position and the second position, it is determined that there is overlap between the target object and the AR object.
  • the apparatus further includes: a fourth determination part, configured to identify the third image frame, and determine the fifth position of the target object in the third image frame in the AR scene a position, the third image frame includes an image frame in the live video stream following the first image frame; a fifth determining part configured to have an overlap between the fifth position and the second position In the case of , the sixth position of the AR object in the AR scene is determined based on the third position and the preset movement track; the second display part is configured to display the same the second AR scene image corresponding to the third image frame.
  • the movement track includes a circle with the second position as a center and a distance between the third position and the second position as a radius.
  • the apparatus further includes: a sixth determination part, configured to identify the fourth image frame, and determine the seventh position of the target object in the fourth image frame in the AR scene a position, the fourth image frame includes an image frame in the live video stream after the first image frame; a third presentation part, configured so that there is no overlap between the seventh position and the second position In this case, based on the seventh position and the second position, a third AR scene image corresponding to the fourth image frame is displayed.
  • the apparatus further includes: a fourth display part, configured to, in the case where there is no overlap between the target object and the AR object, based on the first position and the In the second position, a fourth AR scene image corresponding to the first image frame is displayed.
  • the first display part specifically includes: rendering the AR object in the first image frame based on the relative positional relationship between the third position and the first position, The first AR scene image is generated and displayed.
  • an electronic device including: a processor; a memory configured to store instructions executable by the processor; wherein the processor is configured to invoke the instructions stored in the memory to execute the above method .
  • a computer-readable storage medium on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the foregoing method is implemented.
  • a computer program including computer-readable code, when the computer-readable code is executed on an electronic device, a processor in the electronic device executes instructions configured to implement the above method.
  • a computer program product includes a computer program or an instruction, and when the computer program or instruction runs on a computer, the computer is caused to execute the above method.
  • the third position of the AR object in the AR scene can be determined, and the third position does not overlap with the first position, thereby enabling In the AR scene image displayed based on the first position and the third position, the AR object and the target object do not overlap, thereby improving the overlapping and interlacing phenomenon between the target object and the AR object in the AR scene, and enhancing the realism and immersion of the AR scene sense.
  • FIG. 1 shows a flowchart of a scene image display method according to an embodiment of the present application
  • FIG. 2 shows a schematic diagram of a system architecture of a scene display method according to an embodiment of the present application
  • FIG. 3 shows a schematic diagram of an AR scene image according to the related art
  • FIG. 4 shows a schematic diagram of an AR scene image according to an embodiment of the present application
  • FIG. 5 shows a schematic diagram of an AR scene image according to an embodiment of the present application
  • FIG. 6 shows a flowchart of an AR scene image display method according to an embodiment of the present application
  • FIG. 7 shows a schematic diagram of a movement trajectory according to an embodiment of the present application.
  • FIG. 8 shows a block diagram of a scene image display apparatus according to an embodiment of the present application.
  • FIG. 9 shows a block diagram of an electronic device according to an embodiment of the present application.
  • FIG. 1 shows a flowchart of a method for displaying a scene image according to an embodiment of the present application.
  • the method for displaying a scene image includes:
  • step S10 obtain a live video stream
  • step S11 the first image frame of the live video stream is identified, and the target object in the first image frame and the first position of the target object in the AR scene are determined;
  • step S12 based on the first position and the second position of the AR object in the AR scene, determine whether there is overlap between the target object and the AR object;
  • step S13 when there is overlap between the target object and the AR object, determine the third position of the AR object in the AR scene, and the third position does not overlap the first position;
  • step S14 based on the third position, the first AR scene image corresponding to the first image frame is displayed.
  • the scene image display method may be performed by an AR device that supports AR technology.
  • the AR device may include a terminal device, and the terminal device may include a user equipment (User Equipment, UE), a mobile device, User terminals, terminals, cellular phones, cordless phones, personal digital assistants (Personal Digital Assistant, PDA), handheld devices, computing devices, in-vehicle devices, wearable devices, etc.
  • the method can be stored in the memory by calling the processor of the terminal device implemented by means of computer readable instructions.
  • the live video stream can be captured in real time by, for example, an image capture device set on the AR device; it can also be transmitted to the AR device for other devices, or the AR device can call a local
  • the photo album is obtained, which is not limited in this embodiment of the present application.
  • the first image frame may be one frame or multiple frames.
  • the multi-frame first image frame for example, may be consecutive multi-frame images in the live video stream, or may be multi-frame images obtained by interval sampling of the live video stream, which is not limited in this embodiment of the present application.
  • an image acquisition device may be provided on the AR device, and the image acquisition device can capture real-time images within the shooting field of view in real time, and use the real-time captured real-life images as a real-life video stream, It is transmitted to the processor of the AR device; the processor of the AR device generates an AR scene image based on each image frame of the acquired live video stream, and displays the AR scene image on the display interface of the AR device, so that the display interface is presented in real time AR scene image.
  • the first image frame containing the object can be determined by identifying the first image frame. After the first image frame containing the object is determined, the target object in the first image frame containing the object may be further determined.
  • the target object can be set according to actual needs, for example, it can include but is not limited to: setting all objects in the first image frame as target objects; or setting objects in the middle area in the first image frame as target objects; or setting as After identifying the object in the first image frame, the user selects the target object by himself or herself, which is not limited in this embodiment of the present application.
  • the image acquisition device may include, but is not limited to, a red, green and blue RGB (Red Green Blue) camera, or a time of flight (TOF) camera, and the like.
  • a red, green and blue RGB (Red Green Blue) camera or a time of flight (TOF) camera, and the like.
  • TOF time of flight
  • the identification of the first image frame may be to use any known human body identification technology to perform human body identification on the first image frame to determine the object in the first image frame.
  • the human body recognition technology includes, but is not limited to, technologies such as portrait segmentation and human key point detection. It can be understood that a person skilled in the art can select one or more human body recognition technologies, as long as the object recognition in the first image frame can be realized, which is not limited in this embodiment of the present application.
  • the AR scene may include a pre-built virtual scene according to actual requirements.
  • the AR scene may include AR objects such as virtual characters and virtual animals.
  • the AR scene may be constructed based on the camera coordinate system of the image acquisition device, and the position of the AR object in the AR scene may be preset based on the 3D registration technology.
  • the embodiment of the present application does not limit the content of the AR scene.
  • the AR device may store data packets corresponding to different AR scenarios; other devices (for example, cloud servers) may also transmit data packets corresponding to different AR scenarios to the AR device; by using different AR scenarios Data packages, which can display different AR scenarios in AR devices.
  • the AR scene may be determined in response to a user's selection operation on the AR scene. For example, an AR scene that can be selected by the user can be presented on the display interface of the AR device, so that the user can trigger the selection of the AR scene through operations such as clicking.
  • This embodiment of the present application does not limit the manner of determining the AR scene.
  • the AR scene may be constructed based on the camera coordinate system of the image acquisition device.
  • step S11 after the target object in the first image frame is determined, based on the imaging principle of the image acquisition device and the position of the target object in the first image frame, it is determined that the target object is in the camera coordinate system
  • the position of the target object is determined, that is, the first position of the target object in the AR scene is determined.
  • the second position of the AR object is preset, that is, the second position may be known.
  • the second position is equivalent to the default initial position of the AR object in the AR scene. In the case where there is no overlap between the target object and the AR object, the AR object may be at the initial position.
  • step S12 it may be determined, based on the camera coordinate system, based on the coordinate values corresponding to the first position and the second position, whether there is overlap between the target object and the AR object. For example, based on the coordinate values corresponding to the first position and the second position, it is determined that the first position is the same as the second position or that the first position and the second position overlap, and it is determined that there is overlap between the target object and the AR object; In the case where the first position and the second position are different or there is no overlap between the first position and the second position, it is determined that there is no overlap between the target object and the AR object. It can be set according to actual needs, which is not limited in this embodiment of the present application.
  • the third position may be another position in the AR scene that does not overlap with the first position.
  • the distance between the third position and the second position can be set according to actual needs; the orientation of the third position relative to the second position can be randomly determined without overlapping with the first position, for example, it can be
  • the azimuth is opposite to the azimuth of the first position relative to the second position; or may be an azimuth deviated by a certain angle from the azimuth of the first position relative to the second position, which is not limited in the embodiment of the present application.
  • the AR scene image can be displayed through the display interface of the AR device.
  • the AR scene image may be an image generated after rendering the special effect of the AR object based on the real scene image, where the real scene image may be any image frame in the real scene video stream.
  • the first image frame may be one or more frames.
  • the AR object at the third position may be rendered in the first image frame based on the third position of the AR object, to generate a first AR scene image, and then display the same image as the first image.
  • the first AR scene image corresponding to the frame. It can be understood that, in this case, what is visually perceived may be the effect of the AR object instantaneously moving from the second position to the third position.
  • a movement effect of the AR object moving from the second position to the third position with a certain duration may be displayed based on the multiple frames of the first image frame, It may include: based on the distance between the second position and the third position, according to a certain distance interval, determining a plurality of positions between the second position and the third position;
  • the AR object is rendered in the first image frame, and multiple first AR real-world images are generated; thus, when the multiple first AR scene images are displayed, it is possible to present the AR object moving from the second position to the third position with a certain duration of movement Effect.
  • the third position of the AR object in the AR scene can be determined, and the third position does not overlap with the first position, thereby enabling In the first AR scene image displayed based on the first position and the third position, the AR object and the target object do not overlap; and when the first AR scene image is multi-frame, it can also be shown that the AR object moves from the second position The effect of moving to the third position, thereby improving the overlapping and staggering phenomenon between the target object and the AR object in the AR scene, and enhancing the sense of reality, immersion and interaction of the AR scene.
  • step S11 the first image frame of the live video stream is identified, and the target object in the first image frame and the first position of the target object in the AR scene are determined, which may be include:
  • the depth value of the target object in the AR scene is determined, and the first position includes the depth value and the human body area.
  • performing human body recognition on the first image frame to determine the human body region where the target object is located in the first image frame may be based on a portrait segmentation technology to segment the human body region where the target object is located; or Based on the human body key point detection technology, the key points of the human body contour are detected, the human body contour is determined based on the key points of the human body contour, and the area formed by the human body contour is used as the human body area, which is not limited in the embodiment of the present application.
  • the camera of the image acquisition device may be determined based on the position of the target object in the first image frame
  • the position in the coordinate system that is, the first position of the target object in the AR scene is determined.
  • the position of the target object in the camera coordinate system may include the depth of the target object in the camera coordinate system.
  • the depth of the target object in the camera coordinate system can be understood as the distance between the target object and the image acquisition device in the real scene; the farther the distance, the greater the depth; otherwise, the smaller the depth.
  • the pixel coordinates of the key points of the human body region in the image frame and the imaging principle of the image acquisition device can be used.
  • determine the distance between the target object and the image acquisition device determines the distance between the target object and the image acquisition device; take the distance between the target object and the image acquisition device as the depth of the target object in the camera coordinate system, that is, as the depth value of the target object in the AR scene.
  • the image acquisition device may include a TOF camera.
  • the image acquisition device is a TOF camera
  • the image data collected by the TOF camera already includes the distance information between the target object in the real scene and the TOF camera
  • the target in the image is determined After the human body area of the object is obtained, the distance between the target object and the TOF camera can be obtained, that is, the depth value of the target object in the AR scene can be obtained.
  • the depth value of the target object may be the depth of the key points of the human body contour in the camera coordinate system; it may also be the depth of the key points of the human body joints in the camera coordinate system; it may also be the depth of the human body area.
  • the depth of all key points in the camera coordinate system; the number of key points may be determined according to actual requirements, the computing performance of the processor, etc., which is not limited in this embodiment of the present application.
  • the average value of the depths of multiple key points may be used as the depth value of the target object; or the minimum value and/or the maximum value among the depths of multiple key points may be used as the target object
  • the depth value of the object; or the depth of all key points in the human body area can be used as the depth value of the target object. It may be determined according to the actual application scenario, the computing performance of the processor, etc., which is not limited in this embodiment of the present application.
  • the human body area and the depth value can be accurately determined, so that whether the target object and the AR object overlap can be accurately determined based on the accurate human body area and the depth value.
  • the second position of the AR object includes the depth interval of the AR object in the AR scene and the AR object area of the AR object in the AR scene, and the AR object in the AR scene is based on the first position and the AR object in the AR scene.
  • the second position of to determine whether there is overlap between the target object and the AR object which can include:
  • the depth value is within the depth interval, and there is an overlapping area between the human body area and the AR object area, it is determined that there is an overlap between the first position and the second position, and it is determined that there is an overlap between the target object and the AR object.
  • AR objects can be three-dimensional in an AR scene, and AR objects can have multiple depths, that is, there are depth intervals.
  • the AR object area may be a stereoscopic area occupied by the AR object in the AR scene. Since AR objects are pre-built in the AR scene, the AR object area can be known.
  • the depth value of the target object may be one or more.
  • determining whether the depth value is within the depth interval may include: determining whether the depth value is within the depth interval, and in the case that the depth value is within the depth interval, determining the depth value One depth value is within the depth interval; in the case of multiple depth values, determining whether the depth value is within the depth interval may include: determining whether there is a depth value within the depth interval among the multiple depth values; When there is a depth value within the depth interval among the depth values, it may be determined that the depth value is within the depth interval. Wherein, some of the depth values (eg, the maximum value and/or the minimum value) among the plurality of depth values may be within the depth interval; or all the depth values may be within the depth interval.
  • FIG. 2 is a schematic diagram of a system architecture to which a scene image display method according to an embodiment of the present application can be applied; as shown in FIG.
  • the video capture device 2001 and the image capture terminal 2003 can establish a communication connection through the network 2002, the video capture device 2001 transmits the captured live video stream to the image capture terminal 2003 through the network 2002, and the image capture terminal 2003 receives the real video stream.
  • the video stream is analyzed, and in the case of overlap between the target object and the AR object, the third position of the AR object in the AR scene is determined, and based on the third position, the first image frame corresponding to the first image frame is displayed.
  • An AR scene image to enhance the realism and immersion of the AR scene.
  • the current scene video capture device 2001 may include an image capture device such as a camera.
  • the image acquisition terminal 2003 may include a computer device with a certain computing capability, for example, the computer device includes a terminal device or a server or other processing devices.
  • the network 2002 can be wired or wireless.
  • the video capture device 2001 when the video capture device 2001 is an image capture device and the image capture terminal 2003 is a server, the video capture device can be connected to the image capture terminal through a wired connection, such as data communication through a bus; when the video capture device 2001 image capture
  • the video acquisition device can communicate with the image acquisition terminal in a wireless connection manner, and then perform data communication.
  • the image acquisition terminal 2003 may be a vision processing device with a video acquisition module, or a host with a camera.
  • the scene image display method of the embodiment of the present application may be executed by the image acquisition terminal 2003, and the above-mentioned system architecture may not include the network 2002 and the video acquisition device 2001.
  • the positional relationship between objects may include: front and rear, left and right, and up and down.
  • the depth value of the target object is within the depth interval, and it can be considered that the target object and the AR object are coincident in the front and rear directions.
  • There is an overlapping area between the human body area and the AR object area and it can be understood that the target object and the AR object also overlap in the left and right and/or up and down directions.
  • FIG. 3 shows a schematic diagram of an AR scene image according to the related art. As shown in Figure 3, there is overlap between the target object and the AR object, that is, there is interlacing.
  • judging that there is an overlapping area between the human body area and the AR object area may be based on the coordinate values of the key points of the human body contour to determine whether there is a key point in the AR object area among the key points of the human body contour; In the case where there are key points within the AR object area among the human body contour key points, it is determined that there is an overlapping area between the human body area and the AR object area.
  • the depth value is within the depth interval, and there is no overlapping area between the human body area and the AR object area.
  • the AR object and the target object are in the front and rear directions
  • there is a certain distance between the AR object and the target object in the left-right and/or up-down directions that is, the AR object and the target object do not overlap in the left-right and/or up-down directions.
  • the AR object may also be flat in the AR scene.
  • the depth interval of the AR object also includes a depth;
  • the AR object area may include the area enclosed by the outline of the AR object.
  • the depth value is in the depth interval, which may include: the depth value of the target object has the same depth value as the depth in the depth interval.
  • the method may also include:
  • Identifying the second image frame determining the fourth position of the target object in the second image frame in the AR scene, where the second image frame includes one or more image frames in the live video stream before the first image frame;
  • the determining whether there is overlap between the target object and the AR object based on the first position and the second position of the AR object in the AR scene may further include:
  • a preset duration can be set, and when there is overlap between the target object and the AR object, and the overlapping duration exceeds the preset duration, it is considered that there is an overlap between the target object and the AR object overlapping.
  • the preset duration may be set according to actual needs, which is not limited in this embodiment of the present application.
  • the duration of the video can be the product of the number of frames and the frame rate. That is to say, under a certain frame rate, the preset duration can be reflected by the number of frames. Since the second image frame may include one or more image frames before the first image frame in the live video stream, there is an overlap between the target object and the AR object, and the overlapping duration exceeds the preset duration, which is equivalent to , in the second image frame with the number of frames corresponding to the preset duration, the target object and the AR object both overlap, and the target object and the AR object in the first image frame also overlap.
  • the first position and the second position which can be understood as an overlap between the target object and the AR object in the first image frame; there is an overlap between the fourth position and the second position Overlap, it can be understood that there is an overlap between the target object and the AR object in the second image frame; through the overlap between the first position and the second position, and the overlap between the fourth position and the second position, it can be reflected that There is an overlap between the target object and the AR object, and the overlapping duration exceeds the preset duration.
  • the number of second image frames may be determined according to a preset duration. For example, if the preset duration is 5 seconds, the number of image frames corresponding to 5 seconds can be used as the number of second image frames .
  • determining whether there is an overlap between the fourth position and the second position may be performed in the same manner as determining whether there is an overlap between the first position and the second position in the above-mentioned embodiment of the present application. This will not be repeated.
  • the AR scene image corresponding to the second image frame may also be displayed on the display interface of the AR device.
  • the user may be shown the target object and the AR objects have an overlapping effect for a certain period of time, so that after moving the AR object, the interaction effect between the target object and the AR object from overlapping to non-overlapping can be presented, and the sense of interaction in the AR scene can be improved.
  • the method may further include:
  • Identifying the third image frame determining the fifth position of the target object in the third image frame in the AR scene, and the third image frame including the image frame after the first image frame in the live video stream;
  • the third image frame is identified to determine the fifth position of the target object in the third image frame in the AR scene, and the same identification method as the above-mentioned identification of the first image frame can be used. , and the same position determination method as determining the second position, which is not repeated here.
  • determining whether there is an overlap between the fifth position and the second position may be the same as determining whether there is an overlap between the first position and the second position in the above-mentioned embodiment of the present application. This will not be repeated.
  • the preset movement track may be a track preset for controlling the movement of the AR object.
  • the movement track may be a line segment, a circle, a square, etc.
  • this application The embodiment is not limited. It can be understood that the third position of the AR object is on the movement track, so that the third position can be used as a starting point to move along the movement track.
  • the movement track may include a circle with the second position as the center and the distance between the third position and the second position as the radius.
  • the circular movement trajectory can better realize the movement effect of AR objects.
  • the sixth position of the AR object in the AR scene is determined based on the third position and the preset movement trajectory, which may be from the third position to determine a plurality of AR objects on the movement trajectory. sixth position.
  • the determination of multiple sixth positions on the movement track may be determined based on a preset movement direction and movement distance.
  • the movement direction may be clockwise or counterclockwise, and the sequence of multiple sixth positions may be determined based on the movement direction; if the length of the movement track is known, the number of sixth positions may be determined based on the movement interval.
  • the third image frame may include multiple consecutive image frames after the first image frame in the live video stream, or may be multiple frames spaced by a certain number of frames after the first image frame in the live video stream An image frame, which is not limited in this embodiment of the present application.
  • the displaying the third AR scene image corresponding to the third image frame based on the sixth position may include: based on multiple sixth positions of the AR object, displaying the third image frame in multiple frames of the third image frame Rendering a plurality of AR objects at the sixth position respectively, and generating a plurality of second AR scene images; and then displaying a plurality of second AR scene images on the display interface of the AR device, which can present the dynamic AR objects moving according to the movement trajectory. Effect.
  • the interactive effect of the AR object moving along the movement track can be presented, thereby improving the interactive feeling of the AR scene.
  • the AR object can be moved under the condition that the target object does not overlap with the AR object in the initial position (ie, the second position). Returning to the second position, it can be understood that the AR object moved back to the second position does not overlap the target object.
  • the method may further include:
  • Identifying the fourth image frame determining the seventh position of the target object in the fourth image frame in the AR scene, where the fourth image frame includes the image frame after the first image frame in the live video stream;
  • a third AR scene image corresponding to the fourth image frame is displayed based on the seventh position and the second position.
  • the fourth image frame is identified to determine the seventh position of the target object in the fourth image frame in the AR scene, and the same identification method as the above-mentioned identification of the first image frame can be used. , and the same position determination method as determining the second position, which is not repeated here.
  • judging whether there is overlap between the seventh position and the second position may be in the same manner as judging whether there is overlap between the first position and the second position in the above-mentioned embodiment of the present application, and in the This is not redundant.
  • the fourth image frame may be the same image frame as the third image frame, or may be an image frame after the third image frame.
  • the fourth image frame may be one frame; it may also be multiple consecutive frames or multiple frames sampled at intervals.
  • the AR object at the second position may be rendered into the fourth image frame based on the second position of the AR object to generate a third AR scene image, and then display the same image as the fourth image.
  • the third AR scene image corresponding to the image frame. It can be understood that, in this case, what is visually perceived may be the effect of the AR object instantaneously moving from the current position to the second position.
  • a movement effect of the AR object moving from the current position to the second position with a certain duration may be displayed based on the multiple fourth image frames.
  • the method includes: determining a plurality of positions between the current position and the second position according to a certain distance interval based on the distance between the current position and the second position;
  • the AR objects are respectively rendered in the frames to generate multiple third AR scene images; and when the multiple third AR scene images are displayed, a movement effect of the AR object moving from the current position to the second position with a certain duration is displayed.
  • the current position of the AR object may be the third position or the sixth position on the movement track, which can be determined according to whether the AR object moves according to the movement track after moving to the third position Sure.
  • the AR object can be moved to the second position under the condition that the target object does not overlap with the AR object in the second position, that is, the AR object can be moved back to the initial position.
  • the frame includes multiple frames, a dynamic moving effect can be presented, thereby enhancing the sense of realism and interaction of the AR scene.
  • the method further includes: in the case where there is no overlap between the target object and the AR object, displaying a fourth AR corresponding to the first image frame based on the first position and the second position scene image.
  • the other cases may at least include: the case where the depth value is not within the depth interval; the case where the depth value is within the depth interval, but there is no overlapping area between the human body area and the AR object area, and the like.
  • the depth value of the target object may be one or more.
  • the depth value is not within the depth range, and it may be that none of one or more depth values is within the depth range. In this case, it can be considered that there is no overlap between the target object and the AR object.
  • FIG. 4 shows a schematic diagram of an AR scene image according to an embodiment of the present application.
  • FIG. 5 shows a schematic diagram of an AR scene image according to an embodiment of the present application. As shown in Figure 4 and Figure 5, there is no overlap between the target object and the AR object, but there is occlusion.
  • the relative positional relationship between the first position of the AR object and the second position of the target object may be determined, and then the relative positional relationship between the first position of the AR object and the second position of the target object may be determined.
  • show the fourth AR scene image which can show the relative position between the AR object and the target object, and can achieve the occlusion effect.
  • the AR scene image can be displayed based on the relative positions of the target object and the AR object under the condition that there is no overlap between the target object and the AR object, so that the relationship between the target object and the AR object in the AR scene can be highlighted.
  • the relative positional relationship of to achieve occlusion effect, enhance the sense of realism and immersion.
  • the displaying the first AR scene image corresponding to the first image frame based on the third position may include:
  • the AR object is rendered in the first image frame, and the first AR scene image is generated and displayed.
  • the first position of the target object in the AR scene may include the depth value of the target object in the AR scene; the second position of the AR object in the AR scene may include the depth interval of the AR object in the AR scene.
  • the third position of the AR object in the AR scene may include a depth interval of the AR object at the third position in the AR scene.
  • the third position does not overlap with the first position, that is, the depth value of the target object is not within the depth interval at the third position.
  • the phase position relationship between the third position and the first position may include: the depth value is smaller than the minimum value in the depth interval, or the depth value is larger than the maximum value in the depth interval.
  • the depth value of the target object may be one or more.
  • the depth value is smaller than the minimum value in the depth interval, which may be that one or more depth values of the target object are all smaller than the minimum value in the depth interval.
  • the distance between the target object and the image acquisition device relative to the AR object is closer, that is, relative to the image acquisition device Device-wise, the AR object is behind the target object. Then, if the depth value is smaller than the minimum value in the depth interval, the AR object may be rendered on the back side relative to the target object in the first image frame, and the first AR scene image may be generated and displayed.
  • the depth value of the target object can be one or more.
  • the depth value is greater than the maximum value in the depth interval, which may be that one or more depth values of the target object are all greater than the maximum value in the depth interval.
  • the distance between the target object and the image acquisition device relative to the AR object is farther, that is, relative to the image As far as the acquisition device is concerned, the AR object is on the front side of the target object.
  • the AR object may be rendered on the front side relative to the target object in the first image frame, and the first AR scene image may be generated and displayed.
  • front side and the rear side in the embodiments of the present application are relative, for example, an object closer to the image acquisition device may be located on the front side of the object farther from the image acquisition device.
  • front side and the back side after reading the embodiments of the present application.
  • the identification of the first image frame may use a portrait segmentation technique.
  • the human body area and the background area in the real-life image can be separated, and then the AR object can be rendered on the back side or the background area relative to the target object based on the separated human body area and background area.
  • the first AR scene image is generated.
  • the AR object is on the rear side relative to the target object, and the target object occludes the AR object.
  • displaying the first AR scene image may be to draw the first AR scene image on the display interface of the AR group photo device for display.
  • the fourth AR scene image corresponding to the first image frame is displayed based on the first position and the second position, which may be The same manner as the display of the first AR scene image based on the third position and the first position in the embodiment of the present application is adopted. That is, in the case where the target object and the AR object do not overlap, the AR scene image can be generated and displayed based on the relative positional relationship between the current target object and the AR object.
  • the relative position between the target object and the AR object can be presented, so as to improve the realism and immersion of the generated AR scene image.
  • FIG. 6 shows a flowchart of an AR scene image display method according to an embodiment of the present application.
  • the AR scene display method includes:
  • step S601 video stream data is acquired in real time.
  • step S602 perform portrait segmentation and human depth estimation for each image frame in the video stream data.
  • step S603 the position rendering relationship between the human body and the virtual object is output in real time.
  • step S604 it is determined whether the original positions of the human body and the virtual object overlap and the overlap exceeds a certain period of time.
  • step S605 if the original positions of the human body and the virtual object overlap and the overlap exceeds a certain period of time, go to step S605; the original positions of the human body and the virtual object do not overlap or do not overlap for a certain period of time, and return to step S603.
  • step S605 when the original positions of the human body and the virtual object overlap and the overlap exceeds a certain period of time, the virtual object is triggered to move in a circle around the real human body.
  • the duration can be set independently according to actual needs.
  • step S606 it is determined whether the human body does not overlap with the virtual object in the original position.
  • step S607 the original positions of the human body and the virtual object overlap, and the process returns to step S604.
  • step S607 when the human body does not overlap with the virtual object in the original position, the virtual object returns to the original position.
  • the portrait segmentation may include: detecting a person in an image frame to obtain a human body area; and then segmenting a range where the human body is located based on key point information of the human body area.
  • the human body depth estimation may include: estimating the actual distance between the human body and the camera in the image frame in .
  • the position rendering relationship may include rendering the contextual relationship by calculating the relative position of the character and the virtual object in the screen.
  • FIG. 7 shows a schematic diagram of a movement trajectory according to an embodiment of the present application.
  • the position of the five-pointed star in the figure is the area of the virtual object
  • the position of the five-pointed star is the predetermined position (initial position) of the virtual object
  • the outer circle can be the position where the virtual object may move;
  • the virtual object is triggered to randomly transfer to the outer circle to reduce the phenomenon of overlapping;
  • the real character leaves the position of the five-pointed star the virtual object returns to its original position (ie, the initial position).
  • the AR scene display method in this embodiment of the present application can be applied to Internet APPs, such as map applications, or application products with IP images, spokespersons, etc.; functional APPs, for example, using AR in most of the products being marketed.
  • Internet APPs such as map applications, or application products with IP images, spokespersons, etc.
  • functional APPs for example, using AR in most of the products being marketed.
  • the portrait segmentation function can be combined to highlight the front and rear positional relationship between real people and virtual objects in the AR scene, so that the entire AR scene is more realistic.
  • the user can walk back and forth in the camera to experience a real front-to-back position relationship with the virtual character.
  • the position information of the human body and the virtual object can be combined, and interactive feedback can be made by changing the position of the virtual object in the AR scene, thereby reducing the abnormal interleaving phenomenon.
  • a brand new interactive experience can be brought to the user in the AR scene, which is more interesting, real and fun.
  • the present application also provides a scene image generation apparatus, electronic equipment, computer-readable storage medium, and programs, all of which can be used to implement any scene image generation method provided by the present application. Corresponding records will not be repeated.
  • FIG. 8 shows a block diagram of a scene image display apparatus according to an embodiment of the present application. As shown in FIG. 8 , the apparatus includes:
  • an acquisition part 81 configured to acquire a live video stream
  • the first determining part 82 is configured to identify the first image frame of the live video stream, and determine the target object in the first image frame and the first position of the target object in the augmented reality AR scene;
  • a judgment part 83 configured to determine whether there is overlap between the target object and the AR object based on the first position and the second position of the AR object in the AR scene;
  • the second determination part 84 is configured to determine a third position of the AR object in the AR scene in the case of an overlap between the target object and the AR object, the third position is the same as the AR object The first position does not overlap;
  • the first display part 85 is configured to display the first AR scene image corresponding to the first image frame based on the third position.
  • the first determination part 82 includes: a human body area determination sub-part, configured to perform human body recognition on the first image frame, and determine the location of the target object in the first image frame.
  • a human body area configured to perform human body recognition on the first image frame, and determine the location of the target object in the first image frame.
  • a human body area configured to perform human body recognition on the first image frame, and determine the location of the target object in the first image frame.
  • a human body area configured to determine a depth value of the target object in the AR scene based on the human body area, and the first position includes the depth value and the human body area.
  • the second position of the AR object includes a depth interval of the AR object in the AR scene and an AR object area of the AR object in the AR scene
  • the determining Section 83 includes: a judgment subsection, configured to determine whether the depth value is within a depth interval, and whether there is an overlapping area between the human body area and the AR object area; a first determination subsection, configured to In the case that the depth value is within the depth interval, and there is an overlapping area between the human body area and the AR object area, it is determined that there is an overlap between the first position and the second position; the second determination A subsection configured to determine that there is an overlap between the target object and the AR object if there is an overlap between the first position and the second position.
  • the apparatus further includes: a third determination part, configured to identify the second image frame, and determine the fourth position of the target object in the second image frame in the AR scene
  • the second image frame includes one or more image frames in the live video stream before the first image frame
  • the judging part 83 further includes: a third determining sub-part, configured to If there is overlap between the first position and the second position, and there is an overlap between the fourth position and the second position, it is determined that there is an overlap between the target object and the AR object.
  • the apparatus further includes: a fourth determination part, configured to identify the third image frame, and determine the fifth position of the target object in the third image frame in the AR scene a position, the third image frame includes an image frame in the live video stream following the first image frame; a fifth determining part configured to have an overlap between the fifth position and the second position In the case of , the sixth position of the AR object in the AR scene is determined based on the third position and the preset movement track; the second display part is configured to display the same the second AR scene image corresponding to the third image frame.
  • the movement track includes a circle with the second position as a center and a distance between the third position and the second position as a radius.
  • the apparatus further includes: a sixth determination part, configured to identify the fourth image frame, and determine the seventh position of the target object in the fourth image frame in the AR scene a position, the fourth image frame includes an image frame in the live video stream after the first image frame; a third presentation part, configured so that there is no overlap between the seventh position and the second position In this case, based on the seventh position and the second position, a third AR scene image corresponding to the fourth image frame is displayed.
  • the judging part further includes: a fourth determining sub-part, configured to determine the relationship between the target object and the AR when the depth value is not within the depth interval There is no overlap between objects.
  • the apparatus further includes: a fourth display part, configured to, in the case where there is no overlap between the target object and the AR object, based on the first position and the In the second position, a fourth AR scene image corresponding to the first image frame is displayed.
  • the first display part 85 specifically includes: rendering the AR object in the first image frame based on the relative positional relationship between the third position and the first position , the first AR scene image is generated and displayed.
  • the third position of the AR object in the AR scene can be determined, and the third position does not overlap with the first position, thereby enabling In the AR scene image displayed based on the first position and the third position, the AR object and the target object do not overlap, thereby improving the overlapping and interlacing phenomenon between the target object and the AR object in the AR scene, and improving the realism and immersion of the AR scene sense.
  • the functions or included parts of the apparatus provided in the embodiments of the present application may be configured to execute the methods described in the above method embodiments, and reference may be made to the descriptions of the above method embodiments for implementation. Repeat.
  • the embodiments of the present application further provide a computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the foregoing method is implemented.
  • the computer-readable storage medium may be a non-volatile computer-readable storage medium.
  • An embodiment of the present application further provides an electronic device, including: a processor; a memory configured to store instructions executable by the processor; wherein the processor is configured to invoke the instructions stored in the memory to execute the above method.
  • Embodiments of the present application further provide a computer program, including computer-readable codes.
  • the computer-readable codes When the computer-readable codes are executed on an electronic device, the execution of the processor in the electronic device is configured to realize the scene image display provided by any of the above embodiments. method instruction.
  • the embodiments of the present application further provide another computer program, configured to store computer-readable instructions, and when the instructions are executed, cause the computer to perform the operations of the scene image display method provided by any of the foregoing embodiments.
  • the embodiments of the present application also provide a computer program product, the computer program product includes a computer program or instructions, and when the computer program or instructions are run on a computer, the computer is made to execute the scene image display method provided by any of the foregoing embodiments.
  • the electronic device may be provided as a terminal or other form of device.
  • FIG. 9 shows a block diagram of an electronic device 900 according to an embodiment of the present application.
  • the electronic device 900 may include an AR device that supports AR technology.
  • an electronic device 900 may include one or more of the following components: a processing component 902, a memory 904, a power supply component 906, a multimedia component 908, an audio component 910, an input/output (I/O) interface 912, a sensor component 914 , and the communication component 916 .
  • the processing component 902 generally controls the overall operation of the electronic device 900, such as operations associated with display, phone calls, data communications, camera operations, and recording operations.
  • the processing component 902 may include one or more processors 920 to execute instructions to perform all or some of the steps of the methods described above. Additionally, processing component 902 may include one or more modules to facilitate interaction between processing component 902 and other components. For example, processing component 902 may include a multimedia module to facilitate interaction between multimedia component 908 and processing component 902.
  • Memory 904 is configured to store various types of data to support operation at electronic device 900 . Examples of such data include instructions for any application or method operating on electronic device 900, contact data, phonebook data, messages, pictures, videos, and the like. Memory 904 may be implemented by any type of volatile or non-volatile storage device or combination thereof, such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), Magnetic Memory, Flash Memory, Magnetic Disk or Optical Disk.
  • SRAM static random access memory
  • EEPROM electrically erasable programmable read only memory
  • EPROM erasable Programmable Read Only Memory
  • PROM Programmable Read Only Memory
  • ROM Read Only Memory
  • Magnetic Memory Flash Memory
  • Magnetic Disk Magnetic Disk or Optical Disk.
  • Power supply assembly 906 provides power to various components of electronic device 900 .
  • Power supply components 906 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 900 .
  • Multimedia component 908 includes a screen that provides an output interface between the electronic device 900 and the user.
  • the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user.
  • the touch panel includes one or more touch sensors to sense touch, swipe, and gestures on the touch panel. The touch sensor may not only sense the boundaries of a touch or swipe action, but also detect the duration and pressure associated with the touch or swipe action.
  • the multimedia component 908 includes a front-facing camera and/or a rear-facing camera. When the electronic device 900 is in an operation mode, such as a shooting mode or a video mode, the front camera and/or the rear camera may receive external multimedia data. Each of the front and rear cameras can be a fixed optical lens system or have focal length and optical zoom capability.
  • Audio component 910 is configured to output and/or input audio signals.
  • audio component 910 includes a microphone (MIC) that is configured to receive external audio signals when electronic device 900 is in operating modes, such as calling mode, recording mode, and voice recognition mode. The received audio signal may be further stored in memory 904 or transmitted via communication component 916 .
  • audio component 910 also includes a speaker for outputting audio signals.
  • the I/O interface 912 provides an interface between the processing component 902 and a peripheral interface module, which may be a keyboard, a click wheel, a button, or the like. These buttons may include, but are not limited to: home button, volume buttons, start button, and lock button.
  • Sensor assembly 914 includes one or more sensors for providing status assessments of various aspects of electronic device 900 .
  • the sensor assembly 914 can detect the open/closed state of the electronic device 900, the relative positioning of the components, such as the display and the keypad of the electronic device 900, the sensor assembly 914 can also detect the electronic device 900 or one of the electronic devices 900 Changes in the position of components, presence or absence of user contact with the electronic device 900 , orientation or acceleration/deceleration of the electronic device 900 and changes in the temperature of the electronic device 900 .
  • Sensor assembly 914 may include a proximity sensor configured to detect the presence of nearby objects in the absence of any physical contact.
  • Sensor assembly 914 may also include a light sensor, such as a complementary metal oxide semiconductor (CMOS) or charge coupled device (CCD) image sensor, for use in imaging applications.
  • CMOS complementary metal oxide semiconductor
  • CCD charge coupled device
  • the sensor assembly 914 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
  • Communication component 916 is configured to facilitate wired or wireless communication between electronic device 900 and other devices.
  • the electronic device 900 may access a wireless network based on a communication standard, such as wireless network (WiFi), second generation mobile communication technology (2G) or third generation mobile communication technology (3G), or a combination thereof.
  • the communication component 916 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel.
  • the communication component 916 also includes a near field communication (NFC) module to facilitate short-range communication.
  • the NFC module may be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.
  • RFID radio frequency identification
  • IrDA infrared data association
  • UWB ultra-wideband
  • Bluetooth Bluetooth
  • electronic device 900 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable A programmed gate array (FPGA), controller, microcontroller, microprocessor or other electronic component implementation is used to perform the above method.
  • ASICs application specific integrated circuits
  • DSPs digital signal processors
  • DSPDs digital signal processing devices
  • PLDs programmable logic devices
  • FPGA field programmable A programmed gate array
  • controller microcontroller, microprocessor or other electronic component implementation is used to perform the above method.
  • a non-volatile computer-readable storage medium such as a memory 904 comprising computer program instructions executable by the processor 920 of the electronic device 900 to perform the above method is also provided.
  • Embodiments of the present application may be systems, methods and/or computer program products.
  • the computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement the embodiments of the present application.
  • a computer-readable storage medium may be a tangible device that can hold and store instructions for use by the instruction execution device.
  • the computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing.
  • Computer readable storage media specifically (non-exhaustive list) include: portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), Static random access memory (SRAM), portable compact disc read only memory (CD-ROM), digital versatile disc (DVD), memory sticks, floppy disks, mechanically coded devices such as punched cards or recesses on which instructions are stored The protrusion structure in the groove, and any suitable combination of the above.
  • RAM random access memory
  • ROM read only memory
  • EPROM or flash memory erasable programmable read only memory
  • SRAM Static random access memory
  • CD-ROM compact disc read only memory
  • DVD digital versatile disc
  • memory sticks floppy disks
  • mechanically coded devices such as punched cards or recesses on which instructions are stored
  • the protrusion structure in the groove and any suitable combination of the above.
  • Computer-readable storage media are not to be construed as transient signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (eg, light pulses through fiber optic cables), or through electrical wires transmitted electrical signals.
  • the computer readable program instructions described herein may be downloaded to various computing/processing devices from a computer readable storage medium, or to an external computer or external storage device over a network such as the Internet, a local area network, a wide area network, and/or a wireless network.
  • the network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and/or edge servers.
  • a network adapter card or network interface in each computing/processing device receives computer-readable program instructions from a network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing/processing device .
  • the computer program instructions for performing the operations of the embodiments of the present application may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state setting data, or programmed in one or more Source or object code written in any combination of languages, including object-oriented programming languages - such as Smalltalk, C++, etc., and conventional procedural programming languages, such as the "C" language or similar programming languages.
  • the computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server implement.
  • the remote computer may be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (eg, using an Internet service provider through the Internet connect).
  • LAN local area network
  • WAN wide area network
  • custom electronic circuits such as programmable logic circuits, field programmable gate arrays (FPGAs), or programmable logic arrays (PLAs) can be personalized by utilizing state information of computer readable program instructions.
  • the computer-readable program instructions are executed to implement the embodiments of the present application.
  • Embodiments of the present application are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the present application. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer readable program instructions.
  • These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer or other programmable data processing apparatus to produce a machine that causes the instructions when executed by the processor of the computer or other programmable data processing apparatus , resulting in means for implementing the functions/acts specified in one or more blocks of the flowchart and/or block diagrams.
  • These computer readable program instructions can also be stored in a computer readable storage medium, these instructions cause a computer, programmable data processing apparatus and/or other equipment to operate in a specific manner, so that the computer readable medium on which the instructions are stored includes An article of manufacture comprising instructions for implementing various aspects of the functions/acts specified in one or more blocks of the flowchart and/or block diagrams.
  • Computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other equipment to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other equipment to produce a computer-implemented process , thereby causing instructions executing on a computer, other programmable data processing apparatus, or other device to implement the functions/acts specified in one or more blocks of the flowcharts and/or block diagrams.
  • each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more functions for implementing the specified logical function(s) executable instructions.
  • the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved.
  • each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations can be implemented in dedicated hardware-based systems that perform the specified functions or actions , or can be implemented in a combination of dedicated hardware and computer instructions.
  • the computer program product can be implemented in hardware, software or a combination thereof.
  • the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK) and the like.
  • the embodiments of the present application disclose a scene image display method, device, device, storage medium, program and product, wherein the scene image display method includes: acquiring a live video stream; identifying a first image frame of the live video stream, determining the target object in the first image frame and the first position of the target object in the augmented reality AR scene; determining the first position based on the first position and the second position of the AR object in the AR scene Whether there is overlap between the target object and the AR object; if there is overlap between the target object and the AR object, determine the third position of the AR object in the AR scene, and the third position of the AR object in the AR scene is determined.
  • the three positions do not overlap the first position; based on the third position, a first AR scene image corresponding to the first image frame is displayed.

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Graphics (AREA)
  • Computer Hardware Design (AREA)
  • Software Systems (AREA)
  • Human Computer Interaction (AREA)
  • User Interface Of Digital Computer (AREA)
  • Processing Or Creating Images (AREA)

Abstract

一种场景图像展示方法、装置、设备、存储介质、程序及产品，所述方法包括：获取实景视频流（S10）；对实景视频流的第一图像帧进行识别，确定第一图像帧中的目标对象以及目标对象在增强现实AR场景中的第一位置（S11）；基于第一位置与AR场景中的AR对象的第二位置，确定目标对象与AR对象之间是否存在重叠（S12）；在目标对象与AR对象之间存在重叠的情况下，确定AR对象在AR场景中的第三位置，第三位置与第一位置不重叠（S13）；基于第三位置，展示与第一图像帧对应的第一AR场景图像（S14）。

Description

场景图像展示方法、装置、设备、存储介质、程序及产品
相关申请的交叉引用
本专利申请要求2021年01月15提交的中国专利申请号为202110056332.1，申请人为深圳市慧鲤科技有限公司，申请名称为“场景图像展示方法及装置、电子设备和存储介质”的优先权，该申请的全文以引用的方式并入本申请中。
技术领域
本申请涉及计算机技术领域，尤其涉及场景图像展示方法、装置、设备、存储介质、程序及产品。
背景技术
增强现实(Augmented Reality，AR)是一种人机交互技术，利用这种技术可以模拟真实的场景。由于在AR场景中，真实人物的位置会发生变化，可能存在真实人物与AR场景中的虚拟物体重叠的情况。在该情况下，可能会降低AR场景的真实感。
发明内容
本申请实施例提出了一种场景图像展示技术方案。
根据本申请实施例，提供了一种场景图像展示方法，包括：获取实景视频流；对实景视频流的第一图像帧进行识别，确定所述第一图像帧中的目标对象以及所述目标对象在增强现实AR场景中的第一位置；基于所述第一位置与所述AR场景中的AR对象的第二位置，确定所述目标对象与所述AR对象之间是否存在重叠；在所述目标对象与所述AR对象之间存在重叠的情况下，确定所述AR对象在所述AR场景中的第三位置，所述第三位置与所述第一位置不重叠；基于所述第三位置，展示与所述第一图像帧对应的第一AR场景图像。
根据本申请实施例，能够使基于第一位置和第三位置展示的第一AR场景图像中，AR对象和目标对象之间不重叠；并且在第一AR场景图像为多帧时，还可以呈现出AR对象从第二位置移动至第三位置的移动效果，从而改善AR场景中目标对象与AR对象之间的重叠交错现象，提升AR场景的真实感、沉浸感和互动感。
在一种可能的实现方法中，所述对所述实景视频流的第一图像帧进行识别，确定所述第一图像帧中的目标对象以及所述目标对象在增强现实AR场景中的第一位置，包括：对所述第一图像帧进行人体识别，确定所述第一图像帧中目标对象所在的人体区域；基于所述人体区域，确定所述目标对象在所述AR场景中的深度值，所述第一位置包括所述深度值和所述人体区域。根据本申请实施例，能够准确地确定出人体区域和深度值，从而可以基于准确的人体区域和深度值，准确地判断出目标对象与AR对象之间是否重叠。
在一种可能的实现方式中，所述AR对象的第二位置包括所述AR对象在所述AR场景中的深度区间和所述AR对象在所述AR场景中的AR对象区域，所述基于所述第一位置与所述AR场景中的AR对象的第二位置，确定所述目标对象与所述AR对象之间是否存在重叠，包括：确定所述深度值是否处于深度区间内，以及所述人体区域与所述AR对象区域之间是否存在重叠区域；在所述深度值处于所述深度区间内，且所述人体区域与所述AR对象区域之间存在重叠区域的情况下，确定所述第一位置与所述第二位置之间存在重叠，并确定所述目标对象与所述AR对象之间存在重叠。根据本申请实施例，能够基于第一位置和第二位置，准确有效的确定出目标对象与AR对象之间是否存在重叠。
在一种可能的实现方式中，所述方法还包括：对第二图像帧进行识别，确定所述第二图像帧中的目标对象在所述AR场景中的第四位置，所述第二图像帧包括所述实景视频流中 处于所述第一图像帧之前的一个或多个图像帧；所述基于所述第一位置与所述AR场景中的AR对象的第二位置，确定所述目标对象与所述AR对象之间是否存在重叠，还包括：在所述第一位置与所述第二位置之间存在重叠，且所述第四位置与所述第二位置之间存在重叠的情况下，确定所述目标对象与所述AR对象之间存在重叠。根据本申请实施例，能够实现在多帧图像帧中目标对象与AR对象之间均存在重叠，也即目标对象与AR对象之间存在一定时长的重叠情况下，确定目标对象与AR对象之间存在有效重叠，从而减少移动AR对象的频率，降低AR设备的计算及运行压力。
在一种可能的实现方式中，所述方法还包括：对第三图像帧进行识别，确定所述第三图像帧中的目标对象在所述AR场景中的第五位置，所述第三图像帧包括所述实景视频流中处于所述第一图像帧之后的图像帧；在所述第五位置与所述第二位置之间存在重叠的情况下，基于所述第三位置以及预设的移动轨迹，确定所述AR对象在所述AR场景中的第六位置；基于所述第六位置，展示与所述第三图像帧对应的第二AR场景图像。根据本申请实施例，能够在AR对象移动至第三位置后，呈现出AR对象沿移动轨迹进行移动的互动效果，提高AR场景的互动感。
在一种可能的实现方式中，所述移动轨迹包括以所述第二位置为中心、以所述第三位置与所述第二位置之间的距离为半径的圆形。根据本申请实施例，圆形的移动轨迹，可以较好的实现AR对象的移动效果。
在一种可能的实现方式中，所述方法还包括：对第四图像帧进行识别，确定所述第四图像帧中的目标对象在所述AR场景中的第七位置，所述第四图像帧包括所述实景视频流中处于所述第一图像帧之后的图像帧；在所述第七位置与所述第二位置不存在重叠的情况下，基于所述第七位置及所述第二位置，展示与所述第四图像帧对应的第三AR场景图像。根据本申请实施例，能够在目标对象与第二位置的AR对象不存在重叠的情况下，将AR对象移动至第二位置，也即，实现将AR对象移动回初始位置；并且在第三AR场景图像包括多帧的情况下，可以呈现出动态的移动效果，从而提升AR场景的真实感和互动感。
在一种可能的实现方式中，所述方法还包括：在所述目标对象与所述AR对象之间不存在重叠的情况下，基于所述第一位置及所述第二位置，展示与所述第一图像帧对应的第四AR场景图像。根据本申请实施例，能够在目标对象与AR对象之间不存在重叠的情况下，基于目标对象和AR对象的相对位置展示AR场景图像，从而可以突出AR场景中目标对象与AR对象之间的相对位置关系，以实现遮挡效果，提升真实感和沉浸感。
在一种可能的实现方式中，所述基于所述第三位置，展示与所述第一图像帧对应的第一AR场景图像，包括：基于所述第三位置与所述第一位置的相对位置关系，在所述第一图像帧中渲染所述AR对象，生成所述第一AR场景图像并进行展示。根据本申请实施例，能够呈现出目标对象与AR对象之间的相对位置，提高生成的AR场景图像的真实感和沉浸感。
根据本申请实施例，提供了一种场景图像展示装置，包括：获取部分，配置为获取实景视频流；第一确定部分，配置为对实景视频流的第一图像帧进行识别，确定所述第一图像帧中的目标对象以及所述目标对象在增强现实AR场景中的第一位置；判断部分，配置为基于所述第一位置与所述AR场景中的AR对象的第二位置，确定所述目标对象与所述AR对象之间是否存在重叠；第二确定部分，配置为在所述目标对象与所述AR对象之间存在重叠的情况下，确定所述AR对象在所述AR场景中的第三位置，所述第三位置与所述第一位置不重叠；第一展示部分，配置为基于所述第三位置，展示与所述第一图像帧对应的第一AR场景图像。
在一种可能的实现方法中，所述第一确定部分，包括：人体区域确定子部分，配置为对所述第一图像帧进行人体识别，确定所述第一图像帧中目标对象所在的人体区域；深度值确定子部分，配置为基于所述人体区域，确定所述目标对象在所述AR场景中的深度值，所述第一位置包括所述深度值和所述人体区域。
在一种可能的实现方式中，所述AR对象的第二位置包括所述AR对象在所述AR场景中的深度区间和所述AR对象在所述AR场景中的AR对象区域，所述判断部分，包括：判断子部分，配置为确定所述深度值是否处于深度区间内，以及所述人体区域与所述AR对象区域之间是否存在重叠区域；第一确定子部分，配置为在所述深度值处于所述深度区间内，且所述人体区域与所述AR对象区域之间存在重叠区域的情况下，确定所述第一位置与所述第二位置之间存在重叠；第二确定子部分，配置为在所述第一位置与所述第二位置之间存在重叠的情况下，确定所述目标对象与所述AR对象之间存在重叠。
在一种可能的实现方式中，所述装置还包括：第三确定部分，配置为对第二图像帧进行识别，确定所述第二图像帧中的目标对象在所述AR场景中的第四位置，所述第二图像帧包括所述实景视频流中处于所述第一图像帧之前的一个或多个图像帧；所述判断部分，还包括：第三确定子部分，配置为在所述第一位置与所述第二位置之间存在重叠，且所述第四位置与所述第二位置之间存在重叠的情况下，确定所述目标对象与所述AR对象之间存在重叠。
在一种可能的实现方式中，所述装置还包括：第四确定部分，配置为对第三图像帧进行识别，确定所述第三图像帧中的目标对象在所述AR场景中的第五位置，所述第三图像帧包括所述实景视频流中处于所述第一图像帧之后的图像帧；第五确定部分，配置为在所述第五位置与所述第二位置之间存在重叠的情况下，基于所述第三位置以及预设的移动轨迹，确定所述AR对象在所述AR场景中的第六位置；第二展示部分，配置为基于所述第六位置，展示与所述第三图像帧对应的第二AR场景图像。
在一种可能的实现方式中，所述移动轨迹包括以所述第二位置为中心、以所述第三位置与所述第二位置之间的距离为半径的圆形。
在一种可能的实现方式中，所述装置还包括：第六确定部分，配置为对第四图像帧进行识别，确定所述第四图像帧中的目标对象在所述AR场景中的第七位置，所述第四图像帧包括所述实景视频流中处于所述第一图像帧之后的图像帧；第三展示部分，配置为在所述第七位置与所述第二位置不存在重叠的情况下，基于所述第七位置及所述第二位置，展示与所述第四图像帧对应的第三AR场景图像。
在一种可能的实现方式中，所述装置还包括：第四展示部分，配置为在所述目标对象与所述AR对象之间不存在重叠的情况下，基于所述第一位置及所述第二位置，展示与所述第一图像帧对应的第四AR场景图像。
在一种可能的实现方式中，所述第一展示部分，具体包括：基于所述第三位置与所述第一位置的相对位置关系，在所述第一图像帧中渲染所述AR对象，生成所述第一AR场景图像并进行展示。
根据本申请实施例，提供了一种电子设备，包括：处理器；配置为存储处理器可执行指令的存储器；其中，所述处理器被配置为调用所述存储器存储的指令，以执行上述方法。
根据本申请实施例，提供了一种计算机可读存储介质，其上存储有计算机程序指令，所述计算机程序指令被处理器执行时实现上述方法。
根据本申请实施例，提供了一种计算机程序，包括计算机可读代码，当所述计算机可读代码在电子设备上运行时，电子设备中的处理器执行配置为实现上述方法的指令。
根据本申请实施例，提供了一种计算机程序产品，所述计算机程序产品包括计算机程序或指令，在所述计算机程序或指令在计算机上运行的情况下，使得所述计算机执行上述方法。
在本申请实施例中，能够实现在目标对象与AR对象之间存在重叠的情况下，确定出AR对象在AR场景中的第三位置，并且第三位置与第一位置不重叠，进而可以使基于第一位置和第三位置展示的AR场景图像中，AR对象和目标对象之间不重叠，从而改善AR场景中目标对象与AR对象之间的重叠交错现象，提升AR场景的真实感和沉浸感。
应当理解的是，以上的一般描述和后文的细节描述仅是示例性和解释性的，而非限制本申请。根据下面参考附图对示例性实施例的详细说明，本申请实施例将变得清楚。
附图说明
此处的附图被并入说明书中并构成本说明书的一部分，这些附图示出了符合本申请的实施例，并与说明书一起用于说明本申请的技术方案。
图1示出根据本申请实施例的场景图像展示方法的流程图；
图2示出根据本申请实施例的场景展示方法的一种系统架构示意图；
图3示出根据相关技术的AR场景图像的示意图；
图4示出根据本申请实施例的AR场景图像的示意图；
图5示出根据本申请实施例的AR场景图像的示意图；
图6示出根据本申请实施例的AR场景图像展示方法的流程图；
图7示出根据本申请实施例的移动轨迹的示意图；
图8示出根据本申请实施例的场景图像展示装置的框图；
图9示出根据本申请实施例的一种电子设备的框图。
具体实施方式
以下将参考附图详细说明本申请的各种示例性实施例。附图中相同的附图标记表示功能相同或相似的元件。尽管在附图中示出了各种实施例，但是除非特别指出，不必按比例绘制附图。
在这里专用的词“示例性”意为“用作例子、实施例或说明性”。这里作为“示例性”所说明的任何实施例不必解释为优于或好于其它实施例。
本文中术语“和/或”，仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。另外，本文中术语“至少一种”表示多种中的任意一种或多种中的至少两种的任意组合，例如，包括A、B、C中的至少一种，可以表示包括从A、B和C构成的集合中选择的任意一个或多个元素。
另外，为了更好地说明本申请，在下文的实施方式中给出了众多的细节。本领域技术人员应当理解，没有某些细节，本申请同样可以实施。在一些实例中，对于本领域技术人员熟知的方法、手段、元件和电路未作详细描述，以便于凸显本申请的主旨。
图1示出根据本申请实施例的场景图像展示方法的流程图，如图1所示，所述场景图像展示方法包括：
在步骤S10中，获取实景视频流；
在步骤S11中，对实景视频流的第一图像帧进行识别，确定第一图像帧中的目标对象以及目标对象在AR场景中的第一位置；
在步骤S12中，基于第一位置与AR场景中的AR对象的第二位置，确定目标对象与AR对象之间是否存在重叠；
在步骤S13中，在目标对象与AR对象之间存在重叠的情况下，确定AR对象在AR场景中的第三位置，第三位置与第一位置不重叠；
在步骤S14中，基于第三位置，展示与第一图像帧对应的第一AR场景图像。
在一种可能的实现方式中，所述场景图像展示方法可以由支持AR技术的AR设备执行，例如，AR设备可以包括终端设备，终端设备可以包括用户设备(User Equipment，UE)、移动设备、用户终端、终端、蜂窝电话、无绳电话、个人数字助理(Personal Digital Assistant，PDA)、手持设备、计算设备、车载设备、可穿戴设备等，所述方法可以通过终端设备的处理器调用存储器中存储的计算机可读指令的方式来实现。
在一种可能的实现方式中，在步骤S10中，实景视频流例如可以通过AR设备上设置的 图像采集装置即时采集的；还可以为其他设备传输给AR设备的，也可以是AR设备调用本地相册获取的，对此本申请实施例不做限制。
在一种可能的实现方式中，在步骤S11中，第一图像帧可以是一帧，也可以是多帧。其中，多帧第一图像帧，例如可以是实景视频流中连续的多帧图像，也可以是对实景视频流进行间隔采样得到的多帧图像，对此本申请实施例不做限制。
在一种可能的实现方式中，如上所述，AR设备上可以设置有图像采集装置，该图像采集装置能够实时捕获拍摄视野范围内的实景图像，并将实时捕获的实景图像作为实景视频流，传输给AR设备的处理器；AR设备的处理器基于获取的实景视频流的每帧图像帧，生成AR场景图像，并将AR场景图像显示在AR设备的显示界面上，进而使得显示界面实时呈现AR场景图像。
在一种可能的实现方式中，对第一图像帧进行识别，可以确定出包含对象的第一图像帧。在确定出包含对象的第一图像帧后，可以进一步确定该包含对象的第一图像帧中的目标对象。目标对象可以根据实际需要进行设置，例如可以包括但不限于：设置为第一图像帧中所有对象均为目标对象；或设置第一图像帧中处于中间区域的对象为目标对象；或者可以设置为识别出第一图像帧中的对象后，用户自行选择目标对象等，对此本申请实施例不做限制。
在一种可能的实现方式中，图像采集装置，可以包括但不限于为红绿蓝RGB(Red Green Blue)摄像头，或飞行时间TOF(Time of flight)摄像头等。
在一种可能的实现方式中，在步骤S11中，对第一图像帧进行识别，可以是采用任何已知人体识别技术对第一图像帧进行人体识别，确定出第一图像帧中的对象。其中，人体识别技术例如包括但不限于：人像分割、人体关键点检测等技术。可以理解的是，本领域技术人员可以选择一种或多种人体识别技术，只要可以实现对第一图像帧中对象的识别即可，对此本申请实施例不做限制。
在一种可能的实现方式中，AR场景可以包括根据实际需求预先构建好的虚拟场景。AR场景中可以包括虚拟人物、虚拟动物等AR对象。AR场景可以是基于图像采集设备的相机坐标系构建的，AR对象在AR场景中的位置可以是基于三维注册技术预先设置好的，本申请实施例对于AR场景的内容不做限制。
在一种可能的实现方式中，AR设备中可以存储有不同AR场景对应的数据包；还可以由其他设备(例如，云端服务器)传输不同AR场景对应的数据包到AR设备；通过使用不同的数据包，可以在AR设备中展示不同的AR场景。
在一种可能的实现方式中，可以响应于用户对AR场景的选择操作，确定AR场景。例如，可以在AR设备的显示界面中呈现可供用户选择的AR场景，从而便于用户通过点选等操作，触发对AR场景的选择。本申请实施例对于AR场景的确定方式不做限制。
在一种可能的实现方式中，如上文所述，AR场景可以是基于图像采集设备的相机坐标系构建的。在步骤S11中，在确定出第一图像帧中的目标对象后，可以基于图像采集设备的成像原理，基于目标对象在第一图像帧中的位置，确定出目标对象在所述相机坐标系中的位置，也即确定出目标对象在AR场景中的第一位置。
在一种可能的实现方式中，在步骤S12中，如上文所述，AR对象的第二位置是预先设置好，也即，第二位置可以是已知的。第二位置相当于AR对象在AR场景中的默认的初始位置。在目标对象与AR对象之间不存在重叠的情况下，AR对象可以是处于该初始位置。
在一种可能的实现方式中，在步骤S12中，可以基于所述相机坐标系，基于第一位置和第二位置对应的坐标值，确定目标对象与AR对象之间是否存在重叠。例如，在基于第一位置和第二位置对应的坐标值，确定第一位置与第二位置相同或第一位置与第二位置存在重叠的情况下，确定目标对象与AR对象之间存在重叠；在第一位置与第二位置不同或第一位置与第二位置不存在重叠的情况下，确定目标对象与AR对象之间不存在重叠。可依据实 际需求进行设定，对此本申请实施例不做限制。
在一种可能的实现方式中，在步骤S13中，第三位置可以是AR场景中与第一位置不重叠的其它位置。其中，第三位置与第二位置之间的距离，可根据实际需求设定；第三位置相对于第二位置的方位，可以在不与第一位置重叠的情况下随机确定，例如，可以是与第一位置相对于第二位置的方位相反的方位；或可以是与第一位置相对于第二位置的方位偏离一定角度的方位，对此本申请实施例不做限制。
在一种可能的实现方式中，如上文所述，可通过AR设备的显示界面展示AR场景图像。AR场景图像可以是基于实景图像渲染AR对象的特效后所生成的图像，其中实景图像可以是实景视频流中的任一图像帧。
在一种可能的实现方式中，如上文所述，第一图像帧可以是一帧或多帧。在第一图像帧为一帧的情况下，可以基于AR对象的第三位置，在该第一图像帧中渲染第三位置的AR对象，生成第一AR场景图像，进而展示与该第一图像帧对应的第一AR场景图像。可以理解的是，在该情况下，视觉上感知的可以是AR对象从第二位置瞬时移动至第三位置的效果。
在一种可能的实现方式中，在第一图像帧为多帧的情况下，可以基于多帧第一图像帧，显示AR对象从第二位置移动至第三位置的具有一定时长的移动效果，可以包括：基于第二位置与第三位置之间的距离，按照一定的距离间隔，确定第二位置与第三位置之间的多个位置；基于该多个位置和第三位置，在对应多帧第一图像帧中渲染AR对象，生成多张第一AR实景图像；从而能够在展示该多张第一AR场景图像时，呈现AR对象从第二位置移动至第三位置的具有一定时长移动效果。
在本申请实施例中，能够实现在目标对象与AR对象之间存在重叠的情况下，确定出AR对象在AR场景中的第三位置，并且第三位置与第一位置不重叠，进而可以使基于第一位置和第三位置展示的第一AR场景图像中，AR对象和目标对象之间不重叠；并且在第一AR场景图像为多帧时，还可以呈现出AR对象从第二位置移动至第三位置的移动效果，从而改善AR场景中目标对象与AR对象之间的重叠交错现象，提升AR场景的真实感、沉浸感和互动感。
在一种可能的实现方式中，在步骤S11中，所述对实景视频流的第一图像帧进行识别，确定第一图像帧中的目标对象以及目标对象在AR场景中的第一位置，可以包括：
对第一图像帧进行人体识别，确定第一图像帧中目标对象所在的人体区域；
基于人体区域，确定目标对象在AR场景中的深度值，第一位置包括深度值和人体区域。
在一种可能的实现方式中，对第一图像帧进行人体识别，确定第一图像帧中的目标对象所在的人体区域，可以是基于人像分割技术，分割出目标对象所在的人体区域；还可以是基于人体关键点检测技术，检测人体轮廓关键点，基于人体轮廓关键点确定人体轮廓，将人体轮廓构成的区域作为人体区域，对此本申请实施例不做限制。
在一种可能的实现方式中，如上文所述，在确定出第一图像帧中的目标对象后，可以基于目标对象在第一图像帧中的位置，确定出目标对象在图像采集装置的相机坐标系中的位置，也即确定出目标对象在AR场景中的第一位置。其中，目标对象在该相机坐标系中的位置可以包括目标对象在相机坐标系中的深度。目标对象在相机坐标系中的深度，可以理解为，实景中目标对象与图像采集装置之间的距离；距离越远，深度越大；反之，深度越小。
在一种可能的实现方式中，在确定出目标对象的人体区域后，可以基于人体区域的关键点在图像帧中的像素坐标，以及结合图像采集装置的成像原理(例如，小孔成像原理)，确定出目标对象与图像采集装置之间的距离；将目标对象与图像采集装置之间的距离，作为目标对象在相机坐标系中的深度，也即作为目标对象在AR场景中的深度值。
如上文所述，图像采集装置可以包括TOF摄像头。在一种可能的实现方式中，在图像采集装置为TOF摄像头的情况下，由于TOF摄像头采集的图像数据中已包括实景中目标对 象与TOF摄像头之间的距离信息，所以在确定出图像中目标对象的人体区域后，即可以得到目标对象与TOF摄像头之间的距离，也即得到目标对象在AR场景中的深度值。
在一种可能的实现方式中，目标对象的深度值，可以是人体轮廓关键点在相机坐标系中的深度；还可以是人体关节关键点在相机坐标系中的深度；还可以是人体区域内全部关键点在相机坐标系中的深度；对于关键点的数量，可根据实际需求设定、处理器的运算性能等确定，对此本申请实施例不做限制。
在一种可能的实现方式中，可以将多个关键点的深度的平均值，作为目标对象的深度值；或还可以将多个关键点的深度中的最小值和/或最大值，作为目标对象的深度值；或可以将人体区域内全部关键点的深度，作为目标对象的深度值。可依据实际应用场景、处理器的运算性能等确定，对此本申请实施例不做限制。
在本申请实施例中，能够准确地确定出人体区域和深度值，从而可以基于准确的人体区域和深度值，准确地判断出目标对象与AR对象之间是否重叠。
在一种可能的实现方式中，AR对象的第二位置包括AR对象在AR场景中的深度区间和AR对象在AR场景中的AR对象区域，所述基于第一位置与AR场景中的AR对象的第二位置，确定目标对象与AR对象之间是否存在重叠，可以包括：
确定深度值是否处于深度区间内，以及人体区域与AR对象区域之间是否存在重叠区域；
在深度值处于深度区间内，且人体区域与AR对象区域之间存在重叠区域的情况下，确定第一位置与第二位置之间存在重叠，并确定目标对象与AR对象之间存在重叠。
可以理解的是，AR对象在AR场景中可以是立体的，AR对象的深度可以存在多个，也即存在深度区间。AR对象区域可以是AR对象在AR场景中所占的立体区域。由于AR对象在AR场景中是预先构建好的，所以AR对象区域可以是已知的。
在一种可能的实现方式中，如上文所述，目标对象的深度值可以是一个或多个。在深度值为一个的情况下，确定所述深度值是否处于深度区间内，可以包括：确定该一个深度值是否处于深度区间内，在该一个深度值处于深度区间内的情况下，可以确定该一个深度值处于深度区间内；在深度值为多个的情况下，确定所述深度值是否处于深度区间内，可以包括：确定多个深度值中是否存在处于深度区间内的深度值；在多个深度值中存在处于深度区间内的深度值的情况下，可以确定深度值处于深度区间内。其中，可以是多个深度值中的部分深度值(如最大值和/或最小值)处于深度区间内；也可以是全部深度值均处于深度区间内。
图2可以应用本申请实施例一种场景图像展示方法的一种系统架构示意图；如图2所示，该系统架构中包括：视频采集设备2001、网络2002和图像获取终端2003。为实现支撑一个示例性应用，视频采集设备2001和图像获取终端2003可以通过网络2002建立通信连接，视频采集设备2001通过网络2002向图像获取终端2003传输获取的实景视频流，图像获取终端2003接收实景视频流，并对视频流进行分析，在目标对象与AR对象之间存在重叠的情况下，确定AR对象在AR场景中的第三位置，基于第三位置，展示与第一图像帧对应的第一AR场景图像，提升AR场景的真实感和沉浸感。
作为示例，当前场景视频采集设备2001可以包括摄像头等图像采集设备。图像获取终端2003可以包括具有一定计算能力的计算机设备，该计算机设备例如包括：终端设备或服务器或其它处理设备。网络2002可以采用有线连接或无线连接方式。其中，当视频采集设备2001为图像采集设备，图像获取终端2003为服务器时，视频采集设备可以通过有线连接的方式与图像获取终端通信连接，例如通过总线进行数据通信；当视频采集设备2001图像采集设备，图像获取终端2003为终端设备时，视频采集设备可以通过无线连接的方式与图像获取终端通信连接，进而进行数据通信。
或者，在一些场景中，图像获取终端2003可以是带有视频采集模组的视觉处理设备，可以是带有摄像头的主机。这时，本申请实施例的场景图像展示方法可以由图像获取终端 2003执行，上述系统架构可以不包含网络2002和视频采集设备2001。
可以理解的是，物体之间的位置关系可以包括：前后、左右和上下。目标对象的深度值处于深度区间内，可以认为目标对象和AR对象在前后方向上是重合的。人体区域与AR对象区域之间存在重叠区域，可以理解为，目标对象和AR对象在左右和/或上下方向也存在重合。图3示出根据相关技术的AR场景图像的示意图。如图3所示，目标对象与AR对象之间存在重叠也即存在交错。
在一种可能的实现方式中，判断人体区域与AR对象区域之间存在重叠区域，可以是基于人体轮廓关键点的坐标值，判断人体轮廓关键点中是否存在处于AR对象区域内的关键点；在人体轮廓关键点中存在处于AR对象区域内的关键点的情况下，确定人体区域与AR对象区域之间存在重叠区域。
在一种可能的实现方式中，还可能存在深度值处于深度区间内，人体区域与AR对象区域之间不存在重叠区域的情况，在该情况下，可以理解为AR对象和目标对象在前后方向上存在重合，但AR对象与目标对象之间在左右方向和/或上下方向存在一定距离，也即，AR对象与目标对象之间在左右方向和/或上下方向不重合，在该情况下，可以认为第一位置与第二位置之间不存在重叠，也即目标对象与AR对象之间不存在重叠。
在一种可能的实现方式中，AR对象在AR场景中还可以是平面的，在该情况下，AR对象的深度区间内也就包含一个深度；AR对象区域可以包括AR对象的轮廓围成的平面区域；所述深度值处于深度区间内，可以包括：目标对象的深度值存在与深度区间内深度相同的深度值。
在本申请实施例中，能够基于第一位置和第二位置，准确有效的确定出目标对象与AR对象之间是否存在重叠。
在一种可能的实现方式中，考虑到在图像采集装置采集实景视频流时，可能采集到人体短暂穿过AR对象的情况，在该情况下，可能存在频繁的移动AR对象而增加AR设备的计算及运行压力的问题，所述方法还可以包括：
对第二图像帧进行识别，确定第二图像帧中的目标对象在AR场景中的第四位置，第二图像帧包括实景视频流中处于第一图像帧之前的一个或多个图像帧；
所述基于第一位置与AR场景中的AR对象的第二位置，确定目标对象与AR对象之间是否存在重叠，还可以包括：
在第一位置与第二位置之间存在重叠，且第四位置与第二位置之间存在重叠的情况下，确定目标对象与AR对象之间存在重叠。
在一种可能的实现方式中，对第二图像帧进行识别，确定目标对象在AR场景中的第四位置，可以采用与上述对第一图像帧进行识别相同的识别方式，以及与确定第二位置相同的位置确定方式，在此不做赘述。
如上所述，可能存在人体短暂穿过AR对象的情况，例如，图像采集设备前偶然经过的路人，在该情况下，可能存在频繁的移动AR对象而增加AR设备的计算及运行压力的问题。在一种可能的实现方式中，可以通过设定预设时长，在目标对象与AR对象之间存在重叠，且重叠的时长超过该预设时长的情况下，认为目标对象与AR对象之间存在重叠。其中，预设时长可以根据实际需求设定，对此本申请实施例不做限制。
可以理解的是，视频的时长，可以是帧数与帧率的乘积。也就是说，在帧率一定下，可以通过帧数反映预设时长。由于第二图像帧可以包括实景视频流中处于第一图像帧之前的一个或多个图像帧，那么目标对象与AR对象之间存在重叠，且重叠的时长超过该预设时长，也就相当于，与预设时长对应帧数的第二图像帧中，目标对象与AR对象均存在重叠，且第一图像帧中目标对象与AR对象也重叠。
在一种可能的实现方式中，第一位置与第二位置之间存在重叠，可以理解为，第一图像帧中目标对象与AR对象之间存在重叠；第四位置与第二位置之间存在重叠，可以理解为， 第二图像帧中目标对象与AR对象之间存在重叠；通过第一位置与第二位置之间存在重叠，且第四位置与第二位置之间存在重叠，可以反映出目标对象与AR对象之间存在重叠，且重叠的时长超过预设时长。
在一种可能的实现方式中，第二图像帧的数量可依据预设时长确定，例如，预设时长是5秒，则5秒对应的图像帧的数量，即可作为第二图像帧的数量。
在一种可能的实现方式中，确定第四位置与第二位置之间是否存在重叠，可以是采用上述本申请实施例中确定第一位置与第二位置之间是否存在重叠相同的方式，在此不做赘述。
在一种可能的实现方式中，与第二图像帧对应的AR场景图像，也可以展示在AR设备的显示界面中，在第二图像帧为多帧的情况下，可以向用户展示目标对象与AR对象之间具有一定时长的重叠效果，由此可以在移动AR对象后，呈现出目标对象与AR对象之间从重叠到不重叠的互动效果，提升AR场景中的互动感。
在本申请实施例中，能够实现在多帧图像帧中目标对象与AR对象之间均存在重叠，也即目标对象与AR对象之间存在一定时长的重叠情况下，确定目标对象与AR对象之间存在有效重叠，从而减少移动AR对象的频率，降低AR设备的计算及运行压力。
在一种可能的实现方式中，在基于第三位置展示第一AR场景图像后，所述方法还可以包括：
对第三图像帧进行识别，确定第三图像帧中的目标对象在AR场景中的第五位置，第三图像帧包括实景视频流中处于第一图像帧之后的图像帧；
在第五位置与第二位置之间存在重叠的情况下，基于第三位置以及预设的移动轨迹，确定AR对象在AR场景中的第六位置；
基于第六位置，展示与第三图像帧对应的第二AR场景图像。
在一种可能的实现方式中，对第三图像帧进行识别，确定第三图像帧中的目标对象在AR场景中的第五位置，可以采用与上述对第一图像帧进行识别相同的识别方式，以及与确定第二位置相同的位置确定方式，在此不做赘述。
在一种可能的实现方式中，确定第五位置与第二位置之间是否存在重叠，可以是采用上述本申请实施例中确定第一位置与第二位置之间是否存在重叠相同的方式，在此不做赘述。
在一种可能的实现方式中，预设的移动轨迹，可以是预先设置用于控制AR对象移动的轨迹，例如，移动轨迹可以是线段、圆形、方形等，对于移动轨迹的形式，本申请实施例不做限制。可以理解的是，AR对象的第三位置处于移动轨迹上，从而能够以第三位置为起点沿移动轨迹移动。
在一种可能的实现方式中，所述移动轨迹可以包括以第二位置为中心、以第三位置与第二位置之间的距离为半径的圆形。圆形的移动轨迹，可以较好的实现AR对象的移动效果。
在一种可能的实现方式中，基于第三位置以及预设的移动轨迹，确定AR对象在AR场景中的第六位置，可以是从第三位置开始，确定AR对象在移动轨迹上的多个第六位置。
在一种可能的实现方式中，确定移动轨迹上的多个第六位置，可以是基于预设的移动方向和移动间距确定。例如，移动方向可以是顺时针或逆时针等，基于移动方向可以确定多个第六位置的先后顺序；在移动轨迹的长度已知的情况下，可以基于移动间隔确定出第六位置的数量。
在一种可能的实现方式中，第三图像帧可以包括实景视频流中第一图像帧之后连续的多帧图像帧，也可以是实景视频流中第一图像帧之后间隔一定帧数的多帧图像帧，对此本申请实施例不做限制。
在一种可能的实现方式中，所述基于第六位置，展示与第三图像帧对应的第三AR场景图像，可以包括：基于AR对象的多个第六位置，在多帧第三图像帧中分别渲染多个第六位 置的AR对象，生成多张第二AR场景图像；进而在AR设备的显示界面展示多张第二AR场景图像，能够呈现动态的AR对象按移动轨迹进行移动的移动效果。
在本申请实施例中，能够在AR对象移动至第三位置后，呈现出AR对象沿移动轨迹进行移动的互动效果，提高AR场景的互动感。
在一种可能的实现方式中，考虑到目标对象在AR场景中的位置是变化的，可以在目标对象与初始位置(即第二位置)的AR对象不存在重叠的情况下，将AR对象移回至第二位置，可以理解的是，移回至第二位置的AR对象与目标对象不重叠。
在一种可能的实现方式中，所述方法还可以包括：
对第四图像帧进行识别，确定第四图像帧中的目标对象在AR场景中的第七位置，第四图像帧包括实景视频流中处于第一图像帧之后的图像帧；
在第七位置与第二位置不存在重叠的情况下，基于第七位置及第二位置，展示与第四图像帧对应的第三AR场景图像。
在一种可能的实现方式中，对第四图像帧进行识别，确定第四图像帧中的目标对象在AR场景中的第七位置，可以采用与上述对第一图像帧进行识别相同的识别方式，以及与确定第二位置相同的位置确定方式，在此不做赘述。
在一种可能的实现方式中，判断第七位置与第二位置之间是否存在重叠，可以是采用上述本申请实施例中判断第一位置与第二位置之间是否存在重叠相同的方式，在此不做赘。在一种可能的实现方式中，第四图像帧可以是与所述第三图像帧相同的图像帧，还可以是处于所述第三图像帧之后的图像帧。
在一种可能的实现方式中，第四图像帧可以是一帧；还可以是连续的多帧或间隔采样的多帧。在第四图像帧为一帧的情况下，可以基于AR对象的第二位置，将第二位置的AR对象渲染至该第四图像帧中，生成第三AR场景图像，进而展示与该第四图像帧对应的第三AR场景图像。可以理解的是，在该情况下，视觉上感知的可以是AR对象从当前位置瞬时移动至第二位置的效果。
在一种可能的实现方式中，在第四图像帧为多帧的情况下，可以基于多帧第四图像帧，显示AR对象从当前位置移动至第二位置的具有一定时长的移动效果，可以包括：基于当前位置与第二位置之间的距离，按照一定的距离间隔，确定当前位置与第二位置之间的多个位置；基于该多个位置和第二位置，在多帧第四图像帧中分别渲染AR对象，生成多张第三AR场景图像；进而在展示该多张第三AR场景图像时，显示AR对象从当前位置移动至第二位置的具有一定时长的移动效果。
需要说明的是，AR对象的当前位置可以是所述第三位置，也可以是所述移动轨迹上的第六位置，可依据AR对象在移动至第三位置处后，是否按移动轨迹进行移动确定。
在本申请实施例中，能够在目标对象与第二位置的AR对象不存在重叠的情况下，将AR对象移动至第二位置，也即，实现将AR对象移动回初始位置，在第四图像帧包括多帧的情况下，可以呈现出动态的移动效果，从而提升AR场景的真实感和互动感。
在一种可能的实现方式中，所述方法还包括：在目标对象与AR对象之间不存在重叠的情况下，基于第一位置及第二位置，展示与第一图像帧对应的第四AR场景图像。
如上所述，可以基于目标对象的第一位置和AR对象的第二位置，确定目标对象与AR对象之间是否重叠；以及在深度值处于深度区间内，且人体区域与AR对象区域之间存在重叠区域的情况下，确定第一位置与第二位置之间存在重叠。在一种可能的实现方式中，可以在其它情况下，确定第一位置与第二位置之间不存在重叠。例如，该其它情况可以至少包括：深度值未处于深度区间内的情况；深度值处于深度区间内，但人体区域与AR对象区域之间不存在重叠区域的情况等。
在一种可能的实现方式中，如上文所述，目标对象的深度值可以是一个或多个。深度值未处于深度区间内，可以是一个或多个深度值均未处于深度区间内。在该情况下，可以 认为，目标对象与AR对象之间不存在重叠。
在一种可能的实现方式中，目标对象与AR对象之间不存在重叠，但可能存在遮挡，该情况下，可以认为是深度值未处于深度区间内，但人体区域与AR对象区域之间存在重叠区域。图4示出根据本申请实施例的AR场景图像的示意图。图5示出根据本申请实施例的AR场景图像的示意图。如图4和图5所示，目标对象与AR对象之间不存在重叠，但存在遮挡。
在一种可能的实现方式中，基于目标对象的第一位置和AR对象的第二位置，可以确定出AR对象的第一位置与目标对象的第二位置之间的相对位置关系，进而在生成并展示第四AR场景图像，可以显示出AR对象和目标对象之间的相对位置，以及可以实现遮挡效果。
在本申请实施例中，能够在目标对象与AR对象之间不存在重叠的情况下，基于目标对象和AR对象的相对位置展示AR场景图像，从而可以突出AR场景中目标对象与AR对象之间的相对位置关系，以实现遮挡效果，提升真实感和沉浸感。
在一种可能的实现方式中，所述基于第三位置，展示与第一图像帧对应的第一AR场景图像，可以包括：
基于第三位置与第一位置的相对位置关系，在第一图像帧中渲染AR对象，生成第一AR场景图像并进行展示。
如上文所述，目标对象在AR场景中的第一位置可以包括目标对象在AR场景中的深度值；AR对象在AR场景中的第二位置可以包括AR对象在AR场景中的深度区间。相应的，AR对象在AR场景中的第三位置可以包括AR对象在AR场景中第三位置处的深度区间。
如上文所述，第三位置与第一位置不重叠，也即，目标对象的深度值未处于第三位置处的深度区间内。在一种可能的实现方式中，第三位置与第一位置的相位位置关系可以包括：深度值小于深度区间中的最小值，或深度值大于深度区间中的最大值。
在一种可能的实现方式中，如上文所述，目标对象的深度值可以是一个或多个。深度值小于深度区间中的最小值，可以是目标对象的一个或多个深度值均小于该深度区间中的最小值。
在一种可能的情况下，针对深度值小于深度区间中的最小值的情况，可以认为该情况下，目标对象相对于AR对象与图像采集设备的距离要近，也就是说，相对于图像采集设备来说，AR对象处于目标对象的后侧。那么在深度值小于深度区间中的最小值的情况下，可以在第一图像帧中将AR对象渲染在相对于目标对象的后侧，生成第一AR场景图像并进行展示。
如上文所述，目标对象的深度值可以是一个或多个。深度值大于深度区间中的最大值，可以是目标对象的一个或多个深度值均大于该深度区间中的最大值。
在一种可能的实现方式中，针对深度值大于深度区间中的最大值的情况，可以认为该情况下，目标对象相对于AR对象与图像采集设备的距离要远，也就是说，相对于图像采集设备来说，AR对象处于目标对象的前侧。在深度值大于深度区间中的最大值的情况下，可以在第一图像帧中将AR对象渲染在相对于目标对象的前侧，生成第一AR场景图像并进行展示。
需要说明的是，本申请实施例中的前侧和后侧具有相对性，例如，距离图像采集设备较近的对象，可以是位于距离图像采集设备较远的对象的前侧。本领域技术人员可以理解，或至少在阅读本申请实施例后可以理解前侧和后侧的含义。
如上文所述，对第一图像帧进行识别可以采用人像分割技术。在一种可能的实现方式中，基于人像分割技术，可以将实景图像中人体区域与背景区域分离，进而可以基于分离的人体区域和背景区域，将AR对象渲染在相对于目标对象的后侧或前侧，生成第一AR场景图像。如图4所述，AR对象处于相对于目标对象的后侧，目标对象遮挡AR对象。
在一种可能的实现方式中，展示第一AR场景图像，可以是将第一AR场景图像绘制在AR合影设备的显示界面上，以进行展示。
在一种可能的实现方式中，所述在目标对象与AR对象之间不存在重叠的情况下，基于第一位置及第二位置，展示与第一图像帧对应的第四AR场景图像，可以采用与本申请实施例中基于第三位置和第一位置展示第一AR场景图像相同的方式。也即，在目标对象与AR对象之间不重叠的情况下，可以基于当前目标对象与AR对象之间的相对位置关系，生成并展示AR场景图像。
在本申请实施例中，能够呈现出目标对象与AR对象之间的相对位置，提高生成的AR场景图像的真实感和沉浸感。
图6示出根据本申请实施例的AR场景图像展示方法的流程图。如图6所示，该AR场景展示方法包括：
在步骤S601中，实时获取视频流数据。
在步骤S602中，对视频流数据中的每帧图像帧进行人像分割和人体深度估计。
在步骤S603中，实时输出人体与虚拟物体的位置渲染关系。
在步骤S604中，判断人体和虚拟物体的原位置是否重叠且重叠超过一定时长。
这里，人体和虚拟物体的原位置重叠且重叠超过一定时长，进入步骤S605；人体和虚拟物体的原位置不重叠或重叠不超过一定时长，返回步骤S603。
在步骤S605中，在人体和虚拟物体的原位置重叠且重叠超过一定时长的情况下，触发虚拟物体围绕真实人体进行转圈移动。
这里，时长可根据实际需求自主设定。
在步骤S606中，判断人体与原位置的虚拟物体之间是否不重叠。
这里，人体和虚拟物体的原位置不重叠，进入步骤S607；人体和虚拟物体的原位置重叠，返回步骤S604。
在步骤S607中，在人体与原位置的虚拟物体之间不重叠的情况下，虚拟物体返回原位置。
这里，人体与原位置的虚拟物体之间不重叠，即，人体已经远离原位置的虚拟物体。
在一种可能的实现方式中，人像分割可以包括：对于图像帧中的人物进行检测，获取人体区域；再基于人体区域的关键点信息，分割出人体所在的范围。
在一种可能的实现方式中，人体深度估计可以包括：估算出中的图像帧中人体距离摄像头的实际距离。
在一种可能的实现方式中，位置渲染关系可以包括通过计算画面中的人物和虚拟物体的相对位置，渲染出前后关系。
图7示出根据本申请实施例的移动轨迹的示意图。如图7所示，假设图中五角星为虚拟物体的区域，五角星的位置为该虚拟物体的既定位置(初始位置)，外围的圆形可以是该虚拟物体可能移动至的位置；当真实人物与五角星的位置重叠时，触发虚拟物体随机转移至所述外围的圆形上，减少交错重叠的现象；当真实人物离开五角星的位置，虚拟物体返回原位(即初始位置)。
一种可能的实现方式中，本申请实施例中的AR场景展示方法，可以应用于互联网APP，例如，地图应用、或具有IP形象、代言人等应用产品中；功能性APP，例如，使用到AR营销的大部分产品中。
相关技术中，由于缺乏人像分割功能，整个AR场景中，虚拟物体永远在前景，没有足够的真实感。在本申请实施例中，能够结合人像分割功能，在AR场景中，凸显出真实人物和虚拟物体的前后位置关系，使得整个AR场景更加真实。用户可以在镜头中前后走动，体验和虚拟人物一种真实的前后位置关系。
相关技术中，在人体与虚拟物体发生位置重叠时，无法减少穿插交错现象，只能比较僵硬地显示异常效果，无法做出互动的反馈，一般在产生交错的时候会直接穿过，与真实世界中的运动轨迹不符合。例如真实世界中，两个人是不可能穿插的，只能出现一个人为 另一个人让开位置的情况。在本申请实施例中，能够结合人体与虚拟物体位置信息，在AR场景中，通过变化虚拟物体的位置，做出互动反馈，从而减少异常的穿插交错现象。
在本申请实施例中，能够在AR场景内为用户带来全新的互动体验，更加有趣、真实和好玩。
可以理解，本申请提及的上述各个方法实施例，在不违背原理逻辑的情况下，均可以彼此相互结合形成结合后的实施例，限于篇幅，本申请不再赘述。本领域技术人员可以理解，在具体实施方式的上述方法中，各步骤的执行顺序应当以其功能和可能的内在逻辑确定。
此外，本申请还提供了场景图像生成装置、电子设备、计算机可读存储介质、程序，上述均可用来实现本申请提供的任一种场景图像生成方法，相应技术方案和描述和参见方法部分的相应记载，不再赘述。
图8示出根据本申请实施例的场景图像展示装置的框图，如图8所示，所述装置包括：
获取部分81，配置为获取实景视频流；
第一确定部分82，配置为对实景视频流的第一图像帧进行识别，确定所述第一图像帧中的目标对象以及所述目标对象在增强现实AR场景中的第一位置；
判断部分83，配置为基于所述第一位置与所述AR场景中的AR对象的第二位置，确定所述目标对象与所述AR对象之间是否存在重叠；
第二确定部分84，配置为在所述目标对象与所述AR对象之间存在重叠的情况下，确定所述AR对象在所述AR场景中的第三位置，所述第三位置与所述第一位置不重叠；
第一展示部分85，配置为基于所述第三位置，展示与所述第一图像帧对应的第一AR场景图像。
在一种可能的实现方法中，所述第一确定部分82，包括：人体区域确定子部分，配置为对所述第一图像帧进行人体识别，确定所述第一图像帧中目标对象所在的人体区域；深度值确定子部分，配置为基于所述人体区域，确定所述目标对象在所述AR场景中的深度值，所述第一位置包括所述深度值和所述人体区域。
在一种可能的实现方式中，所述AR对象的第二位置包括所述AR对象在所述AR场景中的深度区间和所述AR对象在所述AR场景中的AR对象区域，所述判断部分83，包括：判断子部分，配置为确定所述深度值是否处于深度区间内，以及所述人体区域与所述AR对象区域之间是否存在重叠区域；第一确定子部分，配置为在所述深度值处于所述深度区间内，且所述人体区域与所述AR对象区域之间存在重叠区域的情况下，确定所述第一位置与所述第二位置之间存在重叠；第二确定子部分，配置为在所述第一位置与所述第二位置之间存在重叠的情况下，确定所述目标对象与所述AR对象之间存在重叠。
在一种可能的实现方式中，所述装置还包括：第三确定部分，配置为对第二图像帧进行识别，确定所述第二图像帧中的目标对象在所述AR场景中的第四位置，所述第二图像帧包括所述实景视频流中处于所述第一图像帧之前的一个或多个图像帧；所述判断部分83，还包括：第三确定子部分，配置为在所述第一位置与所述第二位置之间存在重叠，且所述第四位置与所述第二位置之间存在重叠的情况下，确定所述目标对象与所述AR对象之间存在重叠。
在一种可能的实现方式中，所述装置还包括：第四确定部分，配置为对第三图像帧进行识别，确定所述第三图像帧中的目标对象在所述AR场景中的第五位置，所述第三图像帧包括所述实景视频流中处于所述第一图像帧之后的图像帧；第五确定部分，配置为在所述第五位置与所述第二位置之间存在重叠的情况下，基于所述第三位置以及预设的移动轨迹，确定所述AR对象在所述AR场景中的第六位置；第二展示部分，配置为基于所述第六位置，展示与所述第三图像帧对应的第二AR场景图像。
在一种可能的实现方式中，所述移动轨迹包括以所述第二位置为中心、以所述第三位 置与所述第二位置之间的距离为半径的圆形。
在一种可能的实现方式中，所述装置还包括：第六确定部分，配置为对第四图像帧进行识别，确定所述第四图像帧中的目标对象在所述AR场景中的第七位置，所述第四图像帧包括所述实景视频流中处于所述第一图像帧之后的图像帧；第三展示部分，配置为在所述第七位置与所述第二位置不存在重叠的情况下，基于所述第七位置及所述第二位置，展示与所述第四图像帧对应的第三AR场景图像。
在一种可能的实现方式中，所述判断部分，还包括：第四确定子部分，配置为在所述深度值未处于所述深度区间内的情况下，确定所述目标对象与所述AR对象之间不存在重叠。
在一种可能的实现方式中，所述装置还包括：第四展示部分，配置为在所述目标对象与所述AR对象之间不存在重叠的情况下，基于所述第一位置及所述第二位置，展示与所述第一图像帧对应的第四AR场景图像。
在一种可能的实现方式中，所述第一展示部分85，具体包括：基于所述第三位置与所述第一位置的相对位置关系，在所述第一图像帧中渲染所述AR对象，生成所述第一AR场景图像并进行展示。
在本申请实施例中，能够实现在目标对象与AR对象之间存在重叠的情况下，确定出AR对象在AR场景中的第三位置，并且第三位置与第一位置不重叠，进而可以使基于第一位置和第三位置展示的AR场景图像中，AR对象和目标对象之间不重叠，从而改善AR场景中目标对象与AR对象之间的重叠交错现象，提升AR场景的真实感和沉浸感。
在一些实施例中，本申请实施例提供的装置具有的功能或包含的部分可以配置为执行上文方法实施例描述的方法，其实现可以参照上文方法实施例的描述，为了简洁，这里不再赘述。
本申请实施例还提出一种计算机可读存储介质，其上存储有计算机程序指令，所述计算机程序指令被处理器执行时实现上述方法。计算机可读存储介质可以是非易失性计算机可读存储介质。
本申请实施例还提出一种电子设备，包括：处理器；配置为存储处理器可执行指令的存储器；其中，所述处理器被配置为调用所述存储器存储的指令，以执行上述方法。
本申请实施例还提供了一种计算机程序，包括计算机可读代码，当计算机可读代码在电子设备上运行时，电子设备中的处理器执行配置为实现如上任一实施例提供的场景图像展示方法的指令。
本申请实施例还提供了另一种计算机程序，配置为存储计算机可读指令，指令被执行时使得计算机执行上述任一实施例提供的场景图像展示方法的操作。
本申请实施例还提供了一种计算机程序产品，计算机程序产品包括计算机程序或指令，在计算机程序或指令在计算机上运行的情况下，使得计算机执行上述任一实施例提供的场景图像展示方法。
电子设备可以被提供为终端或其它形态的设备。
图9示出根据本申请实施例的一种电子设备900的框图。例如，电子设备900可以包括支持AR技术的AR设备。
参照图9，电子设备900可以包括以下一个或多个组件：处理组件902，存储器904，电源组件906，多媒体组件908，音频组件910，输入/输出(I/O)的接口912，传感器组件914，以及通信组件916。
处理组件902通常控制电子设备900的整体操作，诸如与显示，电话呼叫，数据通信，相机操作和记录操作相关联的操作。处理组件902可以包括一个或多个处理器920来执行指令，以完成上述的方法的全部或部分步骤。此外，处理组件902可以包括一个或多个模块，便于处理组件902和其他组件之间的交互。例如，处理组件902可以包括多媒体模块，以方便多媒体组件908和处理组件902之间的交互。
存储器904被配置为存储各种类型的数据以支持在电子设备900的操作。这些数据的示例包括用于在电子设备900上操作的任何应用程序或方法的指令，联系人数据，电话簿数据，消息，图片，视频等。存储器904可以由任何类型的易失性或非易失性存储设备或者它们的组合实现，如静态随机存取存储器(SRAM)，电可擦除可编程只读存储器(EEPROM)，可擦除可编程只读存储器(EPROM)，可编程只读存储器(PROM)，只读存储器(ROM)，磁存储器，快闪存储器，磁盘或光盘。
电源组件906为电子设备900的各种组件提供电力。电源组件906可以包括电源管理系统，一个或多个电源，及其他与为电子设备900生成、管理和分配电力相关联的组件。
多媒体组件908包括在所述电子设备900和用户之间的提供一个输出接口的屏幕。在一些实施例中，屏幕可以包括液晶显示器(LCD)和触摸面板(TP)。如果屏幕包括触摸面板，屏幕可以被实现为触摸屏，以接收来自用户的输入信号。触摸面板包括一个或多个触摸传感器以感测触摸、滑动和触摸面板上的手势。所述触摸传感器可以不仅感测触摸或滑动动作的边界，而且还检测与所述触摸或滑动操作相关的持续时间和压力。在一些实施例中，多媒体组件908包括一个前置摄像头和/或后置摄像头。当电子设备900处于操作模式，如拍摄模式或视频模式时，前置摄像头和/或后置摄像头可以接收外部的多媒体数据。每个前置摄像头和后置摄像头可以是一个固定的光学透镜系统或具有焦距和光学变焦能力。
音频组件910被配置为输出和/或输入音频信号。例如，音频组件910包括一个麦克风(MIC)，当电子设备900处于操作模式，如呼叫模式、记录模式和语音识别模式时，麦克风被配置为接收外部音频信号。所接收的音频信号可以被进一步存储在存储器904或经由通信组件916发送。在一些实施例中，音频组件910还包括一个扬声器，用于输出音频信号。
I/O接口912为处理组件902和外围接口模块之间提供接口，上述外围接口模块可以是键盘，点击轮，按钮等。这些按钮可包括但不限于：主页按钮、音量按钮、启动按钮和锁定按钮。
传感器组件914包括一个或多个传感器，用于为电子设备900提供各个方面的状态评估。例如，传感器组件914可以检测到电子设备900的打开/关闭状态，组件的相对定位，例如所述组件为电子设备900的显示器和小键盘，传感器组件914还可以检测电子设备900或电子设备900一个组件的位置改变，用户与电子设备900接触的存在或不存在，电子设备900方位或加速/减速和电子设备900的温度变化。传感器组件914可以包括接近传感器，被配置用来在没有任何的物理接触时检测附近物体的存在。传感器组件914还可以包括光传感器，如互补金属氧化物半导体(CMOS)或电荷耦合装置(CCD)图像传感器，用于在成像应用中使用。在一些实施例中，该传感器组件914还可以包括加速度传感器，陀螺仪传感器，磁传感器，压力传感器或温度传感器。
通信组件916被配置为便于电子设备900和其他设备之间有线或无线方式的通信。电子设备900可以接入基于通信标准的无线网络，如无线网络(WiFi)，第二代移动通信技术(2G)或第三代移动通信技术(3G)，或它们的组合。在一个示例性实施例中，通信组件916经由广播信道接收来自外部广播管理系统的广播信号或广播相关信息。在一个示例性实施例中，所述通信组件916还包括近场通信(NFC)模块，以促进短程通信。例如，在NFC模块可基于射频识别(RFID)技术，红外数据协会(IrDA)技术，超宽带(UWB)技术，蓝牙(BT)技术和其他技术来实现。
在示例性实施例中，电子设备900可以被一个或多个应用专用集成电路(ASIC)、数字信号处理器(DSP)、数字信号处理设备(DSPD)、可编程逻辑器件(PLD)、现场可编程门阵列(FPGA)、控制器、微控制器、微处理器或其他电子元件实现，用于执行上述方法。
在示例性实施例中，还提供了一种非易失性计算机可读存储介质，例如包括计算机程序指令的存储器904，上述计算机程序指令可由电子设备900的处理器920执行以完成上述方法。
本申请实施例可以是系统、方法和/或计算机程序产品。计算机程序产品可以包括计算机可读存储介质，其上载有用于使处理器实现本申请实施例的计算机可读程序指令。
计算机可读存储介质可以是可以保持和存储由指令执行设备使用的指令的有形设备。计算机可读存储介质例如可以是(但不限于)电存储设备、磁存储设备、光存储设备、电磁存储设备、半导体存储设备或者上述的任意合适的组合。计算机可读存储介质具体(非穷举的列表)包括：便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、静态随机存取存储器(SRAM)、便携式压缩盘只读存储器(CD-ROM)、数字多功能盘(DVD)、记忆棒、软盘、机械编码设备、例如其上存储有指令的打孔卡或凹槽内凸起结构、以及上述的任意合适的组合。这里所使用的计算机可读存储介质不被解释为瞬时信号本身，诸如无线电波或者其他自由传播的电磁波、通过波导或其他传输媒介传播的电磁波(例如，通过光纤电缆的光脉冲)、或者通过电线传输的电信号。
这里所描述的计算机可读程序指令可以从计算机可读存储介质下载到各个计算/处理设备，或者通过网络、例如因特网、局域网、广域网和/或无线网下载到外部计算机或外部存储设备。网络可以包括铜传输电缆、光纤传输、无线传输、路由器、防火墙、交换机、网关计算机和/或边缘服务器。每个计算/处理设备中的网络适配卡或者网络接口从网络接收计算机可读程序指令，并转发该计算机可读程序指令，以供存储在各个计算/处理设备中的计算机可读存储介质中。
用于执行本申请实施例操作的计算机程序指令可以是汇编指令、指令集架构(ISA)指令、机器指令、机器相关指令、微代码、固件指令、状态设置数据、或者以一种或多种编程语言的任意组合编写的源代码或目标代码，所述编程语言包括面向对象的编程语言—诸如Smalltalk、C++等，以及常规的过程式编程语言，诸如“C”语言或类似的编程语言。计算机可读程序指令可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络—包括局域网(LAN)或广域网(WAN)—连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。在一些实施例中，通过利用计算机可读程序指令的状态信息来个性化定制电子电路，例如可编程逻辑电路、现场可编程门阵列(FPGA)或可编程逻辑阵列(PLA)，该电子电路可以执行计算机可读程序指令，从而实现本申请实施例。
这里参照根据本申请实施例的方法、装置(系统)和计算机程序产品的流程图和/或框图描述了本申请实施例。应当理解，流程图和/或框图的每个方框以及流程图和/或框图中各方框的组合，都可以由计算机可读程序指令实现。
这些计算机可读程序指令可以提供给通用计算机、专用计算机或其它可编程数据处理装置的处理器，从而生产出一种机器，使得这些指令在通过计算机或其它可编程数据处理装置的处理器执行时，产生了实现流程图和/或框图中的一个或多个方框中规定的功能/动作的装置。也可以把这些计算机可读程序指令存储在计算机可读存储介质中，这些指令使得计算机、可编程数据处理装置和/或其他设备以特定方式工作，从而，存储有指令的计算机可读介质则包括一个制造品，其包括实现流程图和/或框图中的一个或多个方框中规定的功能/动作的各个方面的指令。
也可以把计算机可读程序指令加载到计算机、其它可编程数据处理装置、或其它设备上，使得在计算机、其它可编程数据处理装置或其它设备上执行一系列操作步骤，以产生计算机实现的过程，从而使得在计算机、其它可编程数据处理装置、或其它设备上执行的指令实现流程图和/或框图中的一个或多个方框中规定的功能/动作。
附图中的流程图和框图显示了根据本申请的多个实施例的系统、方法和计算机程序产 品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段或指令的一部分，所述模块、程序段或指令的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个连续的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或动作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。
该计算机程序产品可以通过硬件、软件或其结合的方式实现。在一个可选实施例中，所述计算机程序产品体现为计算机存储介质，在另一个可选实施例中，计算机程序产品体现为软件产品，例如软件开发包(Software Development Kit，SDK)等等。
以上已经描述了本申请的各实施例，上述说明是示例性的，并非穷尽性的，并且也不限于所披露的各实施例。在不偏离所说明的各实施例的范围和精神的情况下，对于本技术领域的普通技术人员来说许多修改和变更都是显而易见的。本文中所用术语的选择，旨在最好地解释各实施例的原理、实际应用或对市场中的技术的改进，或者使本技术领域的其它普通技术人员能理解本文披露的各实施例。
工业实用性
本申请实施例公开了场景图像展示方法、装置、设备、存储介质、程序及产品，其中，场景图像展示方法，包括：获取实景视频流；对所述实景视频流的第一图像帧进行识别，确定所述第一图像帧中的目标对象以及所述目标对象在增强现实AR场景中的第一位置；基于所述第一位置与所述AR场景中的AR对象的第二位置，确定所述目标对象与所述AR对象之间是否存在重叠；在所述目标对象与所述AR对象之间存在重叠的情况下，确定所述AR对象在所述AR场景中的第三位置，所述第三位置与所述第一位置不重叠；基于所述第三位置，展示与所述第一图像帧对应的第一AR场景图像。

Claims (22)

  1. 一种场景图像展示方法，所述方法由电子设备执行，包括：
    获取实景视频流；
    对所述实景视频流的第一图像帧进行识别，确定所述第一图像帧中的目标对象以及所述目标对象在增强现实AR场景中的第一位置；
    基于所述第一位置与所述AR场景中的AR对象的第二位置，确定所述目标对象与所述AR对象之间是否存在重叠；
    在所述目标对象与所述AR对象之间存在重叠的情况下，确定所述AR对象在所述AR场景中的第三位置，所述第三位置与所述第一位置不重叠；
    基于所述第三位置，展示与所述第一图像帧对应的第一AR场景图像。
  2. 根据权利要求1所述的方法，其中，所述对所述实景视频流的第一图像帧进行识别，确定所述第一图像帧中的目标对象以及所述目标对象在增强现实AR场景中的第一位置，包括：
    对所述第一图像帧进行人体识别，确定所述第一图像帧中目标对象所在的人体区域；
    基于所述人体区域，确定所述目标对象在所述AR场景中的深度值，所述第一位置包括所述深度值和所述人体区域。
  3. 根据权利要求2所述的方法，其中，所述AR对象的第二位置包括所述AR对象在所述AR场景中的深度区间和所述AR对象在所述AR场景中的AR对象区域，
    所述基于所述第一位置与所述AR场景中的AR对象的第二位置，确定所述目标对象与所述AR对象之间是否存在重叠，包括：
    确定所述深度值是否处于深度区间内，以及所述人体区域与所述AR对象区域之间是否存在重叠区域；
    在所述深度值处于所述深度区间内，且所述人体区域与所述AR对象区域之间存在重叠区域的情况下，确定所述第一位置与所述第二位置之间存在重叠，并确定所述目标对象与所述AR对象之间存在重叠。
  4. 根据权利要求2或3所述的方法，其中，所述方法还包括：
    对第二图像帧进行识别，确定所述第二图像帧中的目标对象在所述AR场景中的第四位置，所述第二图像帧包括所述实景视频流中处于所述第一图像帧之前的一个或多个图像帧；
    所述基于所述第一位置与所述AR场景中的AR对象的第二位置，确定所述目标对象与所述AR对象之间是否存在重叠，还包括：
    在所述第一位置与所述第二位置之间存在重叠，且所述第四位置与所述第二位置之间存在重叠的情况下，确定所述目标对象与所述AR对象之间存在重叠。
  5. 根据权利要求1至4任一项所述的方法，其中，所述方法还包括：
    对第三图像帧进行识别，确定所述第三图像帧中的目标对象在所述AR场景中的第五位置，所述第三图像帧包括所述实景视频流中处于所述第一图像帧之后的图像帧；
    在所述第五位置与所述第二位置之间存在重叠的情况下，基于所述第三位置以及预设的移动轨迹，确定所述AR对象在所述AR场景中的第六位置；
    基于所述第六位置，展示与所述第三图像帧对应的第二AR场景图像。
  6. 根据权利要求5所述的方法，其中，所述移动轨迹包括以所述第二位置为中心、以所述第三位置与所述第二位置之间的距离为半径的圆形。
  7. 根据权利要求1至6任一项所述的方法，其中，所述方法还包括：
    对第四图像帧进行识别，确定所述第四图像帧中的目标对象在所述AR场景中的第七位置，所述第四图像帧包括所述实景视频流中处于所述第一图像帧之后的图像帧；
    在所述第七位置与所述第二位置不存在重叠的情况下，基于所述第七位置及所述第二位置，展示与所述第四图像帧对应的第三AR场景图像。
  8. 根据权利要求1至7任一项所述的方法，其中，所述方法还包括：
    在所述目标对象与所述AR对象之间不存在重叠的情况下，基于所述第一位置及所述第二位置，展示与所述第一图像帧对应的第四AR场景图像。
  9. 根据权利要求1至8任一项所述的方法，其中，所述基于所述第三位置，展示与所述第一图像帧对应的第一AR场景图像，包括：
    基于所述第三位置与所述第一位置的相对位置关系，在所述第一图像帧中渲染所述AR对象，生成所述第一AR场景图像并进行展示。
  10. 一种场景图像展示装置，包括：
    获取部分，配置为获取实景视频流；
    第一确定部分，配置为对实景视频流的第一图像帧进行识别，确定所述第一图像帧中的目标对象以及所述目标对象在增强现实AR场景中的第一位置；
    判断部分，配置为基于所述第一位置与所述AR场景中的AR对象的第二位置，确定所述目标对象与所述AR对象之间是否存在重叠；
    第二确定部分，配置为在所述目标对象与所述AR对象之间存在重叠的情况下，确定所述AR对象在所述AR场景中的第三位置，所述第三位置与所述第一位置不重叠；
    第一展示部分，配置为基于所述第三位置，展示与所述第一图像帧对应的第一AR场景图像。
  11. 根据权利要求10所述的场景图像展示装置，其中，所述第一确定部分，包括：
    人体区域确定子部分，配置为对所述第一图像帧进行人体识别，确定所述第一图像帧中目标对象所在的人体区域；
    深度值确定子部分，配置为基于所述人体区域，确定所述目标对象在所述AR场景中的深度值，所述第一位置包括所述深度值和所述人体区域。
  12. 根据权利要求11所述的场景图像展示装置，其中，所述AR对象的第二位置包括所述AR对象在所述AR场景中的深度区间和所述AR对象在所述AR场景中的AR对象区域，所述判断部分，包括：
    判断子部分，配置为确定所述深度值是否处于深度区间内，以及所述人体区域与所述AR对象区域之间是否存在重叠区域；
    第一确定子部分，配置为在所述深度值处于所述深度区间内，且所述人体区域与所述AR对象区域之间存在重叠区域的情况下，确定所述第一位置与所述第二位置之间存在重叠；
    第二确定子部分，配置为在所述第一位置与所述第二位置之间存在重叠的情况下，确定所述目标对象与所述AR对象之间存在重叠。
  13. 根据权利要求11或12所述的场景图像展示装置，其中，所述装置还包括：
    第三确定部分，配置为对第二图像帧进行识别，确定所述第二图像帧中的目标对象在所述AR场景中的第四位置，所述第二图像帧包括所述实景视频流中处于所述第一图像帧之前的一个或多个图像帧；
    所述判断部分，还包括：第三确定子部分，配置为在所述第一位置与所述第二位置之间存在重叠，且所述第四位置与所述第二位置之间存在重叠的情况下，确定所述目标对象与所述AR对象之间存在重叠。
  14. 根据权利要求10至13任一项所述的场景图像展示装置，其中，所述装置还包括：
    第四确定部分，配置为对第三图像帧进行识别，确定所述第三图像帧中的目标对象在所述AR场景中的第五位置，所述第三图像帧包括所述实景视频流中处于所述第一图像帧之后的图像帧；
    第五确定部分，配置为在所述第五位置与所述第二位置之间存在重叠的情况下，基于 所述第三位置以及预设的移动轨迹，确定所述AR对象在所述AR场景中的第六位置；
    第二展示部分，配置为基于所述第六位置，展示与所述第三图像帧对应的第二AR场景图像。
  15. 根据权利要求14所述的场景图像展示装置，其中，所述移动轨迹包括以所述第二位置为中心、以所述第三位置与所述第二位置之间的距离为半径的圆形。
  16. 根据权利要求10至15任一项所述的场景图像展示装置，其中，所述装置还包括：
    第六确定部分，配置为对第四图像帧进行识别，确定所述第四图像帧中的目标对象在所述AR场景中的第七位置，所述第四图像帧包括所述实景视频流中处于所述第一图像帧之后的图像帧；
    第三展示部分，配置为在所述第七位置与所述第二位置不存在重叠的情况下，基于所述第七位置及所述第二位置，展示与所述第四图像帧对应的第三AR场景图像。
  17. 根据权利要求10至16任一项所述的场景图像展示装置，其中，所述装置还包括：
    第四展示部分，配置为在所述目标对象与所述AR对象之间不存在重叠的情况下，基于所述第一位置及所述第二位置，展示与所述第一图像帧对应的第四AR场景图像。
  18. 根据权利要求10至17任一项所述的场景图像展示装置，其中，所述第一展示部分，具体包括：
    基于所述第三位置与所述第一位置的相对位置关系，在所述第一图像帧中渲染所述AR对象，生成所述第一AR场景图像并进行展示。
  19. 一种电子设备，包括：
    处理器；
    配置为存储处理器可执行指令的存储器；
    其中，所述处理器被配置为调用所述存储器存储的指令，以执行权利要求1至9中任意一项所述的方法。
  20. 一种计算机可读存储介质，其上存储有计算机程序指令，所述计算机程序指令被处理器执行时实现权利要求1至9中任意一项所述的方法。
  21. 一种计算机程序，包括计算机可读代码，当所述计算机可读代码在电子设备上运行时，电子设备中的处理器执行配置为实现权利要求1至9中任意一项所述的方法的指令。
  22. 一种计算机程序产品，所述计算机程序产品包括计算机程序或指令，在所述计算机程序或指令在计算机上运行的情况下，使得所述计算机执行权利要求1至9中任意一项所述的方法。
PCT/CN2021/106934 2021-01-15 2021-07-16 场景图像展示方法、装置、设备、存储介质、程序及产品 WO2022151686A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110056332.1 2021-01-15
CN202110056332.1A CN112860061A (zh) 2021-01-15 2021-01-15 场景图像展示方法及装置、电子设备和存储介质

Publications (1)

Publication Number Publication Date
WO2022151686A1 true WO2022151686A1 (zh) 2022-07-21

Family

ID=76006956

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/106934 WO2022151686A1 (zh) 2021-01-15 2021-07-16 场景图像展示方法、装置、设备、存储介质、程序及产品

Country Status (2)

Country Link
CN (1) CN112860061A (zh)
WO (1) WO2022151686A1 (zh)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112860061A (zh) * 2021-01-15 2021-05-28 深圳市慧鲤科技有限公司 场景图像展示方法及装置、电子设备和存储介质
CN113382275B (zh) * 2021-06-07 2023-03-07 广州博冠信息科技有限公司 直播数据的生成方法、装置、存储介质及电子设备
CN113485544A (zh) * 2021-07-20 2021-10-08 歌尔光学科技有限公司 增强现实设备的帧率调整方法、系统、设备及存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108615261A (zh) * 2018-04-20 2018-10-02 深圳市天轨年华文化科技有限公司 增强现实中图像的处理方法、处理装置及存储介质
CN111833458A (zh) * 2020-06-30 2020-10-27 北京市商汤科技开发有限公司 图像显示方法及装置、设备、计算机可读存储介质
CN111860252A (zh) * 2020-07-09 2020-10-30 北京市商汤科技开发有限公司 图像处理方法、设备及存储介质
CN112860061A (zh) * 2021-01-15 2021-05-28 深圳市慧鲤科技有限公司 场景图像展示方法及装置、电子设备和存储介质

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE112017007923B4 (de) * 2017-10-20 2021-06-10 Mitsubishi Electric Corporation Anzeigesteuerungsvorrichtung, Anzeigesteuerungsverfahren und Anzeigesystem
WO2019163129A1 (ja) * 2018-02-26 2019-08-29 三菱電機株式会社 仮想物体表示制御装置、仮想物体表示システム、仮想物体表示制御方法、及び仮想物体表示制御プログラム
CN109358748B (zh) * 2018-09-30 2019-09-10 深圳仓谷创新软件有限公司 一种用手与手机ar虚拟物体交互的设备和方法
CN111640203B (zh) * 2020-06-12 2024-04-12 上海商汤智能科技有限公司 一种图像处理方法及装置
CN111833454B (zh) * 2020-06-30 2023-11-28 北京市商汤科技开发有限公司 展示方法、装置、设备和计算机可读存储介质
CN112150349A (zh) * 2020-09-23 2020-12-29 北京市商汤科技开发有限公司 一种图像处理方法、装置、计算机设备及存储介质

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108615261A (zh) * 2018-04-20 2018-10-02 深圳市天轨年华文化科技有限公司 增强现实中图像的处理方法、处理装置及存储介质
CN111833458A (zh) * 2020-06-30 2020-10-27 北京市商汤科技开发有限公司 图像显示方法及装置、设备、计算机可读存储介质
CN111860252A (zh) * 2020-07-09 2020-10-30 北京市商汤科技开发有限公司 图像处理方法、设备及存储介质
CN112860061A (zh) * 2021-01-15 2021-05-28 深圳市慧鲤科技有限公司 场景图像展示方法及装置、电子设备和存储介质

Also Published As

Publication number Publication date
CN112860061A (zh) 2021-05-28

Similar Documents

Publication Publication Date Title
CN110348524B (zh) 一种人体关键点检测方法及装置、电子设备和存储介质
WO2022151686A1 (zh) 场景图像展示方法、装置、设备、存储介质、程序及产品
US20190221041A1 (en) Method and apparatus for synthesizing virtual and real objects
TWI767596B (zh) 場景深度和相機運動預測方法、電子設備和電腦可讀儲存介質
WO2022043741A1 (zh) 网络训练、行人重识别方法及装置、存储介质、计算机程序
US20210097715A1 (en) Image generation method and device, electronic device and storage medium
CN110928627B (zh) 界面展示方法及装置、电子设备和存储介质
WO2016192325A1 (zh) 视频文件的标识处理方法及装置
WO2022188305A1 (zh) 信息展示方法及装置、电子设备、存储介质及计算机程序
CN111401230B (zh) 姿态估计方法及装置、电子设备和存储介质
TWI767217B (zh) 坐標系對齊的方法及裝置、電子設備和計算機可讀存儲介質
WO2022134475A1 (zh) 点云地图构建方法及装置、电子设备、存储介质和程序
WO2023051356A1 (zh) 一种虚拟对象的显示方法及装置、电子设备和存储介质
CN112184787A (zh) 图像配准方法及装置、电子设备和存储介质
WO2023273499A1 (zh) 深度检测方法及装置、电子设备和存储介质
CN110782532B (zh) 图像生成方法、生成装置、电子设备及存储介质
WO2023273498A1 (zh) 深度检测方法及装置、电子设备和存储介质
CN111680646A (zh) 动作检测方法及装置、电子设备和存储介质
CN114581525A (zh) 姿态确定方法及装置、电子设备和存储介质
WO2022151687A1 (zh) 合影图像生成方法、装置、设备、存储介质、计算机程序及产品
WO2023273050A1 (zh) 活体检测方法及装置、电子设备和存储介质
CN112330721B (zh) 三维坐标的恢复方法及装置、电子设备和存储介质
WO2022110785A1 (zh) 定位方法及装置、电子设备、存储介质、计算机程序产品、计算机程序
EP3848894A1 (en) Method and device for segmenting image, and storage medium
CN114549797A (zh) 画作展示方法、装置、电子设备、存储介质和程序产品

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21918881

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 23.10.2023)

122 Ep: pct application non-entry in european phase

Ref document number: 21918881

Country of ref document: EP

Kind code of ref document: A1