WO2021218547A1

WO2021218547A1 - Method for superimposing live image of person onto real scene, and electronic device

Info

Publication number: WO2021218547A1
Application number: PCT/CN2021/084372
Authority: WO
Inventors: 李江亮; 周硙; 方俊
Original assignee: 北京外号信息技术有限公司
Priority date: 2020-04-26
Filing date: 2021-03-31
Publication date: 2021-11-04
Also published as: CN111242704A; TWI795762B; TW202205176A; CN111242704B

Abstract

Provided are a method for superimposing a live image of a person onto a real scene, and an electronic device. The method comprises: determining the position and attitude of a device in a space, wherein the device has an image collector and a display medium; obtaining a spatial position configured for a live image of a person; on the basis of the position and attitude of the device and the spatial position of the live image of a person, determining a presentation position of the live image of a person on the display medium of the device; presenting, on the display medium of the device, a real scene collected by the image collector of the device; and receiving the live image of a person and superimposing the live image of a person at the presentation position onto the display medium.

Description

Method and electronic equipment for superimposing live image of people in real scene

Technical field

The present invention relates to the field of augmented reality technology, and in particular to a method and electronic device for superimposing live image of a person in a real scene observed by a device.

Background technique

The statements in this section are only for providing background information related to the technical solutions of the present application to help understanding, and they do not necessarily constitute prior art for the technical solutions of the present application.

In government affairs halls, bank branches, exhibition halls, scenic spots, shopping malls, supermarkets, airports, stations and other places, it is often necessary to arrange some service personnel to provide users with on-site explanations or consulting services in certain locations, for example, to provide users with policies in the government affairs hall Consultancy services, provide users with introductions of various wealth management products at bank branches, introduce various products to users next to supermarket shelves, and so on.

However, this traditional service method requires close face-to-face (usually about 1 meter or even closer) verbal communication between service personnel and users, which will greatly increase the risk of cross-infection and affect Many industries (especially those that require on-site explanation personnel or service personnel) have resumed work smoothly. Although forcing service personnel and users to wear protective facilities such as masks can reduce this risk, it will affect the smoothness of communication and require additional protection costs. In addition, psychological research has shown that in the process of face-to-face dialogue, the transmission of information is actually carried out on both verbal and non-verbal levels, usually through non-verbal (for example, facial expressions, looks, postures, gestures). Etc.) The information conveyed accounts for more than 50% of the total amount of information in the entire communication process, and facial expressions and looks are very important parts. In the case of wearing masks and other protective facilities, most of the information conveyed through facial expressions and length is blocked and cannot be transmitted, which affects the effect of face-to-face communication.

In addition, with the above-mentioned traditional service methods, usually the same service person can only be responsible for users in one place. Taking a bank branch as an example, even if staff a at bank branch A is currently idle and staff b at bank branch B is busy, staff a cannot serve other users currently waiting at bank branch B. Therefore, the above-mentioned traditional service methods are inefficient and costly. With the rapid arrival of an aging society and the continuous increase of labor costs, the drawbacks of the above-mentioned traditional service methods will become more and more obvious.

In order to solve at least one of the above-mentioned problems, the present application provides a method and electronic device for superimposing a live image of a person in a real scene observed by the device.

Summary of the invention

One aspect of the present invention relates to a method for superimposing live image of a person in a real scene, including: determining the position and posture of a device in space, wherein the device has an image acquisition device and a display medium; The spatial position of the live character image setting; based on the position and posture of the device and the spatial position of the live character image, determine the presentation position of the live character image on the display medium of the device; display on the device Presenting the real scene collected by the image acquisition device of the device on a medium; and receiving the live image of the person and superimposing the live image of the person at the presentation position on the display medium.

In one embodiment, the live character image received by the device is a live character image with a transparent background or a live character image without a background; or, the device processes the received live character image to generate a transparent background Live images of people or live images of people without background.

In one embodiment, the method further includes: determining a live image of a person to be presented to the device.

In one embodiment, the position of the device in the space is used to determine the live image of the person to be presented to the device.

In one embodiment, the live character image to be presented to the device is determined by the position and posture of the device in space.

In an embodiment, the method further includes: obtaining the posture in the space set for the live broadcast image of the person.

In one embodiment, the method further includes: determining the presentation posture of the live person image on the display medium of the device based on the position and posture of the device and the posture of the live person image.

In one embodiment, the front of the live person image is always facing the device.

In one embodiment, the method further includes: collecting video, audio, or text input of the user of the device; and sending the video, audio, or text input to a live broadcaster who provides the live image of the person.

In one embodiment, the method further includes: after superimposing the live person image on the display medium of the device, determining the live person image according to the new position and posture of the device and the spatial position of the live person image The new presentation position of the live character image on the display medium of the device.

In an embodiment, the method further includes: after superimposing the live character image on the display medium of the device, the presentation position of the live character image on the display medium remains unchanged.

In one embodiment, the method further includes: after superimposing the live character image on the display medium of the device, making the live character image appear on the display medium according to the instruction of the user of the device The location remains the same.

In one embodiment, the determining the position and posture of the device in space includes: scanning the optical communication device deployed in the real scene by the device to determine the initial position and posture of the device in space, and continuously tracking The position and posture of the device in space change.

In one embodiment, the method further includes: the device obtains the identification information of the optical communication device, and determines the live person image to be presented for the device through the identification information.

In one embodiment, at least two live person images are superimposed on the display medium of the device.

In one embodiment, the live image of a person is a two-dimensional image of a person or a three-dimensional image of a person.

In one embodiment, the method further includes: before receiving the live person image, instructing a live broadcaster associated with the live person image to provide the live person image.

Another aspect of the present invention relates to a storage medium in which a computer program is stored, and when the computer program is executed by a processor, it can be used to implement the above-mentioned method.

Another aspect of the present invention relates to an electronic device, which includes a processor and a memory, and a computer program is stored in the memory. When the computer program is executed by the processor, the computer program can be used to implement the above-mentioned method.

Through the solution of the present invention, a live interaction method based on the position in the real scene or bound with the position in the real scene is realized, so that the device user can experience the non-contact scene service similar to the live service, and There is no need for service personnel and users to have face-to-face close verbal communication, which can greatly reduce the risk of cross-infection when there is an epidemic, and help related industries to resume work and production smoothly. In addition, through this solution, the same service personnel can serve users in different locations, which can break geographic limitations, save labor costs, and improve service efficiency.

Description of the drawings

The embodiments of the present invention will be further described below with reference to the accompanying drawings, in which:

Fig. 1 shows a method for superimposing live image of a person in a real scene observed by a device according to an embodiment;

Figure 2 shows a schematic diagram of a user watching live images of people in a real scene;

Fig. 3 shows a live broadcaster and camera equipment used to provide live images of people in the real scene shown in Fig. 2;

Figure 4 shows a schematic image presented on the display medium of the user's device;

Fig. 5 is an example real image for showing the actual effect of the present invention;

Figure 6 shows an exemplary optical label;

Figure 7 shows an exemplary optical label network.

Detailed ways

In order to make the objectives, technical solutions, and advantages of the present invention clearer, the following further describes the present invention in detail through specific embodiments with reference to the accompanying drawings. It should be understood that the specific embodiments described here are only used to explain the present invention, but not to limit the present invention.

Fig. 1 shows a method for superimposing a live image of a person in a real scene observed by a device according to an embodiment. The device may be, for example, a device carried or controlled by a user (for example, a mobile phone, a tablet computer, smart glasses, AR/VR glasses, an AR/VR helmet, a smart watch, etc.), and has an image acquisition device (for example, a camera) and a display medium ( For example screen). The method may include the following steps:

Step 1001: Determine the position and posture of the device in space.

Various feasible methods can be used to determine the location and posture of the device in space, for example, visual signs can be arranged in the space and the location and posture of the device can be determined by analyzing the images of the visual signs collected by the device; real scenes can be established The position and attitude of the equipment can be determined by analyzing the scene images collected by the equipment; the position and attitude of the equipment can be determined by using high-precision gyroscopes; it can be arranged in space that can transmit radio signals Beacons, and determine the location and attitude of the device by analyzing the radio signals received by the device; you can use satellite positioning signals to determine the location of the device and use a gyroscope to determine the attitude of the device; and a combination of the above methods; etc. Wait.

Step 1002: Obtain the spatial position set for the live character image.

For example, service personnel in government halls, bank branches, exhibition halls, scenic spots, shopping malls, supermarkets, airports, stations, etc. (may be referred to as "live broadcasters" in this article) can provide live images of people in real time. Images are used to provide content explanations to device users, answer device users’ inquiries, and so on. Through the use of live images of people, service personnel can explain to users remotely and in real time, and can answer user inquiries, etc., without the need to have close face-to-face communication with users, and there is no need to limit service personnel to a certain Fixed location.

For example, the live character can be represented or defined by the spatial position of a point on the live character image, the spatial positions of multiple points (for example, multiple points on the outline of the live character image), or the spatial position of the entire live character image area The spatial position of the image (that is, the presentation position of the live character image in space). For example, if the live character image is an image with a rectangular shape, for example, the position coordinates of the center point of the rectangular image can be used to define the spatial position of the rectangular image. For example, a certain corner of the rectangular image (for example, the upper left corner , Lower left corner, upper right corner, lower right corner) in space to define the spatial position of the rectangular image. For example, two opposite corners of the rectangular image (for example, the upper left corner and the lower right corner, or the lower left corner and the upper right corner) can be used. The position coordinates in space define the spatial position of the rectangular image, and so on.

Before obtaining the spatial location set for the live broadcast character image, various methods can be used to determine the live broadcast character image to be presented to the device. In one embodiment, the live person image that can be presented to the device can be determined by the location of the device in space and an optional posture. For example, the equipment can scan a certain visual sign installed in the exhibition hall to determine the location of the equipment in the exhibition hall and optional posture. Through the location of the equipment in the exhibition hall and the optional posture, it can be inquired to determine the current It can be a live video of people presented by the device (for example, a live video of people used for the introduction of a certain exhibit).

In one embodiment, other information may be used to determine the live person image to be presented for the device, for example, the identification information of the visual sign obtained by the device may be used to query to determine the live person image that can currently be presented for the device.

In one embodiment, multiple live person images that can be presented to the device may be obtained, and the device user can select from them to determine the live person image currently to be presented. For example, for a device user currently in the government affairs hall, the user can be reminded that there are multiple live images of people involved in a variety of services for presentation, and the user can choose what they are interested in according to their needs (for example, according to the business they want to handle) Live images of people.

In one embodiment, the live person images can be filtered based on information related to the device or the device user (for example, the user’s age, gender, occupation, etc.), so that the device user’s preferences may be presented to him on the live broadcast. image.

In one embodiment, after determining the live person image to be presented to the device user or before receiving the live person image, for example, the device may send a corresponding instruction or message to the live broadcaster for providing the live person image, So that the live broadcaster can start the live broadcast and send the live broadcast image of the person to the device.

In one embodiment, one live broadcaster may be associated with multiple live person images. For example, one live broadcaster may be responsible for multiple live person images corresponding to multiple exhibits in the exhibition hall. In this case, the instruction or message sent to the live broadcaster can identify the corresponding live person image (for example, the instruction or message sent to the broadcaster includes the identification information of the corresponding live person image), so that the broadcaster Knowing, for example, enables the live broadcaster to know which exhibit should be provided with the corresponding live person image.

In one embodiment, a live person image may be associated with multiple live broadcasters, and any idle broadcaster among the multiple live broadcasters may provide the live broadcast person image. In an embodiment, the device user may select his favorite live broadcaster, or the live broadcaster who responds to the device user's request first may provide the live broadcast character image.

In one embodiment, the posture in the space set for the live character image to be presented can also be obtained, which can be used, for example, to define the orientation of the live character image in the space.

Step 1003: Based on the location and posture of the device and the spatial position of the live person image, determine the presentation position of the live person image on the display medium of the device.

After determining the position and posture of the device in space, the current field of view of the image acquisition device of the device can actually be determined. Further, based on the spatial position of the live character image, it can be determined whether the live character image is within the current field of view of the image acquisition device of the device, and where it is located within the field of view, so as to determine the display medium of the live character image on the device The presentation position on the.

In one embodiment, in the case where the live character image has a posture in space, the posture of the live character image presented on the display medium of the device may be further determined based on the position and posture of the device and the posture of the live character image.

In one embodiment, a certain direction of the live character image can always face the device of the user who observes the live character image. For example, for a two-dimensional live image of a person, the front of the live image of the person can always face the user's device, so that even if the device user is in a different position or changes location, he can feel that the person in the live person image is always facing himself explain.

Step 1004: Present the real scene collected by the image acquisition device of the device on the display medium of the device.

The device can collect the real scene in real time through its image acquisition device, and present the image of the real scene on the display medium of the device.

Step 1005: Receive the live person image and superimpose the live person image at the presentation position on the display medium of the device.

In this way, the live character image can actually be superimposed on the appropriate position in the real scene observed by the device, so that the live character image closely integrated with the real scene can be provided to the device user, for example, to explain to the device user, Respond to inquiries, etc.

In one embodiment, the live character image received by the device may be a live character image with a transparent background (for example, a live character image with an alpha transparent channel) or a live character image without a background. For example, the live person image may be processed after collecting the live person image or during the process of transmitting the live person image to generate a live person image with a transparent background, and send it to the device. In one embodiment, the device may receive a live character image with an opaque background and process the live character image to generate a live character image with a transparent background or a live character image without a background. In order to facilitate the generation of a live character image with a transparent background or a live character image without a background, a monochrome background, such as a green cloth, can be arranged for the characters when the live character image is shot. In this way, the live image of people superimposed in the real scene can look like only people, without the original background when shooting people. In this way, when the user observes the live character image through the display medium of the device, only the character will be observed, but the original background of the character will not be observed, as if the character is actually located in the real scene, so that a better user experience can be achieved .

In one embodiment, in order to achieve better communication between the device user and the broadcaster, at least one of the image, sound, or text input of the device user may be collected through the device and sent to the broadcaster to So that both parties can interact in real time.

Figure 2 shows a schematic diagram of a user watching a live image of a person in a real scene. The real scene includes a shelf 202, and the user 201 holds the device 203 and watches the live image of a person who is arranged or embedded in the real scene through the display medium of the device 203. The deployment position of the live person image in the real scene is, for example, The dashed box 204 is shown. The position of the entire dashed frame 204 in space can be defined by the spatial position of one or more points on the dashed frame 204. The dashed frame 204 may have a preset or default posture, for example, the default dashed frame 204 is perpendicular to the ground.

FIG. 3 shows a live broadcaster 302 for providing live person images in the real scene shown in FIG. 2, and a camera device 301 for collecting images of the live broadcaster 302 to generate live broadcast person images.

Figure 4 shows a schematic image presented on the display medium of the device 203 of the user 201, in which an image of a real scene (including the shelf 202) is obtained by the image acquisition device of the device 203 and presented on the device 203 On the display medium. In addition, the device 203 also receives the live person image provided by the camera device 301 of the live broadcaster 302, and displays it on the display medium of the device 203 according to the location and posture of the device 203 and the spatial position set for the live person image. The position is superimposed on the live broadcast character image containing the live broadcaster 302 with a transparent background, thereby realizing the perfect integration of the live broadcaster 302 and the real scene.

Fig. 5 is an example real image for showing the actual effect of the present invention. The real scene shown by the real image includes a shelf. When the user uses a mobile phone to observe the real scene, the real scene presented on the screen of the user's mobile phone can be superimposed with a live character image containing an instructor with a transparent background. In this way, the user feels as if there is a real explainer introducing various products to him in front of the shelf.

In one embodiment, two or more characters may be included in the live character image, and the two or more characters may interact verbally or physically to provide the user with a more detailed explanation.

In one embodiment, at least two live person images may be arranged for the real scene, and at least two live person images may be superimposed on the display medium of the device. At least two live person images can be presented simultaneously or sequentially on the display medium of the device.

In one embodiment, the live image of a person may be a two-dimensional image of a person. In one embodiment, the live character image may be a three-dimensional character image. For example, when shooting images of people, multiple camera devices located around the people can be used to shoot from multiple different angles to provide three-dimensional images of people.

In one embodiment, the size of the live character image can also be set or adjusted, for example, adjusted so that the character therein has a size similar to that of a real person.

In one embodiment, after superimposing the live character image on the display medium of the device, the position and posture change of the device can be tracked, and the live character image can be determined in real time according to the new position and posture of the device and the spatial position of the live character image The new presentation position on the display medium of the device. Similarly, according to the new position and posture of the device and the posture in the space set for the live character image, the new presentation posture of the live character image on the display medium of the device can be determined in real time. This method can achieve a good augmented reality effect, making the device user feel that the live broadcaster is actually in the real scene.

In one embodiment, after the live character image is superimposed on the display medium of the device, the live character image can be made to have a fixed presentation position and/or presentation posture on the display medium.

In one embodiment, after the live character image is superimposed on the display medium of the device, the live character image may have a fixed presentation position and/or presentation posture on the display medium according to the instruction of the device user. In this way, even when the device user moves (for example, leaves the current location), the live image of the person can be watched in a desired presentation position and/or presentation posture through the display medium of the device. For example, after a live character image is superimposed on the display medium of the device, the device user can change the position and/or posture of the device in space, so that the live character image superimposed on the device display medium has the presentation position desired by the device user And/or presentation posture. At this time, the device user can send instructions (for example, by clicking a button presented on the device display medium) to make the current presentation position and/or presentation posture of the live character image remain unchanged thereafter, even if the device is in Change position or posture in space.

In one embodiment, the position and posture of the device in the space can be determined by the optical communication device arranged in the space. Optical communication devices are also called optical tags, and these two terms can be used interchangeably in this article. Optical tags can transmit information through different light-emitting methods, which have the advantages of long recognition distance and relaxed requirements for visible light conditions, and the information transmitted by optical tags can change over time, which can provide large information capacity and flexible configuration capabilities.

An optical tag usually includes a controller and at least one light source, and the controller can drive the light source through different driving modes to transmit different information to the outside. Fig. 6 shows an exemplary optical label 100, which includes three light sources (respectively a first light source 101, a second light source 102, and a third light source 103). The optical tag 100 also includes a controller (not shown in FIG. 6), which is used to select a corresponding driving mode for each light source according to the information to be transmitted. For example, in different driving modes, the controller can use different driving signals to control the light emitting mode of the light source, so that when a device with imaging function is used to photograph the light label 100, the imaging of the light source therein can show a different appearance. (For example, different colors, patterns, brightness, etc.). By analyzing the imaging of the light source in the optical label 100, the driving mode of each light source at the moment can be analyzed, so as to analyze the information transmitted by the optical label 100 at the moment. It can be understood that the optical label shown in FIG. 6 is only used as an example, and the optical label may have a different shape from the example shown in FIG. 6 and may have a different number and/or different shape of light sources from the example shown in FIG. 6.

In order to provide users with corresponding services based on optical tags, each optical tag can be assigned an identification information (ID), which is used to uniquely identify or identify the optical tag by the manufacturer, manager, or user of the optical tag . Generally, the light source can be driven by the controller in the optical tag to transmit the identification information outward, and the user can use the device to collect the image of the optical tag to obtain the identification information transmitted by the optical tag, so that the corresponding identification information can be accessed based on the identification information. Services, for example, accessing a webpage associated with the identification information, obtaining other information associated with the identification information (for example, location information of a light tag corresponding to the identification information), and so on. The device can obtain an image containing the optical label through image acquisition device on the optical label, and identify the information conveyed by the optical label by analyzing the imaging of the optical label (or each light source in the optical label) in the image.

The information related to each optical tag can be stored in the server. In reality, a large number of optical labels can also be constructed into an optical label network. Fig. 7 shows an exemplary optical label network, which includes a plurality of optical labels and at least one server. The identification information (ID) or other information of each optical label can be saved on the server, such as service information related to the optical label, description information or attributes related to the optical label, such as location information, model information, and Physical size information, physical shape information, posture or orientation information, etc. The optical label may also have unified or default physical size information and physical shape information. The device can use the identified identification information of the optical tag to query the server to obtain other information related to the optical tag. The location information of the optical tag may refer to the actual location of the optical tag in the physical world, which may be indicated by geographic coordinate information. The server may be a software program running on a computing device, a computing device, or a cluster composed of multiple computing devices. The optical tag may be offline, that is, the optical tag does not need to communicate with the server. Of course, it can be understood that online optical tags that can communicate with the server are also feasible.

In one embodiment, the device can determine its position relative to the optical label by collecting an image including the optical label and analyzing the image (for example, analyzing the imaging size of the optical label in the image, perspective distortion, etc.), and the relative position This can include the distance and orientation of the device relative to the optical tag. In one embodiment, the device can also determine its posture relative to the optical label by collecting an image including the optical label and analyzing the image. For example, when the imaging position or imaging area of the optical tag is located in the center of the imaging field of the device, it can be considered that the device is currently facing the optical tag.

In some embodiments, the device can identify the identification information transmitted by the optical tag by scanning the optical tag, and can obtain (for example, by querying) the position and posture information of the optical tag in the real scene coordinate system through the identification information. The real scene coordinate system may be, for example, a certain place coordinate system (for example, a coordinate system established for a certain room, building, park, etc.) or a world coordinate system. In this way, based on the position and posture information of the light tag in the real scene coordinate system and the position or posture information of the device relative to the light tag, the position or posture information of the device in the real scene coordinate system can be determined. Therefore, the determined position or posture of the device in space may be the position or posture of the device relative to the optical tag, but may also be the position or posture of the device in the real scene coordinate system.

In one embodiment, the device can identify the identification information transmitted by the optical tag by scanning the optical tag, and use the identification information to determine the scene information of the real scene where the optical tag is located. The scene information may be, for example, three-dimensional model information of the real scene, The point cloud information of the real scene, the information of the auxiliary signs around the light tag, and other information, etc. Thereafter, based on the determined scene information and the image of the real scene collected by the device, the position and/or posture of the device in the real scene can be determined through visual positioning.

After scanning the optical tag to determine the position and/or posture of the device in space, the device may be translated and/or rotated. In this case, various sensors built into the device (e.g., acceleration sensor, magnetic Sensors, direction sensors, gravity sensors, gyroscopes, cameras, etc.) measure or track changes in position and/or attitude through methods known in the art (for example, inertial navigation, visual odometry, SLAM, VSLAM, SFM, etc.) , So as to determine the real-time position and/or posture of the device. In one embodiment, the device can rescan the light tag when it is in the field of view of its camera to correct or re-determine its position or posture information.

In one embodiment, the device can obtain the identification information of the light tag. After that, the device can query and determine the live character image to be presented through the identification information, and obtain the spatial position set for the live character image. For example, the device can scan a light tag installed on a shelf in a supermarket and identify the identification information of the light tag. Through the identification information of the light tag, it can be queried and determined that the live image of people currently to be presented to the device is used to introduce the shelf. The live character image of the product of the product, and the spatial location of the live character image can be obtained.

In some embodiments of this application, service personnel are described as live broadcasters, but it is understood that this application is not limited to this, and the live broadcasters can be anyone who wants to provide live broadcast character images to other people, for example, speakers , Presenters, video conference participants, teachers, live broadcasters using various live broadcast apps, etc.

Live images of people can also be images synthesized or generated by a computer. For example, in one embodiment, a plane image or three-dimensional model of character A may be stored in advance, and then the character A can be synthesized or generated through real-time action features, voice features, etc. of character A, and the plane image or three-dimensional model of character A. A's live video. In this way, only real-time action features or voice features of character A can be transmitted, without the need to transmit real-time video of character A, thereby reducing the system's demand for transmission bandwidth and improving efficiency. In one embodiment, a plane image or three-dimensional model of character A may be stored in advance, and then the character A can be synthesized or generated by using the real-time action characteristics, voice features, etc. of character B and the plane image or three-dimensional model of character A. Live video. In this way, the character (for example, character A) in the live broadcast of the character image can be made different from the actual broadcaster (for example, character B). In addition, in this way, the character in the live character image may not be a real character, but a virtual character, such as an animated character.

In an embodiment of the present invention, the present invention can be implemented in the form of a computer program. The computer program can be stored in various storage media (for example, a hard disk, an optical disk, a flash memory, etc.), and when the computer program is executed by a processor, it can be used to implement the method of the present invention.

In another embodiment of the present invention, the present invention may be implemented in the form of an electronic device. The electronic device includes a processor and a memory, and a computer program is stored in the memory. When the computer program is executed by the processor, it can be used to implement the method of the present invention.

References herein to "each embodiment", "some embodiments", "one embodiment", or "an embodiment", etc. refer to the specific features, structures, or properties described in connection with the embodiments that are included in In at least one embodiment. Therefore, the appearances of the phrases "in various embodiments", "in some embodiments", "in one embodiment", or "in an embodiment" in various places throughout this document do not necessarily refer to the same implementation example. In addition, specific features, structures, or properties can be combined in any suitable manner in one or more embodiments. Therefore, a specific feature, structure, or property shown or described in combination with one embodiment can be combined in whole or in part with the feature, structure, or property of one or more other embodiments without limitation, as long as the combination is not incompatible. Logical or not working. Expressions similar to "according to A", "based on A", "through A" or "using A" appearing in this article mean non-exclusive, that is, "according to A" can cover "only according to A" or Covers "according to A and B" unless it is specifically stated to mean "only according to A". For clarity in this application, some illustrative operating steps are described in a certain order, but those skilled in the art can understand that each of these operating steps is not indispensable, and some of the steps can be omitted. Or replaced by other steps. These operation steps do not have to be executed sequentially in the manner shown. On the contrary, some of these operation steps can be executed in a different order according to actual needs, or executed in parallel, as long as the new execution method is not illogical or unable to work.

Thus, several aspects of at least one embodiment of the present invention have been described, and it can be understood that various changes, modifications and improvements can be easily made by those skilled in the art. Such changes, modifications and improvements are intended to be within the spirit and scope of the present invention. Although the present invention has been described through preferred embodiments, the present invention is not limited to the embodiments described here, and also includes various changes and changes made without departing from the scope of the present invention.

Claims

A method for superimposing live images of people in real scenes, including:

Determine the position and posture of the device in space, wherein the device has an image acquisition device and a display medium;

Obtaining the spatial position set for the live broadcast image of the person;

Determine the presentation position of the live person image on the display medium of the device based on the position and posture of the device and the spatial position of the live person image;

Presenting the real scene collected by the image acquisition device of the device on the display medium of the device; and

Receiving the live person image and superimposing the live person image on the presentation position on the display medium.
The method of claim 1, wherein:

The live character image received by the device is a live character image with a transparent background or a live character image without a background; or

The device processes the received live character image to generate a live character image with a transparent background or a live character image without a background.
The method according to claim 1 or 2, further comprising: determining a live image of a person to be presented for the device.
The method according to claim 3, wherein the live image of the person to be presented to the device is determined by the location of the device in the space.
The method according to claim 4, wherein the live image of the person to be presented to the device is determined by the position and posture of the device in the space.
The method according to claim 1 or 2, further comprising:

Obtain the posture in the space set for the live image of the person.
The method according to claim 6, further comprising:

Based on the position and posture of the device and the posture of the live person image, the presentation posture of the live person image on the display medium of the device is determined.
The method according to claim 1 or 2, wherein the front face of the live human image is always facing the device.
The method according to claim 1 or 2, further comprising:

Collect the image, sound or text input of the user of the device; and

The image, sound or text input is sent to the live broadcaster who provides the live broadcast image of the person.
The method according to claim 1 or 2, further comprising:

After the live character image is superimposed on the display medium of the device, it is determined that the live character image is on the display medium of the device according to the new position and posture of the device and the spatial position of the live character image The new presentation position of.
The method according to claim 1 or 2, further comprising:

After the live character image is superimposed on the display medium of the device, the presentation position of the live character image on the display medium remains unchanged.
The method according to claim 1 or 2, further comprising:

After the live broadcast character image is superimposed on the display medium of the device, the presentation position of the live broadcast character image on the display medium remains unchanged according to the instruction of the user of the device.
The method according to claim 1 or 2, wherein the determining the position and posture of the device in space comprises:

The device scans the optical communication device deployed in the real scene to determine the initial position and posture of the device in space, and continuously track the change of the device’s position and posture in the space.
The method according to claim 13, further comprising:

The device obtains the identification information of the optical communication device, and uses the identification information to determine the live image of the person to be presented for the device.
The method according to claim 1 or 2, wherein the live person image comprises a live person image synthesized or generated by a computer.
The method according to claim 1 or 2, wherein the live image of a person is a two-dimensional image of a person or a three-dimensional image of a person.
The method according to claim 1 or 2, further comprising:

Before receiving the live person image, instruct the live broadcaster associated with the live person image to provide the live person image.
A storage medium in which a computer program is stored, and when the computer program is executed by a processor, it can be used to implement the method according to any one of claims 1-17.
An electronic device comprising a processor and a memory, and a computer program is stored in the memory, and when the computer program is executed by the processor, it can be used to implement the method according to any one of claims 1-17.
A computer program product, when it is executed by a processor, can be used to implement the method described in any one of claims 1-17.