WO2023168836A1

WO2023168836A1 - Projection interaction method, and device, medium and program product

Info

Publication number: WO2023168836A1
Application number: PCT/CN2022/095921
Authority: WO
Inventors: 廖春元; 方中慧; 杨浩; 林祥杰
Original assignee: 亮风台（上海）信息科技有限公司
Priority date: 2022-03-11
Filing date: 2022-05-30
Publication date: 2023-09-14
Also published as: CN115185437A

Abstract

The present application aims to provide a projection interaction method, and a device, a medium and a program product. The method specifically comprises: collecting corresponding top image information by means of a top camera apparatus, and transmitting the top image information to a corresponding target user equipment, so that the target user equipment presents the top image information; acquiring image annotation information, about the top image information, of a target user corresponding to the target user equipment, wherein the image annotation information comprises corresponding annotation information and annotation position information of the annotation information, and the annotation information is determined by a first user operation of the target user; and determining corresponding projection position information on the basis of the annotation position information, and projecting and presenting the annotation information on the basis of the projection position information. By means of the present application, an interesting interaction can be performed for a current user, and a more real and natural augmented reality interaction can also be provided for the current user.

Description

A projection interaction method, equipment, medium and program product

This application is based on the application with CN application number 202210241557.9 and a filing date of 2022.03.11, and claims its priority. The disclosure content of the CN application is hereby incorporated into this application as a whole.

Technical field

The present application relates to the field of communications, and in particular to a technology for projection interaction.

Background technique

Augmented Reality: It is a technology that calculates the position and angle of camera images in real time and adds corresponding virtual three-dimensional model animation, video, text, pictures and other digital information. The goal of this technology is to display the The virtual world is nested in the real world and interacts with it. Video call technology usually refers to a communication method based on the Internet and mobile Internet that transmits human voice and images in real time between smart devices. Existing computer equipment lacks other interactive capabilities except video communication.

Contents of the invention

One purpose of this application is to provide a projection interaction method, device, medium and program product.

According to one aspect of the present application, a projection interaction method is provided, wherein the method is applied to a computer device, the computer device includes a top camera device and a projection device, and the method includes:

Collect the corresponding top image information through the top camera device, and transmit the top image information to the corresponding target user equipment, so that the target user equipment can present the top image information;

Obtain image annotation information about the top image information of the target user device corresponding to the target user, wherein the image annotation information includes corresponding annotation information and annotation position information of the annotation information, and the annotation information is composed of the The first user operation of the target user is determined;

Corresponding projection position information is determined based on the annotation position information, and the annotation information is projected and presented based on the projection position information.

According to another aspect of the present application, a projection interaction method is provided, wherein, applied to a target user device, the method includes:

Receive and present the top image information transmitted by the corresponding computer equipment and collected by the corresponding top camera device,

Obtain the first user operation of the target user device corresponding to the target user, and generate annotation information about the top image information based on the first user operation;

The annotation information is returned to the computer device for the computer device to present the annotation information through a corresponding projection device.

According to one aspect of the present application, a computer device for projection interaction is provided. The computer device includes a top camera device and a projection device. The device includes:

A module configured to collect corresponding top image information through the top camera device, and transmit the top image information to the corresponding target user equipment, so that the target user equipment can present the top image information;

One or two modules, configured to obtain image annotation information about the top image information of the target user device corresponding to the target user, wherein the image annotation information includes corresponding annotation information and annotation position information of the annotation information, so The annotation information is determined by the first user operation of the target user;

A third module, configured to determine corresponding projection position information based on the annotation position information, and project and present the annotation information based on the projection position information.

According to another aspect of the present application, a target user device for projection interaction is provided, wherein the device includes:

The two-one module is used to receive and present the top image information transmitted by the corresponding computer equipment and collected by the corresponding top camera device;

The second module is used to obtain the first user operation of the target user device corresponding to the target user, and generate annotation information about the top image information based on the first user operation;

The second and third modules are used to return the annotation information to the computer device, so that the computer device can present the annotation information through a corresponding projection device.

According to one aspect of the present application, a computer device is provided, wherein the device includes:

processor; and

A memory arranged to store computer-executable instructions which, when executed, cause the processor to perform the steps of any of the methods described above.

According to one aspect of the present application, a computer-readable storage medium is provided, on which a computer program/instruction is stored, characterized in that, when executed, the computer program/instruction causes the system to perform any one of the methods described above. step.

According to one aspect of the present application, a computer program product is provided, including a computer program/instruction, characterized in that when the computer program/instruction is executed by a processor, the steps of any of the above methods are implemented.

Compared with the existing technology, this application projects and presents the image annotation information of the target user on the computer device through interaction between the two parties, which can provide the current user with interesting interactions while providing a more realistic and natural augmented reality interaction for the current user. In particular, it can enhance the interaction and participation experience of parent-child companionship for parents who are not with their children.

Description of the drawings

Other features, objects and advantages of the present application will become more apparent by reading the detailed description of the non-limiting embodiments with reference to the following drawings:

Figure 1 shows a system topology diagram of projection interaction according to an embodiment of the present application;

Figure 2 shows a method flow chart of a projection interaction method according to an embodiment of the present application;

Figure 3 shows a method flow chart of a projection interaction method according to an embodiment of the present application;

Figure 4 shows a functional module of a computer device according to an embodiment of the present application;

Figure 5 shows a functional module of a target user equipment according to an embodiment of the present application;

Figure 6 illustrates an example system that may be used to implement various embodiments described in this application.

The same or similar reference numbers in the drawings represent the same or similar parts.

Detailed ways

The present application will be described in further detail below in conjunction with the accompanying drawings.

In a typical configuration of this application, the terminal, the device of the service network and the trusted party all include one or more processors (for example, central processing unit (Central Processing Unit, CPU)), input/output interfaces, network interfaces and Memory.

Memory may include non-permanent memory in computer-readable media, random access memory (Random Access Memory, RAM) and/or non-volatile memory, such as read-only memory (Read Only Memory, ROM) or flash memory ( Flash Memory). Memory is an example of computer-readable media.

Computer-readable media includes both persistent and non-volatile, removable and non-removable media that can be implemented by any method or technology for storage of information. Information may be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, Phase-Change Memory (PCM), Programmable Random Access Memory (PRAM), Static Random-Access Memory (Static Random-Access Memory, SRAM), Dynamic Random Access Memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (Electrically-Erasable Programmable Read) -Only Memory (EEPROM), flash memory or other memory technology, Compact Disc Read-Only Memory (CD-ROM), Digital Versatile Disc (DVD) or other optical storage , magnetic tape cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

The equipment referred to in this application includes but is not limited to user equipment, network equipment, or equipment composed of user equipment and network equipment integrated through a network. The user equipment includes but is not limited to any kind of mobile electronic product that can interact with the user, such as smart phones, tablet computers, smart desk lamps, etc. The mobile electronic product can use any operating system, such as Android operating system, iOS operating system, etc. Among them, the network device includes an electronic device that can automatically perform numerical calculations and information processing according to preset or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC ), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), Digital Signal Processor (Digital Signal Processor, DSP), embedded devices, etc. The network equipment includes but is not limited to a computer, a network host, a single network server, multiple network server sets, or a cloud composed of multiple servers; here, the cloud is composed of a large number of computers or network servers based on cloud computing (Cloud Computing), Among them, cloud computing is a type of distributed computing, a virtual supercomputer composed of a group of loosely coupled computer sets. The network includes but is not limited to the Internet, wide area network, metropolitan area network, local area network, VPN network, wireless self-organizing network (Ad Hoc network), etc. Preferably, the device may also be a program running on the user equipment, network equipment, or a device formed by integrating the user equipment with the network equipment, the network equipment, the touch terminal, or the network equipment and the touch terminal through a network.

Of course, those skilled in the art should understand that the above-mentioned equipment is only an example, and other existing or possible equipment that may appear in the future, if applicable to this application, should also be included in the protection scope of this application, and are included hereby by reference. this.

In the description of this application, "plurality" means two or more, unless otherwise expressly and specifically limited.

Figure 1 shows a typical scenario of this application. The computer device 100 establishes a communication connection with the target user device 200. The computer device 100 transmits the corresponding top image information to the target user device 200; the target user device 200 receives the top image information. The corresponding annotation information or image annotation information is determined based on the top image information, and then the annotation information or image annotation information is returned to the computer device 100 . The computer device 100 includes but is not limited to any electronic product that can perform human-computer interaction with the user, such as smart phones, tablet computers, smart desk lamps, smart projection devices, etc. The computer equipment includes a top camera device for collecting images from above (for example, directly above or diagonally above, etc.) the operating object of the current user of the computer equipment (for example, the book that the user is currently reading or the working parts that the user is operating, etc.) Image information related to the operating object, such as camera or depth camera, etc. The target user equipment includes but is not limited to any mobile electronic product that can perform human-computer interaction with the user, such as smart phones, tablet computers, personal computers, etc. The target user equipment includes a display device for presenting the top Image information, for example, a liquid crystal display screen or a projector, etc.; the target user equipment also includes an input device for collecting the user's annotation information or image annotation information about the top image information. The annotation information includes but is not limited to information about the top image. Mark information such as stickers, text, graphics, videos, graffiti, 2D marks or 3D marks of interactive objects in the interactive object. The corresponding image annotation information includes the above annotation information and the corresponding annotation position information. For example, the annotation information or the interactive object of the annotation information is at the image coordinates The image coordinate information in the system, etc., where the marked position information is only an example and is not limited here. Here, the data transmission between the target user equipment and the computer equipment in this application may be based on a direct communication connection between the two devices, or may be forwarded via a corresponding server, etc.

Figure 2 shows a projection interaction method according to one aspect of the present application. The method is applied to the computer device 100 and can be applied to the system topology shown in Figure 1. The method includes step S101, step S102 and step S103. In step S101, the corresponding top image information is collected through the top camera device, and the top image information is transmitted to the corresponding target user equipment, so that the target user equipment can present the top image information; in step S102, Obtain image annotation information about the top image information of the target user device corresponding to the target user, wherein the image annotation information includes corresponding annotation information and annotation position information of the annotation information, and the annotation information is composed of the The first user operation of the target user is determined; in step S103, corresponding projection position information is determined based on the annotation position information, and the annotation information is projected and presented based on the projection position information.

Specifically, in step S101, the corresponding top image information is collected through the top camera device, and the top image information is transmitted to the corresponding target user equipment, so that the target user equipment can present the top image information. For example, the current user (for example, user A) holds a computer device, through which the computer device can communicate with the target user device held by the target user (for example, user B), such as establishing a connection between the computer device and the target through a wired or wireless manner. Communication connection between user equipment, or data communication and transmission between the two via network equipment. The computer equipment includes a top camera device, which is used to collect image information related to the operating object of the current user. For example, the top camera device collects image information related to the operating object from above the operating object (for example, directly above or diagonally). above, etc.), in some cases, the top camera device is disposed directly above the operation object, and the optical axis of the top camera device can pass through the shape or center of the operation object, etc. Of course, in order for the top camera device to meet the requirements, the computer equipment is usually provided with a forward extension rod, and the top camera device is placed directly below on the extension rod to collect the image information below. Under the extension rod, the computer equipment There is an operation area corresponding to the operation object between the user and the operation area. The operation area can be an extension area below the computer device itself (for example, the top image information can be obtained more accurately through the extension area of the computer device itself) , or by setting a blank area between the computer device and the user as the corresponding operation area (for example, setting a blank desktop as the operation area, etc.). Of course, in some cases, the base of the computer equipment remains horizontal to maintain the stability of the computer equipment. Correspondingly, the optical axis corresponding to the top camera device is vertically downward, and the surface where the operating area is located remains horizontal, thereby allowing the computer equipment to capture The distances between each area of the operation object in the top image information relative to the top camera device are similar.

The computer device collects top image information about the operating object through the top camera device, and sends the top image information directly or via a network device to the target user device. In some embodiments, the computer device may first identify the top image information (for example, identify and track preset template features through corresponding template features), etc., thereby ensuring that the corresponding operation object exists in the top image information. If there is no operation object in the current top image information, the computer device can adjust the camera angle of the top camera device and collect images from other areas to ensure that there is an operation object in the top image information, for example, by adjusting the extension angle and height of the extension rod. Or directly adjust the camera pose information of the top camera device, thereby achieving the effect of changing the camera angle of the top camera device. If there is no operation object after collecting all angles or there is no operation object after continuously collecting a certain amount of top image information, the computer device displays corresponding prompt information to remind the current user that there is no operation object in the current operation area. If there is an operation object in the current top image information, the top image information is transmitted to the target user device. After receiving the top image information, the target user equipment presents the top image information, for example, displays the top image information through a display screen or displays the top image information through projection through a projector.

The target user device receives and presents the top image information for the target user to perform operations and interactions based on the top image information. While presenting the top image information, the target user device can also collect the target user's annotation information about the interactive object in the top image information through the input device. The interactive position of the interactive object may be predetermined, or may be determined and determined by the current user. The transmission to the target user device may also be determined based on the target user's operation position (for example, touch position, cursor position, gesture recognition result or voice recognition result, etc.). If the interaction position in the top image information is predetermined or determined by the current user, the target user device directly transmits the annotation information to the computer device; if the interaction position in the top image information is determined by the target user's operation position, then The target user device determines the interaction position as the annotation position information, combines the annotation information to generate corresponding image annotation information, and returns the image annotation information to the computer device or the like.

In step S102, obtain image annotation information about the top image information of the target user device corresponding to the target user, wherein the image annotation information includes corresponding annotation information and annotation position information of the annotation information, and the The annotation information is determined by the first user operation of the target user. For example, the top image information includes an image coordinate system established based on the top image information. The image coordinate system uses a certain pixel point (for example, the pixel point in the upper left corner of the image, etc.) as the origin, and uses the horizontal axis as the X-axis, The vertical axis is used as the Y axis to establish a corresponding image/pixel coordinate system, etc. The corresponding annotation position information includes the coordinate position information of the corresponding annotation information or the interactive object of the annotation information in the image coordinate system. The coordinate position information can be used to indicate the annotation information or Mark the center position of the interactive object of the information or the coordinate set of the area range, etc. The annotation location information may be determined based on user operations of the current user and the target user, or may be preset. The corresponding annotation information is determined by the target user device based on the first user operation of the target user on the top image. For example, the corresponding annotation is determined based on the mark information added by the target user on mouse input, keyboard input, touch screen input, gesture input, or voice input operation. Information; in some cases, the annotation position information is determined based on the image coordinate information in the top image information based on the corresponding mouse click position, keyboard input corresponding position, touch screen touch position, gesture recognition, and voice recognition results, or based on the mouse click The position, corresponding position of keyboard input, touch position of the touch screen, gesture recognition, and voice recognition results are first determined in the top image information, and then the corresponding image annotation position is determined based on the interactive object.

The annotation information includes but is not limited to tag information such as stickers, text, graphics, videos, graffiti, 2D marks or 3D marks about interactive objects in the top image information. In some embodiments, based on different types of annotation information, the corresponding image There are also differences in the presentation of location information.

In step S103, corresponding projection position information is determined based on the annotation position information, and the annotation information is projected and presented based on the projection position information. For example, the projection device is placed near the top camera device, such as the corresponding projector and the top camera are placed on the top extension rod, etc. The mapping relationship between the corresponding projection device and the top camera device can be obtained through calculation. For example, s1: use the projection device to project a calibration image containing a specific pattern (such as a checkerboard) onto the operating area; s2: the top camera collects images containing the operating area. Video image; s3: Use the image collected in step s2 to identify the coordinate information of each pattern in the display screen; s4: Establish the relationship between the pattern in the original calibration image projected by s1 and the pattern coordinates of the video image containing the operation area collected in s2 Correspondence; s5: Estimate camera internal and external parameters, as well as distortion parameters based on the two types of coordinates in s4; s6: Use s5 to obtain parameters to achieve mapping between the two images. Among them, those skilled in the art should understand that the above-mentioned calculation method of the mapping relationship between the projection device and the top camera device is only an example, and other existing or future methods for calculating the mapping relationship between the projection device and the top camera device may be applicable. This application shall also be included within the scope of protection of this application and are hereby incorporated by reference. Based on the mapping relationship between the projection device and the top camera device, the annotation position information can be converted to projection position information, such as the projection coordinate information of the annotation information in the coordinate system of the projection image corresponding to the projection image, where the projection position information is only an example, No limitation is made here. The computer device projects the annotation information to the corresponding operation area according to the corresponding projection coordinate information, thereby presenting the annotation information in the corresponding area in the operation object.

In some implementations, the computer device further includes a display device, and the method further includes step S104 (not shown). In step S104, the target image information transmitted by the target user device is received, and the display device The target image information is presented. For example, the computer device further includes a display device, which is used to present image information stored or received by the computer device, such as a liquid crystal display screen. In some cases, to facilitate the current user to view the corresponding image information, the display device is placed directly in front of or near the computer device facing the current user. During the communication process between the computer equipment and the target user equipment, in order to facilitate communication between the two users and improve the immediacy and communication efficiency of the interaction, the target user equipment is equipped with a corresponding camera device for collecting target image information on the target user equipment side. The target image The information includes image information about the target user, the target user device can transmit the target image information to the computer device, and the computer device receives and presents the target image information through the display device. In some cases, the target image information and the corresponding top image information are included in the video stream collected in real time, then both the computer device and the target user equipment can present the top image information, the real-time video stream corresponding to the target image information, etc. through the corresponding display device, The corresponding video stream not only includes images collected by the camera device, but also includes voice information collected by the voice input device; while the corresponding computer equipment and target user equipment display the real-time video stream corresponding to the target image information and the top image information, they also pass The voice output device plays corresponding voice information, etc., that is, the target user equipment and the computer equipment conduct audio and video communication.

In some embodiments, the computer equipment further includes a front-facing camera device, wherein the front-facing camera device is used to collect front-facing image information about the current user of the computer device; wherein the method further includes the steps S105 (not shown), in step S105, the preamble image information is transmitted to the target user equipment, so that the target user equipment can present the preamble image information. For example, the computer device further includes a front-facing camera device for collecting image information related to the current user of the computer device holder. The front-facing camera device is placed on the side of the computer device facing the current user. For example, it can be disposed on the display above the device. In some cases, the front-facing camera device is mainly used to collect image information related to the current user's head, and is used to realize video interactive communication between the current user and the target user. The corresponding front-facing camera device collects front-facing image information about the current user when enabled, and transmits the front-facing image information to the target user device for display by the display device of the target user device. In some cases, the front-facing image information The information is included in the video stream collected in real time, then the target user equipment can present the real-time video stream corresponding to the front image information through the corresponding display device. Here, there may be a certain connection between the activation of the front camera device and the activation of the top camera device. , or they can be mutually independent camera devices, and the corresponding target user equipment can simultaneously present the video streams of the two camera devices, or present one of the video streams of the two camera devices, and so on. The enabled state of the front camera device may be enabled based on a video creation request from the target user equipment or computer device, or switched from the enabled state of the top camera device, or based on an independent enable control of the front camera device. Turn on when triggered.

In some implementations, the method further includes step S106 (not shown). In step S106, a camera switching request regarding the current video interaction between the computer device and the target user device is obtained, wherein the current The image information of the video interaction includes the front image information; wherein, in step S101, in response to the camera switching request, the front camera device is turned off and the top camera device is enabled, and the top camera device collects data through the top camera device. The corresponding top image information is transmitted to the corresponding target user equipment, so that the target user equipment can present the top image information. For example, only one of the front camera device and the top camera device of the computer equipment is enabled at the same time, thereby reducing the bandwidth pressure of video interaction and ensuring the efficiency and orderliness of the video interaction process. In some embodiments, the computer device is provided with a corresponding camera switching control. The camera switching control may be a physical button provided on the computer device or a virtual control presented on the current screen. The camera switching control is used to Realize switching from the enabled state of the front camera device to the enabled state of the top camera device. In some cases, the camera switching control is also used to adjust the computer device from the enabled state of the top camera device to the enabled state of the front camera device, In other words, the camera switching control is used to switch the activation state of the top camera device and the front camera device; in other cases, the camera switching control is only used to switch from the activation state of the front camera device to the top camera device. The computer device is also provided with a corresponding camera restore control, and the camera restore control is used to adjust the computer device from the enabled state of the top camera device to the enabled state of the front camera device. In other embodiments, the computer device determines the camera switching request by recognizing the current user's interactive input operations such as gestures, voice, and head movements. In other embodiments, similarly, the target user device may also be provided with a corresponding camera switching control, or the target user device determines the camera switching request by recognizing the target user's gestures, voice, head movements and other interactive input operations. , which will not be described in detail here. The computer device is in a state of collecting front image information during the video interaction process. When the user (for example, the current user or the target user) touches the camera switching control, the computer device turns off the front camera device and enables the top camera. A device that collects corresponding top image information through the top camera device, and transmits the top image information to the corresponding target user equipment, so that the target user equipment can present the top image information. Further, the target user can perform a first user operation on the top image information to determine the icon annotation information of the top image information. The computer device determines the projection position information based on the annotation position information, and projects and presents the annotation information based on the projection position information. Thus, the target Users can provide intuitive guidance on the current user's operation objects.

In some cases, the activation status of different camera devices corresponds to different interaction modes. For example, when the computer device currently only turns on the front camera device, the computer device is in the video communication mode, which is used to achieve video communication between the current user and the target user, etc. ; When the computer device currently only turns on the top camera device, the computer device is in the video guidance mode, which is used to achieve communication guidance for the target user about the current user's operation object; when the computer device currently turns on both the front camera device and the top camera device, the computer device The device not only enables the target user to communicate and guide the current user's operation object, but also facilitates video communication between the target user and the current user.

In some implementations, in step S105, receive a camera switching request transmitted by the target user equipment regarding the current video interaction between the computer device and the target user equipment, wherein the camera switching request is based on the target user equipment. The second user operation is determined. For example, the target user device may generate a video image of the current video interaction based on the target user's second user operation (for example, a trigger operation on the camera switching control, or an interactive input operation instruction on the camera switch, etc.). Switching request, the target user equipment transmits the camera switching request to the corresponding computer device; or, the target user equipment sends the second user operation to the corresponding server, and the server generates the corresponding camera switching request based on the second user operation, and sends the camera switching request to the corresponding computer device. Handover requests are sent to computer equipment, etc. Here, the first user operation and the second user operation are only used to distinguish the corresponding functions of the user operations, and do not involve the correlation of the order, size and other operations. After the video interaction is established, when the target user equipment obtains and presents the front image information of the front camera device, in some embodiments, the corresponding camera switching control is presented in the current screen, and the target user equipment can collect the second user's information by Operation, such as the target user's touch operation on the camera switching control, determines the camera switching request. In other embodiments, the target user device can collect the second user operation, such as the target user's gesture, voice, head movement, etc. Interactively input operation instructions to confirm the camera switching request. The camera switching request usually occurs after the video is established. When the video is established, the corresponding front-facing camera device can be enabled simultaneously based on the video creation request, or the front-facing camera device can be enabled based on the interaction between the two parties after the target user device and the computer device establish communication. The camera device, etc. can also be switched from the enabled state of the top camera device to the enabled state of the front camera device after the target user equipment establishes communication with the computer device and activates the top camera device.

In some implementations, in step S105, obtain a video establishment request regarding the current video interaction; in response to the video establishment request, establish all connections between the computer device and the target user device based on the video establishment request. For the current video interaction, the corresponding front image information is collected through the front camera device and transmitted to the target user equipment, so that the target user equipment can present the front image information. For example, the current video interaction may be initiated based on a video creation request determined by the user operation of the current user or the target user. At the beginning of the current video interaction, the video stream is transmitted between the target user device and the computer device. For example, the computer device transmits a video stream to the computer device. The target user device transmits the corresponding pre-image information, and the target user device transmits the corresponding target image information to the computer device, or only the computer device transmits the pre-image information to the target user device alone, etc. The video creation request may be determined based on the current user's initiated operation on the computer device (for example, a triggering operation on a video creation control, or an interactive input operation instruction on video creation, etc.), and the video creation request includes a user identification corresponding to the target user. Information, user identification information includes but is not limited to unique tag information used to identify the target user, specifically, such as name, image, ID card, mobile phone number, application serial number or device access control address information, etc.; the computer device can The video creation request is sent to the network device for the network device to forward to the target user device and establish video interaction between the two, or the computer device directly sends the video creation request to the target user device and establishes video interaction between the two. For another example, the video creation request may be determined based on the target user's initiating operation on the target user device (for example, a triggering operation on the video creation control, or an interactive input operation instruction on the video creation, etc.), and the video creation request includes information corresponding to the current User identification information of the user; the target user device can send the video creation request to the network device for the network device to forward to the computer device and establish video interaction between the two, or the target user device can directly send the video creation request to the computer device And establish video interaction between the two.

In some implementations, the method further includes step S107 (not shown). In step S107, obtain a camera restoration request regarding the current video interaction between the computer device and the target user device; respond to the camera restoration request , turn off the top camera device and enable the front camera device, collect corresponding front image information through the front camera device, and transmit the front image information to the corresponding target user equipment. For example, the camera restoration request is used to adjust the computer device from the enabled state of the top camera device to the enabled state of the front camera device. After the restored state, the corresponding top camera device is in a closed state, and only the front image information is collected and Transmission, such as the computer device transmitting the corresponding pre-image information to the target user device, the target user device transmitting the corresponding target image information to the computer device, or only the computer device transmitting the pre-image information to the target user device alone, etc. In some embodiments, the camera restoration request is initiated based on the current user/target user's touch operation on the camera restoration control. In other embodiments, the camera restoration request is based on the current user/target user's touch operation on the camera restoration. Initiated by interactively inputting operation instructions (such as gestures, voice, head movements, etc.), based on the camera restoration request, the computer device turns off the top camera device that is currently enabled for video interaction, and enables the corresponding front camera device, thereby achieving the goal of returning the camera to the camera. The video guidance of the operating object is restored to the video communication between both parties.

In some implementations, the method further includes step S108 (not shown). In step S108, a camera start request regarding video interaction between the computer device and the target user device is obtained; wherein, in step S101, in response to In response to the camera startup request, the corresponding top image information is collected through the top camera device, and the top image information is transmitted to the corresponding target user equipment; in step S105, in response to the camera startup request, through the The front camera device collects corresponding front image information and transmits the front image information to the target user equipment. For example, the camera start request is used to enable the front camera device and the top camera device at the same time. The camera start request can be included in the video creation request to enable the front camera device and the top camera device at the same time during the video creation process. And respectively present the closing controls for the front camera device and the top camera device for the current user and/or the target user to close the front camera device or the top camera device. Of course, if the front camera device and the top camera are closed at the same time, device, turns off video interaction between the computer device and the target user device. In some cases, the camera start request may also be to call the front camera device or the top camera during the video interaction process (during the video interaction process, one of the front camera device and the top camera device is enabled). device to achieve the effect of two camera devices being enabled at the same time. In some embodiments, the camera startup request may be generated based on the touch operation of the current user or the target user on opening the control. In other embodiments, the camera startup request may be generated based on the current user or the target user's touch operation on the camera. The interactive input operation instructions (such as gestures, voice, head movements, etc.) to start are generated, and the two camera devices are enabled simultaneously based on the response of the computer device or the target user device to the camera start request.

In some embodiments, the computer device includes a lighting device; wherein the method further includes step S109 (not shown). In step S109, if an enabling request for the lighting device is obtained, turning on the lighting device. For example, the computer equipment may include a lighting device for adjusting the brightness of the operating area. The activation request is used to turn on the corresponding lighting device and project a certain intensity of light on the operating area to change the brightness of the environment. The activation request can be generated by the computer device based on the current user's operation, or it can also be based on the target user's operation corresponding to the target user's equipment. (such as touch controls on lighting controls, etc.) are generated and transmitted to computer equipment, etc. In some cases, the enable request is included in the corresponding video creation request, which is used to turn on the lighting device while the video interaction is being established; in other cases, the enable request is based on the target user/current user during the video interaction. Or the user operation is determined when there is no video interaction, thereby turning on the lighting device to adjust the ambient brightness.

In some embodiments, the computer device further includes an ambient light detection device, and the method further includes step S110 (not shown). In step S110, the illumination intensity information of the current environment is obtained based on the ambient light detection device, Detect whether the light intensity information meets the preset lighting threshold; if not, adjust the lighting device until the light intensity information meets the preset lighting threshold. For example, the lighting device of the computer equipment includes a lighting device with adjustable brightness. For example, the lighting brightness of the lighting device can be adjusted based on the touch selection operation of the brightness adjustment control by the current user or the target user. For example, the lighting brightness of the lighting device can be adjusted based on the current user's or target user's touch selection operation on the brightness adjustment control. The user or the target user interactively inputs operation instructions (such as gestures, voice, head movements, etc.) regarding brightness adjustment to adjust the lighting brightness of the lighting device. In some cases, the computer equipment includes an ambient light detection device, which is used to cooperate with the lighting device to realize automatic adjustment of lighting and ensure the controllability and applicability of ambient brightness. The computer equipment measures the light intensity information of the current environment based on the ambient light detection device, and compares the light intensity information with a preset lighting threshold. The preset lighting threshold can be a specific light intensity value or multiple light intensities. The range of numerical values, etc., is not limited here. If the illumination intensity information is the same as the illumination threshold information or the intensity difference is less than the preset difference threshold, etc., then it is determined that the illumination intensity information satisfies the preset illumination threshold; or, if the illumination intensity information is within the illumination threshold interval, then It is determined that the light intensity information satisfies a preset lighting threshold, etc. If not, the corresponding lighting adjustment information is calculated based on the lighting intensity information and the lighting threshold information, and the corresponding lighting device is adjusted based on the lighting adjustment information. The lighting adjustment information includes an increasing or decreasing lighting adjustment value corresponding to the current lighting intensity information. wait.

In some embodiments, the method further includes step S111 (not shown). In step S111, determine the image brightness information corresponding to the top image information based on the top image information, and detect whether the image brightness information meets a predetermined Set a brightness threshold; if it is not met, adjust the lighting device until the image brightness information meets the preset brightness threshold. For example, the computer device can also determine the corresponding image brightness information based on the top image information, for example, calculate the average brightness information of the current image information based on the pixel brightness information of part (for example, sampling some pixels, etc.) or all pixels in the top image information. The image brightness information is compared with a preset brightness threshold. The preset brightness threshold can be a specific image brightness value, or an interval composed of multiple image brightness values, etc., which is not limited here. If the image brightness information is the same as the brightness threshold information or the brightness difference is less than the preset difference threshold, etc., then it is determined that the image brightness information satisfies the preset brightness threshold; or if the image brightness information is within the brightness threshold interval, then It is determined that the image brightness information satisfies a preset brightness threshold, etc. If not, the corresponding lighting adjustment information is calculated based on the image brightness information and the brightness threshold information, and the corresponding lighting device is adjusted based on the lighting adjustment information. The lighting adjustment information includes the lighting adjustment value corresponding to the increase or decrease of the current lighting intensity information. wait. In some cases, the computer device can also adjust the brightness of the lighting device based on the image brightness information of a specific area of the top image information. The specific area can be an interaction area determined based on the interactive object (for example, a boundary area, a circumscribed rectangular area etc.) or determined from the top image information based on user operations of the target user/current user.

In some embodiments, the method further includes step S112 (not shown). In step S112, the interactive position information of the current user with respect to the top image information is obtained; and the corresponding virtual position information is determined based on the interactive position information. Present information, and project and present the virtual presentation information through the projection device. For example, during the video interaction process, the interactive position information about the interactive object in the top image information can be obtained based on the user operation of the current user. For example, the current user presents the corresponding top image information on the display device, and based on the user's frame selection , click or touch operations, determine one or more pixel positions or pixel areas from the top image information, and use the one or more pixel positions or pixel areas as corresponding interactive position information. The interactive position information is included in the top image information. Coordinate position information in the image coordinate system, etc. For another example, based on the current user's user operation on the operation object (for example, pointing a finger or pen tip to a certain position, etc.), the computer device determines the pointed position of the finger or pen through image recognition technology, and uses the pointed position as the corresponding interactive position information. .

The computer device can directly present the corresponding virtual presentation information based on the interactive position information. For example, the computer device matches the virtual information corresponding to the interactive position information from the database, or for example, by performing target recognition on the interactive object corresponding to the interactive position information, thereby Match the corresponding virtual information in the database, determine the corresponding virtual information as virtual presentation information, and determine the projection position information of the virtual presentation information through the interactive position information, so as to project the virtual presentation information to the interactive object through the projection device Spatial location, etc. For example, in some embodiments, the computer device includes an infrared measurement device; wherein, obtaining the interactive position information of the current user with respect to the top image information includes: determining the top position through the infrared measurement device. Interactive position information of image information. For example, the computer equipment also includes an infrared measurement device. In some embodiments, the infrared measurement device includes an infrared camera and an infrared emitter. For example, the infrared camera is installed on the top extension pole together with the top camera, and the infrared emitter is installed on On the base of the computer equipment, the infrared emitter forms an invisible light film on the surface of the operating object that is higher than a certain distance threshold on the surface. When a finger or any opaque object touches the surface, the light is reflected to the infrared camera, and then passes through the photoelectric position. Accurate calculation to obtain the position of the finger or any opaque object touching the operating object. In other embodiments, the infrared measurement device includes an infrared camera and an infrared pen. When the current user uses the infrared pen to touch the surface of the operating object, the infrared camera can determine the position where the infrared pen contacts the operating object. Determine the corresponding interactive position information based on the position of the finger, any opaque object or the infrared pen touching the operating object.

In some implementations, in step S102, the interaction location information is sent to the target user equipment, and the annotation information transmitted by the target user equipment and returned based on the interaction location information is received, thereby determining the top Image annotation information of image information, wherein the image annotation information includes the annotation information and the interactive position information, and the annotation information is determined by the first user operation of the target user. For example, the interactive location information is used to prompt the target user to the area where the interactive object is currently located in the top image information. The target user device receives the top image information transmitted by the computer device and also receives the interactive location information, and collects the target user's information about the interactive location information. The interactive position information corresponds to the user operation of the interactive object, so that the corresponding annotation information is determined based on the first user operation. The target user device directly returns the annotation information to the computer device, and the computer device implements projection and presentation of the annotation information based on the received annotation information and combined with the previously determined interactive position information.

In some embodiments, the computer device further includes a distance measurement device; wherein the method further includes step S113 (not shown). In step S113, it is determined by the distance measurement device that the distance between the current user and the Distance information between computer devices. If the distance information does not meet a preset distance threshold, the computer device sends a notification. For example, the distance measurement device is disposed on a side of the computer device parallel to the corresponding operating area and facing the current user, and is used to measure real-time distance information between the computer device and the current user, such as a laser rangefinder. The computer device is provided with a corresponding distance threshold interval. When the distance information between the computer device and the current user is within the distance threshold interval, it is determined that the distance information satisfies the preset distance threshold and the current user's posture meets the requirements. If the distance information between the computer device and the current user is outside the distance threshold interval, it is determined that the distance information does not meet the preset distance threshold, and the computer device issues a corresponding prompt notification, such as reminding the current user through sound, image, vibration, text, etc. The posture needs to be adjusted to ensure that the corresponding distance information meets the preset distance threshold.

Figure 3 shows a projection interaction method according to one aspect of the present application, wherein the method is applicable to the system shown in Figure 1 and applied to the target user equipment 200, and mainly includes steps S201, S202 and S203. In step S201, receive and present the top image information transmitted by the corresponding computer device and collected by the corresponding top camera device; in step S202, obtain the first user operation of the target user corresponding to the target user equipment, and based on the first user operation Generate annotation information about the top image information; in step S203, return the annotation information to the computer device for the computer device to present the annotation information through a corresponding projection device.

For example, the current user (for example, user A) holds a computer device, through which the computer device can communicate with the target user device held by the target user (for example, user B), such as establishing a connection between the computer device and the target through a wired or wireless manner. Communication connection between user equipment, or data communication and transmission between the two via network equipment. The computer equipment includes a top camera device, which is used to collect image information related to the operating object of the current user. For example, the top camera device collects image information related to the operating object from above the operating object (for example, directly above or diagonally). above, etc.), in some cases, the top camera device is disposed directly above the operation object, and the optical axis of the top camera device can pass through the shape or center of the operation object, etc. Of course, in order for the top camera device to meet the requirements, the computer equipment is usually provided with a forward extension rod, and the top camera device is placed directly below on the extension rod to collect the image information below. Under the extension rod, the computer equipment There is an operation area corresponding to the operation object between the user and the operation area. The operation area can be an extension area below the computer device itself (for example, the top image information can be obtained more accurately through the extension area of the computer device itself) , or by setting a blank area between the computer device and the user as the corresponding operation area (for example, setting a blank desktop as the operation area, etc.). Of course, in some cases, the base of the computer equipment remains horizontal to maintain the stability of the computer equipment. Correspondingly, the optical axis corresponding to the top camera device is vertically downward, and the surface where the operating area is located remains horizontal, thereby allowing the computer equipment to capture The distances between each area of the operation object in the top image information relative to the top camera device are similar.

The target user device receives and presents the top image information for the target user to perform operations and interactions based on the top image information. While presenting the top image information, the target user device can also collect the target user's annotation information about the interactive object in the top image information through the input device. The interactive position of the interactive object may be predetermined, or may be determined and determined by the current user. The transmission to the target user device may also be determined based on the target user's operation position (for example, touch position, cursor position, gesture recognition result or voice recognition result, etc.). If the interaction position in the top image information is predetermined or determined by the current user, the target user device directly transmits the annotation information to the computer device; if the interaction position in the top image information is determined by the target user's operation position, then The target user device determines the interaction position as the annotation position information, combines the annotation information to generate corresponding image annotation information, and returns the image annotation information to the computer device or the like. As in some embodiments, the method further includes step S204 (not shown). In step S204, corresponding annotation location information is determined based on the first user operation; wherein, in step S203, the annotation information is and the annotation location information is returned to the computer device for the computer device to present the annotation information through a corresponding projection device based on the annotation location information.

For example, the top image information includes an image coordinate system established based on the top image information. The image coordinate system uses a certain pixel point (for example, the pixel point in the upper left corner of the image, etc.) as the origin, and uses the horizontal axis as the X-axis, The vertical axis is used as the Y axis to establish a corresponding image/pixel coordinate system, etc. The corresponding annotation position information includes the coordinate position information of the corresponding annotation information or the interactive object of the annotation information in the image coordinate system. The coordinate position information can be used to indicate the annotation information or Mark the center position of the interactive object of the information or the coordinate set of the area range, etc. The annotation location information may be determined based on user operations of the current user and the target user, or may be preset. The corresponding annotation information is determined by the target user device based on the first user operation of the target user on the top image. For example, the corresponding annotation is determined based on the mark information added by the target user on mouse input, keyboard input, touch screen input, gesture input, or voice input operation. Information; in some cases, the annotation position information is determined based on the image coordinate information in the top image information based on the corresponding mouse click position, keyboard input corresponding position, touch screen touch position, gesture recognition, and voice recognition results, or based on the mouse click The position, corresponding position of keyboard input, touch position of the touch screen, gesture recognition, and voice recognition results are first determined in the top image information, and then the corresponding image annotation position is determined based on the interactive object.

For example, the projection device is placed near the top camera device, for example, the corresponding projector and the top camera are placed on the top extension rod. The mapping relationship between the corresponding projection device and the top camera device can be obtained through calculation. Based on the mapping relationship between the projection device and the top camera device, the annotation position information can be converted to projection position information. For example, the annotation information is in the coordinate system of the projection image corresponding to the projection image. The projection coordinate information in , where the projection position information is only an example and is not limited here. The computer device projects the annotation information to the corresponding operation area according to the corresponding projection coordinate information, thereby presenting the annotation information in the corresponding area in the operation object.

The above mainly introduces various embodiments of a projection interaction method of the present application. In addition, the present application also provides computer equipment and target user equipment that can implement the above embodiments. We will introduce them below with reference to Figures 4 and 5.

Figure 4 shows a projection interactive computer device 100 according to one aspect of the present application. The device includes a first module 101, a second module 102 and a third module 103. 1. Module 101, configured to collect the corresponding top image information through the top camera device, and transmit the top image information to the corresponding target user equipment, so that the target user equipment can present the top image information; 1. 2 modules 102. Used to obtain image annotation information about the top image information of the target user device corresponding to the target user, wherein the image annotation information includes corresponding annotation information and annotation position information of the annotation information, and the annotation The information is determined by the first user operation of the target user; a third module 103 is configured to determine corresponding projection position information based on the annotation position information, and project and present the annotation information based on the projection position information.

Here, the specific implementation manners corresponding to the first module 101, the first second module 102 and the first third module 103 shown in Figure 4 are the same or similar to the embodiments of step S101, step S102 and step S103 shown in Figure 2. , and therefore will not be described again, but is included here by reference.

In some embodiments, the computer device further includes a display device, and the device further includes a four-module (not shown) for receiving the target image information transmitted by the target user device, and presenting the information through the display device. Describe the target image information.

In some embodiments, the computer device further includes a front-facing camera device, wherein the front-facing camera device is used to collect front-facing image information about the current user of the computer device; wherein the device further includes a Five modules (not shown), configured to transmit the pre-image information to the target user equipment so that the target user equipment can present the pre-image information.

In some implementations, the device further includes a module (not shown) for obtaining a camera switching request regarding a current video interaction between the computer device and the target user device, wherein the current video interaction The image information includes the front image information; wherein, a module 101 is used to respond to the camera switching request, turn off the front camera device and enable the top camera device, and collect data through the top camera device The corresponding top image information is transmitted to the corresponding target user equipment, so that the target user equipment can present the top image information.

In some implementations, a module is configured to receive a camera switching request transmitted by the target user equipment regarding the current video interaction between the computer device and the target user equipment, wherein the camera switching request is based on the target user equipment. The user's second user operation is determined.

In some implementations, a module is configured to obtain a video establishment request regarding the current video interaction; in response to the video establishment request, establish a connection between the computer device and the target user equipment based on the video establishment request. For the current video interaction, the corresponding front image information is collected through the front camera device and transmitted to the target user equipment, so that the target user equipment can present the front image information.

In some implementations, the device further includes a module (not shown) for obtaining a camera restoration request regarding the current video interaction between the computer device and the target user device; in response to the camera restore request, close The top camera images and activates the front camera device, collects corresponding front image information through the front camera device, and transmits the front image information to the corresponding target user equipment.

In some implementations, the device further includes a module (not shown) for obtaining a camera start request regarding the video interaction between the computer device and the target user device; wherein, a module 101 is used to respond In response to the camera start request, collect the corresponding top image information through the top camera device, and transmit the top image information to the corresponding target user equipment; a fifth module, configured to respond to the camera start request, through the The front camera device collects corresponding front image information and transmits the front image information to the target user equipment.

In some implementations, the computer device includes a lighting device; wherein the device further includes a module (not shown) configured to turn on the lighting device if an enabling request for the lighting device is obtained.

In some embodiments, the computer device further includes an ambient light detection device, and the device further includes a module (not shown) for obtaining light intensity information of the current environment based on the ambient light detection device, and detecting the Whether the illumination intensity information satisfies the preset illumination threshold; if not, adjust the lighting device until the illumination intensity information satisfies the preset illumination threshold.

In some embodiments, the device further includes an eleven module (not shown) for determining image brightness information corresponding to the top image information based on the top image information, and detecting whether the image brightness information satisfies a preset The brightness threshold; if it is not met, adjust the lighting device until the image brightness information meets the preset brightness threshold.

In some implementations, the device further includes a module (not shown) for obtaining the interactive position information of the current user regarding the top image information; and determining the corresponding virtual presentation based on the interactive position information. information, and the virtual presentation information is projected and presented through the projection device.

In some embodiments, the computer device includes an infrared measurement device; wherein the obtaining the interactive position information of the current user with respect to the top image information includes: determining the position of the top image information through the infrared measurement device. Interactive location information.

In some implementations, the first and second modules 102 are configured to send the interactive location information to the target user equipment, receive the annotation information transmitted by the target user equipment and returned based on the interactive location information, thereby determining the The image annotation information of the top image information, wherein the image annotation information includes the annotation information and the interactive position information, and the annotation information is determined by the first user operation of the target user.

In some embodiments, the computer device further includes a distance measurement device; wherein the device further includes a thirteenth module (not shown) for determining the distance between the current user and the computer through the distance measurement device. Distance information between devices. If the distance information does not meet the preset distance threshold, the computer device sends a notification.

Here, the corresponding specific implementations of the four to thirteen modules are the same or similar to the foregoing embodiments of steps S104 to S113, and therefore will not be described again, but are included here by reference.

Figure 5 shows a target user device 200 for projection interaction according to one aspect of the present application, which mainly includes a two-one module 201, a two-two module 202 and a two-three module 203. The second module 201 is used to receive and present the top image information transmitted by the corresponding computer device and collected by the corresponding top camera; the second module 202 is used to obtain the first user operation of the target user corresponding to the target user equipment, based on the The first user operation generates annotation information about the top image information; the second and third modules 203 are used to return the annotation information to the computer device for the computer device to present the annotation information through the corresponding projection device.

Here, the specific implementations corresponding to the second one module 201, the second second module 202 and the second third module 203 shown in FIG. 5 are the same or similar to the embodiments of step S201, step S202 and step S203 shown in FIG. 3 , and therefore will not be described again, but is included here by reference.

In some embodiments, the device further includes a second and fourth module (not shown), used to determine the corresponding annotation location information based on the first user operation; wherein the second and third module 203 is used to combine the annotation information and The annotation location information is returned to the computer device for the computer device to present the annotation information through a corresponding projection device based on the annotation location information.

Here, the specific implementation manner corresponding to the two or four modules is the same as or similar to the foregoing embodiment of step S204, and therefore will not be described again, but is included here by reference.

In addition to the methods and devices introduced in the above embodiments, the present application also provides a computer-readable storage medium that stores computer code. When the computer code is executed, as in the previous item The method described is executed.

This application also provides a computer program product. When the computer program product is executed by a computer device, the method described in the previous item is executed.

This application also provides a computer device, which includes:

one or more processors;

Memory for storing one or more computer programs;

When the one or more computer programs are executed by the one or more processors, the one or more processors are caused to implement the method as described in any one of the preceding items.

6 illustrates an exemplary system that may be used to implement various embodiments described in this application;

As shown in Figure 6, in some embodiments, the system 300 can serve as any of the above-mentioned devices in each of the described embodiments. In some embodiments, system 300 may include one or more computer-readable media (eg, system memory or NVM/storage device 320) having instructions coupled thereto and configured to perform Instructions are provided to implement means for one or more processors (eg, processor(s) 305) to perform the actions described herein.

For one embodiment, system control module 310 may include any suitable interface controller to provide information to at least one of processor(s) 305 and/or any suitable device or component in communication with system control module 310 Any appropriate interface.

System control module 310 may include a memory controller module 330 to provide an interface to system memory 315 . Memory controller module 330 may be a hardware module, a software module, and/or a firmware module.

System memory 315 may be used, for example, to load and store data and/or instructions for system 300 . For one embodiment, system memory 315 may include any suitable volatile memory, such as suitable DRAM. In some embodiments, system memory 315 may include double data rate type quad synchronous dynamic random access memory (DDR4SDRAM).

For one embodiment, system control module 310 may include one or more input/output (I/O) controllers to provide interfaces to NVM/storage device 320 and communication interface(s) 325 .

For example, NVM/storage device 320 may be used to store data and/or instructions. NVM/storage device 320 may include any suitable non-volatile memory (e.g., flash memory) and/or may include any suitable non-volatile storage device(s) (e.g., one or more hard drives (e.g., one or more hard drives) HDD), one or more compact disc (CD) drives and/or one or more digital versatile disc (DVD) drives).

NVM/storage device 320 may include storage resources that are physically part of the device on which system 300 is installed, or that may be accessed by the device without necessarily being part of the device. For example, NVM/storage device 320 may be accessed over the network via communication interface(s) 325 .

Communication interface(s) 325 may provide an interface for system 300 to communicate over one or more networks and/or with any other suitable device. System 300 may wirelessly communicate with one or more components of a wireless network in accordance with any of one or more wireless network standards and/or protocols.

For one embodiment, at least one of the processor(s) 305 may be packaged with the logic of one or more controllers of the system control module 310 (eg, memory controller module 330). For one embodiment, at least one of the processor(s) 305 may be packaged together with the logic of one or more controllers of the system control module 310 to form a system-in-package (SiP). For one embodiment, at least one of the processor(s) 305 may be integrated on the same die as the logic of the one or more controllers of the system control module 310 . For one embodiment, at least one of the processor(s) 305 may be integrated on the same die with the logic of one or more controllers of the system control module 310 to form a system on a chip (SoC).

In various embodiments, system 300 may be, but is not limited to, a server, a workstation, a desktop computing device, or a mobile computing device (eg, laptop computing device, handheld computing device, tablet, netbook, etc.). In various embodiments, system 300 may have more or fewer components and/or a different architecture. For example, in some embodiments, system 300 includes one or more cameras, keyboards, liquid crystal display (LCD) screens (including touch screen displays), non-volatile memory ports, multiple antennas, graphics chips, application specific integrated circuits ( ASIC) and speakers.

It should be noted that the present application may be implemented in software and/or a combination of software and hardware, for example, using an application specific integrated circuit (ASIC), a general purpose computer or any other similar hardware device. In one embodiment, the software program of the present application can be executed by a processor to implement the steps or functions described above. Likewise, the software program of the present application (including related data structures) may be stored in a computer-readable recording medium, such as a RAM memory, a magnetic or optical drive or a floppy disk and similar devices. In addition, some steps or functions of the present application may be implemented using hardware, for example, as a circuit that cooperates with a processor to perform each step or function.

In addition, part of the present application may be applied as a computer program product, such as computer program instructions. When executed by a computer, through the operation of the computer, methods and/or technical solutions according to the present application may be invoked or provided. Those skilled in the art will understand that the form in which computer program instructions exist in a computer-readable medium includes but is not limited to source files, executable files, installation package files, etc. Correspondingly, the manner in which computer program instructions are executed by a computer includes but is not limited to Limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium that can be accessed by the computer.

Communication media includes the medium whereby communication signals containing, for example, computer readable instructions, data structures, program modules or other data are transmitted from one system to another system. Communication media may include conducted transmission media, such as cables and wires (e.g., fiber optics, coaxial, etc.) and wireless (unguided transmission) media that can propagate energy waves, such as acoustic, electromagnetic, RF, microwave, and infrared . Computer readable instructions, data structures, program modules, or other data may be embodied, for example, as a modulated data signal in a wireless medium, such as a carrier wave or a similar mechanism such as that embodied as part of spread spectrum technology. The term "modulated data signal" refers to a signal in which one or more characteristics are altered or set in a manner that encodes information in the signal. Modulation can be analog, digital or hybrid modulation techniques.

By way of example, and not limitation, computer-readable storage media may include volatile and nonvolatile, removable, storage media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules or other data. Removable and non-removable media. For example, computer-readable storage media include, but are not limited to, volatile memory such as random access memory (RAM, DRAM, SRAM); and non-volatile memory such as flash memory, various read-only memories (ROM, PROM, EPROM ,EEPROM), magnetic and ferromagnetic/ferroelectric memory (MRAM, FeRAM); and magnetic and optical storage devices (hard disks, tapes, CDs, DVDs); or other media now known or developed in the future that can be stored for computer systems Computer readable information/data used.

Here, one embodiment according to the present application includes a device, the device includes a memory for storing computer program instructions and a processor for executing the program instructions, wherein when the computer program instructions are executed by the processor, triggering The device operates based on the aforementioned methods and/or technical solutions according to multiple embodiments of the present application.

It is obvious to those skilled in the art that the present application is not limited to the details of the above-described exemplary embodiments, and that the present application can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application. Therefore, the embodiments should be regarded as illustrative and non-restrictive from any point of view, and the scope of the application is defined by the appended claims rather than the above description, and it is therefore intended that all claims falling within the claims All changes within the meaning and scope of the equivalent elements are included in this application. Any reference signs in the claims shall not be construed as limiting the claim concerned. Furthermore, it is clear that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. Multiple units or means stated in a device claim can also be implemented by one unit or means by software or hardware. Words such as first and second are used to indicate names and do not indicate any specific order.

Claims

A projection interaction method, wherein the method is applied to computer equipment, the computer equipment includes a top camera device and a projection device, the method includes:

Collect the corresponding top image information through the top camera device, and transmit the top image information to the corresponding target user equipment, so that the target user equipment can present the top image information;

Obtain image annotation information about the top image information of the target user device corresponding to the target user, wherein the image annotation information includes corresponding annotation information and annotation position information of the annotation information, and the annotation information is composed of the The first user operation of the target user is determined;

Corresponding projection position information is determined based on the annotation position information, and the annotation information is projected and presented based on the projection position information.
The method of claim 1, wherein the computer device further includes a display device, the method further comprising:

Receive the target image information transmitted by the target user equipment, and present the target image information through the display device.
The method according to claim 1 or 2, wherein the computer equipment further includes a front-facing camera device, wherein the front-facing camera device is used to collect front-facing image information about the current user of the computer device; wherein , the method also includes:

The pre-image information is transmitted to the target user equipment for the target user equipment to present the pre-image information.
The method of claim 3, further comprising:

Obtain a camera switching request regarding the current video interaction between the computer device and the target user device, wherein the image information of the current video interaction includes the front image information;

Wherein, collecting the corresponding top image information through the top camera device and transmitting the top image information to the corresponding target user equipment for the target user equipment to present the top image information includes:

In response to the camera switching request, turn off the front camera device and enable the top camera device, collect the corresponding top image information through the top camera device, and transmit the top image information to the corresponding target user equipment, The top image information is presented to the target user device.
The method according to claim 4, wherein the obtaining a camera switching request regarding the current video interaction between the computer device and the target user device includes:

Receive a camera switching request transmitted by the target user equipment regarding the current video interaction between the computer device and the target user equipment, wherein the camera switching request is determined based on a second user operation of the target user.
The method according to claim 3 or 4, wherein said transmitting the pre-image information to the target user device for the target user device to present the pre-image information includes:

Obtain a video creation request regarding the current video interaction;

In response to the video creation request, establish the current video interaction between the computer device and the target user device based on the video creation request, collect corresponding front image information through the front camera device and transmit it to the Target user equipment, for the target user equipment to present the pre-image information.
The method of claim 4, further comprising:

Obtain a camera restoration request regarding the current video interaction between the computer device and the target user device;

In response to the camera restoration request, turn off the top camera image and enable the front camera device, collect corresponding front image information through the front camera device, and transmit the front image information to the corresponding target User equipment.
The method of claim 3, further comprising:

Obtain a camera start request regarding the video interaction between the computer device and the target user device;

Wherein, collecting the corresponding top image information through the top camera device and transmitting the top image information to the corresponding target user equipment for the target user equipment to present the top image information includes:

In response to the camera start request, collect the corresponding top image information through the top camera device, and transmit the top image information to the corresponding target user equipment;

Wherein, transmitting the pre-image information to the target user equipment for the target user equipment to present the pre-image information includes:

In response to the camera start request, the corresponding front image information is collected through the front camera device, and the front image information is transmitted to the target user equipment.
The method of claim 1, wherein the computer device includes a lighting device; wherein the method further includes:

If an activation request for the lighting device is obtained, the lighting device is turned on.
The method of claim 9, wherein the computer device further includes an ambient light detection device, the method further comprising:

Obtain the light intensity information of the current environment based on the ambient light detection device, and detect whether the light intensity information meets a preset lighting threshold;

If not, the lighting device is adjusted until the light intensity information meets the preset lighting threshold.
The method of claim 9, further comprising:

Determine the image brightness information corresponding to the top image information based on the top image information, and detect whether the image brightness information meets a preset brightness threshold;

If not, the lighting device is adjusted until the image brightness information meets the preset brightness threshold.
The method of claim 1, further comprising:

Obtain the interactive position information of the current user regarding the top image information;

Corresponding virtual presence information is determined based on the interactive position information, and the virtual presence information is projected and presented through the projection device.
The method according to claim 12, wherein the computer device includes an infrared measurement device; wherein the obtaining the interactive position information of the current user with respect to the top image information includes:

The interactive position information of the top image information is determined by the infrared measurement device.
The method according to claim 12 or 13, wherein the obtaining the image annotation information about the top image information of the target user device corresponding to the target user includes:

Send the interactive location information to the target user equipment, receive the annotation information transmitted by the target user equipment and returned based on the interactive location information, thereby determining the image annotation information of the top image information, wherein, The image annotation information includes the annotation information and the interactive position information, and the annotation information is determined by the first user operation of the target user.
The method of claim 1, wherein the computer device further includes a distance measurement device; wherein the method further includes:

The distance information between the current user and the computer device is determined through the distance measurement device, and if the distance information does not meet a preset distance threshold, the computer device issues a notification.
A projection interaction method, which is applied to a target user device, the method includes:

Receive and present the top image information transmitted by the corresponding computer equipment and collected by the corresponding top camera device;

Obtain the first user operation of the target user device corresponding to the target user, and generate annotation information about the top image information based on the first user operation;

The annotation information is returned to the computer device for the computer device to present the annotation information through a corresponding projection device.
The method of claim 16, wherein the method further includes:

Determine corresponding annotation location information based on the first user operation;

Wherein, returning the annotation information to the computer device for the computer device to present the annotation information through a corresponding projection device includes:

The annotation information and the annotation position information are returned to the computer device for the computer device to present the annotation information through a corresponding projection device based on the annotation position information.
A computer device for projection interaction. The computer device includes a top camera device and a projection device. The device includes:

A module configured to collect corresponding top image information through the top camera device, and transmit the top image information to the corresponding target user equipment, so that the target user equipment can present the top image information;

One or two modules, configured to obtain image annotation information about the top image information of the target user device corresponding to the target user, wherein the image annotation information includes corresponding annotation information and annotation position information of the annotation information, so The annotation information is determined by the first user operation of the target user;

A third module, configured to determine corresponding projection position information based on the annotation position information, and project and present the annotation information based on the projection position information.
A target user device for projection interaction, wherein the device includes:

The two-one module is used to receive and present the top image information transmitted by the corresponding computer equipment and collected by the corresponding top camera device;

The second module is used to obtain the first user operation of the target user device corresponding to the target user, and generate annotation information about the top image information based on the first user operation;

The second and third modules are used to return the annotation information to the computer device, so that the computer device can present the annotation information through a corresponding projection device.
A computer device, wherein the device includes:

processor; and

A memory arranged to store computer-executable instructions which, when executed, cause the processor to perform the steps of a method as claimed in any one of claims 1 to 17.
A computer-readable storage medium with a computer program/instruction stored thereon, characterized in that, when executed, the computer program/instruction causes the system to perform the steps of the method described in any one of claims 1 to 17.
A computer program product, including a computer program/instruction, characterized in that when the computer program/instruction is executed by a processor, the steps of the method described in any one of claims 1 to 17 are implemented.