WO2017185799A1

WO2017185799A1 - Method and apparatus for implementing multi-screen interaction

Info

Publication number: WO2017185799A1
Application number: PCT/CN2016/113212
Authority: WO
Inventors: 张奎
Original assignee: 广州视睿电子科技有限公司
Priority date: 2016-04-29
Filing date: 2016-12-29
Publication date: 2017-11-02
Also published as: CN105892976B; CN105892976A

Abstract

A method and an apparatus for implementing a multi-screen interaction. The method comprises: receiving window screenshot information transmitted by a multi-screen initiating terminal (S11), wherein the window screenshot information comprises window images, window position information corresponding to the window images and a timestamp; writing the received multiple window images into a cache according to the window position information and the timestamp to obtain a corresponding synthetic display picture (S12), wherein the multiple window images in the synthetic display picture are presented in a hierarchical form; and transmitting the synthetic display picture to a corresponding display device (S13) so as to display the synthetic display picture at the display device. The method and the apparatus can display the window pictures of a plurality of multi-screen initiating terminals simultaneously on one display device, so that there are no screen-split issues resulting from different system platforms of the multi-screen initiating terminals.

Description

Method and device for realizing multi-screen interaction

Technical field

The present invention relates to the field of multi-screen interaction technology, and in particular to a method and apparatus for implementing multi-screen interaction.

Background technique

Multi-screen interaction refers to the technology of sharing presentation content on different system platform devices. That is, after several devices are connected through a dedicated connection device, operations such as display (such as audio, video, and picture) can be simultaneously performed between the devices. For example, a movie on a mobile phone can be played on a television, and a picture on a tablet or a computer can be displayed on a television.

The structure of the existing multi-screen interactive system is as shown in FIG. 1. A multi-screen initiating device such as a mobile phone or a PC is connected to an Android system device (as a connecting device) with a TV function by a VGA (Video Graphics Array) method, and the TV is connected. The functional Android system device is also connected to the display via VGA. Through the Android system device with TV function, it is possible to display the content displayed by the multi-screen initiator device such as a mobile phone or a PC on the same display.

Based on the existing multi-screen interactive system, the essence is to display the entire screen of other system devices on the display, and the following problems exist:

The hardware cost is high, and the connected device of the Android motherboard with full function is needed; and the entire screen of the other multi-screen initiating device is displayed on the display, and the splitting problem caused by different systems exists on the display end.

Summary of the invention

Based on this, an embodiment of the present invention provides a method and device for implementing multi-screen interaction, which can simultaneously display window screens of multiple multi-screen initiating terminals on one display device, and there is no multi-screen initiating terminal. Splitting problems caused by different system platforms.

An aspect of the present invention provides a method for implementing multi-screen interaction, including:

Receiving window screenshot information sent by the multi-screen initiating terminal, where the window screenshot information includes a window image, and window position information and a time stamp corresponding to the window image;

The received multi-way window image is drawn to the cache according to the window position information and the time stamp, to obtain a corresponding composite display image, wherein the multi-way window image in the composite display screen is presented in a layered form;

The composite display screen is transmitted to the corresponding display device to display the composite display screen on the display device side.

Preferably, the drawing the received multi-way window image to the cache according to the window position information and the time stamp to obtain a corresponding composite display screen, including:

Extracting the window image, window position information and time stamp included in the received screenshot information of each window, sorting the multi-way window images according to the timestamp from near to far, so that the window images with the closer time stamps are sorted earlier;

The respective window images are sequentially drawn to the cache according to the window position information and before and after the sorting to obtain a corresponding composite display screen, wherein the top-ranked window image is presented at the topmost layer of the composite display screen.

Preferably, the drawing, according to the window position information and the sequential window images, to the cache, to obtain a corresponding composite display screen, comprising:

Decoding each window image to obtain corresponding image data; sequentially drawing each window image to the buffer according to the before and after sorting, the window position information, and the image data, to obtain a corresponding composite display screen.

Preferably, the sending the composite display screen to the corresponding display device comprises:

The composite display screen is stored in a preset display queue, and the composite display screen in the display queue is sequentially sent to the corresponding display device.

Preferably, the window location information includes: a screen resolution of the corresponding multi-screen initiating terminal, and a position coordinate of the window being screened in the multi-screen initiating terminal screen.

The invention also provides a method for implementing multi-screen interaction, comprising:

Monitoring the active window of the multi-screen initiating terminal, and determining whether window switching occurs according to the window screen, the window size, and the window position;

Determining the occurrence of a window switch, taking a screenshot of the current active window, and obtaining a corresponding window image; and recording the time information of the screenshot as the time stamp corresponding to the window image, and obtaining the position information of the current active window in the screen of the multi-screen launching terminal, Window position information corresponding to the window image;

Sending window screenshot information of the multi-screen originating terminal to the corresponding hybrid device, the window screenshot information including a window image, corresponding window position information, and a time stamp.

Preferably, the sending, to the corresponding hybrid device, the window screenshot information of the multi-screen initiating terminal, which previously includes:

The obtained window image is image-encoded, and the window screenshot information includes the encoded window image information.

Preferably, the method further includes: establishing a connection between the multi-screen initiating terminal and the hybrid device, and installing a driver corresponding to the hybrid device in the system of the multi-screen initiating terminal after the connection is established;

The monitoring multi-screen initiates an active window of the terminal, and determines whether a window switch occurs according to the active window screen, the window size, and the window position, including:

The active window of the multi-screen initiating terminal is monitored by the driver, and if at least one of the window screen, the window size, and the window position is detected to be changed, it is determined that the window switching occurs.

Preferably, the acquiring the location information of the current active window in the screen of the multi-screen initiation terminal includes:

Obtain the screen resolution of the multi-screen initiating terminal and the position coordinates of the current active window in the multi-screen initiating terminal screen.

The invention also provides an apparatus for implementing multi-screen interaction, comprising:

An image receiving module, configured to receive window screenshot information sent by the multi-screen initiating terminal, where the window screenshot information includes a window image, and window position information and a time stamp corresponding to the window image;

An image mixing module, configured to draw the received multi-way window image to the cache according to the window position information and the time stamp, to obtain a corresponding composite display image, wherein the multi-way window image in the composite display screen is presented in a layered form;

And a sending module, configured to send the composite display screen to the corresponding display device to display the composite display screen on the display device end.

Preferably, the image mixing module comprises:

An information extraction sub-module, configured to extract a window image, a window position information, and a time stamp included in the received screenshot information of each window;

a sorting sub-module for sorting the multi-way window images according to the timestamps from near to far, so that the window images in which the time stamps are closer are sorted earlier;

The mixing sub-module is configured to sequentially draw each window image to the cache according to the window position information and before and after the sorting, to obtain a corresponding composite display image, wherein the top-ranked window image is presented at the top layer of the composite display screen.

Preferably, the hybrid submodule comprises:

a decoding unit, configured to decode each window image to obtain corresponding image data;

The mixing unit is configured to sequentially draw each window image to the cache according to the before and after sorting, the window position information, and the image data, to obtain a corresponding composite display screen.

Preferably, the sending module includes:

a storage submodule, configured to store the composite display screen in a preset display queue;

a sending submodule, configured to sequentially send the composite display screen in the display queue to a corresponding display device.

a window monitoring module, configured to monitor an active window of the multi-screen initiating terminal, and determine whether a window switching occurs according to the window screen, the window size, and the window position;

a window capture module, configured to determine that a window switch occurs, take a screenshot of the current active window, obtain a corresponding window image; and time information for recording the screenshot, as a time stamp corresponding to the window image; and for acquiring a current active window Positioning information in the screen of the multi-screen is displayed as window position information corresponding to the window image;

The screenshot sending module is configured to send window screenshot information of the multi-screen initiating terminal to the corresponding hybrid device, where the window screenshot information includes a window image, corresponding window position information, and a time stamp.

Preferably, the screenshot sending module is further configured to perform image encoding on the obtained window image before sending the window screenshot information of the multi-screen initiating terminal to the corresponding mixing device.

Preferably, the device for implementing multi-screen interaction further includes: a connection module, configured to establish a connection between the multi-screen initiating terminal and the hybrid device, and install in the system of the multi-screen initiating terminal after the connection is established. a driver corresponding to the hybrid device;

The window monitoring module is configured to monitor, by the driver, an active window of the multi-screen initiating terminal, and if at least one of the window screen, the window size, and the window position is detected to change, it is determined that the window switching occurs.

The above technical solution, by receiving the window shot information sent by the multi-screen initiation terminal, the window screenshot information includes a window image, and window position information and a time stamp corresponding to the window image; and the received according to the window position information and the time stamp The multiplexed window image is drawn to the cache to obtain a corresponding composite display screen, wherein the multiplexed window image is presented in a layered form; the composite display screen is sent to the corresponding display device to be at the display device end The composite display screen is displayed. The solution of the foregoing embodiment of the present invention can simultaneously display window images of multiple multi-screen initiating terminals on one display device, and there is no fragmentation problem caused by different system platforms of the multi-screen initiating terminal, thereby improving the utilization ratio of the display; Moreover, the hardware cost is low, and the problem that all the screens of the multi-screen initiating terminal are displayed on the display device is solved, resulting in a screen splitting on the display device end.

DRAWINGS

Figure 1 is a schematic diagram of a conventional multi-screen interactive system.

2 is a schematic diagram of a working environment of an embodiment of the present invention;

FIG. 3 is a schematic flowchart of a method for implementing multi-screen interaction according to an embodiment; FIG.

4 is a schematic diagram of a composite display screen of an embodiment;

FIG. 5 is a schematic flowchart of a method for implementing multi-screen interaction according to another embodiment; FIG.

6 is a schematic structural diagram of an apparatus for implementing multi-screen interaction according to an embodiment;

FIG. 7 is a schematic structural diagram of an apparatus for implementing multi-screen interaction according to another embodiment.

detailed description

The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It is understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

2 is a schematic diagram of a working environment according to an embodiment of the present invention, including: at least two multi-screen initiating terminals 100, wherein the multi-screen initiating terminal can control the display of the local screen display information to other in the multi-screen interaction. The device on the terminal screen takes a PC as an example. Each multi-screen initiating terminal 100 can be connected to a preset mixing device 200 by a USB method, and the hybrid device refers to an intermediate device that can obtain a composite display screen from a multi-way window image, and the mixing device 200 can have two or more The plurality of USB input ports are connected to the multi-screen initiating terminal; the hybrid device 200 is also connected to a display device 300 by means of VGA or HDMI (High Definition Multimedia Interface). Each of the multi-screen initiating terminals 100 can display its active window screen to the display device 300 through the mixing device 200.

In conjunction with the schematic diagram shown in FIG. 2, various embodiments of a method and apparatus for implementing multi-screen interaction are described below.

FIG. 3 is a schematic flowchart of a method for implementing multi-screen interaction according to an embodiment; in this embodiment, the method is applied to the hybrid device as an example for description.

As shown in FIG. 3, the method for implementing multi-screen interaction in this embodiment includes the following steps:

S11, receiving window screenshot information sent by the multi-screen initiating terminal, where the window screenshot information includes a window image, and window position information and a time stamp corresponding to the window image;

The window position information in this embodiment includes: a screen resolution of the corresponding multi-screen initiating terminal, and a position coordinate of the window being screened in the multi-screen initiating terminal screen. Through the screen resolution and the position coordinates in the screen, the position of the screen being screened in the multi-screen launch terminal screen can be determined.

S12, the received multi-path window image is drawn to the cache according to the window position information and the time stamp, to obtain a corresponding composite display image, wherein the multi-way window image in the composite display screen is presented in a layered form;

The implementation of the step may include: extracting the window image, the window position information, and the timestamp included in the received screenshot information of each window, and sorting the multiple window images according to the timestamp from the near to far, and making the timestamp The closer the window image is, the more advanced it is. In this embodiment, the near time of the time stamp is Taking the current time as a reference, the purpose is to sort the window images whose time stamps are closer to the current time in the received multi-path image. Then, the respective window images are sequentially drawn to the cache according to the window position information and before and after the sorting to obtain a corresponding composite display screen, wherein the top-ranked window image is presented at the top layer of the composite display screen.

Preferably, the received two-way window image is synthesized, and the obtained composite display screen effect can be referred to FIG. 4; wherein, the screen 1 is the active window screen of the multi-screen initiation terminal 1, and the screen 2 is the multi-screen initiation terminal 2 The active window screen, and the time stamp of the window image corresponding to the active window screen of the multi-screen initiating terminal 2 is closer.

As a preferred embodiment, in order to improve transmission efficiency, the window image received in step S11 is an encoded image. Correspondingly, in step S12, each window image is first decoded to obtain corresponding image data. For example, h264 (standard writing method should be H.264, a highly compressed digital video codec standard) decoding technology can be used to decode each window image to obtain YUV stream data corresponding to each window image (color image signal) After color separation, the RGB is respectively amplified and corrected, and then the luminance signal Y and the two color difference signals B-Y (i.e., U) and R-Y (i.e., V) are obtained by the matrix conversion circuit. Then, each window image is sequentially drawn to the cache according to the before and after sorting, the window position information, and the image data, to obtain a corresponding composite display screen.

S13. Send the composite display screen to a corresponding display device to display the composite display screen on the display device end.

Preferably, the obtained composite display screen is stored in a display queue preset by the mixing device, and the composite display screen in the display queue is sequentially sent to the corresponding display device according to the first-in-first-out principle of the queue; and each window is The position of the image on the screen of the display device corresponds to the position of the window in the screen of the corresponding multi-screen originating terminal.

Based on the above method for implementing multi-screen interaction, two methods for implementing multi-screen interaction are given below. Application scenario:

Scenario 1, the users each take a laptop to a meeting, and the content on the personal laptop needs to be displayed on a large screen display. At this time, multiple laptops can be connected to the hybrid device, and then the hybrid device can be connected to the above-mentioned large-screen display. Based on the system structure, the active window of multiple laptops can be simultaneously performed on the large-screen display. display. Further, when the active window of a certain laptop is found to be switched, on the large-screen display, the screen after the switching is displayed at the top layer, just like switching two program windows, and implementing multiple The effect of switching between window screens.

In scenario 2, the users each took the laptop to the meeting. There were two documents to be displayed to everyone, but the two files were on different users' computers. At this time, as long as the two computers are connected to the hybrid device, the files of the two laptops can be displayed simultaneously on the large-screen display; and the position of each file window displayed on the large-screen display is on the screen of the laptop. The location corresponds.

Through the method for implementing multi-screen interaction in the above embodiment, an external hybrid device is used to display window images of multiple multi-screen initiating terminals on the same display device, thereby achieving low cost; meanwhile, the display device can be implemented according to user operations. End window switching. Thereby, the utilization of the display is improved, and the problem that all the screens of the multi-screen initiating terminal are displayed on the display device and the screen is split at the display device end is solved.

FIG. 5 is a schematic flowchart of a method for implementing multi-screen interaction in another embodiment; in this embodiment, an example is applied to a multi-screen initiating terminal.

As shown in FIG. 5, the method for implementing multi-screen interaction in this embodiment includes the following steps:

S21: monitoring an active window of the multi-screen initiating terminal, and determining whether a window switching occurs according to the window screen, the window size, and the window position;

As a preferred implementation, before this step, the method further includes establishing a multi-screen initiator terminal and a hybrid device. The step of the connection; referring to FIG. 2, the connection may be a USB connection.

In addition, in this embodiment, after the connection is established, a driver corresponding to the hybrid device is automatically installed in the system of the multi-screen initiating terminal; and then the active window of the multi-screen initiating terminal is monitored by the driver. Whether or not window switching occurs depends on the active window screen, window size, and window position. Preferably, in this embodiment, if at least one of the window screen, the window size, and the window position is detected to change, it is determined that the window switching occurs.

S22, determining that a window switch occurs, taking a screenshot of the current active window, and obtaining a corresponding window image; and recording time information of the screenshot, as a timestamp corresponding to the window image, obtaining a position of the current active window in the screen of the multi-screen launch terminal Information as window position information corresponding to the window image;

To determine that a window switch occurs, the driver can control the current active window to take a screenshot and cache it, and record the time information of the screenshot. If the active window switches to another window or its size, location sends a change, the driver will control the window shot again.

Preferably, obtaining the location information of the current active window in the multi-screen initiating terminal screen in the step includes: acquiring a screen resolution of the multi-screen initiating terminal, and a position coordinate of the current active window in the multi-screen initiating terminal screen. Correspondingly, the window position information corresponding to the window image includes a screen resolution of the corresponding multi-screen initiating terminal, and a position coordinate of the window being screened in the multi-screen initiating terminal screen.

S23. Send window screenshot information of the multi-screen initiating terminal to the corresponding hybrid device, where the window screenshot information includes a window image, corresponding window location information, and a timestamp.

Preferably, before sending the window screenshot information of the multi-screen initiating terminal to the corresponding mixing device, the method further includes the step of performing image encoding on the obtained window image. Correspondingly, the window screenshot information includes the encoded window image information. For example, the image of the active window image is encoded by the h264 encoding technology, and then transmitted together with the window position information and the time stamp to reduce the amount of communication data and improve the communication speed.

It should be noted that, for the foregoing method embodiments, for the sake of brevity, they are all described as a series of action combinations, but those skilled in the art should understand that the present invention is not limited by the described action sequence, because In accordance with the present invention, certain steps may be performed in other sequences or concurrently.

Based on the same idea as the method for implementing multi-screen interaction in the above embodiment, the present invention also provides an apparatus for implementing multi-screen interaction, which can be used to perform the above-described method for realizing multi-screen interaction. For the convenience of the description, in the structural diagram of the device embodiment for implementing the multi-screen interaction, only the parts related to the embodiment of the present invention are shown. Those skilled in the art can understand that the illustrated structure does not constitute a limitation on the device, and may include More or fewer parts than the illustration, or a combination of some parts, or a different part arrangement.

FIG. 6 is a schematic structural diagram of an apparatus for implementing multi-screen interaction according to an embodiment of the present invention; as shown in FIG. 6, the apparatus for implementing multi-screen interaction in this embodiment includes: an image receiving module 310, an image mixing module 320, and a sending Module 330, each module is detailed as follows:

The image receiving module 310 is configured to receive window screenshot information sent by the multi-screen initiating terminal, where the window screenshot information includes a window image, and window position information and a time stamp corresponding to the window image;

The image mixing module 320 is configured to draw the received multi-path window image to the cache according to the window position information and the time stamp, to obtain a corresponding composite display image, where the multi-way window image is presented in a layered form;

The sending module 330 is configured to send the composite display screen to the corresponding display device to display the composite display screen on the display device end.

As a preferred embodiment, the image mixing module 320 includes:

And a mixing sub-module, configured to sequentially draw each window image to the cache according to the window position information and before and after the sorting, to obtain a corresponding composite display image, wherein the top-ranked window image is presented at the top layer of the composite display screen. For example, the two-way window image is synthesized, and the obtained composite display screen effect can be referred to FIG. 4; wherein, the screen 1 is the active window screen of the multi-screen initiating terminal 1, and the screen 2 is the active window screen of the multi-screen initiating terminal 2. And the time stamp of the window image corresponding to the active window screen of the multi-screen initiating terminal 2 is closer.

Further, the mixing submodule may include:

a decoding unit for decoding each window image to obtain corresponding image data; for example, h264 (standard writing method should be H.264, a highly compressed digital video codec standard) decoding technology can be used for each window image Decoding, obtain the YUV stream data corresponding to each window image (the color image signal is subjected to color separation, respectively amplified and corrected to obtain RGB, and then obtained by the matrix conversion circuit, including the luminance signal Y and the two color difference signals B-Y (ie U), R-Y (ie V) data).

And a mixing unit, configured to sequentially draw each window image to the cache according to the before and after sorting, the window position information, and the image data, to obtain a corresponding composite display screen.

As a preferred implementation manner, the sending module 310 includes:

And a sending submodule, configured to sequentially send the composite display screen in the display queue to the corresponding display device to display the composite display screen on the display device end.

FIG. 7 is a schematic structural diagram of an apparatus for implementing multi-screen interaction according to another embodiment; as shown in FIG. 7, the apparatus for implementing multi-screen interaction in this embodiment may be a computer, and the apparatus includes: a window monitoring module 410, and a window. The screenshot module 420 and the screenshot sending module 430 are detailed as follows:

The window monitoring module 410 is configured to monitor an active window of the multi-screen initiating terminal, and determine whether a window switching occurs according to the window screen, the window size, and the window position;

The window capture module 420 is configured to determine that a window switch occurs, take a screenshot of the current active window, and obtain a corresponding window image; and time information for recording the screenshot, as a time stamp corresponding to the window image; The position information of the active window in the multi-screen launch terminal screen is used as the window position information corresponding to the window image;

In this embodiment, the acquiring location information of the current active window in the screen of the multi-screen initiating terminal includes: acquiring a screen resolution of the multi-screen initiating terminal, and a position coordinate of the current active window in the multi-screen initiating terminal screen.

The screenshot sending module 430 is configured to send window screenshot information of the multi-screen initiating terminal to the corresponding mixing device, where the window screenshot information includes a window image, corresponding window position information, and a time stamp.

As a preferred embodiment, the screenshot sending module 430 is further configured to perform image encoding on the obtained window image before sending the window screenshot information of the multi-screen initiating terminal to the corresponding mixing device. For example, the image of the active window image is encoded by the h264 encoding technology, and then transmitted together with the window position information and the time stamp to reduce the amount of communication data and improve the communication speed.

As a preferred implementation manner, the apparatus for implementing multi-screen interaction may further include a connection module, configured to establish a connection between the multi-screen initiation terminal and the hybrid device, and a system for initiating the terminal on the multi-screen after the connection is established. A driver corresponding to the hybrid device is installed in the middle. Correspondingly, the window monitoring module 310 can monitor the active window of the multi-screen initiating terminal by using the driver, and if it is detected that at least one of the window screen, the window size, and the window position changes, it is determined that the window switching occurs.

It should be noted that, in the implementation of the apparatus for implementing multi-screen interaction in the above examples, the information interaction, execution process, and the like between the modules/units are based on the foregoing method embodiments of the present invention. The technical effect of the present invention is the same as that of the foregoing method embodiment of the present invention. For details, refer to the description in the method embodiment of the present invention, and details are not described herein again.

In addition, in the embodiments of the foregoing apparatus for implementing multi-screen interaction, the logical division of each functional module is merely an example, and the actual application may be considered according to requirements, for example, for configuration requirements of corresponding hardware or convenience of implementation of software. The above function assignment is performed by different functional modules, that is, the internal structure of the device for realizing multi-screen interaction is divided into different functional modules to complete all or part of the functions described above. Each function module can be implemented in the form of hardware or in the form of a software function module.

It will be understood by those skilled in the art that all or part of the processes in the above embodiments may be implemented by a computer program to instruct related hardware, and the program may be stored in a computer readable storage medium as Independent product sales or use. The program, when executed, may perform all or part of the steps of an embodiment of the methods described above. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

In the above embodiments, the descriptions of the various embodiments are all focused, and the parts that are not detailed in a certain embodiment can be referred to the related descriptions of other embodiments.

The above-described embodiments are merely illustrative of several embodiments of the invention and are not to be construed as limiting the scope of the invention. It should be noted that a number of variations and modifications may be made by those skilled in the art without departing from the spirit and scope of the invention. Therefore, the scope of the invention should be determined by the appended claims.

Claims

A method for implementing multi-screen interaction, characterized in that it comprises:

Receiving window screenshot information sent by the multi-screen initiating terminal, where the window screenshot information includes a window image, and window position information and a time stamp corresponding to the window image;

The received multi-way window image is drawn to the cache according to the window position information and the time stamp, to obtain a corresponding composite display image, wherein the multi-way window image in the composite display screen is presented in a layered form;

The composite display screen is transmitted to the corresponding display device to display the composite display screen on the display device side.
The method for implementing multi-screen interaction according to claim 1, wherein the drawing of the received multi-way window image to the cache according to the window position information and the time stamp, to obtain a corresponding composite display screen, comprising:

Extracting the window image, window position information and time stamp included in the received screenshot information of each window, sorting the multi-way window images according to the timestamp from near to far, so that the window images with the closer time stamps are sorted earlier;

The respective window images are sequentially drawn to the cache according to the window position information and before and after the sorting to obtain a corresponding composite display screen, wherein the top-ranked window image is presented at the topmost layer of the composite display screen.
The method for realizing multi-screen interaction according to claim 2, wherein the drawing of each window image to the cache according to the window position information and before and after sorting, to obtain a corresponding composite display screen, comprising:

Decoding each window image to obtain corresponding image data; sequentially drawing each window image to the buffer according to the before and after sorting, the window position information, and the image data, to obtain a corresponding composite display screen.
A method of implementing multi-screen interaction as claimed in claim 1, wherein said corresponding The display device sends the composite display screen, including:

The composite display screen is stored in a preset display queue, and the composite display screen in the display queue is sequentially sent to the corresponding display device.
The method for implementing multi-screen interaction according to claim 1, wherein the window position information comprises: a screen resolution of the corresponding multi-screen initiating terminal, and a position of the screen being screened in the multi-screen initiating terminal screen. coordinate.
A method for implementing multi-screen interaction, characterized in that it comprises:

Monitoring the active window of the multi-screen initiating terminal, and determining whether window switching occurs according to the window screen, the window size, and the window position;

Determining the occurrence of a window switch, taking a screenshot of the current active window, and obtaining a corresponding window image; and recording the time information of the screenshot as the time stamp corresponding to the window image, and obtaining the position information of the current active window in the screen of the multi-screen launching terminal, Window position information corresponding to the window image;

Sending window screenshot information of the multi-screen originating terminal to the corresponding hybrid device, the window screenshot information including a window image, corresponding window position information, and a time stamp.
The method for implementing multi-screen interaction according to claim 6, wherein the sending the window screenshot information of the multi-screen initiating terminal to the corresponding hybrid device comprises:

The obtained window image is image-encoded, and the window screenshot information includes the encoded window image information.
The method for implementing multi-screen interaction according to claim 6, further comprising: establishing a connection between the multi-screen initiating terminal and the hybrid device, and after the connection is established, in the system of the multi-screen initiating terminal Installing a driver corresponding to the hybrid device;

The monitoring multi-screen initiates an active window of the terminal, and determines whether a window switch occurs according to the active window screen, the window size, and the window position, including:

The active window of the multi-screen initiating terminal is monitored by the driver, and if at least one of the window screen, the window size, and the window position is detected to be changed, it is determined that the window switching occurs.
The method for implementing the multi-screen interaction according to claim 6, wherein the obtaining the location information of the current active window in the screen of the multi-screen initiating terminal comprises:

Obtain the screen resolution of the multi-screen initiating terminal and the position coordinates of the current active window in the multi-screen initiating terminal screen.
An apparatus for implementing multi-screen interaction, comprising:

An image receiving module, configured to receive window screenshot information sent by the multi-screen initiating terminal, where the window screenshot information includes a window image, and window position information and a time stamp corresponding to the window image;

An image mixing module, configured to draw the received multi-way window image to the cache according to the window position information and the time stamp, to obtain a corresponding composite display image, wherein the multi-way window image in the composite display screen is presented in a layered form;

And a sending module, configured to send the composite display screen to the corresponding display device to display the composite display screen on the display device end.
The apparatus for implementing multi-screen interaction according to claim 10, wherein the image mixing module comprises:

An information extraction sub-module, configured to extract a window image, a window position information, and a time stamp included in the received screenshot information of each window;

a sorting sub-module for sorting the multi-way window images according to the timestamps from near to far, so that the window images in which the time stamps are closer are sorted earlier;

The mixing sub-module is configured to sequentially draw each window image to the cache according to the window position information and before and after the sorting, to obtain a corresponding composite display image, wherein the top-ranked window image is presented at the top layer of the composite display screen.
The apparatus for implementing multi-screen interaction according to claim 11, wherein the hybrid sub-module comprises:

a decoding unit, configured to decode each window image to obtain corresponding image data;

The mixing unit is configured to sequentially draw each window image to the cache according to the before and after sorting, the window position information, and the image data, to obtain a corresponding composite display screen.
The apparatus for implementing multi-screen interaction according to claim 10, wherein the sending module comprises:

a storage submodule, configured to store the composite display screen in a preset display queue;

And a sending submodule, configured to sequentially send the composite display screen in the display queue to the corresponding display device.
The apparatus for implementing multi-screen interaction according to claim 10, wherein the window position information comprises: a screen resolution of the corresponding multi-screen initiating terminal, and a position of the screen being screened in the multi-screen initiating terminal screen. coordinate.
An apparatus for implementing multi-screen interaction, comprising:

a window monitoring module, configured to monitor an active window of the multi-screen initiating terminal, and determine whether a window switching occurs according to the window screen, the window size, and the window position;

a window capture module, configured to determine that a window switch occurs, take a screenshot of the current active window, obtain a corresponding window image; and time information for recording the screenshot, as a time stamp corresponding to the window image; and for acquiring a current active window Positioning information in the screen of the multi-screen is displayed as window position information corresponding to the window image;

The screenshot sending module is configured to send window screenshot information of the multi-screen initiating terminal to the corresponding hybrid device, where the window screenshot information includes a window image, corresponding window position information, and a time stamp.
The apparatus for implementing multi-screen interaction according to claim 15, wherein said screenshot is sent The sending module is further configured to perform image encoding on the obtained window image before sending the window screenshot information of the multi-screen initiating terminal to the corresponding mixing device.
The apparatus for implementing multi-screen interaction according to claim 15, further comprising: a connection module, configured to establish a connection between the multi-screen initiating terminal and the hybrid device, and after the connection is established, in the multi-screen a driver corresponding to the hybrid device is installed in a system of the initiating terminal;

The window monitoring module is configured to monitor, by the driver, an active window of the multi-screen initiating terminal, and if at least one of the window screen, the window size, and the window position is detected to change, it is determined that the window switching occurs.
The apparatus for implementing multi-screen interaction according to claim 15, wherein the obtaining the location information of the current active window in the screen of the multi-screen initiating terminal comprises:

Obtain the screen resolution of the multi-screen initiating terminal and the position coordinates of the current active window in the multi-screen initiating terminal screen.