CN115705173A

CN115705173A - Multi-screen interaction method and device, storage medium and electronic equipment

Info

Publication number: CN115705173A
Application number: CN202110899280.4A
Authority: CN
Inventors: 谢毅
Original assignee: Guangzhou Shiyuan Electronics Thecnology Co Ltd; Guangzhou Shirui Electronics Co Ltd
Current assignee: Guangzhou Shiyuan Electronics Thecnology Co Ltd; Guangzhou Shirui Electronics Co Ltd
Priority date: 2021-08-05
Filing date: 2021-08-05
Publication date: 2023-02-17

Abstract

The application discloses a multi-screen interaction method, a multi-screen interaction device, a storage medium and electronic equipment, which are applied to a screen expansion system, wherein the screen expansion system comprises main touch equipment, an auxiliary touch screen, a video matrix and display peripherals, the main touch equipment, the auxiliary touch screen and the display peripherals are respectively connected with the video matrix, the main touch equipment is in communication connection with the auxiliary touch screen, a picture of the display peripherals is projected to the main touch screen through the video matrix, interface coordinate information of the main touch screen is determined based on the video matrix, a first signal blocking layer is started based on the interface coordinate information of the main touch screen, and/or the picture of the display peripherals is projected to the auxiliary touch screen through the video matrix, interface coordinate information of the auxiliary touch screen is determined based on the video matrix, and a second signal blocking layer is started based on the interface coordinate information of the auxiliary touch screen.

Description

Multi-screen interaction method and device, storage medium and electronic equipment

Technical Field

The application relates to the technical field of computers, in particular to a multi-screen interaction method, a multi-screen interaction device, a storage medium and electronic equipment.

Background

In the screen expansion scene, a plurality of screens exist, the screens can receive the projection of other display peripherals, when the screen of the display peripherals is projected to at least one screen through a video matrix, a first picture displayed by the screen of the projected screen is replaced by a second picture displayed by the display peripherals after the screen of the display peripherals is projected, because the plurality of screens are in communication connection through a Universal Serial Bus (USB), the display peripherals are not in communication connection with any screen through the USB, and a touch signal generated by the touch operation is transmitted to the first picture of the screen of the projected screen, so that the first picture of the screen of the projected screen responds to the touch signal generated by the touch operation, namely the problem of touch penetration is caused.

Disclosure of Invention

The embodiment of the application provides a multi-screen interaction method and device, a storage medium and electronic equipment, which can prevent the problem of touch penetration. The technical scheme is as follows:

in a first aspect, an embodiment of the present application provides a multi-screen interaction method, which is applied to a screen expansion system, where the screen expansion system includes a main touch device, a secondary touch screen, a video matrix, and a display peripheral, where the main touch device includes a main touch screen and an open pluggable specification module, the video matrix includes at least a first input port, a second input port, a third input port, a first output port, and a second output port, an output port of the main touch screen is connected to the first input port of the video matrix, an output port of the open pluggable specification module is connected to the second input port of the video matrix, an input port of the main touch screen is connected to the first output port of the video matrix, an input port of the secondary touch screen is connected to the second output port of the video matrix, an output port of the display peripheral is connected to the third input port of the video matrix, and the main touch device is in communication connection with the secondary touch screen, where the method includes:

inputting the picture of the display peripheral to the video matrix through a third input port of the video matrix, and then projecting the picture to the main touch screen through a first output port of the video matrix;

determining interface coordinate information of the main touch screen based on a first input port of the video matrix;

starting a first signal blocking layer based on the interface coordinate information of the main touch screen;

and/or the presence of a gas in the gas,

inputting the picture of the display peripheral to the video matrix through a third input port of the video matrix, and then projecting the picture to the auxiliary touch screen through a second output port of the video matrix;

determining interface coordinate information of the secondary touch screen based on a second input port of the video matrix;

and starting a second signal blocking layer based on the interface coordinate information of the auxiliary touch screen.

In a second aspect, an embodiment of the present application provides a multi-screen interaction apparatus, which is applied to a screen expansion system, the screen expansion system includes a main touch device, a sub-touch screen, a video matrix and a display peripheral, wherein the main touch device includes a main touch screen and an open pluggable standard module, the video matrix includes at least a first input port, a second input port, a third input port, a first output port and a second output port, an output port of the main touch screen is connected to the first input port of the video matrix, an output port of the open pluggable standard module is connected to the second input port of the video matrix, an input port of the main touch screen is connected to the first output port of the video matrix, an input port of the sub-touch screen is connected to the second output port of the video matrix, an output port of the display peripheral is connected to the third input port of the video matrix, and the main touch device is in communication connection with the sub-touch screen, the apparatus includes:

the first picture screen projection module is used for inputting pictures of the display peripheral equipment into the video matrix through a third input port of the video matrix and projecting the pictures to the main touch screen through a first output port of the video matrix;

the main screen coordinate determination module is used for determining interface coordinate information of the main touch screen based on a first input port of the video matrix;

the first blocking layer starting module is used for starting a first signal blocking layer based on the interface coordinate information of the main touch screen;

and/or the presence of a gas in the gas,

the second picture screen projection module is used for inputting the picture of the display peripheral equipment into the video matrix through a third input port of the video matrix and projecting the picture to the auxiliary touch screen through a second output port of the video matrix;

the auxiliary screen coordinate determination module is used for determining interface coordinate information of the auxiliary touch screen based on a second input port of the video matrix;

and the second blocking layer starting module is used for starting a second signal blocking layer based on the interface coordinate information of the auxiliary touch screen.

In a third aspect, an embodiment of the present application provides a computer storage medium storing a plurality of instructions adapted to be loaded by a processor and to perform the method steps of the first aspect.

In a fourth aspect, an embodiment of the present application provides an electronic device, which may include: a processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and to perform the method steps of the first aspect described above.

The beneficial effects brought by the technical scheme provided by some embodiments of the application at least comprise:

in the embodiment of the application, when a picture of a display peripheral is projected to a main touch screen in a screen expansion system independently, or projected to an auxiliary touch screen independently, or projected to the main touch screen and the auxiliary touch screen, interface coordinate information corresponding to an input port where a screen of the projected screen is connected with a video matrix is determined, a signal blocking layer is started based on the interface coordinate information of the screen of the projected screen, and at this time, if a touch operation is performed on the screen of the projected screen, the signal blocking layer started on the screen of the projected screen blocks a touch signal generated by the touch operation, so that a first picture of the screen does not respond to the touch operation, and the problem of touch penetration is avoided.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

Fig. 1 is a schematic diagram of an architecture of a screen expansion system according to an embodiment of the present application;

fig. 2 is a schematic flowchart of a multi-screen interaction method according to an embodiment of the present disclosure;

fig. 3 is a flowchart illustrating a multi-screen interaction method according to an embodiment of the present disclosure;

fig. 4 is a schematic flowchart of a multi-screen interaction method according to an embodiment of the present disclosure;

fig. 5 is a schematic structural diagram of a screen expansion system provided in an embodiment of the present application;

fig. 6 is a schematic flowchart of a multi-screen interaction method according to an embodiment of the present disclosure;

fig. 7 is a schematic flowchart of a multi-screen interaction method according to an embodiment of the present disclosure;

fig. 8 is a schematic flowchart of a multi-screen interaction method according to an embodiment of the present disclosure;

fig. 9 is a schematic flowchart of a multi-screen interaction method according to an embodiment of the present disclosure;

fig. 10 is a schematic flowchart illustrating a multi-screen interaction method according to an embodiment of the present disclosure;

FIG. 11 is a schematic structural diagram of a multi-screen interaction device according to an embodiment of the disclosure;

FIG. 12 is a schematic structural diagram of a multi-screen interaction device according to an embodiment of the disclosure;

FIG. 13 is a schematic structural diagram of a multi-screen interaction device according to an embodiment of the disclosure;

FIG. 14 is a schematic structural diagram of a multi-screen interaction device according to an embodiment of the disclosure;

FIG. 15 is a schematic structural diagram of a multi-screen interaction device according to an embodiment of the disclosure;

FIG. 16 is a schematic structural diagram of a home screen coordinate determination module according to an embodiment of the present disclosure;

fig. 17 is a schematic structural diagram of a sub-screen coordinate determination module according to an embodiment of the present application;

FIG. 18 is a schematic structural diagram of a multi-screen interaction device according to an embodiment of the disclosure;

FIG. 19 is a schematic structural diagram of a multi-screen interaction device according to an embodiment of the present disclosure;

fig. 20 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more clear, embodiments of the present application will be described in further detail below with reference to the accompanying drawings.

When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the application, as detailed in the appended claims.

In the description of the present application, it is to be understood that the terms "first," "second," and the like are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. The specific meaning of the above terms in this application will be understood to be a specific case for those of ordinary skill in the art. In addition, in the description of the present application, "a plurality" means two or more unless otherwise specified. "and/or" describes the association relationship 6 system of the associated object, indicating that there may be three relationships, e.g., a and/or B, which may indicate: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship.

The present application will be described in detail with reference to specific examples.

The method can be realized by relying on a computer program and can run on a multi-screen interaction device based on a Von Neumann system. The computer program may be integrated into the application or may run as a separate tool-like application. The multi-screen interaction device in the embodiment of the present application may be a mobile terminal, including but not limited to: smart interactive tablets, personal computers, tablet computers, handheld devices, in-vehicle devices, wearable devices, computing devices or other processing devices connected to a wireless modem, and the like. The user terminals may be called different names in different networks, for example: user equipment, access terminal, subscriber unit, subscriber station, mobile station, remote terminal, mobile device, user terminal, wireless communication device, user agent or user equipment, cellular telephone, cordless telephone, personal Digital Assistant (PDA), terminal equipment in a 5G network or future evolution network, and the like.

As shown IN fig. 1, the multi-screen interaction method is applied to a screen expansion system, the screen expansion system includes a main touch device, a sub touch screen, a video matrix and a display peripheral, wherein the main touch device includes a main touch screen and an OPS module, and the video matrix includes at least a first input port IN1, a second input port IN2, a third input port IN3, a first output port OUT1 and a second output port OUT2.

The connection among the main touch device, the sub touch screen, the Video matrix, and the display peripheral may be connected by a connection manner including, but not limited to, a High Definition Multimedia Interface (HDMI), a Separate Video Interface (S-Video), and the like.

The communication connection between the main touch screen and the auxiliary touch screen can be performed in a wireless or wired manner, wherein the wireless manner includes but is not limited to a cellular network, a wireless local area network, an infrared network, a near field communication network or a bluetooth network, and the wired manner includes but is not limited to USB and the like.

As shown IN fig. 1, an achievable connection manner may be that the output port of the main touch screen is connected to the first input port IN1 through an HDMI data line, the input port of the main touch screen is connected to the first output port OUT1 through an HDMI data line, the output port of the OPS module is connected to the second input port IN2 of the video matrix through an HDMI data line, the second output port OUT2 of the video matrix is connected to the auxiliary touch screen through an HDMI data line, the output port of the display peripheral is connected to the third input port IN3 through an HDMI data line, and the main touch screen is connected to the auxiliary touch screen through a USB communication.

The main touch device refers to a device containing an OPS module in the video expansion system.

The OPS module is a computing module in a plug-in format and can add computing power to the main touch device.

The OPS module may run on a Central Processing Unit (CPU) based on Intel and ARM (Advanced RISC Machines, ARM), and the running operating systems include Microsoft Windows and Google Android.

The main touch screen refers to a physical screen for display in the main touch device.

The sub touch screen refers to a physical screen available for display of the projection screen in addition to the main touch screen in the screen expansion system.

The video matrix refers to an electronic device which outputs m paths of video signals to n paths of monitoring equipment at will by an array switching method, wherein m and n can be the same or different, and the video matrix can be used for transmitting any path of signals of equipment providing a signal source to any path of display terminal.

The first input port IN1, the second input port IN2, and the third input port IN3 refer to ports for receiving input signals IN the video matrix, and the first output port OUT1 and the second output port OUT2 refer to ports for signal output IN the video matrix.

The picture of the display peripheral can be independently projected to the main touch screen, independently projected to the auxiliary touch screen, and simultaneously projected to the main touch screen and the auxiliary touch screen.

S101 to S103 are described with reference to the system shown in fig. 1, taking as an example that a screen displaying peripheral devices is projected onto a main touch panel alone. As shown in fig. 2, a flow diagram of a multi-screen interaction method is provided for an embodiment of the present application, where the multi-screen interaction method may include:

s101, inputting the picture of the display peripheral to the video matrix through a third input port of the video matrix, and then projecting the picture to a main touch screen through a first output port of the video matrix;

specifically, the video matrix refers to an electronic device that outputs m channels of video signals to n channels of monitoring equipment at will by an array switching method, where m and n may be the same or different. IN this embodiment, a video matrix for outputting 3 channels of video signals to 2 channels of devices is taken as an example, and a corresponding relationship between an input port and an output port is preset IN the video matrix, and the corresponding relationship is not limited.

The method comprises the steps that a picture for displaying the peripheral equipment is input to a video matrix through a third input port IN3 of the video matrix and then projected to a main touch screen through a first output port OUT1 of the video matrix, specifically, a video signal for displaying the peripheral equipment is input to the video matrix from the third input port IN3 through an HDMI data line, and then the picture for displaying the peripheral equipment is output to the main touch screen through the connection of the first output port OUT1 of the video matrix and the main touch screen through the HDMI data line, so that the picture for displaying the peripheral equipment is projected to the main touch screen.

Before the display peripheral does not perform screen projection, the picture displayed on the screen projected by the display peripheral is a first picture, and after the display peripheral is connected with the video matrix, the picture displayed on the screen projected by the display peripheral is a second picture.

At the moment, before the display peripheral does not perform screen projection, the display peripheral takes a picture displayed on the main touch screen of the screen projection as a first picture, and the display peripheral takes a picture displayed on the main touch screen of the screen projection as a second picture. The method comprises the steps that a picture for displaying peripheral equipment is input into a video matrix through a third input port IN3 of the video matrix, and then the picture is projected to a main touch screen through a first output port OUT1 of the video matrix, a second picture for replacing the first picture is displayed on a screen of the main touch screen, at the moment, prompt information is output by the main touch screen and used for prompting a user that a first signal blocking layer is started, and if touch operation is carried OUT on the main touch screen, the originally displayed picture of the main touch screen cannot respond to a touch signal generated by the touch operation.

S102, determining interface coordinate information of the main touch screen based on a first input port of the video matrix;

specifically, when a picture displayed by the peripheral device is input to the video matrix through the third input port IN3 of the video matrix and then is projected to the main touch screen through the first output port OUT1 of the video matrix, the input port corresponding to the first output port OUT1 is determined IN the video matrix, and because the port correspondence relationship IN the video matrix is that the first input port IN1 corresponds to the first output port OUT1, the interface coordinate information of the main touch screen is determined based on the first input port IN 1.

The interface coordinate information of the main touch screen refers to X and Y coordinates on the display interface of the main touch screen, and the X and Y coordinates are respectively horizontal and vertical addresses of any pixel or fixed point on the display interface of the main touch screen.

The interface coordinate information of the main touch screen is determined based on the first input port IN1, and the interface coordinate information of the main touch screen may be determined based on the device information of the main touch screen for acquiring the device information of the main touch screen connected to the first input port IN 1.

The device information of the main touch screen can be at least one of device parameters such as a header file, a product serial number, extended display identification data, basic display parameters, feature function support data, preset signal timing, a display name descriptor and the like of the main touch screen, and because the device parameters such as the header file, the product serial number, the extended display identification data, the basic display parameters, the feature function support data, the preset signal timing, the display name descriptor and the like of the main touch screen are unique to the main touch screen, whether a picture screen of a display peripheral is a main touch screen can be determined by acquiring at least one of the device parameters, and when the picture screen of the display peripheral is the main touch screen, the interface coordinate information of the main touch screen is determined.

For example, the acquired device information of the main touch screen is preset as the extended display identification data of the main touch screen, the extended display identification data of the main touch screen is preset as the PC Monitor, and if the acquired and recognized extended display identification data is the PC Monitor, the interface coordinate information of the main touch screen is determined.

S103, starting a first signal blocking layer based on the interface coordinate information of the main touch screen;

the first signal blocking layer is a virtual layer which can block touch signals generated by touch operation on the main touch screen.

The starting of the first signal blocking layer is specifically that after an OPS module in the main touch device determines interface coordinate information of the main touch screen, the OPS module generates a virtual layer which is arranged on the main touch screen in a corresponding coordinate range of the main touch screen.

Before a picture of a display peripheral device is projected to a main touch screen, a first picture of the main touch screen is input to a video matrix through a first input port IN1, then the first picture is transmitted to the main touch screen through a first output port OUT1 of the video matrix, the display is carried OUT on the main touch screen, after a first signal blocking layer is started based on interface coordinate information of the main touch screen, if touch operation is carried OUT on the main touch screen at the moment, the first signal blocking layer blocks the touch operation aiming at the main touch screen, the first picture of the main touch screen cannot receive touch signals generated by the touch operation, and then the first picture of the main touch screen cannot respond to the touch operation.

As shown in fig. 3, a flowchart of a multi-screen interaction method is provided in the embodiment of the present application, and S104-S106 are described with reference to the system shown in fig. 1 by taking an example of displaying a single screen of a peripheral device to a secondary touch screen.

As shown in fig. 4, a flowchart of a multi-screen interaction method is provided for the embodiment of the present application, and when a picture displayed on an external device is simultaneously projected to a primary touch screen and a secondary touch screen, the following steps S104 to S106 are also performed in addition to the steps S101 to S103.

S104, inputting the peripheral display picture into the video matrix through a third input port of the video matrix, and then projecting the peripheral display picture to the auxiliary touch screen through a second output port of the video matrix;

At the moment, before the display peripheral does not project the screen, the display peripheral takes a picture displayed on the secondary touch screen as a first picture, and after the display peripheral is connected with the video matrix, the picture displayed on the secondary touch screen is a second picture. The method comprises the steps that a picture for displaying the peripheral equipment is input into a video matrix through a third input port IN3 of the video matrix and then projected to a secondary touch screen through a second output port OUT2 of the video matrix, specifically, a video signal for displaying the peripheral equipment is input into the video matrix from the third input port IN3 of the video matrix through an HDMI data line, and then the second output port OUT2 of the video matrix is connected with the secondary touch screen through an HDMI data line and output to the secondary touch screen, so that the picture for displaying the peripheral equipment is projected to the secondary touch screen.

And inputting a picture for displaying the peripheral equipment into the video matrix through a third input port IN3 of the video matrix, and after the picture is projected to the auxiliary touch screen through a second output port OUT2 of the video matrix, outputting prompt information by the auxiliary touch screen, wherein the prompt information is used for prompting a user that a second signal blocking layer is started, and if the auxiliary touch screen is subjected to touch operation, a first picture of the auxiliary touch screen cannot respond to a touch signal generated by the touch operation.

S105, determining a second logic screen of the secondary touch screen based on the second input port;

specifically, when a picture displayed by the peripheral device is input to the video matrix through the third input port IN3 of the video matrix and then projected to the secondary touch screen through the second output port OUT2 of the video matrix, the input port corresponding to the second output port OUT2 of the video matrix is determined IN the video matrix, and because the port correspondence relationship IN the video matrix is that the second input port IN2 of the video matrix corresponds to the second output port OUT2 of the video matrix, the interface coordinate information of the secondary touch screen is determined based on the second input port IN2 of the video matrix.

The interface coordinate information of the auxiliary touch screen refers to coordinates of X and Y on the display interface of the auxiliary touch screen, and the X and Y coordinates are respectively the horizontal and vertical addresses of any pixel or fixed point on the display interface of the auxiliary touch screen.

When the display interface coordinate range of the main touch screen is X =0-100, Y =0-100, the display interface logical coordinate range of the auxiliary touch screen with the same size is X =100-200, and Y is 0-100.

Determining the interface coordinate information of the auxiliary touch screen based on the second input port IN2 of the video matrix, namely acquiring the device information of the auxiliary touch screen connected with the second input port IN2, and determining the interface coordinate information of the auxiliary touch screen based on the device information of the auxiliary touch screen.

The device information of the secondary touch screen can be at least one of device parameters such as a header file, a product serial number, extended display identification data, basic display parameters, feature function support data, preset signal timing, a display name descriptor and the like of the secondary touch screen, and because the device parameters such as the header file, the product serial number, the extended display identification data, the basic display parameters, the feature function support data, the preset signal timing, the display name descriptor and the like of the secondary touch screen are unique to the secondary touch screen, whether the picture projection screen displaying the peripheral is the secondary touch screen can be determined by acquiring at least one of the information in the parameters of the secondary touch screen, and when the picture projection screen displaying the peripheral is the secondary touch screen, the interface coordinate information corresponding to the secondary touch screen is determined.

For example, the acquired device information of the secondary touch screen is preset to be extended display identification data of the secondary touch screen, the extended display identification data of the secondary touch screen is preset to be VPO1B, and if the extended display identification data is acquired and identified to be VPO1B, the interface coordinate information of the secondary touch screen is determined.

And S106, starting a second signal blocking layer based on the interface coordinate information of the auxiliary touch screen.

The second signal blocking layer is a virtual layer which can block touch signals generated by touch operation on the auxiliary touch screen.

The starting of the second signal blocking layer is specifically that after the OPS module in the main touch device determines the interface coordinate information of the auxiliary touch screen, the OPS module generates a virtual layer which is arranged on the auxiliary touch screen in a corresponding logical coordinate range of the auxiliary touch screen.

Before a picture displaying the peripheral device is projected to the auxiliary touch screen, a first picture of the auxiliary touch screen is input to the video matrix through a second input port IN2 of the video matrix and then is transmitted to the auxiliary touch screen through a second output port OUT2 of the video matrix, so that the first picture is displayed on the auxiliary touch screen, after the picture displaying the peripheral device is projected to the auxiliary touch screen, a second signal blocking layer is started based on interface coordinate information of the auxiliary touch screen, at the moment, if touch operation is carried OUT on the auxiliary touch screen, the second signal blocking layer blocks first picture touch operation aiming at the auxiliary touch screen, the first picture of the auxiliary touch screen cannot receive touch signals generated by the touch operation, and the first picture of the auxiliary touch screen cannot respond to the touch operation.

As shown in fig. 5, the screen expansion system further includes a touch control screen, the video matrix further includes a third output port OUT3, wherein an input port of the touch control screen is connected to the third output port OUT3 of the video matrix through an HDMI data line, and the touch control screen is connected to the main touch device through USB communication.

The communication connection between the main touch screen and the touch control screen may be wireless or wired, where the wireless communication mode includes, but is not limited to, a cellular network, a wireless local area network, an infrared network, a near field communication network, or a bluetooth network, and the wired communication mode includes, but is not limited to, a USB, and the like. When the main touch screen is in communication connection with the touch control screen, the touch control screen and the auxiliary touch screen can be in indirect communication connection based on the communication connection of the main touch screen and the auxiliary touch screen.

Specifically, the video matrix refers to an electronic device that outputs m paths of video signals to n paths of monitoring equipment at will by an array switching method, the following embodiment takes a video matrix that outputs 3 paths of video signals to 3 paths of equipment as an example, a corresponding relationship between an input port and an output port is preset IN the video matrix, the corresponding relationship between the input port and the output port IN the video matrix is not limited, and the following embodiment takes a case that a first input port IN1 corresponds to a first output port OUT1, and a second input port IN2 corresponds to a second output port OUT2 as an example for explanation.

Please refer to fig. 6, which is a flowchart illustrating a multi-screen interaction method according to an embodiment of the present disclosure.

When the touch control screen follows the main touch screen and projects the picture of the display peripheral to the main touch screen, the method comprises the following steps:

s201, inputting the peripheral display picture into the video matrix through a third input port of the video matrix, then projecting the picture to the main touch screen through a first output port of the video matrix, and projecting the picture to the touch control screen through a third output port of the video matrix;

the process of inputting a picture for displaying peripheral equipment into the video matrix through the third input port IN3 of the video matrix and then projecting the picture to the main touch screen through the first output port OUT1 of the video matrix is detailed IN S101, and details are not repeated here.

The screen projection of the picture for displaying the peripheral to the touch control screen through the third output port IN3 of the video matrix means that when the touch control screen follows the main touch screen, the first picture originally displayed on the main touch screen is replaced by the second picture, the touch control screen follows the main touch screen, and the picture on the touch control screen is also replaced by the second picture.

Specifically, the touch control screen follows the main touch screen means that the main touch screen can be controlled by the touch control screen because the main touch screen is in communication connection with the touch control screen, when the main touch screen is controlled by the touch control screen, if touch operation is performed on the touch control screen, a picture in the main touch screen will respond to a touch signal generated by the touch operation, and a response result is displayed on the touch control screen.

Because the display peripheral does not have communication connection with the touch control screen, the touch control screen can not change along with the main touch screen because the display peripheral is projected to the main touch screen and the touch control screen.

S202, determining interface coordinate information of the main touch screen based on a first input port of the video matrix;

the process of determining the interface coordinate information of the main touch screen based on the first input port IN1 of the video matrix is detailed IN S102, and is not described herein again.

S203, starting a first signal blocking layer based on the interface coordinate information of the main touch screen.

The process of starting the first signal blocking layer based on the interface coordinate information of the main touch screen is detailed in S103, and is not described herein again.

In the embodiment of the application, when a picture for displaying an external device is input to a video matrix through a third input port of the video matrix, and then is projected to a main touch screen through a first output port of the video matrix, and is projected to a touch control screen through a third output port of the video matrix, interface coordinate information of the main touch screen is determined based on the first input port of the video matrix, a first signal blocking layer is started based on the interface coordinate information of the main touch screen, if a touch operation is performed on the main touch screen at this time, the started first signal blocking layer blocks the touch operation aiming at the main touch screen, and an original picture of the main touch screen does not respond to the touch operation, so that the problem of touch penetration on the main touch screen is avoided.

Please refer to fig. 7, which is a flowchart illustrating a multi-screen interaction method according to an embodiment of the present disclosure.

When the touch control screen follows the auxiliary touch screen and the picture of the display peripheral device is projected to the main touch screen, the method comprises the following steps:

s301, inputting the peripheral display picture into the video matrix through a third input port of the video matrix, and then projecting the peripheral display picture to a main touch screen through a first output port of the video matrix;

inputting the peripheral display frame to the video matrix through a third input port IN3 of the video matrix, and then projecting the peripheral display frame to the main touch screen through a first output port OUT1 of the video matrix, for details, see S101, which is not described herein again.

Specifically, the touch control screen follows the auxiliary touch screen, that is, the auxiliary touch screen is in indirect communication connection with the touch control screen, so that the touch control screen can control the auxiliary touch screen.

Because the display peripheral does not have communication connection with the touch control screen, the touch control screen can not change along with the auxiliary touch screen because the display peripheral is projected to the main touch screen and the touch control screen.

S302, determining interface coordinate information of the main touch screen based on a first input port of the video matrix;

the method comprises the steps that a picture for displaying peripheral equipment is input into a video matrix through a third input port IN3 of the video matrix, then the picture is projected to a main touch screen through a first output port OUT1 of the video matrix, touch operation is carried OUT on the main touch screen, a touch signal generated by the touch operation is sent to a first picture of the main touch screen, the first picture of the main touch screen responds to the touch signal, the problem of touch penetration still occurs IN the main touch screen, and therefore interface coordinate information of the main touch screen is determined based on the first input port IN1 of the video matrix.

And S303, starting a first signal blocking layer based on the interface coordinate information of the main touch screen.

After the first signal blocking layer is started based on the interface coordinate information of the main touch screen, the method further comprises the following steps:

s304, when the picture of the display peripheral equipment stops inputting or the screen expansion system is closed, synchronously removing the first signal blocking layer.

The picture stop input of the display peripheral may be removing the connection relation between the display peripheral and the video matrix, for example, removing a data line between the display peripheral and the video matrix.

And synchronously removing the first signal blocking layer when the picture of the display peripheral stops inputting, and if touch operation is performed on the main touch screen at the moment, responding to the touch signal generated by the touch operation by the picture of the main touch screen.

The screen expansion system closing can be the power-off of the screen expansion system or the removal of the connection relationship between the main touch device and the video matrix.

And when the screen expansion system is closed, the first signal blocking layer is synchronously removed, and when the screen expansion system is powered on when no display peripheral equipment is arranged, or when the main touch equipment is continuously used after the connection relation between the main touch equipment and the video matrix is removed, if touch operation is carried out on the main touch screen, the touch signal generated by the touch operation is responded by the picture of the main touch screen.

In the embodiment of the application, when the touch control screen follows the auxiliary touch screen and projects a picture displaying an external device to the main touch screen, the interface coordinate information of the main touch screen is determined based on the first input port of the video matrix, the first signal blocking layer is started based on the interface coordinate information of the main touch screen, at this time, if the main touch screen is touched, the started first signal blocking layer blocks the touch operation aiming at the main touch screen, and the first picture originally displayed on the main touch screen does not respond to the touch operation, so that the problem of touch penetration on the main touch screen is avoided.

As shown in fig. 5, the screen expansion system further includes a touch control panel, wherein an input port of the touch control panel is connected to the third output port OUT3 of the video matrix through an HDMI data line, the touch control panel is in communication with the sub touch panel, and when the touch control panel follows the sub touch panel, the screen of the display peripheral device is projected onto the sub touch panel.

Specifically, as shown in fig. 5, the touch control screen is in USB communication with the main touch device, and the main touch device is in USB communication with the sub touch screen, so that the sub touch screen is in indirect communication with the touch control screen.

Referring to fig. 8, a flowchart of a multi-screen interaction method according to an embodiment of the present application is shown, where the method further includes:

s401, inputting the picture of the display peripheral into the video matrix through a third input port of the video matrix, then projecting the picture to the auxiliary touch screen through a second output port of the video matrix, and projecting the picture to the touch control screen through a third output port of the video matrix;

the process of inputting the peripheral display screen to the video matrix through the third input port IN3 of the video matrix and then projecting the peripheral display screen to the secondary touch screen through the second output port OUT2 of the video matrix is detailed IN S104, and is not described herein again.

Specifically, the touch control screen follows the auxiliary touch screen means that the auxiliary touch screen can be controlled by the touch control screen because the auxiliary touch screen is in communication connection with the touch control screen, when the auxiliary touch screen is controlled by the touch control screen, if touch operation is performed on the touch control screen, a picture in the auxiliary touch screen responds to a touch signal generated by the touch operation, and a response result is displayed on the touch operation screen.

Because the display peripheral and the touch control screen are not in communication connection, the touch control screen and the auxiliary touch screen cannot be changed due to the fact that the display peripheral is projected to the auxiliary touch screen and the touch control screen.

S402, determining interface coordinate information of the auxiliary touch screen based on a second input port of the video matrix;

the process of determining the interface coordinate information of the secondary touch screen based on the second input port of the video matrix is detailed in S105, and is not described herein again.

And S403, starting a second signal blocking layer on the basis of the interface coordinate information of the auxiliary touch screen.

The process of starting the second signal blocking layer based on the interface coordinate information of the secondary touch screen is detailed in S105, and is not described herein again.

In the embodiment of the application, when a picture for displaying an external device is input to the video matrix through the third input port of the video matrix, and then is projected to the secondary touch screen through the second output port of the video matrix, and is projected to the touch control screen through the third output port of the video matrix, the interface coordinate information of the secondary touch screen is determined based on the second input port of the video matrix, the second signal blocking layer is started based on the interface coordinate information of the secondary touch screen, if the secondary touch screen is touched, the started second signal blocking layer blocks the touch operation on the secondary touch screen, and the original picture of the secondary touch screen does not respond to the touch operation, so that the problem of touch penetration on the secondary touch screen is avoided.

Please refer to fig. 9, which is a flowchart illustrating a multi-screen interaction method according to an embodiment of the present disclosure. When the touch control screen follows the main touch screen and the picture of the display peripheral device is projected to the auxiliary touch screen, the method comprises the following steps:

s501, inputting the peripheral display picture into the video matrix through a third input port of the video matrix, and then projecting the peripheral display picture to the auxiliary touch screen through a second output port of the video matrix;

Specifically, the touch control screen follows the main touch screen, and the main touch screen is in communication connection with the touch control screen, so that the touch control screen can control the main touch screen.

Because the display peripheral does not have communication connection with the touch control screen, the touch control screen can not change along with the main touch screen because the display peripheral is projected to the auxiliary touch screen and the touch control screen.

S502, determining interface coordinate information of the auxiliary touch screen based on a second input port of the video matrix;

the process of determining the interface coordinate information of the secondary touch screen based on the second input port IN2 of the video matrix is detailed IN S105, and is not described herein again.

And S503, starting a second signal blocking layer based on the interface coordinate information of the auxiliary touch screen.

After the second signal blocking layer is started based on the interface coordinate information of the auxiliary touch screen, the method further comprises the following steps:

s504, when the picture of the display peripheral equipment stops inputting or the screen expansion system is closed, the second signal blocking layer is synchronously removed.

When the picture of the display peripheral device stops inputting, the connection relation between the display peripheral device and the video matrix can be removed, for example, a data line between the display peripheral device and the video matrix is removed.

The screen expansion system closing can be the power failure of the screen expansion system or the removal of the connection relationship between the auxiliary touch screen and the video matrix.

And when the screen expansion system is closed, the second signal blocking layer is synchronously removed, and when the screen expansion system is powered on when no display peripheral equipment is arranged or the auxiliary touch screen is continuously used after the connection relation between the auxiliary touch screen and the video matrix is removed, if touch operation is carried out on the auxiliary touch screen, the touch signal generated by the touch operation is responded by the picture of the auxiliary touch screen.

In the embodiment of the application, when the touch control screen follows the main touch screen and projects a picture displaying peripheral equipment to the auxiliary touch screen, the interface coordinate information of the auxiliary touch screen is determined based on the second input port of the video matrix, and the second signal blocking layer is started based on the interface coordinate information of the auxiliary touch screen.

Please refer to fig. 10, which is a flowchart illustrating a multi-screen interaction method according to an embodiment of the present disclosure. The method comprises the following steps:

s601, projecting the picture of the display peripheral to at least one screen of a main touch screen or an auxiliary touch screen through a video matrix;

specifically, when the screen expansion system further includes a touch control screen, the screen of the display peripheral may be further projected to the touch control screen, and the manner of displaying the screen of the display peripheral to the touch control screen includes the following conditions.

When the touch control screen follows the main touch screen and projects the picture of the display peripheral to the main touch screen, the picture of the display peripheral is input to the video matrix through a third input port IN3 of the video matrix, projected to the main touch screen through a first output port OUT1 of the video matrix and projected to the touch control screen through a third output port OUT3 of the video matrix. When the touch control screen follows the auxiliary touch screen and displays an external picture to be projected to the main touch screen, the external picture is input to the video matrix through a third input port IN3 of the video matrix and is projected to the main touch screen through a first output port OUT1 of the video matrix. When the touch control screen follows the auxiliary touch screen and displays an external picture to be projected to the auxiliary touch screen, the external picture to be displayed is input to the video matrix through a third input port IN3 of the video matrix, is projected to the auxiliary touch screen through a second output port OUT2 of the video matrix and is projected to the touch control screen through a third output port OUT3 of the video matrix. When the touch control screen follows the main touch screen and displays the peripheral picture to be projected to the auxiliary touch screen, the peripheral picture is input to the video matrix through a third input port IN3 of the video matrix and is projected to the auxiliary touch screen through a second output port OUT2 of the video matrix.

When the picture displayed by the peripheral device is input to the video matrix through the third input port IN3 of the video matrix, and then is projected to the main touch screen through the first output port OUT1 of the video matrix, and then is projected to the touch control screen through the third output port OUT3 of the video matrix, or when the picture displayed by the peripheral device is input to the video matrix through the third input port IN3 of the video matrix, and then is projected to the main touch screen through the first output port OUT1 of the video matrix, S602 is executed.

S602, closing a first touch signal receiving function and a first touch signal response function of the main touch screen;

the first touch signal receiving function refers to a function of receiving a touch signal generated by a touch operation in the main touch device if the main touch screen is subjected to the touch operation.

The first touch signal response function refers to a function of responding to a touch signal generated by performing touch operation on the main touch screen.

Specifically, the extended display identification data of the main touch screen is preset as a PC Monitor, and if the extended display identification data is obtained and identified as the PC Monitor, the first touch signal receiving function and the first touch signal response function of the main touch screen are turned off.

When the picture displayed by the peripheral device is input to the video matrix through the third input port IN3 of the video matrix, and then is projected to the auxiliary touch screen through the second output port OUT2 of the video matrix, and is input to the video matrix through the third input port IN3 of the video matrix, and then is projected to the touch control screen through the third output port OUT3 of the video matrix, or when the picture displayed by the peripheral device is input to the video matrix through the third input port IN3 of the video matrix, and then is projected to the auxiliary touch screen through the second output port OUT2 of the video matrix, S603 is executed.

S603, closing a second touch signal receiving function and a second touch signal response function of the auxiliary touch screen;

the second touch signal receiving function means a function of receiving a touch signal generated by a touch operation if the touch operation is performed on the sub touch screen.

The second touch signal response function refers to a function of responding to a touch signal generated by performing touch operation on the auxiliary touch screen.

Specifically, the extended display identification data of the secondary touch screen is preset to be VPO1B, and if the extended display identification data is identified to be VPO1B, the second touch signal receiving function and the second touch signal response function of the secondary touch screen are turned off.

When the screen of the peripheral device is displayed and simultaneously projected to the main touch panel and the sub touch panel, S604 is performed.

S604, closing the touch signal receiving and touch signal responding functions of the main touch screen, and closing the touch signal receiving and touch signal responding functions of the auxiliary touch screen.

The function of closing the touch signal reception and touch signal response of the primary touch screen is detailed in S602, and the function of closing the touch signal reception and touch signal response of the secondary touch screen is detailed in S603, which is not described herein again.

In the embodiment of the application, when a picture for displaying peripheral equipment is independently projected to the main touch screen through the video matrix, the touch signal receiving and touch signal responding functions of the main touch screen are closed; when a picture of the display peripheral is projected to the auxiliary touch screen through the video matrix, the touch signal receiving and touch signal responding functions of the auxiliary touch screen are closed; when a picture of the display peripheral is projected to the main touch screen and the auxiliary touch screen through the video matrix, closing the touch signal receiving and touch signal responding functions of the main touch screen and closing the touch signal receiving and touch signal responding functions of the auxiliary touch screen; when the touch signal receiving and touch signal responding functions of the screen projected by the display peripheral are closed, the screen projected by the display peripheral cannot receive and respond to the touch signals generated by any touch operation, and therefore the problem of touch penetration can be avoided.

The following are embodiments of the apparatus of the present application that may be used to perform embodiments of the method of the present application. For details which are not disclosed in the embodiments of the apparatus of the present application, reference is made to the embodiments of the method of the present application. The multi-screen interaction device can be realized by software, hardware or a combination of the software and the hardware to be all or part of the terminal. The multi-screen interaction device is applied to a screen expansion system, the screen expansion system comprises a main touch device, an auxiliary touch screen, a video matrix and a display peripheral, wherein the main touch device comprises a main touch screen and an open type pluggable standard module, the video matrix at least comprises a first input port, a second input port, a third input port, a first output port and a second output port, the output port of the main touch screen is connected with the first input port of the video matrix, the output port of the open type pluggable standard module is connected with the second input port of the video matrix, the input port of the main touch screen is connected with the first output port of the video matrix, the input port of the auxiliary touch screen is connected with the second output port of the video matrix, the output port of the display peripheral is connected with the third input port of the video matrix, and the main touch device is in communication connection with the auxiliary touch screen.

The device 1 comprises a first picture projection module 11, a main screen coordinate determination module 12 and a first blocking layer starting module 13 and/or comprises a second picture projection module 14, a secondary screen coordinate determination module 15 and a second blocking layer starting module 16.

Referring to fig. 11, a schematic structural diagram of a multi-screen interaction device according to an exemplary embodiment of the present application is shown.

The first picture screen projection module 11 is configured to input a picture of the display peripheral to the video matrix through a third input port of the video matrix, and project a screen to the main touch screen through a first output port of the video matrix;

a main screen coordinate determination module 12, configured to determine interface coordinate information of the main touch screen based on a first input port of the video matrix;

and a first blocking layer starting module 13, configured to start a first signal blocking layer based on the interface coordinate information of the main touch screen.

Referring to fig. 12, a schematic structural diagram of a multi-screen interaction device according to an exemplary embodiment of the present application is shown.

The second picture screen projection module 14 is configured to input a picture of the display peripheral to the video matrix through a third input port of the video matrix, and project a screen to the auxiliary touch screen through a second output port of the video matrix;

the auxiliary screen coordinate determination module 15 is configured to determine interface coordinate information of the auxiliary touch screen based on a second input port of the video matrix;

and a second blocking layer starting module 16, configured to start a second signal blocking layer to start the second signal blocking layer based on the interface coordinate information of the secondary touch screen.

Referring to fig. 13, a schematic structural diagram of a multi-screen interaction device according to an exemplary embodiment of the present application is shown.

a first blocking layer starting module 13, configured to start a first signal blocking layer based on the interface coordinate information of the main touch screen;

the auxiliary screen coordinate determination module 15 is configured to determine, based on the second input port of the video matrix, interface coordinate information of the auxiliary touch screen to determine a second logical screen of the auxiliary touch screen;

and a second blocking layer starting module 16, configured to start a second signal blocking layer based on the interface coordinate information of the secondary touch screen.

Optionally, as shown in fig. 14, the screen expansion system further includes a touch control screen, the video matrix further includes a third output port, an input port of the touch control screen is connected to the third output port, the touch control screen is in communication connection with the main touch device, and when the touch control screen follows the main touch screen, the apparatus includes:

a third screen projection module 17, configured to input a screen of the display peripheral to the video matrix through a third input port of the video matrix, project the screen to the main touch screen through a first output port of the video matrix, and project the screen to the touch control screen through a third output port of the video matrix;

Optionally, as shown in fig. 14, when the touch control panel follows the secondary touch screen, the apparatus includes:

the third picture projection module 17 is further configured to input a picture of the display peripheral to the video matrix through a third input port of the video matrix, and project the picture to the main touch screen through a first output port of the video matrix;

the main screen coordinate determination module 12 is further configured to determine interface coordinate information of the main touch screen based on the first input port of the video matrix;

the first blocking layer starting module 13 is further configured to start a first signal blocking layer based on the interface coordinate information of the main touch screen.

Optionally, as shown in fig. 15, the screen expansion system further includes a touch control screen, wherein an input port of the touch control screen is connected to a third output port of the video matrix, the touch control screen communicates with the secondary touch screen, and when the touch control screen follows the secondary touch screen, the apparatus includes:

a fourth screen projection module 18, configured to input a screen of the display peripheral to the video matrix through a third input port of the video matrix, project the screen to the auxiliary touch screen through a second output port of the video matrix, and project the screen to the touch control screen through a third output port of the video matrix;

a secondary screen coordinate determination module 15, configured to determine interface coordinate information of the secondary touch screen based on the second input port;

Optionally, as shown in fig. 15, when the touch control screen follows the main touch screen, the apparatus includes:

the fourth picture screen projection module 18 is further configured to input a picture of the display peripheral to the video matrix through a third input port of the video matrix, and project the picture to the auxiliary touch screen through a second output port of the video matrix;

the auxiliary screen coordinate determination module 15 is further configured to determine interface coordinate information of the auxiliary touch screen based on a second input port of the video matrix;

and the second blocking layer starting module 16 is further configured to start a second signal blocking layer based on the interface coordinate information of the secondary touch screen.

Optionally, as shown in fig. 16, the first logic screen determining module 12 includes:

a main screen information obtaining unit 121, configured to obtain device information of a main touch screen connected to a first input port of the video matrix;

a main screen coordinate determining unit 122, configured to determine interface coordinate information of the main touch screen based on the device information of the main touch screen.

Optionally, as shown in fig. 17, the second logical screen determining module 15 includes:

a sub-screen information obtaining unit 151 configured to obtain device information of the sub touch screen to which a second input port of the video matrix is connected;

a sub-screen coordinate determination unit 152 for determining interface coordinate information of the sub-touch screen based on the device information of the sub-touch screen.

Optionally, as shown in fig. 18, the apparatus 1 further includes:

and the first blocking layer removing module is used for synchronously removing the first signal blocking layer when the picture of the display peripheral equipment stops inputting or the screen expansion system is closed.

Optionally, as shown in fig. 19, the apparatus 1 further includes:

and the second blocking layer removing module is used for synchronously removing the second signal blocking layer when the picture of the display peripheral equipment stops inputting or the screen expansion system is closed.

By adopting the embodiment, when the touch control screen follows the main touch screen and projects the picture displaying the peripheral equipment to the main touch screen, or when the touch control screen follows the auxiliary touch screen and projects the picture displaying the peripheral equipment to the main touch screen, the interface coordinate information of the main touch screen is determined based on the first input port of the video matrix, the first signal blocking layer is started based on the interface coordinate information of the main touch screen, at the moment, if the touch operation is carried out on the main touch screen, the started first signal blocking layer will block the touch operation aiming at the main touch screen, the first picture of the main touch screen cannot respond to the touch operation, and therefore the problem of touch penetration on the main touch screen is avoided.

When the touch control screen follows the auxiliary touch screen and projects a picture displaying the peripheral equipment to the auxiliary touch screen, or when the touch control screen follows the main touch screen and projects the picture displaying the peripheral equipment to the auxiliary touch screen, determining interface coordinate information of the auxiliary touch screen based on a second input port of the video matrix, starting a second signal blocking layer based on the interface coordinate information of the auxiliary touch screen, if the touch operation is carried out on the projected auxiliary touch screen, the started second signal blocking layer will block the touch operation aiming at the auxiliary touch screen, and if the touch operation is carried out on the auxiliary touch screen at the moment, a first picture of the auxiliary touch screen cannot respond to the touch operation, so that the problem of touch penetration on the auxiliary touch screen is avoided.

It should be noted that, when the multi-screen interaction apparatus provided in the foregoing embodiment performs the multi-screen interaction method, only the division of the functional modules is illustrated, and in practical applications, the function allocation may be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules, so as to complete all or part of the functions described above. In addition, the multi-screen interaction device and the multi-screen interaction method provided by the above embodiments belong to the same concept, and the implementation process is detailed in the method embodiments, which is not described herein again.

The above-mentioned serial numbers of the embodiments of the present application are merely for description, and do not represent the advantages and disadvantages of the embodiments.

An embodiment of the present application further provides a computer storage medium, where multiple instructions may be stored, where the instructions are suitable for being loaded by a processor and executed in the method steps in the embodiments shown in fig. 1 to fig. 10, and specific execution processes may refer to specific descriptions of the embodiments shown in fig. 1 to fig. 10, which are not described herein again.

The present application further provides a computer program product, where at least one instruction is stored, where the at least one instruction is loaded by the processor and executes the method steps in the embodiments shown in fig. 1 to 10, and a specific execution process may refer to specific descriptions of the embodiments shown in fig. 1 to 10, which are not described herein again.

Fig. 20 is a schematic structural diagram of an electronic device according to an embodiment of the present application. As shown in fig. 20, the mobile terminal 1000 may include: at least one processor 1001, at least one network interface 1004, a user interface 1003, memory 1005, at least one communication bus 1002.

The communication bus 1002 is used to implement connection communication among these components.

The user interface 1003 may include a Display screen (Display) and a Camera (Camera), and the optional user interface 1003 may further include a standard wired interface and a wireless interface.

The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface), among others.

Processor 1001 may include one or more processing cores, among other things. The processor 1001 interfaces various components throughout the electronic device 1000 using various interfaces and lines to perform various functions of the electronic device 1000 and to process data by executing or executing instructions, programs, code sets, or instruction sets stored in the memory 1005 and invoking data stored in the memory 1005. Alternatively, the processor 1001 may be implemented in at least one hardware form of Digital Signal Processing (DSP), field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The processor 1001 may integrate one or more of a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), a modem, and the like. Wherein, the CPU mainly processes an operating system, a user interface, an application program and the like; the GPU is used for rendering and drawing the content required to be displayed by the display screen; the modem is used to handle wireless communications. It is understood that the above modem may not be integrated into the processor 1001, and may be implemented by a single chip.

The Memory 1005 may include a Random Access Memory (RAM) or a Read-Only Memory (Read-Only Memory). Optionally, the memory 1005 includes a non-transitory computer-readable medium. The memory 1005 may be used to store an instruction, a program, code, a set of codes, or a set of instructions. The memory 1005 may include a stored program area and a stored data area, wherein the stored program area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing the various method embodiments described above, and the like; the storage data area may store data and the like referred to in the above respective method embodiments. The memory 1005 may optionally be at least one memory device located remotely from the processor 1001. As shown in fig. 20, the memory 1005, which is a kind of computer storage medium, may include therein an operating system, a network communication module, a user interface module, and a multi-screen interactive application program.

In the mobile terminal 1000 shown in fig. 20, the user interface 1003 is mainly used as an interface for providing input for the user, and acquiring data input by the user; and the processor 1001 may be configured to call a multi-screen interactive application stored in the memory 1005,

the screen expansion system comprises a main touch device, an auxiliary touch screen, a video matrix and a display peripheral, wherein the main touch device comprises a main touch screen and an open type pluggable standard module, the video matrix at least comprises a first input port, a second input port, a third input port, a first output port and a second output port, the output port of the main touch screen is connected with the first input port of the video matrix, the output port of the open type pluggable standard module is connected with the second input port of the video matrix, the input port of the main touch screen is connected with the first output port of the video matrix, the input port of the auxiliary touch screen is connected with the second output port of the video matrix, the output port of the display peripheral is connected with the third input port of the video matrix, the main touch device is in communication connection with the auxiliary touch screen, and the processor 1001 specifically executes the following operations:

and/or the presence of a gas in the gas,

In one embodiment, the processor 1001 further includes a touch control screen in the screen expansion system, the video matrix further includes a third output port, an input port of the touch control screen is connected to the third output port of the video matrix, the touch control screen is communicatively connected to the master touch device, and when the touch control screen follows the master touch screen, the following operations are specifically performed:

inputting the picture of the display peripheral equipment to the video matrix through a third input port of the video matrix, then projecting the picture to the main touch screen through a first output port of the video matrix, and projecting the picture to the touch control screen through a third output port of the video matrix;

and starting a first signal blocking layer based on the interface coordinate information of the main touch screen.

In one embodiment, the processor 1001 specifically performs the following operations when the touch control screen follows the secondary touch screen:

In an embodiment, the processor 1001 further includes a touch control screen in the screen expansion system, wherein an input port of the touch control screen is connected to a third output port of the video matrix, the touch control screen is in communication with the secondary touch screen, and when the touch control screen follows the secondary touch screen, the following operations are specifically performed:

inputting the picture of the display peripheral to the video matrix through a third input port of the video matrix, then projecting the picture to the auxiliary touch screen through a second output port of the video matrix, and projecting the picture to the touch control screen through a third output port of the video matrix;

In one embodiment, the processor 1001 specifically performs the following operations when the touch control screen follows the main touch screen:

In one embodiment, when the processor 1001 determines the interface coordinate information of the master touch screen based on the first input port of the video matrix, specifically perform the following operations:

acquiring equipment information of the main touch screen connected with a first input port of the video matrix;

determining interface coordinate information of the main touch screen based on the device information of the main touch screen.

In one embodiment, when the processor 1001 determines the interface coordinate information of the secondary touch screen based on the second input port of the video matrix, specifically performs the following operations:

acquiring equipment information of the auxiliary touch screen connected with a second input port of the video matrix;

determining interface coordinate information of the secondary touch screen based on the device information of the secondary touch screen.

In one embodiment, after executing the initiating of the first signal blocking layer based on the interface coordinate information of the master touch screen, the processor 1001 further executes the following operations:

and when the picture of the display peripheral equipment stops inputting or the screen expansion system is closed, synchronously removing the first signal blocking layer.

In one embodiment, the processor 1001, after executing the activating of the second signal blocking layer based on the interface coordinate information of the secondary touch screen, further performs the following operations:

and when the picture of the display peripheral equipment stops inputting or the screen expansion system is closed, synchronously removing the second signal blocking layer.

By adopting the embodiment, when the touch control screen follows the main touch screen and projects a picture displaying the peripheral equipment to the main touch screen, or when the touch control screen follows the auxiliary touch screen and projects the picture displaying the peripheral equipment to the main touch screen, the interface coordinate information of the main touch screen is determined based on the first input port of the video matrix, the first signal blocking layer is started based on the interface coordinate information of the main touch screen, at the moment, if the touch operation is carried out on the main touch screen, the started first signal blocking layer blocks the touch operation aiming at the main touch screen, and the first picture of the main touch screen cannot respond to the touch operation, so that the problem of touch penetration on the main touch screen is avoided.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a read-only memory or a random access memory.

The above disclosure is only for the purpose of illustrating the preferred embodiments of the present application and is not to be construed as limiting the scope of the present application, so that the present application is not limited thereto, and all equivalent variations and modifications can be made to the present application.

Claims

1. A multi-screen interaction method is characterized in that the method is applied to a screen expansion system, the screen expansion system comprises a main touch device, an auxiliary touch screen, a video matrix and a display peripheral, wherein the main touch device comprises a main touch screen and an open pluggable standard module, the video matrix at least comprises a first input port, a second input port, a third input port, a first output port and a second output port, the output port of the main touch screen is connected with the first input port of the video matrix, the output port of the open pluggable standard module is connected with the second input port of the video matrix, the input port of the main touch screen is connected with the first output port of the video matrix, the input port of the auxiliary touch screen is connected with the second output port of the video matrix, the output port of the display peripheral is connected with the third input port of the video matrix, and the main touch device is in communication connection with the auxiliary touch screen, and the method comprises the following steps:

and/or the presence of a gas in the atmosphere,

2. The method of claim 1, wherein the screen expansion system further comprises a touch control screen, wherein the video matrix further comprises a third output port, wherein an input port of the touch control screen is connected to the third output port of the video matrix, wherein the touch control screen is communicatively connected to the master touch device, and wherein when the touch control screen follows the master touch screen, the method further comprises:

inputting the picture of the display peripheral to the video matrix through a third input port of the video matrix, then projecting the picture to the main touch screen through a first output port of the video matrix, and projecting the picture to the touch control screen through a third output port of the video matrix;

3. The method of claim 2, wherein when the touch control screen follows the secondary touch screen, the method further comprises:

4. The method of claim 1, wherein the screen expansion system further comprises a touch-control screen, wherein an input port of the touch-control screen is connected to a third output port of the video matrix, wherein the touch-control screen is in communication with the secondary touch-screen, and wherein when the touch-control screen follows the secondary touch-screen, the method further comprises:

5. The method of claim 4, wherein when the touch control screen follows the primary touch screen, the method further comprises:

6. The method of any of claims 1-3, wherein determining interface coordinate information for the primary touchscreen based on the first input port of the video matrix comprises:

7. The method of claim 1, 4 or 5, wherein determining interface coordinate information of the secondary touch screen based on the second input port of the video matrix comprises:

8. The method of any one of claims 1-3, wherein after initiating the first signal blocking layer based on the interface coordinate information of the primary touch screen, further comprising:

9. The method of claim 1, 4 or 5, wherein after activating the second signal blocking layer based on the interface coordinate information of the secondary touch screen, the method further comprises:

10. The utility model provides a many screen interaction device, its characterized in that is applied to screen expansion system, screen expansion system includes main touch equipment, vice touch-sensitive screen, video matrix and shows peripheral hardware, wherein main touch equipment includes main touch-sensitive screen and open but plug standard module, the video matrix includes first input port, second input port, third input port, first output port and second output port at least, the output port of main touch-sensitive screen with the first input port of video matrix is connected, but the output port of open but plug standard module with the second input port of video matrix is connected, the input port of main touch-sensitive screen with the first output port of video matrix is connected, the input port of vice touch-sensitive screen with the second output port of video matrix is connected, the output port that shows peripheral hardware with the third input port of video matrix is connected, main touch equipment and vice touch-sensitive screen communication connection, the device includes:

the first picture screen projection module is used for inputting the picture of the display peripheral equipment into the video matrix through a third input port of the video matrix and projecting the picture to the main touch screen through a first output port of the video matrix;

and/or the presence of a gas in the gas,

11. The apparatus of claim 10, wherein the screen expansion system further comprises a touch control screen, wherein the video matrix further comprises a third output port, wherein an input port of the touch control screen is connected to the third output port of the video matrix, wherein the touch control screen is communicatively connected to the master touch device, and wherein when the touch control screen follows the master touch screen, the apparatus further comprises:

the third picture screen projection module is used for inputting the picture of the display peripheral equipment into the video matrix through a third input port of the video matrix, projecting the picture to the main touch screen through a first output port of the video matrix and projecting the picture to the touch control screen through a third output port of the video matrix;

and the first blocking layer starting module is used for blocking the first signal layer based on the interface coordinate information of the main touch screen.

12. The apparatus of claim 11, wherein when the touch control screen follows the secondary touch screen,

the third picture screen projection module is further configured to input a picture of the display peripheral to the video matrix through a third input port of the video matrix, and project a screen to the main touch screen through a first output port of the video matrix;

the main screen coordinate determination module is further used for determining interface coordinate information of the main touch screen based on the first input port of the video matrix;

the first blocking layer starting module is further used for starting a first signal blocking layer based on the interface coordinate information of the main touch screen.

13. The apparatus of claim 10, wherein the screen expansion system further comprises a touch control screen, wherein an input port of the touch control screen is connected to a third output port of the video matrix, the touch control screen is in communication with the primary touch device, and when the touch control screen follows the secondary touch screen, the apparatus further comprises:

the fourth picture screen projection module is used for inputting the picture of the display peripheral equipment into the video matrix through a third input port of the video matrix, projecting the picture to the auxiliary touch screen through a second output port of the video matrix, and projecting the picture to the touch control screen through a third output port of the video matrix;

the auxiliary screen coordinate determination module is used for determining interface coordinate information of the auxiliary touch screen based on the second input port;

14. The method of claim 13, wherein when the touch control screen follows the primary touch screen,

the fourth picture screen projection module is further configured to input a picture of the display peripheral to the video matrix through a third input port of the video matrix, and project the picture to the auxiliary touch screen through a second output port of the video matrix;

the auxiliary screen coordinate determination module is further used for determining interface coordinate information of the auxiliary touch screen based on a second input port of the video matrix;

and the second blocking layer starting module is also used for starting a second signal blocking layer based on the interface coordinate information of the auxiliary touch screen.

15. The apparatus of any of claims 10-12, wherein the first coordinate determination module comprises:

the main screen information acquisition unit is used for acquiring the equipment information of the main touch screen connected with the first input port of the video matrix;

and the main screen coordinate determining unit is used for determining the interface coordinate information of the main touch screen based on the equipment information of the main touch screen.

16. The apparatus of claim 10, 13 or 14, wherein the coordinate determination module comprises:

the auxiliary screen information acquisition unit is used for acquiring the equipment information of the auxiliary touch screen connected with the second input port of the video matrix;

and the auxiliary screen coordinate determination unit is used for determining the interface coordinate information of the auxiliary touch screen based on the equipment information of the auxiliary touch screen.

17. The apparatus of any one of claims 10-12, wherein the multi-screen interaction apparatus further comprises:

18. The apparatus of claim 10, 13 or 14, wherein the multi-screen interaction apparatus further comprises:

19. A computer storage medium, characterized in that it stores a plurality of instructions adapted to be loaded by a processor and to perform the method steps according to any of claims 1-9.

20. An electronic device, comprising: a processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and to perform the method steps of any of claims 1-9.