WO2018184443A1

WO2018184443A1 - Terminal screen display processing method and apparatus and storage medium

Info

Publication number: WO2018184443A1
Application number: PCT/CN2018/078425
Authority: WO
Inventors: 罗刚; 谢守军
Original assignee: 博彦科技股份有限公司
Priority date: 2017-04-05
Filing date: 2018-03-08
Publication date: 2018-10-11
Also published as: CN108694029A

Abstract

A terminal screen display processing method and apparatus and a storage medium. Said method comprises: capturing images of each terminal of a plurality of terminals, the images of each terminal being used for showing contents displayed by the screen of the terminal, the images of each terminal including a plurality of images, the plurality of images being images taken when there are changes on the terminal screen (S102); sending the images (S104); using the images to show, in an interface, the contents displayed by the screen of each terminal of the plurality of terminals (S106). Said method solves the technical problem of bandwidth wastage when displaying the terminal screen using related technologies.

Description

Terminal screen display processing method, device and storage medium

This application claims the priority of the Chinese Patent Application, filed on Apr. 05, 2017, the application Serial No. In the application.

Technical field

The present invention relates to the field of network technologies, and in particular, to a terminal screen display processing method, apparatus, and storage medium.

Background technique

With the development of electronic and software technologies, the functions and interfaces of terminal devices have been continuously optimized, and corresponding, user-friendly interfaces and intuitive operating experiences have become important issues in the field of terminal devices. With the increase in the number and variety of terminal devices, users have also pursued new conveniences in controlling terminal devices. However, in the existing terminal control mode, the display of the screen of the controlled terminal is real-time, and the screen information of the terminal to be controlled is transmitted to the master device. In this case, the bandwidth of the connection established between the master device and the controlled terminal is always occupied, which seriously causes waste of bandwidth, and further causes a jam when transmitting other data, so that The user experience is also poor.

In view of the above-mentioned related art, the problem of wasting bandwidth when displaying a terminal screen has not yet proposed an effective solution.

Summary of the invention

The embodiment of the invention provides a method, a device and a storage medium for processing a terminal screen display, so as to at least solve the technical problem of wasting bandwidth when the related art displays the screen of the terminal.

According to an aspect of the present invention, a method for processing a terminal screen display includes: capturing a picture of each of the plurality of terminals, wherein the picture of each terminal is used to display the terminal The content displayed on the screen, the picture of each terminal includes a plurality of pictures, the multiple pictures are pictures when the terminal screen changes; the pictures are sent; and the plurality of pictures are used to display the plurality of pictures in one interface The content displayed on each terminal screen in the terminal.

Optionally, the capturing the picture of each terminal includes: acquiring the number of terminals displayed in the one interface; setting a resolution according to the number of the terminals; and capturing each terminal according to the resolution The picture, or, after the picture of each terminal is captured, is converted according to the resolution.

Optionally, the capturing the picture of each terminal includes: acquiring a picture of each terminal according to a predetermined time interval; determining whether a change of the acquired two adjacent pictures exceeds a threshold, and exceeding a threshold In the case, a picture captured later in the two pictures is used as a picture displayed in the interface.

Optionally, determining whether the acquired change of the two pictures exceeds the threshold includes: acquiring pixel points of each of the two pictures; comparing whether each pixel in the two pictures is A change occurs, the pixel point at which the change occurs is recorded; and it is determined whether the number of changed pixel points exceeds the threshold.

Optionally, in the case that the picture is captured and the picture is displayed in the same device, sending the picture includes: acquiring between the device that grabs the picture and the device that uses the picture for display a network condition; processing the captured picture according to the network condition; and transmitting the processed picture.

Optionally, the network status includes at least one of the following: a packet loss rate, a bandwidth.

In order to achieve the above object, according to another aspect of the present invention, a storage medium is provided, the storage medium including a stored program, wherein, when the program is running, controlling a device in which the storage medium is located to execute the terminal described above The screen displays the processing method.

In order to achieve the above object, according to another aspect of the present invention, a processor is provided, the processor being configured to execute a program, wherein the program is executed to execute the terminal screen display processing method described above.

According to another aspect of the embodiments of the present invention, there is also provided a terminal screen display processing apparatus, including: a capture module, configured to capture a picture of each of the plurality of terminals, wherein each of the terminals The picture is used to display the content displayed on the screen of the terminal, the picture of each terminal includes a plurality of pictures, the multiple pictures are pictures when the terminal screen changes, and the sending module is configured to send the picture; The first display module is configured to display content displayed by each of the plurality of terminals in an interface using the picture.

Optionally, the capturing module includes: a first acquiring unit configured to acquire a quantity of terminals displayed in the one interface; a setting unit configured to set a resolution according to the number of the terminals; a grabbing unit, And being configured to capture the picture of each terminal according to the resolution, or to perform the conversion according to the resolution after the picture of each terminal is captured.

In the embodiment of the present invention, a picture of each terminal of the multiple terminals is captured, where the picture of each terminal is set to display the content displayed on the screen of the terminal, and the picture of each terminal includes multiple pictures and multiple pictures. For the picture when the terminal screen changes; send a picture; use the picture to display the content displayed on each of the plurality of terminals in one interface. The technical problem of wasting bandwidth when displaying the screen of the terminal in the related art is solved by the embodiment of the invention, and the bandwidth of the screen of the display terminal is saved.

DRAWINGS

The drawings described herein are intended to provide a further understanding of the invention, and are intended to be a part of the invention. In the drawing:

1 is a flowchart of a terminal screen display processing method according to an embodiment of the present invention;

2 is a flowchart of a terminal control method according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of a preferred acquisition of a first terminal according to an embodiment of the present invention; FIG.

4 is a schematic illustration of a preferred pre-set area in accordance with an embodiment of the present invention;

Figure 5 is a schematic illustration of a preferred selection control in accordance with an embodiment of the present invention;

FIG. 6 is a schematic diagram of a terminal screen display processing apparatus according to an embodiment of the present invention; FIG.

FIG. 7 is a schematic diagram of a terminal control apparatus according to an embodiment of the present invention; FIG.

FIG. 8 is a block diagram showing the hardware structure of a computer terminal according to an embodiment of the present invention;

FIG. 9 is a second block diagram of a hardware structure of a computer terminal according to an embodiment of the invention.

detailed description

The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention. It is an embodiment of the invention, but not all of the embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts shall fall within the scope of the present invention.

It is to be understood that the terms "first", "second" and the like in the specification and claims of the present invention are used to distinguish similar objects, and are not necessarily used to describe a particular order or order. It is to be understood that the data so used may be interchanged where appropriate, so that the embodiments of the invention described herein can be implemented in a sequence other than those illustrated or described herein. In addition, the terms "comprises" and "comprises" and "the" and "the" are intended to cover a non-exclusive inclusion, for example, a process, method, system, product, or device that comprises a series of steps or units is not necessarily limited to Those steps or units may include other steps or units not explicitly listed or inherent to such processes, methods, products or devices.

Example 1

According to an embodiment of the present invention, there is also provided a method embodiment of a terminal screen display process, wherein the steps shown in the flowchart of the drawing may be performed in a computer system such as a set of computer executable instructions. Also, although logical sequences are shown in the flowcharts, in some cases the steps shown or described may be performed in a different order than the ones described herein.

At present, the operating systems used on mobile terminals (or mobile devices) are Microsoft's WP operating system, Android system, and Apple's iOS system. In addition to these operating systems, there may be other types of operating systems.

A variety of operating systems can also be used for computing (eg, a personal computer PC), such as a Windows operating system, an Apple operating system, a Linux operating system, or a Unix operating system.

At present, there are various control modes for the terminals. These control methods have different processing methods for the controlled terminals. For example, in a control mode, when the screen of the controlled terminal is displayed on the main control device, it is necessary to The data for displaying the screen of the controlled terminal is transmitted to the master device in real time. For this control method, the bandwidth for controlling the connection of the data transmitted by both parties is kept in an occupied state, which not only wastes bandwidth but also affects other data. Transmission.

Based on the above considerations, the present embodiment describes a method for controlling the terminal from the perspective of the display of the terminal screen. The technical solution involved in this embodiment can be made into an application in the application market, and the user downloads the application and installs it when using it. Or, if you want to promote an operating system, you can also build this function into the operating system. As a selling point of the operating system, increase the installed capacity of the operating system.

In the above operating environment, the embodiment provides a terminal screen display processing method as shown in FIG. 1 . FIG. 1 is a flowchart of a terminal screen display processing method according to an embodiment of the present invention. As shown in FIG. 1, the method includes the following steps:

Step S102: Capture a picture of each terminal of the multiple terminals, where the picture of each terminal is used to display the content displayed on the screen of the terminal, and the picture of each terminal includes multiple pictures, and multiple pictures are the terminal screen. The picture when the change occurred;

Step S104, sending a picture;

Step S106, using the picture to display the content displayed on each of the plurality of terminals in one interface.

In the above steps, the user may first select a plurality of terminals that need to be controlled. Generally, the operation of the current terminal is accessible to the network. In the above steps, the controlled terminal can be connected through the network.

The above steps describe the operation of displaying the screen of the plurality of terminals by the terminal screen display processing method, and the control application performs operations such as capturing and displaying the screen display contents of the plurality of terminals.

As an alternative embodiment, a server can be used to connect terminals of different operating systems. For example, an Android phone and an iOS phone can be connected through a Mac Mini.

The processing of displaying content on multiple terminal screens is supported in general operating systems, for example, even as the most restrictive iOS system is supported, for other relatively open systems, such as Android or Linux systems. It is also supported by the operating system. Therefore, the technical solutions involved in the above steps can support a wider range of operating systems.

As an optional embodiment, the multiple terminals may be smart terminal devices such as a computer, a notebook, a tablet computer, a mobile phone, a smart wearable device, or a smart home device such as an air conditioner, a refrigerator, a washing machine, or a television. It should be noted that the smart device, such as the smart terminal device and the smart home device, refers to a device that has a computing processing capability and can perform connection control. The foregoing smart device is only an example of the embodiment, which is not limited in this embodiment.

Based on the solutions disclosed in the foregoing steps S102 to S106, multiple pictures from the plurality of terminals for displaying the screen display content of the terminal are detected and captured in real time, and the picture is sent and used. The picture displays the content displayed on each of the terminal screens in an interface, so that the content displayed on the screen of the terminal is captured when the screen of the terminal changes, and the captured picture is displayed. In an alternative embodiment, the operations of capturing a picture and displaying a picture may occur on the same device or on different devices.

In an optional embodiment, the captured picture may be, but is not limited to, a key frame of terminal screen data, and a screen of multiple terminals is displayed on one interface, and the screens of the displayed terminals are key frames instead of Streaming media, the key frames of multiple terminals are captured as pictures, and the screen of the terminal is displayed through the transmission of pictures.

As can be seen from the above, in this embodiment, the picture when the screen of each of the plurality of terminals changes is captured, and the content displayed on each of the plurality of terminals is displayed and displayed in an interface. Therefore, when the screen of the terminal is displayed, the picture for indicating the change of the screen is transmitted only when the screen of the terminal being displayed changes, so that the connection for controlling the plurality of terminals does not need to transmit the screen data in real time, thereby reducing the screen display process. The occupation of bandwidth.

Through the solution disclosed in the above embodiment of the present application, the content displayed on multiple terminal screens is displayed on one interface, and the purpose of the picture when the terminal screen changes is captured, thereby realizing the technology for saving the bandwidth of the display terminal screen. effect. Therefore, the above embodiment of the present application solves the technical problem of wasting bandwidth when displaying the screen of the terminal in the related art.

In this embodiment, the image of each terminal can be captured according to the number of terminals displayed on the interface. In this embodiment, an optional implementation manner is provided, and the optional implementation manner may include The following steps:

Step 202: Acquire a number of terminals displayed in one interface;

Step 204, setting a resolution according to the number of terminals;

Step 206: Grab the picture of each terminal according to the resolution, or capture the picture of each terminal and convert according to the resolution.

In the above optional implementation manner, in order to adapt the captured image to the interface for displaying the picture, the resolution may be set for the captured picture according to the number of terminals displayed on the interface, and may be in one of two ways. Obtain the picture of the resolution: mode one, capture the picture of the terminal according to the above resolution; mode two, grab the picture of each terminal, and then convert the picture according to the above resolution.

In an optional implementation manner, the resolution of the displayed picture may be set according to the bandwidth status of the control side. When the network condition is not good, the lower resolution may be set to display the picture. The resolution when the image is captured on the terminal may be set according to the resolution of the terminal itself. In the case of poor network conditions, the control side may display with a lower resolution to adapt to the condition of the network. This makes the terminal screen display more efficient.

In an optional implementation manner, after the image is captured, the image may be processed into a corresponding resolution according to requirements. For example, the size of the image after processing may be determined according to the number of terminals displayed on one page; The size to determine the size of the image after processing.

For example, if you want to display 9 terminals on an interface with a resolution of 1920×1080 pixels, you can set the resolution of the terminal to 640×360 pixels, and then grab the picture of each terminal with a resolution of 640×360 pixels. Or, after capturing the picture of each terminal, the resolution of the captured picture is converted to 640×360 pixels.

In an optional implementation manner, the image resolution when the image is captured may be, but is not limited to, a fixed value.

In an optional implementation manner, the resolution of the captured image may be fixed (for example, 1920×1080), and when the captured image is displayed, the resolution of the displayed image may be based on the current network condition. Make settings. For example, the resolution of the displayed picture can be set according to the current data transmission rate. For example: obtaining the data transmission rate of the current network; setting the resolution for the displayed picture according to the data transmission rate; displaying the picture according to the set resolution.

Through the above embodiments, the purpose of capturing the picture of each terminal through the number of terminals displayed by the interface is achieved.

In the process of capturing the image of each terminal, the image displayed on the interface may be determined according to the change of the two images captured before and after. In an optional embodiment, the method may further include the following steps:

Step 302: Obtain a picture of each terminal according to a predetermined time interval.

Step 304: Determine whether the acquired change of two adjacent pictures exceeds a threshold, and if the threshold value is exceeded, use one picture captured in the two pictures as the picture displayed in the interface.

For example, the picture of each terminal is obtained every 2 seconds, and it is determined whether the change of the two pictures acquired before and after each terminal exceeds a preset threshold. For the terminal whose change exceeds the threshold, the picture that is captured later is taken as The image displayed in the interface can keep the image displayed on the current interface unchanged for images that do not exceed the threshold.

In this embodiment, when the terminal screen display processing method is applied to the test scenario of the application, comparing the adjacent images captured before and after is helpful to discover problems in the running of the application, and may be based on the application. The stage of operation on the device: installation, startup, verification, operation, uninstallation, etc. intercept the screen image to test the various stages of the application.

Through the above steps, the picture of each terminal is captured according to a predetermined time interval, and the picture displayed on the interface is determined according to the change of the two pictures captured before and after, thereby saving the bandwidth occupied by the screen display of the terminal.

In an optional embodiment, determining whether the acquired changes of the two pictures exceed a threshold may include the following steps:

Step 402: Obtain pixel points of each picture in the two pictures;

Step 404, comparing whether each pixel in the two pictures changes, and recording the changed pixel point;

Step 406: Determine whether the number of pixel points that have changed exceeds a threshold.

In this alternative embodiment, the change of the picture may be determined according to the change of the pixel points in the two pictures, thereby determining the picture displayed on the interface. For example, obtaining the pixels of each picture in the two pictures, and comparing whether each pixel in the two pictures changes, recording the changed pixel points, and obtaining the number of changed pixels is 15000, and The preset threshold is 10000 pixels, and the number of currently changed pixels has exceeded the threshold, so that the change of the obtained two pictures has exceeded the threshold, and the two pictures can be grasped later. An image taken is used as the image displayed in the interface.

In an optional implementation manner, the image may also be captured by acquiring a picture of each terminal according to a preset time, and using the currently captured picture as the picture displayed in the interface.

Optionally, the preset time may be, but is not limited to, selected by a user or set by a system.

Optionally, in this embodiment, before the picture is captured, the time interval for capturing the picture and the number of the captured pictures may be preset, and when the picture is captured, the preset capture is performed. The time interval of the image is captured, and the image is stopped when the number of captured images reaches the preset number of captured images. For example, before the image is captured, the time interval for capturing the image is set to 5s and the number of pre-set images is 20, and when the image is captured, one image is captured every 5 seconds. Stop crawling when 20 images are taken.

Optionally, in this embodiment, before the picture is captured, it may be determined whether the number of pictures that have been captured has reached a preset number. If the preset number is not reached, the picture of each terminal is obtained. If the preset number has been reached, stop capturing the image.

Optionally, the preset number may be, but is not limited to, selected by a user or set by a system.

In an optional implementation manner, in the case that the picture is captured and the picture is displayed in the same device, the network condition between the device that captures the picture and the device that uses the picture is displayed, and according to The network condition processes the captured image and then sends the processed image. The foregoing network status may include, but is not limited to, at least one of the following: a packet loss rate and a bandwidth.

In order to achieve the above object, in accordance with another aspect of the present invention, an embodiment of the present invention further provides a storage medium, the storage medium including a stored program, wherein, when the program is running, controlling a device where the storage medium is executed The terminal screen display processing method described above.

In order to achieve the above object, according to another aspect of the present invention, an embodiment of the present invention further provides a processor, where the processor is configured to run a program, where the program is executed to execute the terminal screen display processing method described above. .

According to an embodiment of the present invention, there is also provided a method embodiment of terminal control, and it should be noted that the steps shown in the flowchart of the accompanying drawings may be executed in a computer system such as a set of computer executable instructions, and Although a logical order is shown in the flowcharts, in some cases the steps shown or described may be performed in a different order than the ones described herein.

At present, there are various control modes for the terminals. These control methods are different for the controlled terminal. For example, one control mode is to display the interface of a controlled terminal on the control interface. Since the controlled terminal may have many, for This control method can only control a single controlled terminal.

However, the operation efficiency of this method is rather low for the case where there are a large number of controlled terminals. In addition, there are some drawbacks to the manner of interface display. For example, the screen size of the main control terminal and the controlled terminal used by the user may not be the same, or even the gap is large. When the user opens the interface of the controlled terminal, the user will find the The interface is full of the screen of the main control terminal, which may cause an uncomfortable feeling under certain circumstances, which may affect the user's experience.

Based on the above considerations, the present embodiment describes a method of controlling the terminal from the perspective of the controller in the terminal control. The technical solution involved in this embodiment can be made into an application in the application market, and the user downloads the application and installs it when using it. Or, if you want to promote an operating system, you can also build this function into the operating system. As a selling point of the operating system, increase the installed capacity of the operating system.

In addition, the technical solution involved in this embodiment may also be configured as a website, and the user may access the website when using a browser or using other applications, thereby implementing control of the terminal on the webpage.

In the above operating environment, the embodiment provides a terminal control method as shown in FIG. 2. FIG. 2 is a flowchart of a method for controlling a terminal according to an embodiment of the present invention. As shown in FIG. 2, the method includes the following steps:

Step S202, acquiring multimedia data from each of the plurality of terminals, wherein the multimedia data of each terminal is used to display the content displayed on the screen of the terminal;

Step S204, using multimedia data to display content displayed on each terminal screen of the plurality of terminals in one interface;

Step S206, receiving an operation signal input by the user through the interface;

Step S208, acquiring a first terminal corresponding to the operation signal;

Step S210: Convert the operation signal into a corresponding instruction and send it to the first terminal.

In the above steps, the terminal that needs to be controlled may be selected by the user first. Generally, the operation of the current terminal is accessible to the network. In the above steps, the controlled terminal can be connected through the network.

The above steps describe the operation of the control application to acquire a connection of a plurality of terminals, and the control application performs operations such as acquisition and connection of a plurality of terminals. If the content in the above steps is built into the operating system as a function, it is the operation of acquiring and connecting a plurality of terminals by the function.

As an alternative embodiment, a server can be used to connect mobile phones of different operating systems. For example, an Android phone and an iOS phone can be connected through a Mac Mini. Alternatively, the terminal can be directly connected to the hub through the USB cable, and the hub is connected to the server, thereby realizing the connection between the terminal and the server.

As an optional implementation manner, authorization may be set for the activation of the control application or the use of the function of the operating system. For example, the use of the control application may require the user to input a password or use biometric information to identify the user, if the user password The input is correct or it is recognized that the authorized user is allowed to use the function. Identification of biological information is currently widely used for fingerprint recognition. Of course, other biometric technologies can also be used, for example, iris recognition, facial recognition, voiceprint recognition, and the like.

As an optional implementation manner, the user can obtain the terminal control function by purchasing, and the price of the purchase can be determined according to the number of terminals that can be controlled. The more the number of terminals that can be controlled, the higher the price, and the user can provide the user after purchasing. A license is used as a proof of purchase.

For terminal control, it is supported in general operating systems. For example, even the most restrictive iOS system is supported. For other relatively open systems, such as Android or Linux, the operating system also supports it. . Therefore, the technical solutions involved in the above steps can support a wider range of operating systems.

Based on the solutions disclosed in the above steps S202 to S210, the multimedia data from each of the plurality of terminals is detected and acquired in real time, and the content displayed on each of the plurality of terminals is displayed in an interface using the multimedia data. The multimedia data of each terminal is used to display the content displayed on the screen of the corresponding terminal. In an optional embodiment, the acquired multimedia data may be, but is not limited to, displayed on the screen by the plurality of terminals. The multimedia data that is converted and transmitted by the content, or obtained, may be, but is not limited to, obtained by converting the data sent by the plurality of terminals for displaying the content displayed on the screen.

In an optional embodiment, the operation signal may be, but is not limited to, a signal generated by a user's touch on the screen, and may be, but is not limited to, a signal input by a user through some input devices. The above input device may include, but is not limited to, a mouse, a keyboard, a touchpad, and the like. For example, taking the user's touch on the screen as an example, the signal generated by the user's touch interface can be received, and the signal is converted into an operation signal.

As can be seen from the above, in the embodiment, the multimedia data for displaying the content displayed by the screen from each of the plurality of terminals is acquired, and the screens of the plurality of terminals are displayed in an interface using the acquired multimedia data. Displaying content, thereby realizing display of a plurality of terminal screens on one interface, receiving an operation signal input by the user through the interface, acquiring a first terminal corresponding to the operation signal, and converting the operation signal into a corresponding instruction, the instruction Send to the first terminal to implement control of the terminal displayed on the interface.

Through the solution disclosed in the above embodiment of the present application, the purpose of displaying a plurality of terminals on one interface and transmitting instructions to the terminal that needs to be controlled is achieved, thereby achieving the technical effect of improving the control efficiency of the terminal control. Therefore, the above embodiments of the present application solve the technical problem of low control efficiency of the related art terminal control.

In this embodiment, the first terminal may be acquired according to the location of the operation signal on the interface and the location of each terminal displayed on the interface. In this embodiment, an optional implementation manner is provided. An optional implementation may include the following steps:

Step 502: Acquire first location information input by the operation signal in the interface.

Step 504: Acquire second location information displayed by each terminal in the interface, where the second location information is a range occupied by the terminal in the interface.

Step 506: Determine, according to the first location information and the second location information, the first terminal corresponding to the operation signal.

In the above optional embodiment, the first location information and the second location information may be, but are not limited to, represented by coordinate values, or may be, but not limited to, represented by location information of pixel points, for example, may be used in The center of the area where the operation signal is generated on the interface to represent the operation signal, obtain the coordinate value of the center in the coordinate system of the interface as the first position information, and the second position information may be the area displayed by the terminal on the interface. The coordinate interval corresponding to the boundary determines the first terminal corresponding to the operation signal by determining a coordinate interval corresponding to the second position information in which the coordinate value corresponding to the first position information falls.

In an optional implementation manner, the terminal corresponding to the range in which the first location information falls within the range occupied by each terminal in the interface is used as the first terminal, where the first terminal includes one or more terminals. For example, the interface may be pre-divided into a plurality of areas, and each area displays a terminal, and the first terminal is obtained by determining an area in which the operation signal falls. FIG. 3 is a schematic diagram of a preferred acquisition of a first terminal according to an embodiment of the present invention. As shown in FIG. 3, the interface is divided into nine areas, and the screens of the terminals 1 to 9 are respectively displayed, and an operation signal is obtained. The first location information input in the interface falls into the area where the terminal 4 is located, and then the terminal 4 can be determined to be the first terminal.

With the above embodiment, the purpose of acquiring the first terminal to be controlled by the operation signal and the position information of the terminal is achieved.

In this embodiment, when the number of the first terminals is multiple, the function of multi-screen synchronization of the operation signals input by the user can also be implemented. That is to say, when the number of the first terminals is plural, the operation signal input by the user is converted into a corresponding instruction and transmitted to the plurality of terminals, thereby controlling the plurality of terminals to perform the same operation.

For example, when the application is tested, after logging in to the DevicePass test platform, the user can simultaneously control multiple test phones to test the application. After the user selects the multi-screen control function, the selected test phone will remain synchronized in any operation behavior initiated by one of the test phones.

Optionally, in this embodiment, the iOS and Android test mobile phones can simultaneously perform multi-screen synchronization operations. Android test phone multi-screen operation synchronization response time can be controlled within 0.1 seconds. The iOS test phone multi-screen operation synchronization response time can be controlled within 0.5 seconds.

Optionally, in this embodiment, when performing multi-screen synchronization control, the test mobile phone screen in which any synchronous operation is performed may be hidden or closed.

Optionally, in this embodiment, a multi-screen synchronous click, slide, text input, and the like test operation of the tested application may be performed. It is also possible to perform multi-screen synchronized screenshots, horizontal screen, vertical screen, return to the previous step, and return to the main screen for the tested application.

As an optional implementation manner, the screen of the first terminal to be controlled may be enlarged to be a full screen display. For example, in the process of application testing, after logging in to the DevicePass test platform, the user can control multiple terminals. When the user needs to perform a control operation on one of the test terminals, the user can select the test terminal and Test the terminal screen for full-screen zoom operation.

Optionally, in this embodiment, the full screen magnification of the single device screen of the mobile phone can be tested for iOS and Android.

Optionally, in this embodiment, when performing full-screen enlargement of the terminal screen, the control device may perform operations such as clicking, sliding, etc. to enlarge the screen of the controlled terminal. For example, a full-screen zoom operation can be identified by double-clicking, two-finger dragging, or other gestures.

Optionally, in this embodiment, when performing full-screen enlargement of the terminal screen, a test operation such as clicking, sliding, text input, etc. of the tested application is supported.

Optionally, in this embodiment, when performing full-screen enlargement of the terminal screen, the tested application may perform a function operation such as taking a screenshot, a horizontal screen, a vertical screen, returning to the previous step, and returning to the home screen.

Optionally, in this embodiment, when performing full-screen enlargement of the terminal screen, the keyboard ESC key may be clicked to quickly exit the full-screen zoom mode. You can also manually turn off full screen zoom mode.

In the conversion process of the instruction, the operation signal may be converted into its corresponding instruction according to the type of the operating system of the first terminal or the type of the application on the first terminal. In an optional embodiment, the foregoing method may also Including the following steps:

Step 602: Convert the operation signal into an instruction corresponding to the type of the operating system or the type of the application according to the type of the operating system of the first terminal acquired in advance or the type of the application on the first terminal;

Step 604, sending an instruction to the first terminal.

For example, the type of the operating system of the first terminal acquired in advance is an Android system, and the operation signal is converted into an instruction corresponding to the Android system, and the instruction is sent to the first terminal. Alternatively, if the type of the application on the first terminal acquired in advance is an instant messaging application, the operation signal is converted into an instruction corresponding to the instant messaging application, and the instruction is sent to the first terminal.

Through the above steps, if the conversion of the instruction is performed according to the type of the operating system of the first terminal acquired in advance, the operation signal is converted into an instruction corresponding to the type of the operating system, if the operation signal is according to the type of the application on the first terminal When the conversion is performed, the operation signal is converted into an instruction corresponding to the type of the application, so that the conversion of the operation signal can be applied to various operating systems or applications.

In an optional embodiment, the method may further include the following steps in order to implement the function of converting and transmitting the command:

Step 702: Convert an operation signal into an operation signal to the first terminal.

Step 704: Send an operation signal to the first terminal to the first terminal, where the operation signal to the first terminal is used as an input to the first terminal.

For example, the operation signal is to input the content "auto show" to be searched in the search field on the browser application opened by the first terminal, and perform a search. Then, the operation signal is converted into an operation signal to the first terminal, and the operation signal to the first terminal may be inputting "auto show" in the search bar of the browser, and clicking the search button. And transmitting the operation signal of the first terminal to the first terminal, thereby controlling the first terminal to perform the foregoing operation.

Through the above steps, the operation signal is converted into an operation signal to the first terminal as an input to the first terminal, so that the first terminal can be input on the interface, thereby making the control of the terminal more convenient and improving the user's Experience.

In an optional implementation manner, an operation signal input by the user may be received by, but not limited to, an area on the interface that does not overlap with the screen content of the displayed terminal. For example, the operation signal input by the user through the interface may be received by, but not limited to, receiving an operation signal input by the user in a preset area, wherein the area does not overlap with an area displaying the screen content of each terminal.

In this optional embodiment, FIG. 4 is a schematic diagram of a preferred pre-set area according to an embodiment of the present invention. As shown in FIG. 4, the interface may be divided into two areas, and an area above the interface is used. The screen of the terminal is displayed, and the area below the interface can be used as the above-mentioned preset area for receiving an operation signal.

In an optional implementation manner, the selection control for selecting the first terminal may be preset on the interface, and the terminal selected by the user through the selection control is received, wherein the selected terminal is used as the first terminal. Acquiring the first terminal and transmitting an operation signal to the first terminal.

Alternatively, in the above embodiment, the selection control may be, but is not limited to, an option that provides one of the following: an option for a single terminal, an option for a group of terminals, and an option for all terminals. That is to say, the above multiple terminals can be controlled individually or in batch by selecting a control.

As an alternative embodiment, FIG. 5 is a schematic diagram of a preferred selection control according to an embodiment of the present invention. As shown in FIG. 5, a selection control may be disposed in the preset area, and the selection control is used. In selecting the first terminal.

In order to achieve the above object, according to another aspect of the present invention, an embodiment of the present invention further provides a storage medium, the storage medium including a stored program, wherein the terminal control method described above is executed when the program is running .

In order to achieve the above object, in accordance with another aspect of the present invention, an embodiment of the present invention further provides a processor configured to execute a program, wherein the program is executed to execute the terminal control method described above.

It should be noted that, for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations, but those skilled in the art should understand that the present invention is not limited by the described action sequence. Because certain steps may be performed in other sequences or concurrently in accordance with the present invention. In addition, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

Through the description of the above embodiments, those skilled in the art can clearly understand that the application restriction method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, can also be through hardware, but in many cases The former is a better implementation. Based on such understanding, the technical solution of the present invention in essence or the contribution to the related art can be embodied in the form of a software product stored in a storage medium (such as ROM/RAM, disk, CD-ROM). The instructions include a number of instructions for causing a terminal device (which may be a cell phone, computer, server, or network device, etc.) to perform the methods described in various embodiments of the present invention.

Example 2

An embodiment of the apparatus for performing the above-described terminal screen display processing method is provided according to an embodiment of the present invention. FIG. 6 is a schematic diagram of a terminal screen display processing apparatus according to an embodiment of the present invention. The device includes a grab module 61, a transmitting module 63, and a first display module 65.

The capture module 61 is configured to capture a picture of each terminal of the multiple terminals, where the picture of each terminal is used to display the content displayed on the screen of the terminal, and the picture of each terminal includes multiple pictures, and more The picture is a picture when the terminal screen changes; the sending module 63 is configured to send a picture; the first display module 65 is configured to display the content displayed by each of the plurality of terminals in one interface using the picture.

It should be noted that the above-mentioned capture module 61, the sending module 63, and the first display module 65 may correspond to the steps S102 to S106 in the first embodiment, and the examples and application scenarios implemented by the three modules and corresponding steps. The same, but not limited to, the contents disclosed in the above embodiment 1.

In an optional embodiment, the capturing module includes: a first acquiring unit configured to acquire the number of terminals displayed in one interface; a setting unit configured to set a resolution according to the number of terminals; the capturing unit , set to capture the picture of each terminal according to the resolution, or, after grabbing the picture of each terminal, convert according to the resolution.

It should be noted that the first acquiring unit, the setting unit, and the capturing unit may correspond to steps 202 to 206 in Embodiment 1, and the three modules are the same as the examples and application scenarios implemented by the corresponding steps, but It is not limited to the contents disclosed in the above embodiment 1.

In an optional embodiment, the capturing unit includes: a second acquiring unit, configured to acquire a picture of each terminal according to a predetermined time interval; and the processing unit is configured to determine the acquired two adjacent pictures. Whether the change exceeds the threshold, and if the threshold is exceeded, a post-fetched picture in the two pictures is used as the picture displayed in the interface.

It should be noted that the foregoing second obtaining unit and the processing unit may correspond to step 302 to step 304 in Embodiment 1, and the two modules are the same as the examples and application scenarios implemented by the corresponding steps, but are not limited to the above. The content disclosed in Embodiment 1.

In an optional embodiment, the processing unit is configured to: acquire pixel points of each picture in the two pictures; compare whether each pixel point in the two pictures changes, and record the changed pixel point; Determine if the number of pixels that have changed exceeds the threshold.

It should be noted that the foregoing processing unit may correspond to step 402 to step 406 in Embodiment 1, which is the same as the example and application scenario implemented by the corresponding step, but is not limited to the content disclosed in Embodiment 1 above. .

In an optional embodiment, in the case that the captured picture and the used picture are not displayed in the same device, the sending module is configured to: acquire a network condition between the device that captures the picture and the device that uses the picture for display. ; Process the captured image according to the network status; send the processed image.

In an optional embodiment, the foregoing network condition includes at least one of the following: a packet loss rate, a bandwidth.

According to an embodiment of the present invention, an apparatus embodiment configured to implement the foregoing terminal control method is further provided. FIG. 7 is a schematic diagram of a terminal control apparatus according to an embodiment of the present invention. As shown in FIG. 7, the apparatus includes a first The acquisition module 71, the second presentation module 73, the receiving module 75, the second acquisition module 77, and the processing module 79.

The first obtaining module 71 is configured to acquire multimedia data from each of the plurality of terminals, wherein the multimedia data of each terminal is used to display the content displayed on the screen of the terminal; and the second display module 73 is configured to The multimedia data is used to display the content displayed on each of the plurality of terminals in an interface; the receiving module 75 is configured to receive an operation signal input by the user through the interface; and the second obtaining module 77 is configured to acquire the first corresponding to the operation signal. a terminal; the processing module 79 is configured to convert the operation signal into a corresponding instruction and send the signal to the first terminal.

It should be noted that the first acquiring module 71, the second displaying module 73, the receiving module 75, the second obtaining module 77, and the processing module 79 may correspond to steps S202 to S210 in the first embodiment, and three modules. The examples and application scenarios implemented by the corresponding steps are the same, but are not limited to the contents disclosed in the above embodiment 1.

In an optional embodiment, the second obtaining module 77 includes: a first acquiring unit, configured to acquire first position information input by the operation signal in the interface; and a second acquiring unit configured to acquire Second location information displayed by the terminal in the interface, wherein the second location information is a range occupied by the terminal in the interface; and the determining unit is configured to be based on the first location information and the location The second location information determines the first terminal corresponding to the operation signal.

It should be noted that, the foregoing first obtaining unit, the second obtaining unit, and the determining unit may correspond to steps 502 to 506 in Embodiment 1, and the three modules are the same as the examples and application scenarios implemented by the corresponding steps. However, it is not limited to the contents disclosed in the above embodiment 1.

In an optional implementation manner, the determining unit may use, as the first terminal, a terminal corresponding to a range in which the first location information falls within a range occupied by each terminal in the interface, where the first terminal includes one or more Terminals.

In an optional embodiment, the processing module 79 is configured to: convert the operation signal into the type or application of the operating system according to the type of the operating system of the first terminal acquired in advance or the type of the application on the first terminal. The instruction corresponding to the type; the instruction is sent to the first terminal.

It should be noted that the foregoing processing module may also correspond to step 602 to step 604 in Embodiment 1, which is the same as the example and application scenario implemented by the corresponding step, but is not limited to the one disclosed in Embodiment 1 above. content.

In an optional embodiment, the processing module is further configured to: convert the operation signal into an operation signal to the first terminal; and send an operation signal to the first terminal to the first terminal, where the first terminal is The operation signal is used as an input to the first terminal.

It should be noted that the foregoing processing module may also correspond to step 702 to step 704 in Embodiment 1, which is the same as the example and application scenario implemented by the corresponding step, but is not limited to the one disclosed in Embodiment 1 above. content.

In an optional embodiment, the receiving module is configured to: receive an operation signal input by the user in a preset area, where the area does not overlap with an area displaying the screen content of each terminal; the second acquiring module The method is configured to: receive a terminal selected by the user by using a selection control, where the selected terminal is used as the first terminal; and send an operation signal to the first terminal.

In an alternative embodiment, the selection control provides one of the following options: an option for a single terminal, an option for a group of terminals, an option for all terminals.

Example 3

Embodiments of the present invention may provide a computer terminal, which may be any one of computer terminal groups. Optionally, in this embodiment, the foregoing computer terminal may also be replaced with a terminal device such as a mobile terminal.

Optionally, in this embodiment, the computer terminal may be located in at least one network device of the plurality of network devices of the computer network.

In this embodiment, the computer terminal may execute the program code of the following steps in the application restriction method of the application: fetching a picture of each of the plurality of terminals, wherein the picture of each terminal is used to display the screen of the terminal The displayed content, the picture of each terminal includes a plurality of pictures, the multiple pictures are pictures when the screen of the terminal changes; the pictures are sent; the pictures are displayed in an interface to display the contents displayed on each of the plurality of terminals.

In this embodiment, the computer terminal may execute the program code of the following steps in the data transmission processing method of the application program: acquiring the number of terminals displayed in one interface; setting the resolution according to the number of terminals; and crawling each according to the resolution A picture of the terminal, or, after capturing the picture of each terminal, converting according to the resolution.

Optionally, FIG. 8 shows a hardware structure block diagram of an optional computer terminal. As shown in FIG. 8, the computer terminal 8 may include one or more (81a, 81b, ..., 81n are used in the figure). Processor 81 (processor 81 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 83 configured to store data, and a transmission device 85 configured as a communication function. In addition, it can also include: display, input/output interface (I/O interface), universal serial bus (USB) port (which can be included as one of the ports of the I/O interface), network interface, power supply And / or camera. It will be understood by those skilled in the art that the structure shown in FIG. 8 is merely illustrative and does not limit the structure of the above electronic device. For example, computer terminal 8 may also include more or fewer components than shown in FIG. 8, or have a different configuration than that shown in FIG.

It should be noted that one or more of the above described processors 81 and/or other data processing circuits may be referred to herein generally as "data processing circuits." The data processing circuit may be embodied in whole or in part as software, hardware, firmware or any other combination. Moreover, the data processing circuitry can be a single, separate processing module, or incorporated in whole or in part into any of the other components in computer terminal 8. As referred to in the embodiments of the present application, the data processing circuit is controlled as a processor (e.g., selection of a variable resistance terminal path connected to the interface).

The memory 83 can be configured as a software program and a module for storing application software, such as a program instruction/data storage device corresponding to the data transmission processing method in the embodiment of the present invention, and the processor 81 runs the software program and module stored in the memory 83. Thereby, various functional applications and data processing are performed, that is, a data transmission processing method of the above-described application is implemented. Memory 83 may include high speed random access memory and may also include non-volatile memory such as one or more magnetic storage devices, flash memory, or other non-volatile solid state memory. In some examples, memory 83 may further include memory remotely located relative to processor 81, which may be coupled to computer terminal 8 via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

Transmission device 85 is arranged to receive or transmit data via a network. The above specific network example may include a wireless network provided by a communication provider of the computer terminal 8. In one example, transmission device 85 includes a Network Interface Controller (NIC) that can be connected to other network devices through a base station to communicate with the Internet. In one example, transmission device 85 can be a Radio Frequency (RF) module configured to communicate with the Internet wirelessly.

The display can be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with the user interface of computer terminal 8.

It should be noted that, in some optional embodiments, the computer terminal 8 shown in FIG. 8 above may include hardware components (including circuits), software components (including computer code stored on a computer readable medium), or A combination of both hardware and software components. It should be noted that FIG. 8 is only one example of a specific specific example, and is intended to show the types of components that may be present in the above computer terminal.

It should be noted here that in some embodiments, the computer terminal 8 shown in FIG. 8 described above has a touch display (also referred to as a "touch screen" or a "touch display screen"). In some embodiments, the computer terminal 8 shown in FIG. 8 above has an image user interface (GUI), and the user can interact with the GUI by touching a finger contact and/or gesture on the touch-sensitive surface, where the person is The optional machine interaction includes the following interactions: creating web pages, drawing, word processing, making electronic documents, games, video conferencing, instant messaging, emailing, calling interfaces, playing digital video, playing digital music and/or web browsing, etc. The executable instructions for performing the human-machine interaction functions described above are configured/stored in one or more processor-executable computer program products or readable storage media.

The processor may call the information stored in the memory and the application program by the transmission device to perform the following steps: acquiring a picture of each terminal according to a predetermined time interval; determining whether the acquired change of the two adjacent pictures exceeds a threshold, and exceeding In the case of a threshold, a picture captured after the two pictures is used as a picture displayed in the interface.

The processor may call the memory stored information and the application by the transmission device to perform the following steps: acquiring pixels of each picture in the two pictures; comparing whether each pixel in the two pictures changes, the recording occurs. A pixel that changes; determines if the number of pixels that have changed exceeds a threshold.

Optionally, the foregoing processor may further execute the following program code: obtain a network condition between the device that captures the image and the device that uses the image to display; and process the captured image according to the network condition; after the sending process picture of.

A person skilled in the art can understand that the structure shown in FIG. 8 is only an illustration, and the computer terminal can also be a smart phone (such as an Android mobile phone, an iOS mobile phone, etc.), a tablet computer, an applause computer, and a mobile Internet device (Mobile Internet Devices, MID). ), PAD and other terminal devices. FIG. 8 does not limit the structure of the above electronic device. For example, computer terminal 8 may also include more or fewer components (e.g., network interfaces, display devices, etc.) than shown in Figure 8, or have a different configuration than that shown in Figure 8.

A person of ordinary skill in the art may understand that all or part of the steps of the foregoing embodiments may be completed by a program to instruct terminal device related hardware, and the program may be stored in a computer readable storage medium, and the storage medium may be Including: flash disk, read-only memory (ROM), random access memory (RAM), disk or optical disk.

In this embodiment, the computer terminal may execute the program code of the following steps in the application restriction method of the application: acquiring multimedia data from each of the plurality of terminals, wherein the multimedia data of each terminal is used for displaying the terminal The content displayed on the screen; displaying the content displayed on each terminal screen of the plurality of terminals in an interface; receiving an operation signal input by the user through the interface; acquiring the first terminal corresponding to the operation signal; converting the operation signal into a corresponding The instructions are sent to the first terminal.

In this embodiment, the computer terminal may execute the program code of the following steps in the data transmission processing method of the application program: acquiring the first location information input by the operation signal in the interface; and acquiring the second location displayed by each terminal in the interface. Information, wherein the second location information is a range occupied by the terminal in the interface; and determining, by the first location information and the second location information, the first terminal corresponding to the operation signal.

Optionally, FIG. 9 shows a hardware structure block diagram of an optional computer terminal. As shown in FIG. 9, the computer terminal 9 may include one or more (91a, 91b, ..., 91n are used in the figure). Processor 91 (processor 91 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 93 configured to store data, and a transmission device 95 configured as a communication function. In addition, it can also include: display, input/output interface (I/O interface), universal serial bus (USB) port (which can be included as one of the ports of the I/O interface), network interface, power supply And / or camera. It will be understood by those skilled in the art that the structure shown in FIG. 9 is merely illustrative and does not limit the structure of the above electronic device. For example, computer terminal 9 may also include more or fewer components than shown in FIG. 9, or have a different configuration than that shown in FIG.

It should be noted that one or more of the above described processors 91 and/or other data processing circuits may be referred to herein generally as "data processing circuits." The data processing circuit may be embodied in whole or in part as software, hardware, firmware or any other combination. Moreover, the data processing circuitry can be a single, separate processing module, or incorporated in whole or in part into any of the other components in computer terminal 9. As referred to in the embodiments of the present application, the data processing circuit is controlled as a processor (e.g., selection of a variable resistance terminal path connected to the interface).

The memory 93 can be configured as a software program and a module for storing application software, such as a program instruction/data storage device corresponding to the data transmission processing method in the embodiment of the present invention, and the processor 91 runs the software program and the module stored in the memory 93. Thereby, various functional applications and data processing are performed, that is, a data transmission processing method of the above-described application is implemented. Memory 93 may include high speed random access memory and may also include non-volatile memory such as one or more magnetic storage devices, flash memory, or other non-volatile solid state memory. In some examples, memory 93 may further include memory remotely located relative to processor 91, which may be connected to computer terminal 9 via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

Transmission device 95 is arranged to receive or transmit data via a network. The above specific network example may include a wireless network provided by a communication provider of the computer terminal 9. In one example, transmission device 95 includes a Network Interface Controller (NIC) that can be connected to other network devices through a base station to communicate with the Internet. In one example, the transmission device 95 can be a Radio Frequency (RF) module for communicating with the Internet wirelessly.

The display can be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with the user interface of computer terminal 9.

It should be noted that, in some optional embodiments, the computer terminal 9 shown in FIG. 9 above may include hardware components (including circuits), software components (including computer code stored on a computer readable medium), or A combination of both hardware and software components. It should be noted that FIG. 9 is only one example of a specific specific example, and is intended to show the types of components that may be present in the above computer terminal.

It should be noted here that in some embodiments, the computer terminal 9 shown in FIG. 9 described above has a touch display (also referred to as a "touch screen" or a "touch display screen"). In some embodiments, the computer terminal 9 shown in FIG. 9 above has an image user interface (GUI), and the user can interact with the GUI by touching a finger contact and/or gesture on the touch-sensitive surface, where the person is The optional machine interaction includes the following interactions: creating web pages, drawing, word processing, making electronic documents, games, video conferencing, instant messaging, emailing, calling interfaces, playing digital video, playing digital music and/or web browsing, etc. The executable instructions for performing the human-machine interaction functions described above are configured/stored in one or more processor-executable computer program products or readable storage media.

The processor may call the information and the application stored in the memory by the transmission device to perform the following steps: the terminal corresponding to the range in which the first location information falls within the range occupied by each terminal in the interface is used as the first terminal, where The first terminal includes one or more terminals.

The processor may call the information and the application stored by the memory through the transmission device to perform the following steps: converting the operation signal into an operating system according to the type of the operating system of the first terminal acquired in advance or the type of the application on the first terminal. The type or the instruction corresponding to the type of the application; the instruction is sent to the first terminal.

Optionally, the foregoing processor may further execute the following steps: converting the operation signal into an operation signal to the first terminal; and transmitting an operation signal to the first terminal to the first terminal, where The operational signal acts as an input to the first terminal.

Optionally, the processor may further execute the following program code: receiving an operation signal input by the user in a preset area, where the area does not overlap with an area displaying the screen content of each terminal; The first terminal includes: receiving the terminal selected by the user through the selection control, wherein the selected terminal is used as the first terminal; and the operation signal is sent to the first terminal.

Optionally, the selection control provides one of the following options: options for a single terminal, options for a group of terminals, options for all terminals.

A person skilled in the art can understand that the structure shown in FIG. 9 is only an illustration, and the computer terminal can also be a smart phone (such as an Android mobile phone, an iOS mobile phone, etc.), a tablet computer, an applause computer, and a mobile Internet device (Mobile Internet Devices, MID). ), PAD and other terminal devices. FIG. 9 does not limit the structure of the above electronic device. For example, computer terminal 9 may also include more or fewer components (such as a network interface, display device, etc.) than shown in FIG. 9, or have a different configuration than that shown in FIG.

Example 4

Embodiments of the present invention also provide a storage medium. Optionally, in this embodiment, the foregoing storage medium may be configured to save the program code executed by the terminal screen display processing method provided in Embodiment 1 above.

Optionally, in this embodiment, the foregoing storage medium may be located in any one of the computer terminal groups in the computer network, or in any one of the mobile terminal groups.

Optionally, in the embodiment, the storage medium is configured to store program code for performing the steps of: capturing a picture of each of the plurality of terminals, wherein the picture of each terminal is used to display the terminal The content displayed on the screen, the picture of each terminal includes a plurality of pictures, the plurality of pictures are pictures when the screen of the terminal changes; the pictures are sent; the pictures are displayed in an interface to display the contents displayed on each of the plurality of terminals .

Optionally, in the embodiment, the storage medium is configured to store program code for performing the steps of: acquiring the number of terminals displayed in one interface; setting the resolution according to the number of terminals; crawling each according to the resolution A picture of the terminal, or, after capturing the picture of each terminal, converting according to the resolution.

Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: acquiring a picture of each terminal according to a predetermined time interval; determining whether the acquired change of the two adjacent pictures exceeds Threshold, if a threshold is exceeded, a picture captured in the two pictures is used as the picture displayed in the interface.

Optionally, in the embodiment, the storage medium is configured to store program code for performing the following steps: acquiring pixel points of each picture in the two pictures; comparing whether each pixel point in the two pictures occurs Change, record the pixel that has changed; determine if the number of pixels that have changed exceeds the threshold.

Optionally, in the embodiment, the storage medium is configured to store program code for performing the following steps: obtaining a network condition between the device that captures the picture and the device that uses the picture for display; and fetching according to the network condition The image to be processed is processed; the processed image is sent.

Optionally, in this embodiment, the network status includes at least one of the following: a packet loss rate, and a bandwidth.

Embodiments of the present invention also provide a storage medium. Optionally, in this embodiment, the foregoing storage medium may be used to save the program code executed by the terminal control method provided in Embodiment 1 above.

Optionally, in the embodiment, the storage medium is configured to store program code for performing the following steps: acquiring multimedia data from each of the plurality of terminals, wherein the multimedia data of each terminal is used for displaying The content displayed on the screen of the terminal; displaying the content displayed on each terminal screen of the plurality of terminals in one interface by using the multimedia data; receiving an operation signal input by the user through the interface; acquiring the first terminal corresponding to the operation signal; converting the operation signal into The corresponding instruction is sent to the first terminal.

Optionally, in the embodiment, the storage medium is configured to store program code for performing the following steps: acquiring first location information input by the operation signal in the interface; acquiring a second location displayed by each terminal in the interface Information, wherein the second location information is a range occupied by the terminal in the interface; and determining, by the first location information and the second location information, the first terminal corresponding to the operation signal.

Optionally, in the embodiment, the storage medium is configured to store program code for performing the following steps: the terminal corresponding to the range in which the first location information falls within the range occupied by each terminal in the interface as the first A terminal, wherein the first terminal comprises one or more terminals.

Optionally, in the embodiment, the storage medium is configured to store program code for performing the following steps: operating signals according to the type of the operating system of the first terminal acquired in advance or the type of the application on the first terminal The instruction corresponding to the type of the operating system or the type of the application; the instruction is sent to the first terminal.

Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: converting the operation signal into an operation signal to the first terminal; transmitting an operation signal to the first terminal to the first a terminal, wherein an operation signal to the first terminal is used as an input to the first terminal.

Optionally, in the embodiment, the storage medium is configured to store program code for performing the following steps: receiving an operation signal input by the user in a preset area, wherein the area and the screen content of each terminal are displayed. The area does not overlap; the first terminal corresponding to the operation signal includes: receiving the terminal selected by the user through the selection control, wherein the selected terminal is used as the first terminal; and the operation signal is sent to the first terminal.

Optionally, in the present embodiment, the selection control provides one of the following options: an option for a single terminal, an option for a group of terminals, and an option for all terminals.

The serial numbers of the embodiments of the present invention are merely for the description, and do not represent the advantages and disadvantages of the embodiments.

In the above-mentioned embodiments of the present invention, the descriptions of the various embodiments are different, and the parts that are not detailed in a certain embodiment can be referred to the related descriptions of other embodiments.

In the several embodiments provided by the present application, it should be understood that the disclosed technical contents may be implemented in other manners. The device embodiments described above are merely illustrative. For example, the division of the unit is only a logical function division. In actual implementation, there may be another division manner. For example, multiple units or components may be combined or may be Integrate into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, unit or module, and may be electrical or otherwise.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a standalone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may contribute to the related art or all or part of the technical solution may be embodied in the form of a software product stored in a storage medium. A number of instructions are included to cause a computer device (which may be a personal computer, server or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: a U disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic disk, or an optical disk, and the like. .

The above description is only a preferred embodiment of the present invention, and it should be noted that those skilled in the art can also make several improvements and retouchings without departing from the principles of the present invention. It should be considered as the scope of protection of the present invention.

Industrial applicability

The solution provided by the embodiment of the present invention can be applied to the display process of the terminal screen. The technical problem of wasting bandwidth when displaying the screen of the terminal in the related art is solved by the embodiment of the invention, and the bandwidth of the screen of the display terminal is saved.

Claims

A terminal screen display processing method includes:

Grasing a picture of each of the plurality of terminals, wherein the picture of each terminal is used to display content displayed on a screen of the terminal, the picture of each terminal includes a plurality of pictures, the multiple sheets The picture is the picture when the screen of the terminal changes;

Send the picture;

The content displayed by each of the plurality of terminals is displayed in an interface using the picture.
The method of claim 1, wherein the capturing a picture of each of the terminals comprises:

Obtaining the number of terminals displayed in the one interface;

Setting a resolution according to the number of terminals;

The picture of each terminal is captured according to the resolution, or the picture of each terminal is captured and converted according to the resolution.
The method of claim 1, wherein the capturing a picture of each of the terminals comprises:

Obtaining a picture of each terminal according to a predetermined time interval;

Determining whether the acquired change of two adjacent pictures exceeds a threshold, and if the threshold is exceeded, using a picture captured in the two pictures as a picture displayed in the interface .
The method according to claim 3, wherein determining whether the acquired change of the two pictures exceeds the threshold comprises:

Obtaining pixels of each of the two pictures;

Comparing whether each pixel in the two pictures changes, and recording a pixel that changes;

It is determined whether the number of pixels that have changed exceeds the threshold.
The method of claim 1, wherein the capturing a picture of each of the terminals comprises:

Obtaining a picture of each terminal according to a preset time;

A picture currently captured is used as the picture displayed in the interface.
The method according to any one of claims 1 to 5, wherein, in the case that the picture is captured and the picture is displayed using the picture is not in the same device, transmitting the picture comprises:

Obtaining a network condition between a device that grabs the picture and a device that uses the picture to display;

Processing the captured image according to the network condition;

Send the processed picture.
The method of claim 6, wherein the network condition comprises at least one of: a packet loss rate, a bandwidth.
A terminal screen display processing device includes:

a capture module, configured to capture a picture of each of the plurality of terminals, where the picture of each terminal is used to display content displayed on a screen of the terminal, where the picture of each terminal includes multiple a picture, the plurality of pictures being pictures when the screen of the terminal changes;

a sending module, configured to send the picture;

The first display module is configured to display content displayed by each of the plurality of terminals in an interface using the picture.
A storage medium comprising a stored program, wherein the terminal screen display processing method according to any one of claims 1 to 7 is executed while the program is running.
A processor configured to execute a program, wherein the program is executed to execute the terminal screen display processing method according to any one of claims 1 to 7.