WO2022111732A1

WO2022111732A1 - Application control method for terminal device, and related device thereof

Info

Publication number: WO2022111732A1
Application number: PCT/CN2021/134494
Authority: WO
Inventors: 陈普
Original assignee: 华为云计算技术有限公司
Priority date: 2020-11-30
Filing date: 2021-11-30
Publication date: 2022-06-02
Also published as: CN114595013A

Abstract

The present application discloses an application control method for a terminal device, and a related device thereof, which can effectively utilize device resources and improve the utilization rate of a device. The method of the present application comprises: providing a user interaction interface, the user interaction interface being configured to provide, for at least two applications of a terminal device, options of a plurality of input devices and a plurality of output devices, respectively, and receive a first selection set by a user for a first application of the terminal device on the basis of the option, the first selection comprising a first input device and a first output device; in cases where the first application is enabled, determining a first input device and a first output device of the first application according to the first selection, running the first application according to the instruction acquired from the first input device, and sending, to the first output device, display screen data generated by running the first application, and displaying same.

Description

Application control method for terminal equipment and related equipment

technical field

The present application relates to the field of computer technologies, and in particular, to an application control method for a terminal device and related devices.

Background technique

At present, the terminal device itself usually has a set of input devices (for example, a built-in keyboard and mouse, etc.) and output devices (for example, a display, etc.), through which the user can operate various components on the terminal device. application.

However, if the user wants to use the external input device and the external output device to operate the application of the terminal device, the set of input device and output device can be connected to the terminal device. At this time, the user needs to perform a series of operations on the operating system of the terminal device, so as to connect the terminal device to the set of input devices and output devices, and then the input device can be used to control the terminal device, and the terminal device outputs to the output device, such as display Output signals such as image and/or sound.

In the above process, the terminal device is bound to the external input device and output device, and all applications on the terminal device must use the set of input devices and output devices, resulting in poor user experience.

SUMMARY OF THE INVENTION

Embodiments of the present application provide an application control method for a terminal device and related devices, which can effectively utilize device resources and improve device utilization.

A first aspect of the embodiments of the present application provides an application control method for a terminal device, and the method includes:

On the operating system of the terminal device, multiple applications, such as game software, office software, and the like, may run. Further, the terminal device is connected with multiple input devices and multiple output devices. For example, the terminal device is connected with multiple input devices such as a mouse, a keyboard, and a camera, and the terminal device is also connected with multiple output devices such as a monitor and a TV. The operating system of the terminal device can centrally manage multiple input devices and multiple output devices to allocate these devices to different applications.

Specifically, the operating system of the terminal device may first provide a user interaction interface, where the user interaction interface is used to respectively provide options of multiple input devices and multiple output devices for at least two applications, and receive user settings for the first application based on the options. The first selection includes a first input device and a first output device. Then, when the first application is started, the operating system of the terminal device determines the first input device and the first output device of the first application according to the first selection, and runs the first application according to the instruction obtained from the first input device, The display screen data generated by running the first application is sent to the first output device for display.

For example, let’s say a smartphone has its own keyboard and display and is connected to additional devices such as gamepads and TVs. The operating system of a smartphone can provide the user with a visual interface that includes options such as the phone's keyboard, display, gamepad, and TV for the user to choose from. The user can set the gamepad and the TV set to a certain game software on the visual interface, so that the gamepad and the TV set exclusively serve the game software. In this way, the user can input operation instructions through the gamepad, and the operating system can run the game software based on the operation instructions from the gamepad, and send display screen data generated by running the game software to the TV for display. At this time, the display screen of the smart phone will not display the display screen data generated by running the game software, that is, the keyboard of the smart phone and the display screen of the smart phone can serve other applications.

It can be seen from the above method that the operating system of the terminal device can provide a user interaction interface, and the user interaction interface is used to provide options of multiple input devices and multiple output devices respectively for at least two applications, so the user can A first selection is input on the interactive interface, and the operating system binds the first input device and the first output device to the first application based on the first selection, so that the first input device and the first output device serve the first application. When the first application has been started, the operating system can run the first application according to the operation instruction from the first input device, and send display screen data generated by running the first application to the first output device for display. In the foregoing process, the user only needs to input the first selection on the user interaction interface provided by the operating system, and the first input device and the first output device can be bound to the first application, that is, the process only requires the user to perform a simple The operation does not need to perform multiple complex operations, which can improve the user experience.

Further, the remaining input devices except the first input device and the remaining output devices except the first output device can serve other applications, thereby effectively utilizing device resources and improving device utilization.

In a possible implementation manner, the user interaction interface is further configured to receive a second selection set by the user for the second application of the terminal device based on the option, where the second selection includes a second input device and a second output device, and the method further Including: when the second application is started, determining the second input device and the second output device of the second application according to the second selection, running the second application according to the instruction obtained from the second input device, and running the second application The generated display screen is sent to the second input device for display. As in the above example, the user can set the keyboard of the smartphone and the display of the smartphone to a certain office software on the visual interface provided by the operating system, so that the keyboard of the smartphone and the display of the smartphone are specially designed for the office. software service. In this way, the user can input operating instructions through the keyboard of the smart phone, and the operating system can run the office software based on the operating instructions from the keyboard of the smart phone, and send the display screen data generated by running the office software to the display screen of the smart phone for display. . In the foregoing implementation manner, the user may also input a second selection on the user interaction interface, and the operating system binds the second input device and the second output device to the second application based on the second selection, so that the second input device and the first The second output device serves the second application. When the second application has been started, the operating system can run the second application according to the operation instruction from the second input device, and send display screen data generated by running the second application to the second output device for display. In this way, the user can make the first input device and the first output device serve the first application, and make the second input device and the second output device serve the second application, thereby effectively utilizing each input device and each output of the terminal device. equipment to improve the utilization of equipment.

In a possible implementation manner, the first application and the second application run simultaneously in the terminal device. In the foregoing implementation manner, the operating system of the terminal device can run the first application and the second application at the same time, that is, while a user can operate the first application through the first input device and the first output device, another user can also operate the first application through the first input device and the first output device. The second input device and the second output device operate the second application, so the operation requirements of different users on different applications on the same terminal device can be met. Still as in the above example, when the operating system receives an operation instruction from the gamepad, it sends the operation instruction to the game software for processing, and sends the display screen data fed back by the game software to the TV, so as to display the game screen on the TV. . At the same time, if the operating system also receives an operation instruction from the keyboard of the mobile phone, the operation instruction is sent to the office software for processing, and the display screen data fed back by the office software is sent to the display screen of the mobile phone, so that the display screen of the mobile phone can be displayed on the mobile phone. The office screen is displayed on the display. In this way, the operating system on one terminal device can simultaneously satisfy the operation requirements of different users for different software, that is, simultaneously realize the game operation of one user and the office operation of another user.

In a possible implementation manner, the user interaction interface is further configured to receive a third selection set by the user for the first application of the terminal device based on the option, where the third selection includes a third input device and a third output device, and the method further Including: when the first application is started, determining the third input device and the third output device of the first application according to the third selection, running the first application according to the instruction obtained from the third input device, and running the first application The generated display screen is sent to the third input device for display. In the foregoing implementation manner, the user may also input a third selection on the user interaction interface, and the operating system binds the third input device and the third output device to the first application based on the third selection, so that the third input device and the third The three output devices serve the first application, thereby replacing the first input device and the first output device already bound to the first application. Then, when the first application has been started, the operating system can run the first application according to the operation instruction from the third input device, and send the display screen data generated by running the first application to the third output device for display. Therefore, the user can modify the input and output devices bound to the first application according to actual needs, thereby improving the comprehensiveness of the solution.

In a possible implementation manner, the first input device is one of a local input device of the terminal device and an external input device of the terminal device; the second input device is a local input device of the terminal device and an external input device of the terminal device The other of the first output device is one of the local output device of the terminal device and the external output device of the terminal device; the second output device is the other of the local output device of the terminal device and the external output device of the terminal device. one.

In a possible implementation manner, the first input device includes a first keyboard device and a first mouse device; the second input device includes a second keyboard device and a second mouse device.

In a possible implementation manner, the local output device is an output device provided by the terminal device, and the external output device is an output device connected to the terminal device.

A second aspect of the embodiments of the present application provides an application control device for a terminal device, the device includes: an interaction module configured to provide a user interaction interface, where the user interaction interface is configured to provide multiple an option of an input device and a plurality of output devices, and receive a first selection set by the user for the first application of the terminal device based on the option, where the first selection includes the first input device and the first output device; the operation module is used for the first selection in the first application. When an application is started, determine the first input device and the first output device of the first application according to the first selection, run the first application according to the instruction obtained from the first input device, and execute the display generated by the first application The screen data is sent to the first output device for display.

It can be seen from the above device that the operating system of the terminal device can provide a user interaction interface, and the user interaction interface is used to respectively provide multiple input devices and multiple output devices for at least two applications, so the user can use the user interaction interface On inputting the first selection, the operating system binds the first input device and the first output device to the first application based on the first selection, so that the first input device and the first output device serve the first application. When the first application has been started, the operating system can run the first application according to the operation instruction from the first input device, and send display screen data generated by running the first application to the first output device for display. Then, the remaining input devices except the first input device and the remaining output devices except the first output device can serve other applications, thereby effectively utilizing device resources and improving device utilization.

In a possible implementation manner, the user interaction interface is further configured to receive a second selection set by the user for the second application of the terminal device based on the option, the second selection includes a second input device and a second output device, and the running module further When the second application is started, determine the second input device and the second output device of the second application according to the second selection, run the second application according to the instruction obtained from the second input device, and will run the second application The generated display screen is sent to the second input device for display.

In a possible implementation manner, the first application and the second application run simultaneously in the terminal device.

In a possible implementation manner, the user interaction interface is further configured to receive a third selection set by the user for the first application of the terminal device based on the option, the third selection includes a third input device and a third output device, and the running module further When the first application is started, determine the third input device and the third output device of the first application according to the third selection, run the first application according to the instruction obtained from the third input device, and run the first application The generated display screen is sent to the third input device for display.

A third aspect of the embodiments of the present application provides a terminal device, the terminal device includes: a processor and a memory; the memory is used for storing computer-executed instructions; the processor is used for executing the computer-executed instructions stored in the memory, so that the communication apparatus realizes the following: The method described in the first aspect or any possible implementation manner of the first aspect.

A fourth aspect of the embodiments of the present application provides a computer storage medium, which is characterized in that it includes computer-readable instructions, and when the computer-readable instructions are executed, the first aspect or any one of the possibilities of the first aspect is implemented. implement the method described.

A fifth aspect of the embodiments of the present application provides a computer program product containing instructions, which, when run on a computer, causes the computer to execute the method described in the first aspect or any possible implementation manner of the first aspect .

In this embodiment of the present application, the operating system of the terminal device may provide a user interaction interface, and the user interaction interface is used to respectively provide multiple input devices and multiple output devices for at least two applications, so the user can use the user interaction interface on the user interaction interface. The first selection is input, and the operating system binds the first input device and the first output device to the first application based on the first selection, so that the first input device and the first output device serve the first application. When the first application has been started, the operating system can run the first application according to the operation instruction from the first input device, and send display screen data generated by running the first application to the first output device for display. Then, the remaining input devices except the first input device and the remaining output devices except the first output device can serve other applications, thereby effectively utilizing device resources and improving device utilization.

Description of drawings

FIG. 1 is a schematic structural diagram of a terminal device provided by an embodiment of the present application;

2 is a schematic flowchart of an application control method for a terminal device provided by an embodiment of the present application;

3 is a schematic diagram of a user interaction interface provided by an embodiment of the present application;

FIG. 4 is a schematic structural diagram of an application control apparatus of a terminal device provided by an embodiment of the present application;

FIG. 5 is another schematic structural diagram of a terminal device provided by an embodiment of the present application.

Detailed ways

The terms "first", "second" and the like in the description and claims of the present application and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged under appropriate circumstances, and this is only a distinguishing manner adopted when describing objects with the same attributes in the embodiments of the present application. Furthermore, the terms "comprising" and "having" and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, product or device comprising a series of elements is not necessarily limited to those elements, but may include no explicit or other units inherent to these processes, methods, products, or devices.

The embodiments of the present application may be implemented by a terminal device, and the terminal device may be a personal computer (personal computer, PC), a smart phone, a tablet computer (portable android device, PAD) and other devices. Such terminal devices are usually shipped with local input devices and local output devices connected. In order to meet the needs of some special scenarios, for example, when the user needs to control the terminal device from a distance, the local input devices and local output devices already provided by the terminal device are often unable to meet the needs, so the terminal device can also be connected to an external input device. and external output devices.

Specifically, the local input device is usually an input device provided by the terminal device. For example, when the terminal device is a PC, the local input device of the PC may include a wired mouse, a wired keyboard, and the like provided by the PC. For another example, when the terminal device is a smart phone, the local input device of the smart phone may include a keyboard (physical keypad or virtual numeric keypad, etc.), a camera, a microphone, and the like provided in the smart phone. Similarly, the local output device is usually an output device provided by the terminal device. For example, when the terminal device is a PC, the local output device of the PC may include a monitor, a speaker, and the like provided by the PC. For another example, when the terminal device is a smart phone, the local output device of the smart phone may include a display screen, a speaker, and the like built in the smart phone. In addition, the external input device is usually an input device additionally added (or additionally connected) to the terminal device, for example, a wired mouse, wired keyboard, wireless mouse, wireless keyboard, touch screen, camera, memory, etc. added to the terminal device. The external output device is usually an additional output device added to the terminal device, for example, a TV set, a projector, a printer, a speaker, etc., added to the terminal device.

Therefore, a terminal device can be connected with multiple input devices and multiple output devices. The terminal device can realize centralized management of multiple input devices and multiple output devices through the operating system. For the convenience of description and understanding, the following description will be given by taking the terminal device as a smart phone. It is assumed that the smart phone is connected with a local input device, an external input device, a local output device and an external output device, which are the keyboard of the mobile phone, the Bluetooth keyboard and mouse, the display screen of the mobile phone and the TV. When a user inputs a certain operation command to an application running by the operating system through a Bluetooth keyboard and mouse (it can also be the keyboard of a mobile phone, here only the Bluetooth keyboard and mouse are used as an example for schematic illustration), the operating system can make the display screen and the The TV can synchronously display the display screen data generated by the application based on the operation instructions, so the user can enjoy better visual effects on the TV.

Specifically, the operating system usually runs multiple applications, but the operating system generally focuses on the most front-end applications. Therefore, the operating instructions input by the user through the keyboard of the mobile phone and the Bluetooth keyboard and mouse will be sent to the application for processing by the operating system. The display screen data fed back by the application is sent to the display screen of the mobile phone and the TV for display. That is, the operating system will make all input and output devices serve the application. However, when the user actually operates the application, the devices that are effectively used are the Bluetooth keyboard, mouse and TV. At this time, if other users need to use the keyboard of the mobile phone and the display screen of the mobile phone to operate another application, the operating system cannot meet this requirement, resulting in a certain waste of equipment resources and a low utilization rate of the equipment.

In order to solve the above problem, the embodiments of the present application provide a new operating system implementation, so as to effectively utilize the device resources of the terminal device. For ease of understanding, the operating system provided by the embodiment of the present application is further introduced below with reference to FIG. 1 . FIG. 1 is a schematic structural diagram of a terminal device provided by an embodiment of the present application. As shown in FIG. 1 , an operating system of the terminal device includes three modules, namely a binding module, an input adapter, and an output adapter, and the operating system is loaded with Drivers for local input devices, drivers for external input devices, and display drivers. In addition, a plurality of applications (also referred to as application programs) also run on the operating system, and the terminal is connected with a local input device, an external input device, a local output device, and an external output device. Each of the aforementioned modules will be introduced separately below:

An application can be a piece of software or a cloud instance. For example, the application may be game software, office software, desktop cloud client, and so on. For another example, the application may be a virtual machine (virtual machine, VM), a bare metal server (bare metal server, BMS), an application container engine (docker), and the like.

The driver of the local input device is deployed in the operating system, and the driver of the local input device can be used to support the input function of the local input device, that is, the command input by the local input device can be sent to the driver of the local input device, and the driver of the local input device The instruction is sent to the operating system, and the operating system can process the instruction by itself or send the instruction to the application for processing, which will not be expanded here. Similarly, the driver of the external input device is also deployed in the operating system, and the driver of the external input device can be used to support the input function of the external input device.

The display driver is deployed in the operating system, and the display driver is used to support the output function of the local output device. When the terminal device is a PC, the operating system can send the display screen data fed back by the application to the display driver, and the display driver will then send the display screen data to the graphics card, and then the graphics card can complete graphics rendering based on the display screen data, and render the rendered image data. Graphics are output to a local output device (eg, a display) for display. When the terminal device is a smart phone, the operating system can send the display screen data fed back by the application to the display driver. Since the local output device of the mobile phone (for example, the display screen of the mobile phone) has a high degree of integration, the display driver can directly send the data to the display driver. The display screen data is sent to the local output device for display. For the convenience of drawing, FIG. 1 takes the terminal device as a PC as an example for schematic illustration, and does not limit the type of the terminal device in this application.

For the convenience of description, the process of the driver forwarding the instruction is omitted below. For example, "the local input device sends the instruction to the driver of the local input device, and the driver sends the instruction to the operating system" is equivalent to "the operating system receives the instruction from the local input device." Instructions” and so on, which will not be repeated in the following.

The binding module of the operating system can be used as a configuration entry for the correspondence between the application and the input and output devices (including the local input device, the external input device, the local output device and the external output device) during the application startup process. For example, during application startup, the binding module may provide the user with a user-interactive interface (eg, a visual interface) to receive selections entered by the user in the interface and send the selections to the input and output adapters, Therefore, the input adapter builds the corresponding relationship between the application and the input device based on the selection, and the output adapter builds the corresponding relationship between the application and the output device based on the selection. For another example, in the process of application startup, the binding module may receive the selection from the application, and send the selection to the input adapter and the output adapter, so that the input adapter builds the corresponding relationship between the application and the input device based on the selection, The output adapter builds the correspondence between the application and the output device based on the selection, and so on. In addition, the binding module can also be used as an entry for modifying the corresponding relationship between the application and the input and output devices. For example, after receiving the user-input selection for modifying the corresponding relationship, the binding module may send the selection to the input adapter and the output adapter, so that the input adapter can determine the corresponding relationship between the application and the input device based on the selection. Modification is performed, and the output adapter modifies the correspondence between the application and the output device based on the selection.

The input adapter of the operating system can establish the corresponding relationship between the application and the input device, and based on the corresponding relationship between the application and the input device, the operation instructions from the input device (for example, the keyboard event of the operation application, the mouse event of the operation application etc.) are sent to the corresponding application for processing. Generally, in an input adapter, an application can correspond to at least one input device (either a local input device or an external input device). For example, a certain game software on the mobile phone can correspond to an additional gamepad, so after the input adapter receives the operation command input by the gamepad, the input adapter can send the operation command to the game software for processing. Another example is that a certain office software of the PC can bind a wired mouse that comes with the PC and an additional wireless keyboard, etc., so when the input adapter receives the operation command input by the wired mouse and wireless keyboard, the input adapter can The operation instruction is sent to the office software for processing.

The output adapter of the operating system can establish the corresponding relationship applied to the output devices, and based on the corresponding relationship applied to the output devices, the display screen data from the application (for example, the data obtained after the application processes keyboard events or mouse events) etc.) to the corresponding output device for display. Generally, in an output adapter, an application can correspond to at least one output device (either a local output device or an external output device). For example, a certain game software on the mobile phone can correspond to an additional TV, so after the input adapter receives the display screen data from the game software, the input adapter can send the display screen data to the TV for display. Another example is that a certain office software of a mobile phone can be bound to a display screen of a mobile phone, etc. Therefore, after the input adapter receives the display screen data from the office software, the input adapter can send the display screen data to the display screen of the mobile phone for processing. show.

It is worth noting that the display screen data can be transmitted wirelessly between the output adapter and the external output device, so the operating system does not need to load the driver of the external output device. Specifically, the output adapter can send the display screen data from the application to an external output device for display based on the network video streaming technology or the image transmission technology. Among them, the network video streaming transmission technology may be a digital living network alliance (DLNA) technology, Miracast technology or other custom video transmission protocols, etc., and the image transmission technology may be a desktop image transmission technology and so on.

Based on the above brief introduction, it can be seen that the operating system provided by the embodiments of the present application can make different input and output devices serve different applications, so different users can operate different applications on different input and output devices of a terminal device at the same time , thereby improving the utilization rate of the equipment. In order to further understand the solution provided by the present application, further introduction will be made below with reference to FIG. 2 ( FIG. 2 is a schematic flowchart of an application control method for a terminal device provided by an embodiment of the present application). In this application, among the multiple applications running on the operating system of the terminal device, two of the applications are used as examples for introduction below. Specifically, the two applications may be referred to as a first application and a second application, respectively. Further, among the multiple input devices and multiple output devices connected to the terminal device, the input device and the output device bound to the first application are called the first input device and the first output device, and the input device and the output device bound to the second application are called the first input device and the first output device. The input device and output device of the application are called the second input device and the second output device, wherein the first input device can be either a local input device or an external input device, and the first output device can be either a local output device or a second output device. It is an external output device, the second input device can be either a local input device or an external input device, and the second output device can be either a local output device or an external output device. As shown in Figure 2, the method includes:

201. In the process of starting the first application, a user interaction interface is provided, and a first selection input by the user is received through the user interaction interface, where the first selection is used to indicate the first input device and the first output set by the user for the first application equipment.

When the user needs to use the first application on the terminal device, the first application on the terminal device may be activated. After the binding module of the operating system of the terminal device detects that the first application starts to be started, it can provide a user interaction interface to the user during the process of starting the first application, and the user interaction interface is used to respectively provide multiple inputs for at least two applications device and multiple output devices. Specifically, the binding module can send the user interaction interface to at least one unbound output device for display, and the user interaction interface can be presented as a visual interface that displays at least one unbound input device and at least one unbound input device. Options for unbound output devices for the user to choose from. It can be understood that, in a terminal device, unbound input devices and output devices refer to input devices and output devices that are not bound to an application, and correspondingly, input devices and output devices that have been bound to an application. Devices are bound input and output devices.

After the binding module provides the user interaction interface, the user can select a certain (or some) output devices and A certain (or some) input device is bound to the first application, that is, the first input device and the first output device are set for the first application. Specifically, the user can input the first selection to the user interaction interface through any unbound input device, and the first selection is used to indicate the first input device and the first output device (ie, the first selection) set by the user for the first application. including a first input device and a first output device). After receiving the first selection input by the user through the user interaction interface, the binding module forwards the first selection to the input adapter and the output adapter.

The above process will be further described below with reference to FIG. 3 . FIG. 3 is a schematic diagram of a user interaction interface provided by an embodiment of the present application. As shown in FIG. 3 , if a user starts a certain game software on a PC, the binding module can provide a visualization during the start-up process of the game software. interface that displays a Bluetooth keyboard, a universal serial bus (USB) keyboard, a Bluetooth mouse, a USB mouse, a TV (which can also be displayed as a network video stream), and a monitor (which can also be displayed as a network card) and other options. Therefore, the user can select the input device and the output device bound to the game software in the interface. When the user sets the Bluetooth keyboard, Bluetooth mouse and TV as devices bound to the game software in the interface, it is equivalent to inputting a selection into the interface, and the selection can be sent to the binding module through the interface.

202. According to the first selection, construct a correspondence between the first application and the first input device and a correspondence between the first application and the first output device.

After receiving the first selection, the binding module sends the first selection to the input adapter and the output adapter. After receiving the first selection, the input adapter can determine that the first input device needs to be bound to the first application, so the input adapter builds a correspondence between the first application and the first input device, and stores the correspondence, which is is equivalent to successfully binding the first input device to the first application. Similarly, after receiving the first selection, the output adapter may determine that the first output device needs to be bound to the first application, so the output adapter builds a correspondence between the first application and the first output device, and stores the correspondence The relationship, at this time, is equivalent to successfully binding the first output device to the first application.

Still as in the above example, after the binding module forwards the selection to the input adapter and the output adapter, the input adapter can determine that the Bluetooth keyboard and the Bluetooth mouse need to be bound to the game software, and then build a correspondence between the Bluetooth keyboard and the game software. relationship, and the corresponding relationship between the Bluetooth mouse and the game software. Similarly, the output adapter can determine that the TV needs to be bound to the game software, and then establish a corresponding relationship between the TV and the game software. In this way, the bluetooth keyboard, bluetooth mouse and TV are successfully bound to the game software.

It should be noted that the corresponding relationship in this embodiment may be presented in the form of various data structures, such as a Map data structure, etc., which is not specifically limited here.

203. When the first application has been started, run the first application according to the first operation instruction obtained from the first input device, and send display screen data generated by running the first application to the first output device for display.

In the case that the first application has been started (that is, the first application has been started), the user can operate the first application through the first input device and the first output device. Specifically, when the first application has been started, if the user needs to operate the first application, the user may first send a first operation instruction to the input adapter through the first input device. After receiving the first operation instruction from the first input device, the input adapter sends the first operation instruction to the first application based on the correspondence between the first input device and the first application, so that the first application can The operation command is processed. After the first application completes the processing of the first operation instruction, it can generate corresponding display screen data, and send the display screen data to the output adapter. Finally, the output adapter sends the display screen data to the first output device for display based on the corresponding relationship between the first output device and the first application. In this way, when the user operates the first application through the first input device and the first output device, the remaining input devices and output devices of the terminal device are in an idle state, that is, the remaining input devices and output devices of the terminal device can serve other applications. , which can effectively utilize the input and output device resources of the terminal device.

Still like the above example, after successfully binding the Bluetooth keyboard, Bluetooth mouse and TV to the game software, if the user starts to send operation commands through the Bluetooth keyboard and Bluetooth mouse (hereinafter collectively referred to as the Bluetooth keyboard and mouse), the input adapter can receive input from the game software. The operation instructions of the Bluetooth keyboard and mouse are sent to the game software based on the corresponding relationship between the Bluetooth keyboard and mouse and the game software. After the game software processes the operation instructions, it can generate corresponding display screen data and send the display screen data to the output adapter. Finally, based on the corresponding relationship between the TV and the game software, the output adapter sends the display screen data to the TV for display, thereby displaying the game screen on the TV. At this point, the PC's USB keyboard, USB keyboard and monitor are unbound devices, so the PC's monitor will not display the game screen.

For another example, suppose the smartphone has its own keyboard and display screen, and is connected with additional devices such as a Bluetooth keyboard and mouse and a TV, and the Bluetooth keyboard, mouse and TV are bound to the desktop cloud client running on the smartphone. When the user needs to operate the desktop cloud client, they can send the operation command through the Bluetooth keyboard and mouse, and the input adapter can receive the operation command from the Bluetooth keyboard and mouse, and send the operation command based on the corresponding relationship between the Bluetooth keyboard and mouse and the game software. to the desktop cloud client. The desktop cloud client sends the operation instruction to the remote desktop cloud server (located outside the terminal device), and the desktop cloud server can return the display screen data to the desktop cloud client after processing the operation instruction. The desktop cloud client sends the display data to the output adapter. Finally, based on the corresponding relationship between the TV and the game software, the output adapter sends the display screen data to the TV for display, thereby displaying the screen of the desktop cloud client on the TV. In this way, the user can operate the desktop cloud client through a Bluetooth keyboard, mouse and TV, making the use of the desktop cloud very simple and convenient. At this time, the keyboard and display screen of the smartphone are unbound devices, that is, the smartphone can be used normally, and the display screen of the smartphone will not display the screen of the desktop cloud client.

204. In the process of starting the second application, provide a user interaction interface, and receive a second selection input by the user through the user interaction interface, where the second selection is used to indicate the second input device and the second output set by the user for the second application equipment.

205. According to the second selection, construct the correspondence between the second application and the second input device and the correspondence between the second application and the second output device.

206. In the case that the second application has been started, run the second application according to the second operation instruction obtained from the second input device, and send display screen data generated by running the second application to the second output device for display.

On the basis that the first application is already running, if the user still needs to use the second application, the second application on the terminal device may be started. Specifically, for the description of steps 204 to 206, reference may be made to the above-mentioned relevant descriptions of steps 201 to 203, which will not be repeated here. It should be noted that, step 203 and step 206 may be repeatedly performed, so step 203 and step 206 may be performed simultaneously, that is, the operating system of the terminal device may run the first application and the second application at the same time. In this way, while a user operates the first application through the first input device and the first output device, another user can operate the second application through the second input device and the second output device, that is, on the same terminal device. The operating system can meet the operation needs of different users for different applications at the same time.

Still in the example shown in Figure 3, if another user starts a certain office software on the PC, in the process of starting the office software, the binding module can provide a visual interface, which displays the USB keyboard, Options such as USB mouse and monitor (since the Bluetooth keyboard, Bluetooth mouse and monitor are all bound devices, these devices are no longer displayed in the visual interface). Therefore, the user can select the input device and the output device bound to the office software in the interface. When the user sets the USB keyboard, USB mouse and monitor as devices bound to the office software in this interface, it is equivalent to inputting a selection into this interface, and the selection can be sent to the binding module through this interface.

After the binding module forwards the selection to the input adapter and the output adapter, the input adapter can determine that the USB keyboard and USB mouse need to be bound to the office software, and then builds the corresponding relationship between the USB keyboard and the office software, and the USB mouse Correspondence with the office software. Likewise, the output adapter can determine that the monitor needs to be bound to the office software, and then establish a corresponding relationship between the monitor and the office software. In this way, the USB keyboard, USB mouse and monitor are successfully bound to the office software.

After successfully binding the USB keyboard, USB mouse and monitor to the office software, if another user starts to send operation commands through the USB keyboard and USB mouse (hereinafter collectively referred to as USB keyboard and mouse), the input adapter can receive the data from the USB keyboard and mouse. operation instructions, and based on the corresponding relationship between the USB keyboard and mouse and the office software, the operation instructions are sent to the office software. After the office software processes the operation instructions, it can generate corresponding display screen data and send the display screen data to the output adapter. Finally, based on the correspondence between the display and the office software, the output adapter sends the display image data to the display for display, thereby displaying the office image on the display.

Therefore, on the same terminal device, one user can operate the game software through the Bluetooth keyboard and mouse and the TV, and at the same time, another user can operate the office software through the USB keyboard and mouse and the monitor, and the two do not interfere with each other. In this way, the same terminal device can realize the game operation of one user and the office operation of another user at the same time.

207. Provide the user interaction interface again when the first application has been started, and receive a third selection input by the user through the user interaction interface, where the third selection is used to indicate the third input device and the third input device set by the user for the first application. Three output devices.

After steps 201 to 203, the first input device and the first output device are bound to the first application. If the user needs to modify the input/output device bound to the first application, a trigger instruction may be sent to the binding module through the first input device. After receiving the trigger instruction, the binding module can provide the user interaction interface again, that is, the binding module sends the user interaction interface to the first output device for display. At this point, the user interaction interface may display options such as remaining unbound input devices and remaining unbound output devices for the user to select.

After the binding module provides the user interaction interface, the user can select a certain (or some) output device and a certain (or some) input device to bind to the first application in the options displayed on the user interaction interface, that is, The first application sets the third input device and the third output device, thereby replacing the first input device and the first output device that were originally bound to the first application. Specifically, the user may input a third selection to the user interaction interface through the first input device, and the third selection is used to indicate the third input device and the third output device set by the user for the first application. After receiving the third selection input by the user through the user interaction interface, the binding module forwards the third selection to the input adapter and the output adapter.

208. According to the third selection, modify the correspondence between the first application and the first input device to the correspondence between the first application and the third input device, and change the correspondence between the first application and the first output device. The corresponding relationship is modified to the corresponding relationship between the first application and the third output device.

After receiving the third selection, the input adapter can determine that the third input device needs to be bound to the first application, and then modify the corresponding relationship between the first application and the first input device to the relationship between the first application and the third input device. Correspondence between. Similarly, after the output adapter receives the third selection, it can determine that the third output device needs to be bound to the first application, and then modify the correspondence between the first application and the first output device to the first application and the third output device. Correspondence between output devices. At this time, it is equivalent to successfully binding the third output device and the third input device to the first application.

209. Run the first application according to the third operation instruction obtained from the third input device, and send display screen data generated by running the first application to the third output device for display.

If the user needs to continue to operate the first application, the user needs to operate the first application through the third input device and the third output device. For the process of operating the first application through the third input device and the third output device, reference may be made to the relevant description part of operating the first application through the first input device and the first output device in step 203, which will not be repeated here.

In this embodiment, the operating system of the terminal device may provide a user interaction interface, and the user interaction interface is used to respectively provide multiple input devices and multiple output devices for at least two applications. The first input device and the first output device are bound to the first application, so that the first input device and the first output device serve the first application. When the first application has been started, the operating system can run the first application according to the operation instruction from the first input device, and send display screen data generated by running the first application to the first output device for display. Then, the remaining input devices except the first input device and the remaining output devices except the first output device can serve other applications, thereby effectively utilizing device resources and improving device utilization.

The above is a detailed description of the application control method of the terminal device provided by the embodiment of the present application, and the application control apparatus of the terminal device provided by the embodiment of the present application will be introduced below. FIG. 4 is a schematic structural diagram of an application control apparatus of a terminal device provided by an embodiment of the present application. Please refer to FIG. 4 . The apparatus includes:

The interaction module 401 is configured to provide a user interaction interface, where the user interaction interface is used to respectively provide options of multiple input devices and multiple output devices for at least two applications of the terminal device, and receive a user's first selection of the terminal device based on the options. a first selection of application settings, where the first selection includes a first input device and a first output device;

The running module 402 is configured to, when the first application starts, determine the first input device and the first output device of the first application according to the first selection, run the first application according to the instruction obtained from the first input device, and The display screen data generated by running the first application is sent to the first output device for display.

In a possible implementation manner, the user interaction interface is further configured to receive a second selection set by the user for the second application of the terminal device based on the option, where the second selection includes a second input device and a second output device, and the operation module 402 It is also used to determine the second input device and the second output device of the second application according to the second selection when the second application is started, run the second application according to the instruction obtained from the second input device, and run the second application. The display screen generated by the application is sent to the second input device for display.

In a possible implementation manner, the user interaction interface is further configured to receive a third selection set by the user for the first application of the terminal device based on the option. The third selection includes a third input device and a third output device, and the module 402 is executed. It is also used to determine the third input device and the third output device of the first application according to the third selection when the first application is started, run the first application according to the instruction obtained from the third input device, and run the first application. The display screen generated by the application is sent to the third input device for display.

It should be noted that the information exchange, execution process and other contents among the modules/units of the above-mentioned apparatus are based on the same concept as the method embodiments of the present application, and the technical effects brought by them are the same as those of the method embodiments of the present application, and the specific contents can be Reference is made to the descriptions in the method embodiments shown in the foregoing embodiments of the present application, which will not be repeated here.

FIG. 5 is another schematic structural diagram of a terminal device provided by an embodiment of the present application. As shown in FIG. 5 , one embodiment of the offload card in this embodiment of the present application may include one or more central processing units 501 , a memory 502 , an input/output interface 503 , a wired or wireless network interface 504 , and a power supply 505 .

The memory 502 may be ephemeral storage or persistent storage. Furthermore, the central processing unit 501 may be configured to communicate with the memory 502 to execute a series of instruction operations in the memory 502 on the offload card.

In this embodiment, the central processing unit 501 may perform the operations performed by the operating system of the terminal device in the foregoing embodiment shown in FIG. 2 , and details are not repeated here.

In this embodiment, the division of specific functional modules in the central processing unit 501 may be similar to the division of modules such as binding modules, input adapters, and output adapters described in FIG. 2 , which will not be repeated here.

In this embodiment, the division of specific functional modules in the central processing unit 501 may also be similar to the division of modules such as the interaction module and the operation module described in FIG. 4 , which is not repeated here.

Embodiments of the present application also relate to a computer storage medium, including computer-readable instructions, when the computer-readable instructions are executed, the method described in FIG. 2 is implemented.

Embodiments of the present application also relate to a computer program product containing instructions, which, when run on a computer, cause the computer to execute the method described in FIG. 2 .

Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system, device and unit described above may refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

In the several embodiments provided in this application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the apparatus embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units.

The integrated unit, if implemented in the form of a software functional unit and sold or used as an independent product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the present application can be embodied in the form of software products in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, and the computer software products are stored in a storage medium , including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, Read-Only Memory (ROM, Read-Only Memory), Random Access Memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program codes .

Claims

An application control method for a terminal device, characterized in that the method includes:

Provide a user interaction interface, the user interaction interface is used to respectively provide options of multiple input devices and multiple output devices for at least two applications of the terminal device, and receive the user as the first application of the terminal device based on the options a first selection of settings, the first selection including a first input device and a first output device;

When the first application is started, the first input device and the first output device of the first application are determined according to the first selection, and run according to an instruction acquired from the first input device the first application, and send display screen data generated by running the first application to the first output device for display.
The method according to claim 1, wherein the user interaction interface is further configured to receive a second selection set by the user for the second application of the terminal device based on the options, the second selection comprising: a second input device and a second output device, the method further comprising:

When the second application is started, the second input device and the second output device of the second application are determined according to the second selection, and run according to an instruction obtained from the second input device the second application, and send the display screen generated by running the second application to the second input device for display.
The method according to claim 2, wherein the first application and the second application run simultaneously in the terminal device.
The method according to claim 2 or 3, wherein the first input device is one of a local input device of the terminal device and an external input device of the terminal device;

The second input device is the other of a local input device of the terminal device and an external input device of the terminal device;

The first output device is one of a local output device of the terminal device and an external output device of the terminal device;

The second output device is the other of a local output device of the terminal device and an external output device of the terminal device.
The method according to claim 4, wherein the first input device comprises a first keyboard device and a first mouse device;

The second input device includes a second keyboard device and a second mouse device.
The method according to claim 4, wherein the local output device is an output device provided by the terminal device, and the external output device is an output device connected to the terminal device.
The method according to any one of claims 1 to 6, wherein the user interaction interface is further configured to receive a third selection set by the user for the first application of the terminal device based on the options, the The third selection includes a third input device and a third output device, and the method further includes:

When the first application is started, the third input device and the third output device of the first application are determined according to the third selection, and run according to an instruction obtained from the third input device the first application, and send the display screen generated by running the first application to the third input device for display.
An application control device for terminal equipment, characterized in that the device includes:

An interaction module, configured to provide a user interaction interface, the user interaction interface is used to respectively provide a selection of multiple input devices and multiple output devices for at least two applications of the terminal device, and receive the user for the terminal based on the options. a first selection of a first application setting of a device, the first selection including a first input device and a first output device;

an operation module, configured to determine the first input device and the first output device of the first application according to the first selection when the first application is started, and determine the first input device and the first output device of the first application according to the first selection. The instruction obtained by the device runs the first application, and sends display screen data generated by running the first application to the first output device for display.
The apparatus according to claim 8, wherein the user interaction interface is further configured to receive a second selection set by the user for the second application of the terminal device based on the selection, the second selection comprising: a second input device and a second output device, and the running module is further configured to determine the second input device and the second input device of the second application according to the second selection when the second application is started The second output device runs the second application according to the instruction obtained from the second input device, and sends a display screen generated by running the second application to the second input device for display.
The apparatus according to claim 9, wherein the first application and the second application run simultaneously in the terminal device.
The apparatus according to claim 9 or 10, wherein the first input device is one of a local input device of the terminal device and an external input device of the terminal device;

The second input device is the other of a local input device of the terminal device and an external input device of the terminal device;

The first output device is one of a local output device of the terminal device and an external output device of the terminal device;

The second output device is the other of a local output device of the terminal device and an external output device of the terminal device.
The apparatus according to claim 11, wherein the first input device comprises a first keyboard device and a first mouse device;

The second input device includes a second keyboard device and a second mouse device.
The apparatus according to claim 11, wherein the local output device is an output device provided by the terminal device, and the external output device is an output device connected to the terminal device.
The apparatus according to any one of claims 8 to 13, wherein the user interaction interface is further configured to receive a third selection set by the user for the first application of the terminal device based on the options, the The third selection includes a third input device and a third output device, and the running module is further configured to determine the third selection of the first application according to the third selection when the first application is started The input device and the third output device run the first application according to the instruction obtained from the third input device, and send a display screen generated by running the first application to the third input device for display.
A terminal device, characterized in that the terminal device comprises: a processor and a memory;

the memory is used to store computer-executable instructions;

The processor is configured to execute computer-implemented instructions stored in the memory to cause the communication device to implement the method of any one of claims 1 to 7.
A computer storage medium, characterized by comprising computer-readable instructions, which, when executed, implement the method according to any one of claims 1 to 7.
A computer program product comprising instructions which, when run on a computer, cause the computer to perform the method of any one of claims 1 to 7.