WO2022111732A1 - Procédé de commande d'applications pour dispositif terminal et dispositif associé - Google Patents

Procédé de commande d'applications pour dispositif terminal et dispositif associé Download PDF

Info

Publication number
WO2022111732A1
WO2022111732A1 PCT/CN2021/134494 CN2021134494W WO2022111732A1 WO 2022111732 A1 WO2022111732 A1 WO 2022111732A1 CN 2021134494 W CN2021134494 W CN 2021134494W WO 2022111732 A1 WO2022111732 A1 WO 2022111732A1
Authority
WO
WIPO (PCT)
Prior art keywords
application
input device
input
output
terminal device
Prior art date
Application number
PCT/CN2021/134494
Other languages
English (en)
Chinese (zh)
Inventor
陈普
Original Assignee
华为云计算技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为云计算技术有限公司 filed Critical 华为云计算技术有限公司
Publication of WO2022111732A1 publication Critical patent/WO2022111732A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/451—Execution arrangements for user interfaces
    • G06F9/452—Remote windowing, e.g. X-Window System, desktop virtualisation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/1454—Digital output to display device ; Cooperation and interconnection of the display device with other functional units involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/445—Program loading or initiating
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/445—Program loading or initiating
    • G06F9/44505—Configuring for program initiating, e.g. using registry, configuration files
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/451—Execution arrangements for user interfaces

Definitions

  • the present application relates to the field of computer technologies, and in particular, to an application control method for a terminal device and related devices.
  • the terminal device itself usually has a set of input devices (for example, a built-in keyboard and mouse, etc.) and output devices (for example, a display, etc.), through which the user can operate various components on the terminal device. application.
  • input devices for example, a built-in keyboard and mouse, etc.
  • output devices for example, a display, etc.
  • the set of input device and output device can be connected to the terminal device.
  • the user needs to perform a series of operations on the operating system of the terminal device, so as to connect the terminal device to the set of input devices and output devices, and then the input device can be used to control the terminal device, and the terminal device outputs to the output device, such as display Output signals such as image and/or sound.
  • the terminal device is bound to the external input device and output device, and all applications on the terminal device must use the set of input devices and output devices, resulting in poor user experience.
  • Embodiments of the present application provide an application control method for a terminal device and related devices, which can effectively utilize device resources and improve device utilization.
  • a first aspect of the embodiments of the present application provides an application control method for a terminal device, and the method includes:
  • the terminal device On the operating system of the terminal device, multiple applications, such as game software, office software, and the like, may run. Further, the terminal device is connected with multiple input devices and multiple output devices. For example, the terminal device is connected with multiple input devices such as a mouse, a keyboard, and a camera, and the terminal device is also connected with multiple output devices such as a monitor and a TV. The operating system of the terminal device can centrally manage multiple input devices and multiple output devices to allocate these devices to different applications.
  • the operating system of the terminal device may first provide a user interaction interface, where the user interaction interface is used to respectively provide options of multiple input devices and multiple output devices for at least two applications, and receive user settings for the first application based on the options.
  • the first selection includes a first input device and a first output device.
  • the operating system of the terminal device determines the first input device and the first output device of the first application according to the first selection, and runs the first application according to the instruction obtained from the first input device, The display screen data generated by running the first application is sent to the first output device for display.
  • a smartphone has its own keyboard and display and is connected to additional devices such as gamepads and TVs.
  • the operating system of a smartphone can provide the user with a visual interface that includes options such as the phone's keyboard, display, gamepad, and TV for the user to choose from.
  • the user can set the gamepad and the TV set to a certain game software on the visual interface, so that the gamepad and the TV set exclusively serve the game software.
  • the user can input operation instructions through the gamepad, and the operating system can run the game software based on the operation instructions from the gamepad, and send display screen data generated by running the game software to the TV for display.
  • the display screen of the smart phone will not display the display screen data generated by running the game software, that is, the keyboard of the smart phone and the display screen of the smart phone can serve other applications.
  • the operating system of the terminal device can provide a user interaction interface, and the user interaction interface is used to provide options of multiple input devices and multiple output devices respectively for at least two applications, so the user can A first selection is input on the interactive interface, and the operating system binds the first input device and the first output device to the first application based on the first selection, so that the first input device and the first output device serve the first application.
  • the operating system can run the first application according to the operation instruction from the first input device, and send display screen data generated by running the first application to the first output device for display.
  • the user only needs to input the first selection on the user interaction interface provided by the operating system, and the first input device and the first output device can be bound to the first application, that is, the process only requires the user to perform a simple The operation does not need to perform multiple complex operations, which can improve the user experience.
  • the remaining input devices except the first input device and the remaining output devices except the first output device can serve other applications, thereby effectively utilizing device resources and improving device utilization.
  • the user interaction interface is further configured to receive a second selection set by the user for the second application of the terminal device based on the option, where the second selection includes a second input device and a second output device, and the method further Including: when the second application is started, determining the second input device and the second output device of the second application according to the second selection, running the second application according to the instruction obtained from the second input device, and running the second application
  • the generated display screen is sent to the second input device for display.
  • the user can set the keyboard of the smartphone and the display of the smartphone to a certain office software on the visual interface provided by the operating system, so that the keyboard of the smartphone and the display of the smartphone are specially designed for the office. software service.
  • the user can input operating instructions through the keyboard of the smart phone, and the operating system can run the office software based on the operating instructions from the keyboard of the smart phone, and send the display screen data generated by running the office software to the display screen of the smart phone for display.
  • the user may also input a second selection on the user interaction interface, and the operating system binds the second input device and the second output device to the second application based on the second selection, so that the second input device and the first The second output device serves the second application.
  • the operating system can run the second application according to the operation instruction from the second input device, and send display screen data generated by running the second application to the second output device for display.
  • the user can make the first input device and the first output device serve the first application, and make the second input device and the second output device serve the second application, thereby effectively utilizing each input device and each output of the terminal device. equipment to improve the utilization of equipment.
  • the first application and the second application run simultaneously in the terminal device.
  • the operating system of the terminal device can run the first application and the second application at the same time, that is, while a user can operate the first application through the first input device and the first output device, another user can also operate the first application through the first input device and the first output device.
  • the second input device and the second output device operate the second application, so the operation requirements of different users on different applications on the same terminal device can be met.
  • the operating system receives an operation instruction from the gamepad, it sends the operation instruction to the game software for processing, and sends the display screen data fed back by the game software to the TV, so as to display the game screen on the TV.
  • the operating system also receives an operation instruction from the keyboard of the mobile phone, the operation instruction is sent to the office software for processing, and the display screen data fed back by the office software is sent to the display screen of the mobile phone, so that the display screen of the mobile phone can be displayed on the mobile phone.
  • the office screen is displayed on the display.
  • the user interaction interface is further configured to receive a third selection set by the user for the first application of the terminal device based on the option, where the third selection includes a third input device and a third output device, and the method further Including: when the first application is started, determining the third input device and the third output device of the first application according to the third selection, running the first application according to the instruction obtained from the third input device, and running the first application The generated display screen is sent to the third input device for display.
  • the user may also input a third selection on the user interaction interface, and the operating system binds the third input device and the third output device to the first application based on the third selection, so that the third input device and the third The three output devices serve the first application, thereby replacing the first input device and the first output device already bound to the first application.
  • the operating system can run the first application according to the operation instruction from the third input device, and send the display screen data generated by running the first application to the third output device for display. Therefore, the user can modify the input and output devices bound to the first application according to actual needs, thereby improving the comprehensiveness of the solution.
  • the first input device is one of a local input device of the terminal device and an external input device of the terminal device;
  • the second input device is a local input device of the terminal device and an external input device of the terminal device
  • the other of the first output device is one of the local output device of the terminal device and the external output device of the terminal device;
  • the second output device is the other of the local output device of the terminal device and the external output device of the terminal device.
  • the first input device includes a first keyboard device and a first mouse device
  • the second input device includes a second keyboard device and a second mouse device.
  • the local output device is an output device provided by the terminal device
  • the external output device is an output device connected to the terminal device.
  • a second aspect of the embodiments of the present application provides an application control device for a terminal device, the device includes: an interaction module configured to provide a user interaction interface, where the user interaction interface is configured to provide multiple an option of an input device and a plurality of output devices, and receive a first selection set by the user for the first application of the terminal device based on the option, where the first selection includes the first input device and the first output device; the operation module is used for the first selection in the first application.
  • an application determine the first input device and the first output device of the first application according to the first selection, run the first application according to the instruction obtained from the first input device, and execute the display generated by the first application
  • the screen data is sent to the first output device for display.
  • the operating system of the terminal device can provide a user interaction interface, and the user interaction interface is used to respectively provide multiple input devices and multiple output devices for at least two applications, so the user can use the user interaction interface
  • the operating system binds the first input device and the first output device to the first application based on the first selection, so that the first input device and the first output device serve the first application.
  • the operating system can run the first application according to the operation instruction from the first input device, and send display screen data generated by running the first application to the first output device for display. Then, the remaining input devices except the first input device and the remaining output devices except the first output device can serve other applications, thereby effectively utilizing device resources and improving device utilization.
  • the user interaction interface is further configured to receive a second selection set by the user for the second application of the terminal device based on the option, the second selection includes a second input device and a second output device, and the running module further When the second application is started, determine the second input device and the second output device of the second application according to the second selection, run the second application according to the instruction obtained from the second input device, and will run the second application The generated display screen is sent to the second input device for display.
  • the first application and the second application run simultaneously in the terminal device.
  • the first input device is one of a local input device of the terminal device and an external input device of the terminal device;
  • the second input device is a local input device of the terminal device and an external input device of the terminal device
  • the other of the first output device is one of the local output device of the terminal device and the external output device of the terminal device;
  • the second output device is the other of the local output device of the terminal device and the external output device of the terminal device.
  • the first input device includes a first keyboard device and a first mouse device
  • the second input device includes a second keyboard device and a second mouse device.
  • the local output device is an output device provided by the terminal device
  • the external output device is an output device connected to the terminal device.
  • the user interaction interface is further configured to receive a third selection set by the user for the first application of the terminal device based on the option, the third selection includes a third input device and a third output device, and the running module further When the first application is started, determine the third input device and the third output device of the first application according to the third selection, run the first application according to the instruction obtained from the third input device, and run the first application The generated display screen is sent to the third input device for display.
  • a third aspect of the embodiments of the present application provides a terminal device, the terminal device includes: a processor and a memory; the memory is used for storing computer-executed instructions; the processor is used for executing the computer-executed instructions stored in the memory, so that the communication apparatus realizes the following: The method described in the first aspect or any possible implementation manner of the first aspect.
  • a fourth aspect of the embodiments of the present application provides a computer storage medium, which is characterized in that it includes computer-readable instructions, and when the computer-readable instructions are executed, the first aspect or any one of the possibilities of the first aspect is implemented. implement the method described.
  • a fifth aspect of the embodiments of the present application provides a computer program product containing instructions, which, when run on a computer, causes the computer to execute the method described in the first aspect or any possible implementation manner of the first aspect .
  • the operating system of the terminal device may provide a user interaction interface, and the user interaction interface is used to respectively provide multiple input devices and multiple output devices for at least two applications, so the user can use the user interaction interface on the user interaction interface.
  • the first selection is input, and the operating system binds the first input device and the first output device to the first application based on the first selection, so that the first input device and the first output device serve the first application.
  • the operating system can run the first application according to the operation instruction from the first input device, and send display screen data generated by running the first application to the first output device for display. Then, the remaining input devices except the first input device and the remaining output devices except the first output device can serve other applications, thereby effectively utilizing device resources and improving device utilization.
  • FIG. 1 is a schematic structural diagram of a terminal device provided by an embodiment of the present application.
  • FIG. 2 is a schematic flowchart of an application control method for a terminal device provided by an embodiment of the present application
  • FIG. 3 is a schematic diagram of a user interaction interface provided by an embodiment of the present application.
  • FIG. 4 is a schematic structural diagram of an application control apparatus of a terminal device provided by an embodiment of the present application.
  • FIG. 5 is another schematic structural diagram of a terminal device provided by an embodiment of the present application.
  • Embodiments of the present application provide an application control method for a terminal device and related devices, which can effectively utilize device resources and improve device utilization.
  • the embodiments of the present application may be implemented by a terminal device, and the terminal device may be a personal computer (personal computer, PC), a smart phone, a tablet computer (portable android device, PAD) and other devices.
  • Such terminal devices are usually shipped with local input devices and local output devices connected.
  • the terminal device can also be connected to an external input device. and external output devices.
  • the local input device is usually an input device provided by the terminal device.
  • the terminal device when the terminal device is a PC, the local input device of the PC may include a wired mouse, a wired keyboard, and the like provided by the PC.
  • the terminal device when the terminal device is a smart phone, the local input device of the smart phone may include a keyboard (physical keypad or virtual numeric keypad, etc.), a camera, a microphone, and the like provided in the smart phone.
  • the local output device is usually an output device provided by the terminal device.
  • the local output device of the PC may include a monitor, a speaker, and the like provided by the PC.
  • the local output device of the smart phone may include a display screen, a speaker, and the like built in the smart phone.
  • the external input device is usually an input device additionally added (or additionally connected) to the terminal device, for example, a wired mouse, wired keyboard, wireless mouse, wireless keyboard, touch screen, camera, memory, etc. added to the terminal device.
  • the external output device is usually an additional output device added to the terminal device, for example, a TV set, a projector, a printer, a speaker, etc., added to the terminal device.
  • a terminal device can be connected with multiple input devices and multiple output devices.
  • the terminal device can realize centralized management of multiple input devices and multiple output devices through the operating system.
  • the terminal device For the convenience of description and understanding, the following description will be given by taking the terminal device as a smart phone. It is assumed that the smart phone is connected with a local input device, an external input device, a local output device and an external output device, which are the keyboard of the mobile phone, the Bluetooth keyboard and mouse, the display screen of the mobile phone and the TV.
  • the operating system can make the display screen and the The TV can synchronously display the display screen data generated by the application based on the operation instructions, so the user can enjoy better visual effects on the TV.
  • the operating system usually runs multiple applications, but the operating system generally focuses on the most front-end applications. Therefore, the operating instructions input by the user through the keyboard of the mobile phone and the Bluetooth keyboard and mouse will be sent to the application for processing by the operating system.
  • the display screen data fed back by the application is sent to the display screen of the mobile phone and the TV for display. That is, the operating system will make all input and output devices serve the application.
  • the devices that are effectively used are the Bluetooth keyboard, mouse and TV. At this time, if other users need to use the keyboard of the mobile phone and the display screen of the mobile phone to operate another application, the operating system cannot meet this requirement, resulting in a certain waste of equipment resources and a low utilization rate of the equipment.
  • FIG. 1 is a schematic structural diagram of a terminal device provided by an embodiment of the present application.
  • an operating system of the terminal device includes three modules, namely a binding module, an input adapter, and an output adapter, and the operating system is loaded with Drivers for local input devices, drivers for external input devices, and display drivers.
  • a plurality of applications also run on the operating system, and the terminal is connected with a local input device, an external input device, a local output device, and an external output device.
  • applications also referred to as application programs
  • the terminal is connected with a local input device, an external input device, a local output device, and an external output device.
  • An application can be a piece of software or a cloud instance.
  • the application may be game software, office software, desktop cloud client, and so on.
  • the application may be a virtual machine (virtual machine, VM), a bare metal server (bare metal server, BMS), an application container engine (docker), and the like.
  • the driver of the local input device is deployed in the operating system, and the driver of the local input device can be used to support the input function of the local input device, that is, the command input by the local input device can be sent to the driver of the local input device, and the driver of the local input device
  • the instruction is sent to the operating system, and the operating system can process the instruction by itself or send the instruction to the application for processing, which will not be expanded here.
  • the driver of the external input device is also deployed in the operating system, and the driver of the external input device can be used to support the input function of the external input device.
  • the display driver is deployed in the operating system, and the display driver is used to support the output function of the local output device.
  • the operating system can send the display screen data fed back by the application to the display driver, and the display driver will then send the display screen data to the graphics card, and then the graphics card can complete graphics rendering based on the display screen data, and render the rendered image data.
  • Graphics are output to a local output device (eg, a display) for display.
  • the terminal device is a smart phone
  • the operating system can send the display screen data fed back by the application to the display driver. Since the local output device of the mobile phone (for example, the display screen of the mobile phone) has a high degree of integration, the display driver can directly send the data to the display driver.
  • the display screen data is sent to the local output device for display.
  • FIG. 1 takes the terminal device as a PC as an example for schematic illustration, and does not limit the type of the terminal device in this application.
  • the process of the driver forwarding the instruction is omitted below.
  • the local input device sends the instruction to the driver of the local input device, and the driver sends the instruction to the operating system
  • the operating system receives the instruction from the local input device. Instructions” and so on, which will not be repeated in the following.
  • the binding module of the operating system can be used as a configuration entry for the correspondence between the application and the input and output devices (including the local input device, the external input device, the local output device and the external output device) during the application startup process.
  • the binding module may provide the user with a user-interactive interface (eg, a visual interface) to receive selections entered by the user in the interface and send the selections to the input and output adapters, Therefore, the input adapter builds the corresponding relationship between the application and the input device based on the selection, and the output adapter builds the corresponding relationship between the application and the output device based on the selection.
  • the binding module may receive the selection from the application, and send the selection to the input adapter and the output adapter, so that the input adapter builds the corresponding relationship between the application and the input device based on the selection,
  • the output adapter builds the correspondence between the application and the output device based on the selection, and so on.
  • the binding module can also be used as an entry for modifying the corresponding relationship between the application and the input and output devices. For example, after receiving the user-input selection for modifying the corresponding relationship, the binding module may send the selection to the input adapter and the output adapter, so that the input adapter can determine the corresponding relationship between the application and the input device based on the selection. Modification is performed, and the output adapter modifies the correspondence between the application and the output device based on the selection.
  • the input adapter of the operating system can establish the corresponding relationship between the application and the input device, and based on the corresponding relationship between the application and the input device, the operation instructions from the input device (for example, the keyboard event of the operation application, the mouse event of the operation application etc.) are sent to the corresponding application for processing.
  • an application can correspond to at least one input device (either a local input device or an external input device).
  • a certain game software on the mobile phone can correspond to an additional gamepad, so after the input adapter receives the operation command input by the gamepad, the input adapter can send the operation command to the game software for processing.
  • a certain office software of the PC can bind a wired mouse that comes with the PC and an additional wireless keyboard, etc., so when the input adapter receives the operation command input by the wired mouse and wireless keyboard, the input adapter can The operation instruction is sent to the office software for processing.
  • the output adapter of the operating system can establish the corresponding relationship applied to the output devices, and based on the corresponding relationship applied to the output devices, the display screen data from the application (for example, the data obtained after the application processes keyboard events or mouse events) etc.) to the corresponding output device for display.
  • an application can correspond to at least one output device (either a local output device or an external output device).
  • a certain game software on the mobile phone can correspond to an additional TV, so after the input adapter receives the display screen data from the game software, the input adapter can send the display screen data to the TV for display.
  • a certain office software of a mobile phone can be bound to a display screen of a mobile phone, etc. Therefore, after the input adapter receives the display screen data from the office software, the input adapter can send the display screen data to the display screen of the mobile phone for processing. show.
  • the display screen data can be transmitted wirelessly between the output adapter and the external output device, so the operating system does not need to load the driver of the external output device.
  • the output adapter can send the display screen data from the application to an external output device for display based on the network video streaming technology or the image transmission technology.
  • the network video streaming transmission technology may be a digital living network alliance (DLNA) technology, Miracast technology or other custom video transmission protocols, etc.
  • the image transmission technology may be a desktop image transmission technology and so on.
  • FIG. 2 is a schematic flowchart of an application control method for a terminal device provided by an embodiment of the present application.
  • the two applications are used as examples for introduction below. Specifically, the two applications may be referred to as a first application and a second application, respectively.
  • the input device and the output device bound to the first application are called the first input device and the first output device
  • the input device and the output device bound to the second application are called the first input device and the first output device.
  • the input device and output device of the application are called the second input device and the second output device, wherein the first input device can be either a local input device or an external input device, and the first output device can be either a local output device or a second output device. It is an external output device, the second input device can be either a local input device or an external input device, and the second output device can be either a local output device or an external output device.
  • the method includes:
  • a user interaction interface is provided, and a first selection input by the user is received through the user interaction interface, where the first selection is used to indicate the first input device and the first output set by the user for the first application equipment.
  • the first application on the terminal device may be activated.
  • the binding module of the operating system of the terminal device detects that the first application starts to be started, it can provide a user interaction interface to the user during the process of starting the first application, and the user interaction interface is used to respectively provide multiple inputs for at least two applications device and multiple output devices.
  • the binding module can send the user interaction interface to at least one unbound output device for display, and the user interaction interface can be presented as a visual interface that displays at least one unbound input device and at least one unbound input device. Options for unbound output devices for the user to choose from.
  • unbound input devices and output devices refer to input devices and output devices that are not bound to an application, and correspondingly, input devices and output devices that have been bound to an application. Devices are bound input and output devices.
  • the binding module After the binding module provides the user interaction interface, the user can select a certain (or some) output devices and A certain (or some) input device is bound to the first application, that is, the first input device and the first output device are set for the first application. Specifically, the user can input the first selection to the user interaction interface through any unbound input device, and the first selection is used to indicate the first input device and the first output device (ie, the first selection) set by the user for the first application. including a first input device and a first output device). After receiving the first selection input by the user through the user interaction interface, the binding module forwards the first selection to the input adapter and the output adapter.
  • FIG. 3 is a schematic diagram of a user interaction interface provided by an embodiment of the present application.
  • the binding module can provide a visualization during the start-up process of the game software.
  • interface that displays a Bluetooth keyboard, a universal serial bus (USB) keyboard, a Bluetooth mouse, a USB mouse, a TV (which can also be displayed as a network video stream), and a monitor (which can also be displayed as a network card) and other options. Therefore, the user can select the input device and the output device bound to the game software in the interface.
  • the user sets the Bluetooth keyboard, Bluetooth mouse and TV as devices bound to the game software in the interface, it is equivalent to inputting a selection into the interface, and the selection can be sent to the binding module through the interface.
  • the first selection construct a correspondence between the first application and the first input device and a correspondence between the first application and the first output device.
  • the binding module After receiving the first selection, the binding module sends the first selection to the input adapter and the output adapter.
  • the input adapter can determine that the first input device needs to be bound to the first application, so the input adapter builds a correspondence between the first application and the first input device, and stores the correspondence, which is is equivalent to successfully binding the first input device to the first application.
  • the output adapter may determine that the first output device needs to be bound to the first application, so the output adapter builds a correspondence between the first application and the first output device, and stores the correspondence The relationship, at this time, is equivalent to successfully binding the first output device to the first application.
  • the input adapter can determine that the Bluetooth keyboard and the Bluetooth mouse need to be bound to the game software, and then build a correspondence between the Bluetooth keyboard and the game software. relationship, and the corresponding relationship between the Bluetooth mouse and the game software.
  • the output adapter can determine that the TV needs to be bound to the game software, and then establish a corresponding relationship between the TV and the game software. In this way, the bluetooth keyboard, bluetooth mouse and TV are successfully bound to the game software.
  • the user can operate the first application through the first input device and the first output device.
  • the user may first send a first operation instruction to the input adapter through the first input device.
  • the input adapter After receiving the first operation instruction from the first input device, the input adapter sends the first operation instruction to the first application based on the correspondence between the first input device and the first application, so that the first application can The operation command is processed.
  • the first application completes the processing of the first operation instruction, it can generate corresponding display screen data, and send the display screen data to the output adapter.
  • the output adapter sends the display screen data to the first output device for display based on the corresponding relationship between the first output device and the first application.
  • the remaining input devices and output devices of the terminal device are in an idle state, that is, the remaining input devices and output devices of the terminal device can serve other applications. , which can effectively utilize the input and output device resources of the terminal device.
  • the input adapter can receive input from the game software.
  • the operation instructions of the Bluetooth keyboard and mouse are sent to the game software based on the corresponding relationship between the Bluetooth keyboard and mouse and the game software.
  • the game software processes the operation instructions, it can generate corresponding display screen data and send the display screen data to the output adapter.
  • the output adapter sends the display screen data to the TV for display, thereby displaying the game screen on the TV.
  • the PC's USB keyboard, USB keyboard and monitor are unbound devices, so the PC's monitor will not display the game screen.
  • the smartphone has its own keyboard and display screen, and is connected with additional devices such as a Bluetooth keyboard and mouse and a TV
  • the Bluetooth keyboard, mouse and TV are bound to the desktop cloud client running on the smartphone.
  • the user can send the operation command through the Bluetooth keyboard and mouse, and the input adapter can receive the operation command from the Bluetooth keyboard and mouse, and send the operation command based on the corresponding relationship between the Bluetooth keyboard and mouse and the game software.
  • the desktop cloud client sends the operation instruction to the remote desktop cloud server (located outside the terminal device), and the desktop cloud server can return the display screen data to the desktop cloud client after processing the operation instruction.
  • the desktop cloud client sends the display data to the output adapter.
  • the output adapter sends the display screen data to the TV for display, thereby displaying the screen of the desktop cloud client on the TV.
  • the user can operate the desktop cloud client through a Bluetooth keyboard, mouse and TV, making the use of the desktop cloud very simple and convenient.
  • the keyboard and display screen of the smartphone are unbound devices, that is, the smartphone can be used normally, and the display screen of the smartphone will not display the screen of the desktop cloud client.
  • step 203 and step 206 may be repeatedly performed, so step 203 and step 206 may be performed simultaneously, that is, the operating system of the terminal device may run the first application and the second application at the same time.
  • the operating system can meet the operation needs of different users for different applications at the same time.
  • the binding module can provide a visual interface, which displays the USB keyboard, Options such as USB mouse and monitor (since the Bluetooth keyboard, Bluetooth mouse and monitor are all bound devices, these devices are no longer displayed in the visual interface). Therefore, the user can select the input device and the output device bound to the office software in the interface.
  • the user sets the USB keyboard, USB mouse and monitor as devices bound to the office software in this interface, it is equivalent to inputting a selection into this interface, and the selection can be sent to the binding module through this interface.
  • the input adapter can determine that the USB keyboard and USB mouse need to be bound to the office software, and then builds the corresponding relationship between the USB keyboard and the office software, and the USB mouse Correspondence with the office software.
  • the output adapter can determine that the monitor needs to be bound to the office software, and then establish a corresponding relationship between the monitor and the office software. In this way, the USB keyboard, USB mouse and monitor are successfully bound to the office software.
  • USB keyboard and mouse After successfully binding the USB keyboard, USB mouse and monitor to the office software, if another user starts to send operation commands through the USB keyboard and USB mouse (hereinafter collectively referred to as USB keyboard and mouse), the input adapter can receive the data from the USB keyboard and mouse. operation instructions, and based on the corresponding relationship between the USB keyboard and mouse and the office software, the operation instructions are sent to the office software. After the office software processes the operation instructions, it can generate corresponding display screen data and send the display screen data to the output adapter. Finally, based on the correspondence between the display and the office software, the output adapter sends the display image data to the display for display, thereby displaying the office image on the display.
  • the same terminal device can realize the game operation of one user and the office operation of another user at the same time.
  • the first input device and the first output device are bound to the first application. If the user needs to modify the input/output device bound to the first application, a trigger instruction may be sent to the binding module through the first input device. After receiving the trigger instruction, the binding module can provide the user interaction interface again, that is, the binding module sends the user interaction interface to the first output device for display. At this point, the user interaction interface may display options such as remaining unbound input devices and remaining unbound output devices for the user to select.
  • the binding module After the binding module provides the user interaction interface, the user can select a certain (or some) output device and a certain (or some) input device to bind to the first application in the options displayed on the user interaction interface, that is, The first application sets the third input device and the third output device, thereby replacing the first input device and the first output device that were originally bound to the first application.
  • the user may input a third selection to the user interaction interface through the first input device, and the third selection is used to indicate the third input device and the third output device set by the user for the first application.
  • the binding module After receiving the third selection input by the user through the user interaction interface, the binding module forwards the third selection to the input adapter and the output adapter.
  • the third selection modify the correspondence between the first application and the first input device to the correspondence between the first application and the third input device, and change the correspondence between the first application and the first output device.
  • the corresponding relationship is modified to the corresponding relationship between the first application and the third output device.
  • the input adapter can determine that the third input device needs to be bound to the first application, and then modify the corresponding relationship between the first application and the first input device to the relationship between the first application and the third input device. Correspondence between.
  • the output adapter receives the third selection, it can determine that the third output device needs to be bound to the first application, and then modify the correspondence between the first application and the first output device to the first application and the third output device. Correspondence between output devices. At this time, it is equivalent to successfully binding the third output device and the third input device to the first application.
  • the user needs to operate the first application through the third input device and the third output device.
  • the relevant description part of operating the first application through the first input device and the first output device in step 203 which will not be repeated here.
  • the operating system of the terminal device may provide a user interaction interface, and the user interaction interface is used to respectively provide multiple input devices and multiple output devices for at least two applications.
  • the first input device and the first output device are bound to the first application, so that the first input device and the first output device serve the first application.
  • the operating system can run the first application according to the operation instruction from the first input device, and send display screen data generated by running the first application to the first output device for display. Then, the remaining input devices except the first input device and the remaining output devices except the first output device can serve other applications, thereby effectively utilizing device resources and improving device utilization.
  • FIG. 4 is a schematic structural diagram of an application control apparatus of a terminal device provided by an embodiment of the present application. Please refer to FIG. 4 .
  • the apparatus includes:
  • the interaction module 401 is configured to provide a user interaction interface, where the user interaction interface is used to respectively provide options of multiple input devices and multiple output devices for at least two applications of the terminal device, and receive a user's first selection of the terminal device based on the options. a first selection of application settings, where the first selection includes a first input device and a first output device;
  • the running module 402 is configured to, when the first application starts, determine the first input device and the first output device of the first application according to the first selection, run the first application according to the instruction obtained from the first input device, and The display screen data generated by running the first application is sent to the first output device for display.
  • the user interaction interface is further configured to receive a second selection set by the user for the second application of the terminal device based on the option, where the second selection includes a second input device and a second output device, and the operation module 402 It is also used to determine the second input device and the second output device of the second application according to the second selection when the second application is started, run the second application according to the instruction obtained from the second input device, and run the second application.
  • the display screen generated by the application is sent to the second input device for display.
  • the first application and the second application run simultaneously in the terminal device.
  • the first input device is one of a local input device of the terminal device and an external input device of the terminal device;
  • the second input device is a local input device of the terminal device and an external input device of the terminal device
  • the other of the first output device is one of the local output device of the terminal device and the external output device of the terminal device;
  • the second output device is the other of the local output device of the terminal device and the external output device of the terminal device.
  • the first input device includes a first keyboard device and a first mouse device
  • the second input device includes a second keyboard device and a second mouse device.
  • the local output device is an output device provided by the terminal device
  • the external output device is an output device connected to the terminal device.
  • the user interaction interface is further configured to receive a third selection set by the user for the first application of the terminal device based on the option.
  • the third selection includes a third input device and a third output device, and the module 402 is executed. It is also used to determine the third input device and the third output device of the first application according to the third selection when the first application is started, run the first application according to the instruction obtained from the third input device, and run the first application.
  • the display screen generated by the application is sent to the third input device for display.
  • FIG. 5 is another schematic structural diagram of a terminal device provided by an embodiment of the present application.
  • the offload card in this embodiment of the present application may include one or more central processing units 501 , a memory 502 , an input/output interface 503 , a wired or wireless network interface 504 , and a power supply 505 .
  • the memory 502 may be ephemeral storage or persistent storage. Furthermore, the central processing unit 501 may be configured to communicate with the memory 502 to execute a series of instruction operations in the memory 502 on the offload card.
  • the central processing unit 501 may perform the operations performed by the operating system of the terminal device in the foregoing embodiment shown in FIG. 2 , and details are not repeated here.
  • the division of specific functional modules in the central processing unit 501 may be similar to the division of modules such as binding modules, input adapters, and output adapters described in FIG. 2 , which will not be repeated here.
  • the division of specific functional modules in the central processing unit 501 may also be similar to the division of modules such as the interaction module and the operation module described in FIG. 4 , which is not repeated here.
  • Embodiments of the present application also relate to a computer storage medium, including computer-readable instructions, when the computer-readable instructions are executed, the method described in FIG. 2 is implemented.
  • Embodiments of the present application also relate to a computer program product containing instructions, which, when run on a computer, cause the computer to execute the method described in FIG. 2 .
  • the disclosed system, apparatus and method may be implemented in other manners.
  • the apparatus embodiments described above are only illustrative.
  • the division of the units is only a logical function division. In actual implementation, there may be other division methods.
  • multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented.
  • the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be in electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.
  • each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.
  • the above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units.
  • the integrated unit if implemented in the form of a software functional unit and sold or used as an independent product, may be stored in a computer-readable storage medium.
  • the technical solutions of the present application can be embodied in the form of software products in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, and the computer software products are stored in a storage medium , including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application.
  • the aforementioned storage medium includes: U disk, mobile hard disk, Read-Only Memory (ROM, Read-Only Memory), Random Access Memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program codes .

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

La présente demande divulgue un procédé de commande d'applications pour un dispositif terminal et un dispositif associé, susceptibles d'utiliser efficacement des ressources de dispositif et d'améliorer le facteur d'utilisation d'un dispositif. Le procédé de la présente demande comprend les étapes consistant à : fournir une interface d'interaction avec l'utilisateur, l'interface d'interaction avec l'utilisateur étant configurée pour fournir, pour au moins deux applications d'un dispositif terminal, des options d'une pluralité de dispositifs d'entrée et d'une pluralité de dispositifs de sortie, respectivement ; et pour recevoir un premier ensemble de sélection par un utilisateur pour une première application du dispositif terminal en fonction de l'option, la première sélection comprenant un premier dispositif d'entrée et un premier dispositif de sortie ; si la première application est activée, déterminer un premier dispositif d'entrée et un premier dispositif de sortie de la première application selon la première sélection, exécuter la première application selon l'instruction acquise à partir du premier dispositif d'entrée et envoyer, au premier dispositif de sortie, des données d'écran d'affichage générées par exécution de la première application ; et afficher celles-ci.
PCT/CN2021/134494 2020-11-30 2021-11-30 Procédé de commande d'applications pour dispositif terminal et dispositif associé WO2022111732A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011376395.7 2020-11-30
CN202011376395.7A CN114595013A (zh) 2020-11-30 2020-11-30 一种终端设备的应用控制方法及其相关设备

Publications (1)

Publication Number Publication Date
WO2022111732A1 true WO2022111732A1 (fr) 2022-06-02

Family

ID=81754036

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/134494 WO2022111732A1 (fr) 2020-11-30 2021-11-30 Procédé de commande d'applications pour dispositif terminal et dispositif associé

Country Status (2)

Country Link
CN (1) CN114595013A (fr)
WO (1) WO2022111732A1 (fr)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2001097022A2 (fr) * 2000-06-13 2001-12-20 Checkout Holdings Limited Ordinateur multi-utilisateurs
CN102073616A (zh) * 2009-11-23 2011-05-25 仁宝电脑工业股份有限公司 个人计算机系统及其多重输出的操作方法
US20110246690A1 (en) * 2010-03-31 2011-10-06 Thomas Lang Computing arrangement
CN108632448A (zh) * 2017-03-17 2018-10-09 三星电子株式会社 用于控制音频输出的电子设备及其操作方法
CN109308175A (zh) * 2018-11-02 2019-02-05 盛子望 利用移动设备及显示设备进行同屏显示的电子装置及方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2001097022A2 (fr) * 2000-06-13 2001-12-20 Checkout Holdings Limited Ordinateur multi-utilisateurs
CN102073616A (zh) * 2009-11-23 2011-05-25 仁宝电脑工业股份有限公司 个人计算机系统及其多重输出的操作方法
US20110246690A1 (en) * 2010-03-31 2011-10-06 Thomas Lang Computing arrangement
CN108632448A (zh) * 2017-03-17 2018-10-09 三星电子株式会社 用于控制音频输出的电子设备及其操作方法
CN109308175A (zh) * 2018-11-02 2019-02-05 盛子望 利用移动设备及显示设备进行同屏显示的电子装置及方法

Also Published As

Publication number Publication date
CN114595013A (zh) 2022-06-07

Similar Documents

Publication Publication Date Title
US20230289131A1 (en) Method and system for dynamically generating different user environments with secondary devices with displays of various form factors
WO2020221056A1 (fr) Procédé de dessin d'image, appareil, dispositif électronique et support de stockage informatique
JP3172006U (ja) スマートフォン・ディスプレイの画面転送コントロール装置
US20110066971A1 (en) Method and apparatus for providing application interface portions on peripheral computing devices
EP3188006B1 (fr) Interface graphique composite ayant des objets de données partagés
EP3787301B1 (fr) Procédé et système de partage d'un dispositif de sortie entre des dispositifs multimédias à des fins d'émission et de réception de données
US9864644B2 (en) System and method for transport layer agnostic programming interface for use with smartphones
WO2022135156A1 (fr) Procédé de collaboration distribuée inter-appareils, et dispositif électronique et système de communication
CN112316417B (zh) 控制设备连接方法、装置、设备及计算机可读存储介质
CN114281288A (zh) 投屏处理方法、装置及电子设备
EP3704861B1 (fr) Découverte de canal de retour d'interface utilisateur en réseau par connexion vidéo filaire
CN112274910A (zh) 虚拟按键配置方法、虚拟按键方法及相关装置
WO2022160612A1 (fr) Procédé d'interaction avec un système monté sur véhicule, support de stockage et terminal mobile
JP7018110B1 (ja) モバイル機器ドック
JP2023547821A (ja) 制御方法及び装置、及び電子デバイス
CN111143017B (zh) 云操作系统交互处理方法、客户端及云操作系统
WO2022111732A1 (fr) Procédé de commande d'applications pour dispositif terminal et dispositif associé
CN115421846A (zh) 跨设备控制方法、控制装置、电子设备和可读存储介质
CN112272223B (zh) 应用的运行方法、装置、服务器及计算机可读存储介质
KR102626168B1 (ko) 디스플레이 장치 및 그 제어 방법
CN116347148A (zh) 一种投屏方法及显示设备
JPWO2016080428A1 (ja) 情報処理システム、情報処理方法、情報処理装置およびその制御方法と制御プログラム
US11544083B2 (en) Display apparatus and control method thereof
EP4383732A1 (fr) Dispositif électronique et procédé de partage d'écrans et de signaux audio correspondant aux écrans
US20240004600A1 (en) Display apparatus and operating method of the display apparatus

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21897232

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21897232

Country of ref document: EP

Kind code of ref document: A1