WO2023066395A1 - 一种应用运行方法以及相关设备 - Google Patents

一种应用运行方法以及相关设备 Download PDF

Info

Publication number
WO2023066395A1
WO2023066395A1 PCT/CN2022/126849 CN2022126849W WO2023066395A1 WO 2023066395 A1 WO2023066395 A1 WO 2023066395A1 CN 2022126849 W CN2022126849 W CN 2022126849W WO 2023066395 A1 WO2023066395 A1 WO 2023066395A1
Authority
WO
WIPO (PCT)
Prior art keywords
task
terminal device
information
running
execution
Prior art date
Application number
PCT/CN2022/126849
Other languages
English (en)
French (fr)
Inventor
王伟
欧阳黜霏
林俊如
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2023066395A1 publication Critical patent/WO2023066395A1/zh

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/48Program initiating; Program switching, e.g. by interrupt
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]

Definitions

  • the present application relates to the field of computers, and in particular to an application running method and related equipment.
  • the embodiment of the present application provides a distributed operation method of an application, the method is applied to a first terminal device, and the first terminal device communicates with the second terminal device, and the second terminal device can be locally Alternatively, at the remote end, the method includes: when running the target application, acquiring device information and/or software execution environment of the second terminal device; based on the device information and/or software execution environment of the second terminal device satisfying the The target application executes the first task, and transmits task operation information related to the first task to the second terminal device, so that the second terminal device executes the first task according to the task operation information, And the first terminal device does not execute the first task.
  • the second terminal device can execute part of the tasks of the target application
  • transferring part of the tasks of the target application to the second terminal device for execution can reduce the computing load of the first terminal device. If the hardware performance of the first terminal device, such as display resolution, CPU, and GPU computing power, is not as good as that of the second terminal hardware device, the tasks transferred by this method can obtain better application performance on the second terminal device.
  • the first task may be one of a UI display task, an image rendering task, a logic status update task, an AI task, and a data processing or analysis task.
  • the first task may also be other tasks that require large computing resources and/or storage resources and or communication resources.
  • the first task may be a task to be executed when the currently available computing resources and/or storage resources or communication resources of the first terminal device are insufficient, for example, whether the first terminal device can run the first task normally (such as It takes a lot of time, etc.), or whether the first terminal device will affect other tasks when running the first task, or whether the first terminal device will get hot when running the first task, and so on.
  • the first task may be a pre-divided task category, for example, a game application is pre-divided into a logic state update task and a rendering task, or the rendering task is divided into multiple subtasks.
  • a game application is pre-divided into a logic state update task and a rendering task, or the rendering task is divided into multiple subtasks.
  • WeChat Chinese text chat and video call are two subtasks that can be split. Generally, only one task can be executed at a time on a single computer. In this example, it can be split into two subtasks, and one of them can be transferred to the second The second terminal device is running.
  • first terminal device may also transfer multiple tasks to the second terminal device, or transfer multiple tasks to different terminal devices for execution, which is not limited herein.
  • the second terminal device may not only execute the first task according to the task information of the first task, but may also consider the balance between information such as the status of the second terminal device, application execution effect, and user experience, such as display effect Balance with the dynamic resource utilization rate of the device, and finally achieve the best display, and the execution effect and execution efficiency of the second terminal device in performing the first task can be enhanced compared with the first device.
  • device information and software operating environment can change according to changes in application scenarios, but there are also invariants, such as device power, CPU, memory usage, network load status, and the runtime status of the application process on the current device; change
  • the part includes but is not limited to, for example, the divided subtasks of the target application of the first device may contain UI window information, and the first device also needs to collect the window information on the second device.
  • the first terminal device and the second terminal device can be connected through near-field communication or far-field communication, that is, Internet network, WIFI, Bluetooth, ultra-wideband, ZigBee , RFID, 4G, 5G, LoRa, SIGFOX, Z-Wave and NB-loT at least one way of communication.
  • the connection between the first terminal device and the second terminal device is through Internet, WIFI, Bluetooth, ultra-wideband, ZigBee, RFID, 4G, 5G, LoRa, SIGFOX, Z-Wave and NB-loT connected in various ways.
  • connection between the first terminal device and the second terminal device may also be performed through other wireless connection methods with low delay (such as 3GPP, non-3GPP, and connection methods based on short-distance technology). Not limited.
  • the first task is a rendering task
  • the display parameters such as refresh rate, screen size, refresh rate, etc.
  • the second terminal device may execute the above task according to the task running information related to the first task and the display parameters of the display device of the second terminal device (such as the resolution of the display device, the screen refresh rate, etc.).
  • the first task can further display display content adapted to the display device of the second terminal device.
  • the method before transmitting the task execution information related to the first task to the second terminal device, the method further includes: presenting first selection information, the first selection information indicating The target application has at least one task, the device information and/or software execution environment of the second terminal device meet the execution requirements of the at least one task, and a first input fed back by the user according to the first selection information is received; Based on the first input including the first task, enabling the action of transmitting task execution information related to the first task to the second terminal device.
  • the target task may include a plurality of subtasks, and the first terminal device may determine a task ( at least one task) and presented to the user for selection.
  • the method further includes: transmitting the status data related to the first task to the second terminal device.
  • the state data is state data required by the second terminal device to execute the first task.
  • the state data may be an intermediate state of the UI engine, or data state information of the application itself.
  • the status data may be input by the user on the first terminal device, or may be generated by the first terminal device itself.
  • the first terminal device may acquire the device information of the terminal device that has a communication connection, and based on the analysis of the device information, determine whether the terminal device has the ability to run the task to be transferred, and determine whether the Which part of the task is offloaded to the terminal device.
  • the software operating environment information includes component information, and the component information is used to indicate whether the target application is deployed on the second terminal device.
  • component the running component is configured to have the capability of executing the first task.
  • the running component can be an AI engine, and the AI engine can be used to perform AI tasks.
  • the running component can be a rendering engine, and the rendering engine can be used to perform rendering tasks.
  • the running component can be a logic engine and a physical engine. To execute the logic state update task, if the second terminal device is not deployed with an operating component capable of executing the first task, it is not necessary to transfer the first task to the second terminal device for execution.
  • the first task may be one of a UI display task, an image rendering task, a logic state update task, an AI task, and a data processing or analysis task.
  • the running component can be an AI engine, and the AI engine can be used to perform AI tasks.
  • the running component can be a rendering engine, and the rendering engine can be used to perform rendering tasks.
  • the running component can be a logic engine and a physical engine. For executing the logic state update task, if the second terminal device is not deployed with an operating component capable of executing the first task, the first task does not need to be transferred to the second terminal device for execution.
  • the software operating environment may include the system environment for running applications, which require both devices to have some necessary operating components. For example, if it is a game, a graphics card component may be required, and if it is a mobile application, an operating environment of Android 4.0 or above may be required.
  • the first terminal device may be deployed with a running component.
  • the first terminal device may execute the first task through the running component, and the running component deployed by the first terminal device It is consistent (or basically consistent) with the running component deployed on the second terminal device.
  • the device information may include power information, and the power information may be the available power and charging mode of the second terminal device (such as active input or battery power supply), for example, when the power supply information indicates that the available power of the second terminal device is low, it may be determined that the first task does not need to be transferred to the second terminal device for execution.
  • the power information may be the available power and charging mode of the second terminal device (such as active input or battery power supply), for example, when the power supply information indicates that the available power of the second terminal device is low, it may be determined that the first task does not need to be transferred to the second terminal device for execution.
  • the device information may include processor information
  • the processor may be CPU, GPU, TPU, NPU, etc.
  • the processor information may be but not limited to CPU model/main frequency, GPU model/main frequency, current CPU/GPU load, etc. Since some tasks have certain requirements for the type of processor, processing power, and currently available resources, if the task requirements are not met, it can be determined that the first task does not need to be transferred to the second terminal device for execution; or the current device processes If the processor load is relatively high, but the second terminal device has sufficient processor resources, it may also decide to perform task transfer immediately.
  • the device information may include memory information, such as memory size.
  • memory information such as memory size.
  • the device information may include transmission bandwidth, and the transmission bandwidth may be an available bandwidth of the current network.
  • the transmission bandwidth between devices needs to meet the requirements corresponding to the first task. When the requirement is not met, it may be determined that the first task does not need to be transferred to the second terminal device for execution.
  • the device information may include input device information
  • the input device information may include the type of input device
  • the input device may be audio input, mouse input, or keyboard input. wait.
  • the first task also needs to support user input, and since the first task is transferred to the second terminal device for execution, the second terminal device is required to have the input required by the first task Equipment type. When the requirement is not met, it may be determined that the first task does not need to be transferred to the second terminal device for execution.
  • the device information may include output device information
  • the output device information may include the output device type
  • the output device may be audio output, image/video output, etc.
  • the output device information may also display device display properties such as screen resolution and refresh rate.
  • the first task also needs to support output (for example, the rendering task requires the presence of a display device, and the parameter requirements of the display device), and since the first task is transferred to the second terminal device for execution, Then the second terminal device needs to have the output device type and the attributes of the output device required by the first task. When the requirement is not met, it may be determined that the first task does not need to be transferred to the second terminal device for execution.
  • the device information may include network load information
  • the network load information may include the current device network card working status, network bandwidth occupancy rate, and end-to-end delay time wait. Since some tasks have certain requirements for network processing capabilities and currently available network resources, if the transfer task requirements are not met, it can be determined that the first task does not need to be transferred to the second terminal device for execution; or the current device has a high network load , and the second terminal device may decide to execute the task transfer immediately if the network resources are sufficient.
  • the device information includes: the device state of the terminal device and/or the running state of the running component, the device state means that the device has been powered on, and may be running or not running certain applications , the corresponding non-device status includes device power off, device sleep or screen off, etc.;
  • the task running information related to the first task includes first indication information, and the first indication information is used to trigger the second terminal device to start , instructing the second terminal device to run the running component, and execute the first task through the running component; or,
  • the task running information related to the first task includes a second indication information
  • the second instruction information is used to instruct the second terminal device to run the running component and execute the first task through the running component
  • the task running information related to the first task includes a third indication information
  • the third indication information is used to instruct the second terminal device to execute the first task through the running component.
  • the first terminal device determines multiple terminal devices meeting the requirements, some (or all) terminal devices may be selected from the multiple terminal devices as transfer objects of the first task.
  • device information of multiple terminal devices connected to the first terminal device may be acquired, where the multiple terminal devices include the second terminal device, and the Selection of said second terminal device among the terminal devices.
  • second selection information may be presented, the second selection information is used to instruct the user to select from the plurality of terminal devices, and the second selection information fed back by the user according to the second selection information is received. an input; including the second terminal device based on the second input, enabling selection for the second terminal device of the plurality of terminal devices.
  • the first terminal device may also select, from the multiple terminal devices, the device information that best meets the execution requirements of the first task based on the device information and or application information of the second terminal device and or application information, enabling selection of the second terminal device among the plurality of terminal devices.
  • the so-called best performance requirements can be understood as selecting a terminal device with the best performance among terminal devices that meet the execution requirements of the first task.
  • the target application task includes the first task and the second task; the method further includes: executing the second task.
  • the synchronization frequency needs to be very high. For example, each frame of data needs to be synchronized once. If For a 60fps game, it takes about 16ms to perform a synchronization operation, so the delay requirement for synchronization is very high.
  • the first terminal device and the second terminal device are heterogeneous devices, the first terminal device needs to serialize the data to be sent to obtain binary data, and transfer the binary data To the second terminal device, the second terminal device needs to deserialize the binary data to obtain data that can be recognized by itself. The above sequence deserialization operation will cause a certain delay, which is required in games and other games that require high synchronization requirements. In the scene, it is not allowed.
  • the task running information related to the first task may include: execution data and index information of the execution data; wherein, the execution data is the data required to execute the first task, and the The index information includes a memory address, the memory address is an address of a storage space available on the second terminal device, and the index information is used to instruct the second terminal device to store the execution data to the memory address corresponding to in the storage space.
  • the available storage space on the second terminal device can be obtained first, and the task running information together with the address of the storage space selected by the first terminal device can be transmitted to the second terminal device, and the second terminal device can information, storing the execution data in the storage space corresponding to the memory address, and acquiring the execution data from the storage space corresponding to the memory address when executing the first task. Since the task component on the second terminal device can directly execute the first task, there is no need to perform format conversion on the execution data, and no sequence and deserialization operations are required, which reduces the processing delay.
  • the first task is one of a UI display task, an image rendering task, a logic state update task, an AI task, and a data processing or analysis task.
  • the present application provides an application running method, the method is applied to a target system, the target system includes a first terminal device and a second terminal device, the first terminal device and the second terminal device A communication connection, the method comprising:
  • the first terminal device acquires device information and/or software execution environment of the second terminal device when running the target application;
  • the first terminal device meets the execution requirements of the first task of the target application based on the device information and/or software execution environment of the second terminal device, and transmits the task running information of the first task to the second terminal device. a terminal device, and the first terminal device does not perform the first task;
  • the second terminal device executes the first task according to task running information related to the first task.
  • the method before transmitting the task execution information related to the first task to the second terminal device, the method further includes: the first terminal device presents the first selection information, so The first selection information indicates at least one task of the target application, the device information and/or software execution environment of the second terminal device meet the execution requirements of the at least one task, and the user receives the information based on the first selection information the first input of the feedback;
  • the first terminal device includes the first task based on the first input, and enables the action of transmitting task execution information related to the first task to the second terminal device.
  • the method further includes:
  • the first terminal device transmits state data related to the first task to the second terminal device; the state data is state data required by the second terminal device to execute the first task.
  • the first terminal device and the second terminal device are connected through near field communication, Internet network, WIFI, Bluetooth, ultra-wideband, ZigBee, RFID, 4G, 5G, LoRa, At least one of SIGFOX, Z-Wave and NB-loT communicates.
  • the software execution environment includes:
  • Component information where the component information is used to indicate whether the second terminal device is deployed with an operating component of the target application, and the operating component is configured to have the ability to execute the first task;
  • the implementation requirements include:
  • the second terminal device is deployed with the running component.
  • the device information includes at least one of the following information:
  • Power supply information processor information, memory information, transmission bandwidth, input device information, output device information, network load.
  • the software execution environment includes: the device state of the terminal device and/or the running state of the running component;
  • the task running information related to the first task includes first indication information, and the first indication information uses To indicate at least one of the following information: the second terminal device is powered on, ends dormancy, ends screen off or unlocks the screen, instructs the second terminal device to run the running component, and executes the running component through the running component. first task; or,
  • the task running information related to the first task includes a second indication information, the second indication information is used to indicate that at least one of the second terminal devices runs the running component and executes the first task through the running component; or,
  • the task running information related to the first task includes a third indication information
  • the third indication information is used to instruct the second terminal device to execute the first task through the running component.
  • the synchronization frequency needs to be very high. For example, each frame of data needs to be synchronized once. If For a 60fps game, it takes about 16ms to perform a synchronization operation, so the delay requirement for synchronization is very high.
  • the first terminal device and the second terminal device are heterogeneous devices, the first terminal device needs to serialize the data to be sent to obtain binary data, and transfer the binary data To the second terminal device, the second terminal device needs to deserialize the binary data to obtain data that can be recognized by itself. The above sequence deserialization operation will cause a certain delay, which is required in games and other games that require high synchronization requirements. In the scene, it is not allowed.
  • the task running information related to the first task may include: execution data and index information of the execution data; wherein, the execution data is the data required to execute the first task, and the The index information includes a memory address, the memory address is an address of a storage space available on the second terminal device, and the index information is used to instruct the second terminal device to store the execution data to the memory address corresponding to in the storage space.
  • the available storage space on the second terminal device can be obtained first, and the task running information together with the address of the storage space selected by the first terminal device can be transmitted to the second terminal device, and the second terminal device can information, storing the execution data in the storage space corresponding to the memory address, and acquiring the execution data from the storage space corresponding to the memory address when executing the first task. Since the task component on the second terminal device can directly execute the first task, there is no need to perform format conversion on the execution data, and no sequence and deserialization operations are required, which reduces the processing delay.
  • the second terminal device does not execute other tasks of the target application based on completing the execution of the first task and not receiving an execution trigger for other tasks in the target application by the first terminal device.
  • a lockstep synchronization mechanism based on heterogeneous memory arrangement is adopted, a unified state synchronization data interface is designed, and the data addresses of different platforms are sequenced (for example, there is a connection relationship between multiple terminal devices and the first terminal device, Then the first terminal device can transfer the task running information to different terminal devices according to the preset order), pack and notify the opposite side, and at the same time perform a strictly limited synchronization method according to the lockstep method (as a terminal device receiving task running information, except for the received task running information, and will not perform other processing related to the target application).
  • the first task is one of a UI display task, an image rendering task, a logic state update task, an AI task, and a data processing or analysis task.
  • the target application task includes the first task and the second task; the first task and the second task are rendering tasks; the first task is used to draw the first application Interface, the second task is used to draw a second application interface, the first application interface and the second application interface are different interfaces; the first terminal device can execute the second task and trigger the display The device displays the second application interface; the second terminal device may trigger the display device to display the first application interface. It is equivalent to dividing the content originally displayed on the same display screen into multiple interfaces, which can be rendered and displayed on different terminal devices.
  • the second terminal device may execute the task according to the task execution information related to the first task, and the display parameters of the display device of the second terminal device (such as the resolution of the display device). rate, screen refresh rate, etc.), execute the first task.
  • the second terminal device may also feed back the execution result to the first terminal device.
  • the present application also provides an application running method, the method is applied to a first terminal device, and the first terminal device communicates with the second terminal device, and the method includes: running a target application; the target application Including a first task; the first task is a part of the task in the target application; wherein, the target application is an application with a dialogue function; the first task is the UI display of the chat box of some or all chat objects 1.
  • the target application is a painting application, text editing application, image editing application or audio and video editing application
  • the first task is the UI display of the editing tool interface , or edit the UI display of the object
  • the application with the dialogue function may include an instant chat application, a social application, etc., and through the application, a text dialogue, an audio dialogue, a video dialogue, etc. can be conducted with other contacts.
  • the current tasks of an application with a dialogue function may include UI display of a chat interface and a video call, and the first terminal device may transfer task execution information related to the video call to the second terminal device for execution.
  • the current task of an application with a dialogue function may include UI display of a chat interface
  • the chat interface may include multiple contacts (that is, chat objects)
  • the first terminal device may display the chat boxes of some contacts in the multiple contacts.
  • the UI display is transferred to the second terminal device for execution.
  • the target application further includes a second task different from the first task
  • the target application is an application with a dialogue function
  • the second task is a UI display of a chat object different from the first task, a UI display of a video chat, or a UI display of an audio chat;
  • the target application is a painting application, a text editing application, an image editing application, or an audio and video editing application;
  • the second task is the UI display of the editing tool interface, or the UI display of the editing object;
  • the method also includes:
  • the present application provides an application running device, the device is applied to a first terminal device, and the first terminal device communicates with the second terminal device, and the device includes:
  • An acquisition module configured to acquire device information and/or software execution environment of the second terminal device when running the target application
  • a sending module configured to transmit task running information related to the first task to the second terminal device when the device information and/or software execution environment of the second terminal device meet the execution requirements of the first task of the target application. and two terminal devices, so that the second terminal device executes the first task according to the task running information, and the first terminal device does not execute the first task.
  • the target application further includes a second task; the device further includes:
  • a task execution module configured to keep the execution of the second task on the first terminal device when the second terminal device executes the first task according to the task running information.
  • the device also includes:
  • a receiving module configured to receive a first input from a user before transmitting task execution information related to the first task to the second terminal device, the first input indicating that the first task is related to The task running information is transmitted to the second terminal device.
  • the device also includes:
  • a presenting module configured to present first selection information indicating at least one task of the target application before transmitting the task execution information related to the first task to the second terminal device,
  • the device information and/or software execution environment of the second terminal device meet the execution requirements of the at least one task
  • the receiving module is specifically used for:
  • a first input fed back by the user according to the first selection information is received; the first input includes the first task.
  • the sending module is further configured to, after the task running information related to the first task is transferred to the second terminal device, transfer the status data related to the first task to the The second terminal device; the state data is state data required by the second terminal device to execute the first task.
  • the first terminal device and the second terminal device are connected through near field communication, Internet network, WIFI, Bluetooth, ultra-wideband, ZigBee, RFID, 4G, 5G, LoRa, At least one of SIGFOX, Z-Wave and NB-loT communicates.
  • the software execution environment includes:
  • Component information where the component information is used to indicate whether the second terminal device is deployed with an operating component of the target application, and the operating component is configured to have the ability to execute the first task;
  • the implementation requirements include:
  • the second terminal device is deployed with the running component.
  • the device information includes at least one of the following information:
  • Power supply information processor information, memory information, transmission bandwidth, input device information, output device information, network load.
  • the software execution environment includes: the device state of the terminal device and/or the running state of the running component;
  • the task running information related to the first task includes first indication information, and the first indication information uses To indicate at least one of the following information: the second terminal device is powered on, ends dormancy, ends screen off or unlocks the screen, instructs the second terminal device to run the running component, and executes the running component through the running component. first task; or,
  • the task running information related to the first task includes a second indication information, the second indication information is used to indicate that at least one of the second terminal devices runs the running component and executes the first task through the running component; or,
  • the task running information related to the first task includes a third indication information
  • the third indication information is used to instruct the second terminal device to execute the first task through the running component.
  • the acquiring module is specifically used for:
  • the multiple terminal devices include the second terminal device
  • the device also includes:
  • a terminal selection module configured to enable selection of the second terminal device among the plurality of terminal devices before the task execution information of the first task is transmitted to the second terminal device.
  • the terminal selection module is specifically configured to:
  • Second selection information is used to instruct the user to select from the plurality of terminal devices, and receiving a second input fed back by the user according to the second selection information;
  • a selection for the second terminal device of the plurality of terminal devices is enabled based on the second input including the second terminal device.
  • the terminal selection module is specifically configured to:
  • a second input from the user is received, the second input indicating to select the second device from the plurality of terminal devices.
  • the terminal selection module is specifically configured to:
  • a second input fed back by the user according to the second selection information is received; the second input includes the second terminal device.
  • the terminal selection module is specifically configured to:
  • the target application task includes the first task and the second task
  • the device also includes:
  • a task execution module configured to keep executing the second task when the device information of the second terminal device does not meet the execution requirements of the target application second task.
  • the task running information related to the first task includes:
  • Execution data and index information of the execution data wherein, the execution data is data required to execute the first task, and the index information includes a memory address, and the memory address is available on the second terminal device The address of the storage space, the index information is used to instruct the second terminal device to store the execution data in the storage space corresponding to the memory address.
  • the first task is one of a UI display task, an image rendering task, a logic state update task, an AI task, and a data processing or analysis task.
  • the present application provides an application running device, the device is applied to a target system, the target system includes a first terminal device and a second terminal device, the first terminal device and the second terminal device
  • the communication connection, said first terminal device comprises:
  • An acquisition module configured to acquire device information and/or software execution environment of the second terminal device when running the target application
  • a sending module configured to satisfy the execution requirements of the first task of the target application based on the device information and/or software execution environment of the second terminal device, and transmit the task running information of the first task to the second terminal device, and the first terminal device does not perform the first task;
  • the second terminal device includes:
  • a first task execution module configured to execute the first task according to task running information related to the first task.
  • the target application further includes a second task;
  • the second terminal device includes:
  • a second task execution module configured to keep the second task executed by the first terminal device when the second terminal device executes the first task according to the task execution information.
  • the first terminal device further includes:
  • a receiving module configured to receive a first input from a user before transmitting task execution information related to the first task to the second terminal device, the first input indicating that the first task is related to The task running information is transmitted to the second terminal device.
  • the first terminal device further includes:
  • a presenting module configured for the first terminal device to present first selection information, the first selection information indicating at least one task of the target application, and the device information and/or software execution environment of the second terminal device satisfying the The execution requirements of at least one task;
  • the receiving module is specifically used for:
  • a first input fed back by the user according to the first selection information is received; the first input includes the first task.
  • the sending module is further configured to, after the task running information related to the first task is transmitted to the second terminal device, the first terminal device sends the first task Relevant state data is transmitted to the second terminal device; the state data is state data required by the second terminal device to execute the first task.
  • the sending module is further configured to, after the task running information related to the first task is transmitted to the second terminal device, the first terminal device sends the first task Relevant state data is transmitted to the second terminal device; the state data is state data required by the second terminal device to execute the first task.
  • the first terminal device and the second terminal device are connected through near-field communication, WIFI, Bluetooth, ultra-wideband, ZigBee, RFID, 4G, 5G, LoRa, SIGFOX, Z -At least one of Wave and NB-loT communication.
  • the device information includes:
  • Component information where the component information is used to indicate whether the second terminal device is deployed with an operating component of the target application, and the operating component is configured to have the ability to execute the first task;
  • the implementation requirements include:
  • the second terminal device is deployed with the running component.
  • the device information includes at least one of the following information:
  • Power supply information processor information, memory information, transmission bandwidth, input device information, output device information, network load.
  • the software execution environment includes: the device state of the terminal device and/or the running state of the running component;
  • the task running information related to the first task includes first indication information, and the first indication information uses To indicate at least one of the following information: the second terminal device is powered on, ends dormancy, ends screen off or unlocks the screen, instructs the second terminal device to run the running component, and executes the running component through the running component. first task; or,
  • the task running information related to the first task includes a second indication information, the second indication information is used to indicate that at least one of the second terminal devices runs the running component and executes the first task through the running component; or,
  • the task running information related to the first task includes a third indication information
  • the third indication information is used to instruct the second terminal device to execute the first task through the running component.
  • the task running information related to the first task includes:
  • Execution data and index information of the execution data wherein, the execution data is data required to execute the first task, and the index information includes a memory address, and the memory address is available on the second terminal device the address of the storage space, the index information is used to instruct the second terminal device to store the execution data in the storage space corresponding to the memory address;
  • the first task execution module is specifically used for:
  • the execution data is acquired from the storage space corresponding to the memory address.
  • the second terminal device does not execute the first task based on completing execution of the first task and not receiving execution triggers for other tasks in the target application by the first terminal device. Other tasks for the target application described above.
  • the first task is one of a UI display task, an image rendering task, a logic state update task, an AI task, and a data processing or analysis task.
  • the target application task includes the first task and the second task; the first task and the second task are rendering tasks; the first task is used to draw the first application interface, the second task is used to draw a second application interface, the first application interface and the second application interface are different interfaces;
  • the first terminal device also includes:
  • a second task execution module configured to execute the second task, and trigger a display device to display the second application interface
  • the first task execution module is specifically used for:
  • the second terminal device triggers the display device to display the first application interface.
  • the first task is a rendering task
  • the first task execution module is specifically used for:
  • the present application provides a terminal device, the device includes a processor, a memory, and a bus, wherein:
  • the processor and the memory are connected through the bus;
  • the memory is used to store computer programs or instructions
  • the processor is used to call or execute the programs or instructions stored in the memory to implement the steps described in the above-mentioned first aspect and any one of the possible implementation modes of the first aspect, as well as the second aspect and the second aspect Any one of the steps described in the possible implementation manners.
  • the present application provides a computer storage medium, including computer instructions.
  • the computer instructions When the computer instructions are run on an electronic device or a server, the steps described in any one of the above-mentioned first aspect and possible implementation modes of the first aspect are executed. , and the second aspect and the steps described in any one possible implementation manner of the second aspect.
  • the present application provides a computer program product.
  • the computer program product When the computer program product is run on an electronic device or a server, it executes the steps described in any one of the above-mentioned first aspect and possible implementation modes of the first aspect, and the first aspect. The steps described in the possible implementation manners of any one of the second aspect and the second aspect.
  • the present application provides a chip system
  • the chip system includes a processor, configured to support the terminal device to implement the functions involved in the above aspect, for example, send or process the data involved in the above method; or, information .
  • the chip system further includes a memory, and the memory is used for storing necessary program instructions and data of the execution device or the training device.
  • the system-on-a-chip may consist of chips, or may include chips and other discrete devices.
  • An embodiment of the present application provides an application running method, the method is applied to a target system, the target system includes a first terminal device and a second terminal device, and the first terminal device is communicatively connected to the second terminal device , the method includes: when the first terminal device is running a target application, acquiring device information of a second terminal device; based on the device information of the second terminal device, the first terminal device satisfies the first The execution of the task requires that the task execution information of the first task be transmitted to the second terminal device, and the first terminal device does not execute the first task; the second terminal device The task execution information related to the task executes the first task.
  • the second terminal device can execute a part of the tasks of the target application, transferring a part of the tasks of the target application to the second terminal device for execution can reduce the computing load of the first terminal device.
  • FIG. 1 is a schematic diagram of an application architecture provided by an embodiment of the present application
  • FIG. 2 is a schematic diagram of an application architecture provided by an embodiment of the present application.
  • FIG. 3 is a schematic structural diagram of a terminal device provided in an embodiment of the present application.
  • FIG. 4 is a block diagram of a software structure of a terminal device according to an embodiment of the present application.
  • FIG. 5 is a schematic diagram of an embodiment of an application running method provided in an embodiment of the present application.
  • FIG. 6 is a schematic diagram of an interface provided by an embodiment of the present application.
  • FIG. 7 is a schematic diagram of an interface provided by an embodiment of the present application.
  • FIG. 8 is a schematic diagram of index information provided by an embodiment of the present application.
  • FIG. 9 is a schematic diagram of an embodiment of a synchronization method provided by an embodiment of the present application.
  • FIG. 10 is a schematic diagram of an embodiment of an application running method provided in an embodiment of the present application.
  • FIG. 11 is a schematic diagram of an embodiment of an application running method provided by the embodiment of the present application.
  • FIG. 12 is a schematic diagram of an embodiment of an application running device provided in an embodiment of the present application.
  • FIG. 13 is a schematic diagram of an embodiment of an application running device provided in an embodiment of the present application.
  • FIG. 14 is a schematic diagram of an embodiment of an application running device provided in an embodiment of the present application.
  • FIG. 15 is a schematic diagram of an embodiment of an application running device provided in an embodiment of the present application.
  • FIG. 16 is a schematic diagram of an embodiment of a terminal device provided in an embodiment of the present application.
  • Fig. 17 is a schematic diagram of transfer of an application task.
  • FIG. 1 is a schematic diagram of a device network relationship architecture diagram provided by an embodiment of the present application, wherein the system may include a terminal device 1 and a terminal device 2 .
  • the terminal device 1 may be connected to the terminal device 2 in a wired or wireless manner.
  • wireless methods include General Packet Radio Service (General Packet Radio Service, GPRS), Wireless Local Area Networks (Wireless Local Area Networks, WLAN), ZigBee (ZigBee), Bluetooth (Bluetooth), Near Field Communication (Near Field Communication, NFC) and so on. It should be understood that the examples of the above communication methods are only for illustration and should not be construed as specific limitations.
  • the terminal device 1 and the terminal device 2 may be mobile terminals.
  • the mobile terminal can be a smart phone, a handheld processing device, a tablet computer, a mobile notebook, a virtual reality device, an integrated handheld device, and the like.
  • the mobile terminal can also be a wearable device, and the wearable device can also be called a wearable smart device. Clothing and shoes etc.
  • a wearable device can be a portable device that is worn directly on the body, or integrated into the user's clothing or accessories. Wearable devices can achieve powerful functions through software support, data interaction, and cloud interaction.
  • Generalized wearable smart devices include full-featured, large-sized, complete or partial functions without relying on smart phones, such as smart watches or smart glasses, etc., and only focus on a certain type of application functions, and need to cooperate with other devices such as smart phones Use, such as various smart bracelets and smart jewelry for physical sign monitoring.
  • the terminal device 1 and the terminal device 2 may also be vehicle-mounted devices, smart conference devices, smart advertising devices, smart home appliances, and the like.
  • a smart home appliance may be a home appliance with built-in audio equipment, such as a smart TV, or a home appliance without an audio equipment, such as a smart projector, a smart display, etc., which are not specifically limited here.
  • terminal device 1 may be a human-computer interaction device with a relatively small display screen
  • terminal device 2 may be a device with a relatively large display screen.
  • terminal device 1 may be a mobile phone
  • terminal device 2 may be a TV. machine or PC.
  • the system architecture may also include a greater number of terminals.
  • the terminal device 1 may communicate with multiple terminal devices.
  • the above-mentioned multiple terminal devices may also communicate with each other. connect.
  • FIG. 3 is a schematic structural diagram of a terminal device provided by an embodiment of the present application.
  • the terminal device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (universal serial bus, USB) interface 130, a charging management module 140, a power management module 141, and a battery 142 , antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, earphone jack 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193 , a display screen 194, and a subscriber identification module (subscriber identification module, SIM) card interface 195, etc.
  • SIM subscriber identification module
  • the sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, bone conduction sensor 180M, etc.
  • the structure shown in the embodiment of the present application does not constitute a specific limitation on the terminal device 100 .
  • the terminal device 100 may include more or fewer components than shown in the figure, or combine certain components, or separate certain components, or arrange different components.
  • the illustrated components can be realized in hardware, software or a combination of software and hardware.
  • the processor 110 may include one or more processing units, for example: the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processing unit (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), controller, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural network processor (neural-network processing unit, NPU), etc. Wherein, different processing units may be independent devices, or may be integrated in one or more processors.
  • application processor application processor, AP
  • modem processor graphics processing unit
  • GPU graphics processing unit
  • image signal processor image signal processor
  • ISP image signal processor
  • controller video codec
  • digital signal processor digital signal processor
  • baseband processor baseband processor
  • neural network processor neural-network processing unit
  • the controller can generate an operation control signal according to the instruction opcode and timing signal, and complete the control of fetching and executing the instruction.
  • a memory may also be provided in the processor 110 for storing instructions and data.
  • the memory in processor 110 is a cache memory.
  • the memory may hold instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to use the instruction or data again, it can be called directly from the memory. Repeated access is avoided, and the waiting time of the processor 110 is reduced, thus improving the efficiency of the system.
  • processor 110 may include one or more interfaces.
  • the interface may include an integrated circuit (inter-integrated circuit, I2C) interface, an integrated circuit built-in audio (inter-integrated circuit sound, I2S) interface, a pulse code modulation (pulse code modulation, PCM) interface, a universal asynchronous transmitter (universal asynchronous receiver/transmitter, UART) interface, mobile industry processor interface (mobile industry processor interface, MIPI), general-purpose input and output (general-purpose input/output, GPIO) interface, subscriber identity module (subscriber identity module, SIM) interface, and /or universal serial bus (universal serial bus, USB) interface, etc.
  • I2C integrated circuit
  • I2S integrated circuit built-in audio
  • PCM pulse code modulation
  • PCM pulse code modulation
  • UART universal asynchronous transmitter
  • MIPI mobile industry processor interface
  • GPIO general-purpose input and output
  • subscriber identity module subscriber identity module
  • SIM subscriber identity module
  • USB universal serial bus
  • the I2C interface is a bidirectional synchronous serial bus, including a serial data line (serial data line, SDA) and a serial clock line (derail clock line, SCL).
  • processor 110 may include multiple sets of I2C buses.
  • the processor 110 can be respectively coupled to the touch sensor 180K, the charger, the flashlight, the camera 193 and the like through different I2C bus interfaces.
  • the processor 110 may be coupled to the touch sensor 180K through an I2C interface, so that the processor 110 communicates with the touch sensor 180K through an I2C bus interface to realize the touch function of the terminal device 100 .
  • the I2S interface can be used for audio communication.
  • processor 110 may include multiple sets of I2S buses.
  • the processor 110 may be coupled to the audio module 170 through an I2S bus to implement communication between the processor 110 and the audio module 170 .
  • the audio module 170 can transmit audio signals to the wireless communication module 160 through the I2S interface, so as to realize the function of answering calls through the Bluetooth headset.
  • the PCM interface can also be used for audio communication, sampling, quantizing and encoding the analog signal.
  • the audio module 170 and the wireless communication module 160 may be coupled through a PCM bus interface.
  • the audio module 170 can also transmit audio signals to the wireless communication module 160 through the PCM interface, so as to realize the function of answering calls through the Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.
  • the UART interface is a universal serial data bus used for asynchronous communication.
  • the bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication.
  • a UART interface is generally used to connect the processor 110 and the wireless communication module 160 .
  • the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to realize the Bluetooth function.
  • the audio module 170 can transmit audio signals to the wireless communication module 160 through the UART interface, so as to realize the function of playing music through the Bluetooth headset.
  • the MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193 .
  • MIPI interface includes camera serial interface (camera serial interface, CSI), display serial interface (display serial interface, DSI), etc.
  • the processor 110 communicates with the camera 193 through a CSI interface to realize the shooting function of the terminal device 100 .
  • the processor 110 communicates with the display screen 194 through the DSI interface to realize the display function of the terminal device 100 .
  • the GPIO interface can be configured by software.
  • the GPIO interface can be configured as a control signal or as a data signal.
  • the GPIO interface can be used to connect the processor 110 with the camera 193 , the display screen 194 , the wireless communication module 160 , the audio module 170 , the sensor module 180 and so on.
  • the GPIO interface can also be configured as an I2C interface, I2S interface, UART interface, MIPI interface, etc.
  • the USB interface 130 is an interface conforming to the USB standard specification, specifically, it can be a Mini USB interface, a Micro USB interface, a USB Type C interface, and the like.
  • the USB interface 130 can be used to connect a charger to charge the terminal device 100, and can also be used to transmit data between the terminal device 100 and peripheral devices. It can also be used to connect headphones and play audio through them. This interface can also be used to connect other electronic devices, such as AR devices.
  • the interface connection relationship between the modules shown in the embodiment of the present application is only a schematic illustration, and does not constitute a structural limitation of the terminal device 100 .
  • the terminal device 100 may also adopt different interface connection modes in the foregoing embodiments, or a combination of multiple interface connection modes.
  • the charging management module 140 is configured to receive a charging input from a charger.
  • the charger may be a wireless charger or a wired charger.
  • the charging management module 140 can receive charging input from the wired charger through the USB interface 130 .
  • the charging management module 140 may receive wireless charging input through the wireless charging coil of the terminal device 100 . While the charging management module 140 is charging the battery 142 , it can also provide power for electronic devices through the power management module 141 .
  • the power management module 141 is used for connecting the battery 142 , the charging management module 140 and the processor 110 .
  • the power management module 141 receives the input from the battery 142 and/or the charging management module 140 to provide power for the processor 110 , the internal memory 121 , the display screen 194 , the camera 193 , and the wireless communication module 160 .
  • the power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle times, and battery health status (leakage, impedance).
  • the power management module 141 may also be disposed in the processor 110 .
  • the power management module 141 and the charging management module 140 may also be set in the same device.
  • the wireless communication function of the terminal device 100 may be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, a modem processor, a baseband processor, and the like.
  • Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals.
  • Each antenna in the terminal device 100 can be used to cover single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization of the antennas.
  • Antenna 1 can be multiplexed as a diversity antenna of a wireless local area network.
  • the antenna may be used in conjunction with a tuning switch.
  • the mobile communication module 150 can provide wireless communication solutions including 2G/3G/4G/5G applied on the terminal device 100 .
  • the mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (low noise amplifier, LNA) and the like.
  • the mobile communication module 150 can receive electromagnetic waves through the antenna 1, filter and amplify the received electromagnetic waves, and send them to the modem processor for demodulation.
  • the mobile communication module 150 can also amplify the signals modulated by the modem processor, and convert them into electromagnetic waves through the antenna 1 for radiation.
  • at least part of the functional modules of the mobile communication module 150 may be set in the processor 110 .
  • at least part of the functional modules of the mobile communication module 150 and at least part of the modules of the processor 110 may be set in the same device.
  • a modem processor may include a modulator and a demodulator.
  • the modulator is used for modulating the low-frequency baseband signal to be transmitted into a medium-high frequency signal.
  • the demodulator is used to demodulate the received electromagnetic wave signal into a low frequency baseband signal. Then the demodulator sends the demodulated low-frequency baseband signal to the baseband processor for processing.
  • the low-frequency baseband signal is passed to the application processor after being processed by the baseband processor.
  • the application processor outputs sound signals through audio equipment (not limited to speaker 170A, receiver 170B, etc.), or displays images or videos through display screen 194 .
  • the modem processor may be a stand-alone device.
  • the modem processor may be independent from the processor 110, and be set in the same device as the mobile communication module 150 or other functional modules.
  • the wireless communication module 160 can provide wireless local area networks (wireless local area networks, WLAN) (such as wireless fidelity (Wireless Fidelity, Wi-Fi) network), bluetooth (bluetooth, BT), global navigation satellite, etc. System (global navigation satellite system, GNSS), frequency modulation (frequency modulation, FM), near field communication technology (near field communication, NFC), infrared technology (infrared, IR) and other wireless communication solutions.
  • the wireless communication module 160 may be one or more devices integrating at least one communication processing module.
  • the wireless communication module 160 receives electromagnetic waves via the antenna 2 , frequency-modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110 .
  • the wireless communication module 160 can also receive the signal to be sent from the processor 110 , frequency-modulate it, amplify it, and convert it into electromagnetic waves through the antenna 2 for radiation.
  • the antenna 1 of the terminal device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the terminal device 100 can communicate with the network and other devices through wireless communication technology.
  • the wireless communication technology may include global system for mobile communications (GSM), general packet radio service (general packet radio service, GPRS), code division multiple access (code division multiple access, CDMA), broadband Code division multiple access (wideband code division multiple access, WCDMA), time division code division multiple access (time-division code division multiple access, TD-SCDMA), long term evolution (long term evolution, LTE), BT, GNSS, WLAN, NFC , FM, and/or IR techniques, etc.
  • GSM global system for mobile communications
  • GPRS general packet radio service
  • code division multiple access code division multiple access
  • CDMA broadband Code division multiple access
  • WCDMA wideband code division multiple access
  • time division code division multiple access time-division code division multiple access
  • TD-SCDMA time-division code division multiple access
  • the GNSS may include a global positioning system (global positioning system, GPS), a global navigation satellite system (global navigation satellite system, GLONASS), a Beidou navigation satellite system (beidou navigation satellite system, BDS), a quasi-zenith satellite system (quasi -zenith satellite system (QZSS) and/or satellite based augmentation systems (SBAS).
  • GPS global positioning system
  • GLONASS global navigation satellite system
  • Beidou navigation satellite system beidou navigation satellite system
  • BDS Beidou navigation satellite system
  • QZSS quasi-zenith satellite system
  • SBAS satellite based augmentation systems
  • the terminal device 100 implements a display function through a GPU, a display screen 194, an application processor, and the like.
  • the GPU is a microprocessor for image processing, and is connected to the display screen 194 and the application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering.
  • Processor 110 may include one or more GPUs that execute program instructions to generate or change display information. Specifically, one or more GPUs in the processor 110 may implement an image rendering task.
  • the display screen 194 is used to display images, videos and the like.
  • the display screen 194 includes a display panel.
  • the display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active matrix organic light emitting diode or an active matrix organic light emitting diode (active-matrix organic light emitting diode, AMOLED), flexible light-emitting diode (flex light-emitting diode, FLED), Miniled, MicroLed, Micro-oLed, quantum dot light emitting diodes (quantum dot light emitting diodes, QLED), etc.
  • the terminal device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.
  • the terminal device 100 can realize the shooting function through the ISP, the camera 193 , the video codec, the GPU, the display screen 194 and the application processor.
  • the ISP is used for processing the data fed back by the camera 193 .
  • the light is transmitted to the photosensitive element of the camera through the lens, the light signal is converted into an electrical signal, and the photosensitive element of the camera transmits the electrical signal to the ISP for processing, and converts it into an image visible to the naked eye.
  • ISP can also perform algorithm optimization on image noise, brightness, and skin color.
  • ISP can also optimize the exposure, color temperature and other parameters of the shooting scene.
  • the ISP may be located in the camera 193 .
  • Camera 193 is used to capture still images or video.
  • the object generates an optical image through the lens and projects it to the photosensitive element.
  • the photosensitive element may be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor.
  • CMOS complementary metal-oxide-semiconductor
  • the photosensitive element converts the light signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal.
  • the ISP outputs the digital image signal to the DSP for processing.
  • DSP converts digital image signals into standard RGB, YUV and other image signals.
  • the terminal device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.
  • Video codecs are used to compress or decompress digital video.
  • the terminal device 100 may support one or more video codecs.
  • the terminal device 100 can play or record videos in various encoding formats, for example: moving picture experts group (moving picture experts group, MPEG) 1, MPEG2, MPEG3, MPEG4, etc.
  • the NPU is a neural-network (NN) computing processor.
  • NN neural-network
  • the NPU can quickly process input information and continuously learn by itself.
  • Applications such as intelligent cognition of the terminal device 100 can be implemented through the NPU, such as image recognition, face recognition, speech recognition, text understanding, and the like.
  • the external memory interface 120 may be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the terminal device 100.
  • the external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. Such as saving music, video and other files in the external memory card.
  • the internal memory 121 may be used to store computer-executable program codes including instructions.
  • the internal memory 121 may include an area for storing programs and an area for storing data.
  • the stored program area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, etc.) and the like.
  • the storage data area can store data created during the use of the terminal device 100 (such as audio data, phonebook, etc.) and the like.
  • the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, flash memory device, universal flash storage (universal flash storage, UFS) and the like.
  • the processor 110 executes various functional applications and data processing of the terminal device 100 by executing instructions stored in the internal memory 121 and/or instructions stored in a memory provided in the processor.
  • the terminal device 100 may implement an audio function through an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, and an application processor. Such as music playback, recording, etc.
  • the audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signal.
  • the audio module 170 may also be used to encode and decode audio signals.
  • the audio module 170 may be set in the processor 110 , or some functional modules of the audio module 170 may be set in the processor 110 .
  • Speaker 170A also referred to as a "horn" is used to convert audio electrical signals into sound signals.
  • the terminal device 100 can listen to music through the speaker 170A, or listen to hands-free calls.
  • Receiver 170B also called “earpiece” is used to convert audio electrical signals into sound signals.
  • the receiver 170B can be placed close to the human ear to listen to the voice.
  • the microphone 170C also called “microphone” or “microphone” is used to convert sound signals into electrical signals. When making a phone call or sending a voice message, the user can put his mouth close to the microphone 170C to make a sound, and input the sound signal to the microphone 170C.
  • the terminal device 100 may be provided with at least one microphone 170C. In some other embodiments, the terminal device 100 may be provided with two microphones 170C, which may also implement a noise reduction function in addition to collecting sound signals. In some other embodiments, the terminal device 100 can also be provided with three, four or more microphones 170C to realize sound signal collection, noise reduction, identify sound sources, and realize directional recording functions, etc.
  • the earphone interface 170D is used for connecting wired earphones.
  • the earphone interface 170D can be a USB interface 130, or a 3.5mm open mobile terminal platform (OMTP) standard interface, or a cellular telecommunications industry association of the USA (CTIA) standard interface.
  • OMTP open mobile terminal platform
  • CTIA cellular telecommunications industry association of the USA
  • the pressure sensor 180A is used to sense the pressure signal and convert the pressure signal into an electrical signal.
  • pressure sensor 180A may be disposed on display screen 194 .
  • pressure sensors 180A such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors.
  • a capacitive pressure sensor may be comprised of at least two parallel plates with conductive material.
  • the terminal device 100 determines the intensity of pressure according to the change in capacitance.
  • the terminal device 100 detects the intensity of the touch operation according to the pressure sensor 180A.
  • the terminal device 100 may also calculate the touched position according to the detection signal of the pressure sensor 180A.
  • touch operations acting on the same touch position but with different touch operation intensities may correspond to different operation instructions. For example: when a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, an instruction to view short messages is executed. When a touch operation whose intensity is greater than or equal to the first pressure threshold acts on the icon of the short message application, the instruction of creating a new short message is executed.
  • the gyroscope sensor 180B can be used to determine the motion posture of the terminal device 100 .
  • the angular velocity of the terminal device 100 around three axes ie, x, y and z axes
  • the gyro sensor 180B can be used for image stabilization.
  • the gyro sensor 180B detects the shaking angle of the terminal device 100, calculates the distance that the lens module needs to compensate according to the angle, and allows the lens to counteract the shaking of the terminal device 100 through reverse motion to achieve anti-shake.
  • the gyro sensor 180B can also be used for navigation and somatosensory game scenes.
  • the air pressure sensor 180C is used to measure air pressure.
  • the terminal device 100 calculates the altitude based on the air pressure value measured by the air pressure sensor 180C to assist positioning and navigation.
  • the magnetic sensor 180D includes a Hall sensor.
  • the terminal device 100 may use the magnetic sensor 180D to detect the opening and closing of the flip holster.
  • the terminal device 100 may detect opening and closing of the clamshell according to the magnetic sensor 180D.
  • features such as automatic unlocking of the flip cover are set.
  • the acceleration sensor 180E can detect the acceleration of the terminal device 100 in various directions (generally three axes). When the terminal device 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of electronic devices, and can be used in applications such as horizontal and vertical screen switching, pedometers, etc.
  • the distance sensor 180F is used to measure the distance.
  • the terminal device 100 can measure the distance by infrared or laser. In some embodiments, when shooting a scene, the terminal device 100 may use the distance sensor 180F for distance measurement to achieve fast focusing.
  • Proximity light sensor 180G may include, for example, light emitting diodes (LEDs) and light detectors, such as photodiodes.
  • the light emitting diodes may be infrared light emitting diodes.
  • the terminal device 100 emits infrared light through the light emitting diode.
  • the terminal device 100 detects infrared reflected light from nearby objects using a photodiode. When sufficient reflected light is detected, it can be determined that there is an object near the terminal device 100 . When insufficient reflected light is detected, the terminal device 100 may determine that there is no object near the terminal device 100 .
  • the terminal device 100 can use the proximity light sensor 180G to detect that the user holds the terminal device 100 close to the ear to make a call, so as to automatically turn off the screen to save power.
  • the proximity light sensor 180G can also be used in leather case mode, automatic unlock and lock screen in pocket mode.
  • the ambient light sensor 180L is used for sensing ambient light brightness.
  • the terminal device 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness.
  • the ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures.
  • the ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the terminal device 100 is in the pocket to prevent accidental touch.
  • the fingerprint sensor 180H is used to collect fingerprints.
  • the terminal device 100 can use the collected fingerprint characteristics to realize fingerprint unlocking, access to the application lock, take pictures with fingerprints, answer incoming calls with fingerprints, and so on.
  • the temperature sensor 180J is used to detect temperature.
  • the terminal device 100 uses the temperature detected by the temperature sensor 180J to implement a temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds the threshold, the terminal device 100 may reduce the performance of a processor located near the temperature sensor 180J, so as to reduce power consumption and implement thermal protection.
  • the terminal device 100 when the temperature is lower than another threshold, the terminal device 100 heats the battery 142 to avoid abnormal shutdown of the terminal device 100 due to low temperature.
  • the terminal device 100 boosts the output voltage of the battery 142 to avoid abnormal shutdown caused by low temperature.
  • the touch sensor 180K is also called “touch device”.
  • the touch sensor 180K can be disposed on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also called a “touch screen”.
  • the touch sensor 180K is used to detect a touch operation on or near it.
  • the touch sensor can pass the detected touch operation to the application processor to determine the type of touch event.
  • Visual output related to the touch operation can be provided through the display screen 194 .
  • the touch sensor 180K may also be disposed on the surface of the terminal device 100 , which is different from the position of the display screen 194 .
  • the bone conduction sensor 180M can acquire vibration signals. In some embodiments, the bone conduction sensor 180M can acquire the vibration signal of the vibrating bone mass of the human voice. The bone conduction sensor 180M can also contact the human pulse and receive the blood pressure beating signal. In some embodiments, the bone conduction sensor 180M can also be disposed in the earphone, combined into a bone conduction earphone.
  • the audio module 170 can analyze the voice signal based on the vibration signal of the vibrating bone mass of the vocal part acquired by the bone conduction sensor 180M, so as to realize the voice function.
  • the application processor can analyze the heart rate information based on the blood pressure beating signal acquired by the bone conduction sensor 180M, so as to realize the heart rate detection function.
  • the keys 190 include a power key, a volume key and the like.
  • the key 190 may be a mechanical key. It can also be a touch button.
  • the terminal device 100 may receive key input and generate key signal input related to user settings and function control of the terminal device 100 .
  • the motor 191 can generate a vibrating reminder.
  • the motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback.
  • touch operations applied to different applications may correspond to different vibration feedback effects.
  • the motor 191 may also correspond to different vibration feedback effects for touch operations acting on different areas of the display screen 194 .
  • Different application scenarios for example: time reminder, receiving information, alarm clock, games, etc.
  • the touch vibration feedback effect can also support customization.
  • the indicator 192 can be an indicator light, and can be used to indicate charging status, power change, and can also be used to indicate messages, missed calls, notifications, and the like.
  • the SIM card interface 195 is used for connecting a SIM card.
  • the SIM card can be connected and separated from the terminal device 100 by inserting it into the SIM card interface 195 or pulling it out from the SIM card interface 195 .
  • the terminal device 100 may support 1 or N SIM card interfaces, where N is a positive integer greater than 1.
  • SIM card interface 195 can support Nano SIM card, Micro SIM card, SIM card etc. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the multiple cards may be the same or different.
  • the SIM card interface 195 is also compatible with different types of SIM cards.
  • the SIM card interface 195 is also compatible with external memory cards.
  • the terminal device 100 interacts with the network through the SIM card to implement functions such as calling and data communication.
  • the terminal device 100 adopts an eSIM, that is, an embedded SIM card.
  • the eSIM card can be embedded in the terminal device 100 and cannot be separated from the terminal device 100 .
  • the software system of the terminal device 100 may adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture.
  • an Android system with a layered architecture is taken as an example to illustrate the software structure of the terminal device 100 .
  • FIG. 4 is a block diagram of a software structure of a terminal device 100 according to an embodiment of the present disclosure.
  • the layered architecture divides the software into several layers, and each layer has a clear role and division of labor. Layers communicate through software interfaces.
  • the Android system is divided into four layers, which are respectively the application program layer, the application program framework layer, the Android runtime (Android runtime) and the system library, and the kernel layer from top to bottom.
  • the application layer can consist of a series of application packages.
  • the application package may include application programs such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and short message.
  • application programs such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and short message.
  • the application framework layer provides an application programming interface (application programming interface, API) and a programming framework for applications in the application layer.
  • the application framework layer includes some predefined functions.
  • the application framework layer can include window manager, content provider, view system, phone manager, resource manager, notification manager, etc.
  • a window manager is used to manage window programs.
  • the window manager can get the size of the display screen, determine whether there is a status bar, lock the screen, capture the screen, etc.
  • Content providers are used to store and retrieve data and make it accessible to applications.
  • Said data may include video, images, audio, calls made and received, browsing history and bookmarks, phonebook, etc.
  • the view system includes visual controls, such as controls for displaying text, controls for displaying pictures, and so on.
  • the view system can be used to build applications.
  • a display interface can consist of one or more views.
  • a display interface including a text message notification icon may include a view for displaying text and a view for displaying pictures.
  • the phone manager is used to provide the communication function of the terminal device 100 .
  • the management of call status including connected, hung up, etc.).
  • the resource manager provides various resources for the application, such as localized strings, icons, pictures, layout files, video files, and so on.
  • the notification manager enables the application to display notification information in the status bar, which can be used to convey notification-type messages, and can automatically disappear after a short stay without user interaction.
  • the notification manager is used to notify download completion, message reminders, etc.
  • the notification manager can also be a notification that appears on the top status bar of the system in the form of a chart or scroll bar text, such as a notification of an application running in the background, or a notification that appears on the screen in the form of a dialog window.
  • prompting text information in the status bar issuing a prompt sound, vibrating the electronic device, and flashing the indicator light, etc.
  • the Android Runtime includes core library and virtual machine. The Android runtime is responsible for the scheduling and management of the Android system.
  • the core library consists of two parts: one part is the function function that the java language needs to call, and the other part is the core library of Android.
  • the application layer and the application framework layer run in virtual machines.
  • the virtual machine executes the java files of the application program layer and the application program framework layer as binary files.
  • the virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.
  • a system library can include multiple function modules. For example: surface manager (surface manager), media library (Media Libraries), 3D graphics processing library (eg: OpenGL ES), 2D graphics engine (eg: SGL), etc.
  • the surface manager is used to manage the display subsystem and provides the fusion of 2D and 3D layers for multiple applications.
  • the media library supports playback and recording of various commonly used audio and video formats, as well as still image files, etc.
  • the media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.
  • the 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing, etc.
  • 2D graphics engine is a drawing engine for 2D drawing.
  • the kernel layer is the layer between hardware and software.
  • the kernel layer includes at least a display driver, a camera driver, an audio driver, and a sensor driver.
  • FIG. 5 is a schematic flowchart of an application running method provided by an embodiment of the present application.
  • the method is applied to a target system, and the target system includes a first terminal device and a second terminal device.
  • the first terminal The device communicates with the second terminal device.
  • an application running method provided in an embodiment of the present application includes:
  • the first terminal device acquires device information and/or a software execution environment of a second terminal device when running a target application.
  • the user may run the target application on the first terminal device, and the target application may be a game application, an application with a dialogue function, a data analysis application, an audio and video playback application, and the like.
  • the target application is an application with high computing power and storage overhead.
  • the computing capability of the first terminal device is limited, which will cause the computing resources occupied by the first terminal device to be excessive when running the target application. High, in order to reduce the resource load of the first terminal device when running the target application, part of the computing tasks of the target application can be transferred to other terminal devices and executed by other terminal devices.
  • the first terminal device when running the target application, may present options for the user to select to determine whether the user needs or allows the first terminal device to transfer part of the computing tasks of the target application to other terminal devices .
  • the first terminal device may display an interface as shown in the figure, and the interface may include two options ("stand-alone mode” option and " “distributed mode” option), the user can determine whether to enter the stand-alone mode or the distributed mode by selecting on the interface. If the user selects the stand-alone mode, the first terminal device can run the target application, and will not use a part of the target application to calculate tasks Transferring to other terminal devices, if the user selects the distributed mode, the first terminal device can run the target application, and transfer a part of the computing tasks of the target application to other terminal devices.
  • the first terminal device may acquire terminal devices that have a communication connection with the first terminal device, and select a terminal device that may be used as a task transfer object from the terminal devices that have a communication connection.
  • the communication connection can be at least one of Internet network, WIFI, Bluetooth, ultra-wideband, ZigBee, (radio frequency identification, RFID), 4G, 5G, LoRa, SIGFOX, Z-Wave and NB-loT communication .
  • the first terminal device and the second terminal device are connected by near field communication, WIFI, Bluetooth, ultra-wideband, ZigBee, RFID, 4G, 5G, LoRa, SIGFOX, Z-Wave and NB-loT Connect in various ways.
  • connection between the first terminal device and the second terminal device may also be performed through other wireless connection methods with low delay (such as 3GPP, non-3GPP, and connection methods based on short-distance technology). Not limited.
  • the first terminal device may be a mobile phone
  • the second terminal device may be a PC
  • the phone and the desktop interact with each other through wireless means, such as WiFi Direct or WiFi hotspots
  • the Desktop and the Monitor are connected through a dedicated cable, such as GMSL.
  • GMSL dedicated cable
  • the difference is that the current distributed applications can be mainly classified into three categories.
  • the first category is Mapreduce-like distributed multi-task frameworks. This type of framework is mostly used in homogeneous systems, especially in server clusters, for concurrent execution of multiple tasks.
  • there is an essential difference in the dynamic division of application tasks and at the same time, less consideration is given to task execution and synchronization in heterogeneous environments, which cannot achieve the ultimate goal of this example, which is to share the load pressure of the first device; the streaming computing of the second type of image , the main technical point is to divide the image task to the second terminal device, and then return to the first device for display.
  • This process can realize part of the computing power transfer, but the application scenarios are limited, especially the large amount of image data itself, which cannot support high real-time application scenarios.
  • this example can deal with the communication bottleneck problem after segmentation.
  • the third type of cloud game applications this type of application renders images through the cloud and transmits them back to the local area to realize partial computing power transfer. Compared with the previous type, it has poorer support for real-time performance and does not have the ability to split application tasks. Just simple image task rendering callbacks.
  • the first terminal device may acquire the device information of the terminal device that has a communication connection, and based on the analysis of the device information, determine whether the terminal device has the ability to run the task to be transferred, and determine whether the Which part of the task is offloaded to the terminal device.
  • the device information and/or application information may include the current device running status and possible application running status information of the second terminal device.
  • selective synchronization is also required.
  • it can be called application runtime information, which includes device information, existing application information, and transferred application running information.
  • the first task may be one of a UI display task, an image rendering task, a logic state update task, an AI task, and a data processing or analysis task.
  • the AI engine can be used to perform AI tasks
  • the rendering engine can be used to perform rendering tasks
  • the logic engine and physics engine can be used to perform logic state update tasks. If the second terminal device is not deployed with the ability to perform the first task If the running component does not need to transfer the first task to the second terminal device for execution.
  • the running components can include a logic state update engine, a physics engine, an AI engine, a rendering engine, etc.
  • the logic state update engine and the physics engine can be called simulation engines, and the simulation engine can execute game logic and control The execution of the game setting simulation.
  • the rendering engine may be referred to as a rendering engine, and the rendering engine may control the execution of the rendering of the game settings frame and the output of the rendering of the frame.
  • the simulation engine can read in game rules and generate game states based on input received from one or more users.
  • the simulation engine can control the execution of individual objects (such as virtual components, virtual effects, and/or virtual characters) within the gaming application.
  • the simulation engine can manage and determine the character movement, character state, collision detection, and can derive the expected movement of the character based on the collision.
  • the simulation engine receives user input and determines character events, such as actions, collisions, runs, throws, attacks, and other events appropriate to the game. Character events may be governed by character movement rules that determine the appropriate movement a character should make in response to an event.
  • the simulation engine can include a physics engine that can determine new poses for the character.
  • the physics engine can take as its input: skeletal models of various characters, environment settings, character states such as current pose (e.g., position of body parts expressed as positions, joint angles, or other specifications), and The velocities (linear and/or angular velocities) of the body parts and motions, which can be in the form of a collection of force/torque vectors for some or all of the body parts. Based on this information, the physics engine generates new poses for the character using physics rules, and these new poses can be used to update the character state.
  • the simulation engine provides user input according to defined game rules to control various aspects of the game application. Examples of game rules include scoring rules, possible inputs, actions/events, moves in response to inputs, and the like. Other components can control what input is accepted and how the game plays and other aspects of game setup.
  • the simulation engine can output graphics state data that is used by the rendering engine to generate and render frames within the gaming application.
  • Each virtual object can be configured as a state flow process through the simulation engine.
  • Each state flow process can generate graphics state data for the rendering engine.
  • a state flow process may include various virtual objects such as emitters, lights, models, occluders, terrain, visual environments, and other virtual objects that affect the state of the game along with the game application. This article further details the execution of the simulation engine.
  • the rendering engine can use the graphics state data within the gaming application to generate and render frames for output to the display.
  • the rendering engine can combine virtual objects, such as virtual characters, animated objects, non-animated objects, background objects, lighting, reflections, etc., to generate complete scenes and new frames for display.
  • the rendering engine takes into account surfaces, color textures, and other parameters during the rendering process.
  • the rendering engine can combine virtual objects (eg, lighting and virtual character images in a virtual environment with non-animated objects and background objects) to generate and render frames. This document further details the execution of the rendering engine.
  • the target application can be split into multiple tasks.
  • the game application can be dynamically split into two tasks, a game logic state update task and a rendering task.
  • the requirements for the game logic update task on the equipment can be CPU equipment, CPU main frequency above 2.1Ghz, memory greater than 2GB, Android system or Windows system, audio input and output, touch screen input is required, and the transmission bandwidth is above 50Mbps; rendering tasks require CPU Equipment, CPU frequency above 2.1Ghz, high demand for GPU equipment, display device above 1080P, above 60Hz, receiving bandwidth requirements, etc.
  • the device information may include power information, and the power information may be the available power and charging mode of the second terminal device (such as active input or battery power supply), for example, when the power supply information indicates that the available power of the second terminal device is low, it may be determined that the first task does not need to be transferred to the second terminal device for execution.
  • the power information may be the available power and charging mode of the second terminal device (such as active input or battery power supply), for example, when the power supply information indicates that the available power of the second terminal device is low, it may be determined that the first task does not need to be transferred to the second terminal device for execution.
  • the device information may include processor information
  • the processor may be CPU, GPU, TPU, NPU, etc.
  • the processor information may be but not limited to CPU model/main frequency, GPU model/main frequency, current CPU/GPU load, etc. Since some tasks have certain requirements on the processor type, processing capability, and currently available resources, if the task requirements are not met, it can be determined that the first task does not need to be transferred to the second terminal device for execution.
  • the device information may include memory information, such as memory size.
  • memory information such as memory size.
  • the device information may include transmission bandwidth, and the transmission bandwidth may be an available bandwidth of the current network.
  • the transmission bandwidth between devices needs to meet the requirements corresponding to the first task. When the requirement is not met, it may be determined that the first task does not need to be transferred to the second terminal device for execution.
  • the device information may include input device information
  • the input device information may include the type of input device
  • the input device may be audio input, mouse input, or keyboard input. wait.
  • the first task also needs to support user input, and since the first task is transferred to the second terminal device for execution, the second terminal device is required to have the input required by the first task Equipment type. When the requirement is not met, it may be determined that the first task does not need to be transferred to the second terminal device for execution.
  • the device information may include output device information
  • the output device information may include the output device type
  • the output device may be audio output, image/video output, etc.
  • the output device information may also display device display properties such as screen resolution and refresh rate.
  • the first task also needs to support output (for example, the rendering task requires the presence of a display device, and the parameter requirements of the display device), and since the first task is transferred to the second terminal device for execution, Then the second terminal device needs to have the output device type and the attributes of the output device required by the first task. When the requirement is not met, it may be determined that the first task does not need to be transferred to the second terminal device for execution.
  • the device information may include network load information
  • the network load information may include the current device network card working status, network bandwidth occupancy rate, and end-to-end delay time wait. Since some tasks have certain requirements for network processing capabilities and currently available network resources, if the transfer task requirements are not met, it can be determined that the first task does not need to be transferred to the second terminal device for execution; or the current device has a high network load , and the second terminal device may decide to execute the task transfer immediately if the network resources are sufficient.
  • the first terminal device may initiate a request to obtain device information from the second terminal device.
  • the first terminal device is a mobile phone
  • the processor of the mobile phone initiates the acquisition of device specifications and status through the information collection module.
  • Information request to obtain connected devices such as PC
  • PC returns device information
  • device information can include [CPU model/main frequency, GPU model/main frequency, memory size, screen resolution/refresh rate, input device type, Audio output, network interface/capability, whether connected to the Internet, battery capacity, current CPU/GPU load, current available network bandwidth, current remaining memory], for example, the information returned by the PC can be [Inteli711260/3.2Ghz, GTX3060, 16GB, 4K/ 120, [audio input, mouse input, keyboard input], audio output, WIFI802.11ax, connected to the Internet, active, CPU load 30%/GPU load 5%, current available network bandwidth 200Mbps, current remaining memory 14GB], phone returns
  • the device information can be [ARMv8/2.6Ghz, Mali65, 4GB, 10
  • the first terminal device determines multiple terminal devices meeting the requirements, some (or all) terminal devices may be selected from the multiple terminal devices as transfer objects of the first task.
  • device information of multiple terminal devices connected to the first terminal device may be acquired, where the multiple terminal devices include the second terminal device, and the Selection of said second terminal device among the terminal devices.
  • second selection information may be presented, the second selection information is used to instruct the user to select from the plurality of terminal devices, and the second selection information fed back by the user according to the second selection information is received. an input; including the second terminal device based on the second input, enabling selection for the second terminal device of the plurality of terminal devices.
  • FIG. 7 is a schematic diagram of a second selection information presentation method.
  • FIG. 7 shows the display interface of the first terminal device, which may include options for multiple terminal devices, and the user can select from the options of multiple terminal devices Select the transfer object of the first task.
  • the first terminal device may also select, based on the device information of the second terminal device, the device information that best meets the execution requirements of the first task among the multiple terminal devices, enabling the Selection of the second terminal device among the plurality of terminal devices.
  • the so-called best performance requirements can be understood as selecting a terminal device with the best performance among terminal devices that meet the execution requirements of the first task.
  • the first terminal device can match the device list according to the logic state update task and the rendering display task requirements, and output the best scheduling result.
  • the rendering and display tasks require higher screen resolution, refresh rate, and GPU specifications, and the PC GPU specifications, screen resolution, and refresh rate specifications of the peripheral devices are higher, and the game logic update task requires touch screen input. Only mobile devices can meet the requirements, so the best task scheduling result in this case is that the logic update task is scheduled locally on the mobile phone, and the rendering task is scheduled on the PC.
  • the first terminal device can determine that the second terminal device is the transfer object of the first task. It should be understood that, in addition to the second terminal device, the first terminal device can also determine other terminal devices as the transfer object of the first task , is not limited here.
  • the first terminal device Based on the device information of the second terminal device meeting the execution requirements of the first task of the target application, the first terminal device transmits task execution information of the first task to the second terminal device, and The first terminal device does not execute the first task.
  • the first terminal device when the first terminal device transfers the first task, it needs to transfer data related to the execution of the first task (or called task running information) to the second terminal device.
  • the device information and or application information may further include: the device status of the terminal device and/or the running status of the running component; when the device status indicates that the second terminal device If it is not started, the task running information related to the first task includes first indication information, and the first indication information is used to trigger the second terminal device to start and instruct the second terminal device to run the running component, and execute the first task through the running component; or, when the device state indicates that the second terminal device is started, and the running state indicates that the second terminal device is not running the running component
  • the task running information related to the first task includes second instruction information, and the second instruction information is used to instruct the second terminal device to run the running component and execute the first task through the running component.
  • the information includes third indication information, where the third indication information is used to instruct the second terminal device to execute the first task through the running component.
  • the task running information can include the task list and the startup parameters of the distributed game, for example, game.exe–task render (and other information related to rendering), through the network Send to the second terminal device, and start the task.
  • the first terminal device can start the game locally, and start the game logic state update task.
  • task execution information may include execution data.
  • the execution data may include game data, game state data, and so on.
  • the game data may include game rules, pre-recorded motion capture poses/paths, environment settings, environment objects, constraints, skeleton models, route information and/or other game application information. At least a portion of the game data may be stored in the application data store. In some embodiments, a portion of the game data may be received and/or stored remotely, such as in a data store, and in such embodiments, the game data may be received during runtime of the game application.
  • the game state data may include game state, character state, environment state, scene object storage, route information and/or other information associated with the runtime state of the game application.
  • game state data may identify the state of a game application at a particular point in time, such as character position, character orientation, character actions, game level attributes, and other information that contributes to the state of the game application.
  • Game state data may include simulation state data and graphics state data.
  • the simulation state data includes game state data used by the simulation engine to perform a simulation of the game application.
  • Graphics state data includes game state data generated based on simulation state data and used by the rendering engine to generate and render frames for output, such as to a display of a user computing system.
  • the first terminal device may be responsible for logic tasks in the game application (may be referred to as the second task), and the logic tasks are mainly the processing logic of the game application itself, such as processing game input and moving the corresponding game object position.
  • the first terminal device can synchronize the state set to the second terminal device through its own synchronization module, and the state set can be a logical entity: the real data is stored in Among the scattered specific objects, it can also be a physical entity: it is a continuous storage space to save specific state data. State data can include information such as position, rotation, and animation state.
  • the state set of the physical entity is mainly described.
  • the synchronization module can synchronize the physical entity without perceiving the specific state, so the serialization and deserialization operations can be reduced during the synchronization process, and a series of states can be synchronized at one time without serializing and deserializing each state separately.
  • Serialization work in a complex game scene, the state can be thousands, which can save a lot of overhead. This requires that the design of physical entities can span heterogeneous devices.
  • the synchronization frequency needs to be very high. For example, each frame of data needs to be synchronized once. If For a 60fps game, it takes about 16ms to perform a synchronization operation, so the delay requirement for synchronization is very high.
  • the first terminal device and the second terminal device are heterogeneous devices, the first terminal device needs to serialize the data to be sent to obtain binary data, and transfer the binary data To the second terminal device, the second terminal device needs to deserialize the binary data to obtain data that can be recognized by itself. The above sequence deserialization operation will cause a certain delay, which is required in games and other games that require high synchronization requirements. In the scene, it is not allowed.
  • the task running information related to the first task may include: execution data and index information of the execution data; wherein, the execution data is the data required to execute the first task, and the The index information includes a memory address, the memory address is an address of a storage space available on the second terminal device, and the index information is used to instruct the second terminal device to store the execution data to the memory address corresponding to in the storage space.
  • the available storage space on the second terminal device can be obtained first, and the task running information together with the address of the storage space selected by the first terminal device can be transmitted to the second terminal device, and the second terminal device can information, storing the execution data in the storage space corresponding to the memory address, and acquiring the execution data from the storage space corresponding to the memory address when executing the first task. Since the task component on the second terminal device can directly execute the first task, there is no need to perform format conversion on the execution data, and no sequence and deserialization operations are required, which reduces the processing delay.
  • a layout module of a state set is shown in Figure 8 below, where the index area stores a series of index information, which may include: a pointer to a synchronization object, a pointer to a specific physical synchronization object in the game logic, through which the pointer can be Get the real synchronization object.
  • a type index indicating the type of this pointer.
  • the specific type information is static, and the first terminal device and the second terminal device are globally synchronized when they are deployed.
  • Offset represents the offset of the synchronization object in the state set.
  • the main steps of the synchronization process on the phone side are as follows: in the initial stage of the game, when the Go1 object is generated, when it exists and needs to be synchronized ( It can be identified by annotations, etc.), and apply for state set synchronization information from the synchronization module. Including index information ⁇ position pointer, postion type index, 0 (status concentration offset) ⁇ , and the specific space for saving postion rawdata. And the information is synchronized to the peer, and the game logic is executed, during which the object position information will be modified.
  • Execute the synchronization module, and the synchronization module performs synchronization information collection, and performs the following processing on the index information in turn: according to the type index, obtain the specific type. Convert the pointer to a pointer of this type, and save the content pointed to by the pointer to position rawdata.
  • the memory layout of this position rawdata can directly copy the original position, or use other forms to avoid the layout of different devices and achieve compatibility with heterogeneous devices. Encapsulate the rawdata part directly, and send the data to the peer.
  • the main steps of the synchronization process on the desktop side can be as follows: the synchronization module performs receiving work to obtain the state data of the peer end, and the synchronization module overwrites the state data with the rawdata part of the state set of the local end; Do the following: According to the type index, get the specific type, convert the pointer to this type of pointer, and update the rawdata data corresponding to the offset to the specific content pointed to by the pointer.
  • the rendering module refreshes the game object GO, its position information It has been updated and GO will be displayed in the corresponding position.
  • the second terminal device executes the first task according to task running information related to the first task.
  • the second terminal device does not execute other tasks of the target application based on completing the execution of the first task and not receiving an execution trigger for other tasks in the target application by the first terminal device.
  • a lockstep synchronization mechanism based on heterogeneous memory arrangement is adopted, a unified state synchronization data interface is designed, and the data addresses of different platforms are sequentially (for example, there are connections between multiple terminal devices and the first terminal device) relation, then the first terminal device can transmit the task running information to different terminal devices in the preset order), pack and notify the opposite side, and at the same time carry out a strictly limited synchronization method according to the lockstep method (as a terminal device receiving task running information, except for receiving The received task running information will not perform other processing related to the target application).
  • the second terminal device may execute the task according to the task execution information related to the first task, and the display parameters of the display device of the second terminal device (such as the resolution of the display device). rate, screen refresh rate, etc.), execute the first task.
  • the state synchronization sequence flow chart may be as shown in FIG. 9 , and may include the following steps.
  • the game state is modified accordingly to obtain state a.
  • the synchronization module of device A synchronizes state a to the synchronization module of device B. This step is specifically involved in subsequent embodiments.
  • Device B directly uses state a to perform the rendering task, and transmits the picture to the display device.
  • the rendering task of device B is completed, and no other calculations are performed, waiting for the next state of device A.
  • the target application task includes the first task and the second task; the first task and the second task are rendering tasks; the first task is used to draw the first application Interface, the second task is used to draw a second application interface, the first application interface and the second application interface are different interfaces; the first terminal device can execute the second task and trigger the display The device displays the second application interface; the second terminal device may trigger the display device to display the first application interface. It is equivalent to dividing the content originally displayed on the same display screen into multiple interfaces, which can be rendered and displayed on different terminal devices.
  • the main interface of the game can be transferred to the large screen display, the small map is displayed on the PAD side, the task interface is displayed on the notebook, and the mobile phone side displays the control interface at the same time, and then the multi-screen and multi-angle display mode can be realized, bringing consumers Better game application experience.
  • the second terminal device may also feed back the execution result to the first terminal device.
  • the above steps 501 to 503 can be executed regularly (for example, every 1s) to reacquire device information, check whether the optimal scheduling has changed, and reschedule if so.
  • the mobile phone the first terminal device
  • the PC the second terminal device
  • the network between the mobile phone and the PC becomes poor and the bandwidth cannot meet the requirements, or there is no network connection
  • the information of the PC device is obtained in the fourth step
  • the available bandwidth is 10Mbps or the PC device information cannot be obtained
  • the fifth and sixth steps are entered, the PC side cannot meet the rendering task requirements, and the scheduling algorithm will schedule both tasks back to the mobile phone.
  • the mobile phone is in the logical state at the same time Update and render the display, and return to the stand-alone game mode; when the mobile phone is closer to the PC, and the PC can meet the rendering task requirements again, the user will be prompted whether to enter the distributed mode, re-schedule the task, and enter the distributed mode.
  • An embodiment of the present application provides an application running method, the method is applied to a target system, the target system includes a first terminal device and a second terminal device, and the first terminal device is communicatively connected to the second terminal device , the method includes: when the first terminal device is running a target application, acquiring device information of a second terminal device; based on the device information of the second terminal device, the first terminal device satisfies the first The execution of the task requires that the task execution information of the first task be transmitted to the second terminal device, and the first terminal device does not execute the first task; the second terminal device The task execution information related to the task executes the first task.
  • the second terminal device can execute a part of the tasks of the target application, transferring a part of the tasks of the target application to the second terminal device for execution can reduce the computing load of the first terminal device.
  • Table 1 compares the power consumption and performance test results of the stand-alone and distributed versions of mobile phones:
  • the above table shows the effect of the embodiment of this application in a distributed environment. It can be seen that the single-ended game breaks through the upper limit of the performance bottleneck, and realizes the improvement of display resolution and real-time frame rate. At the same time, the peak temperature drops, which shows that this technology can effectively reduce the single-ended CPU load and save energy consumption.
  • FIG. 10 and FIG. 11 are descriptions of the flow of the application running method provided by the embodiment of the present application, taking a game application as an example.
  • FIG. 17 Another example is a simple application with chat and video functions.
  • the left device in the pre-transfer stage, the left device can only interact with the user on one function when running the application. The possible situation is that the user is currently If you are using the chat function, the video function is either turned off, or implemented according to the current general implementation. In order to use the two functions at the same time, the video function can only be displayed in the foreground in a small window mode. After confirming that the peer device has the transfer conditions , you can tell the video function to run in the right application. If the video function of the device on the left has been running in the small window mode before, you can seamlessly switch to the running video function on the right and exit the small window mode. In this way, the user can implement the chat function and video function in parallel in the two devices, without being limited by a single device.
  • the basic modules of the application running device may include modules such as rule information collection, task scheduling, task synchronization, and task execution.
  • the information collection module is used to collect the status of the distributed equipment itself, the execution site and status of the distributed application program;
  • the task scheduling module is responsible for determining the specific measures of task scheduling, task segmentation strategies and scheduling objects according to the collected information;
  • the task synchronization module It is a module that ensures that the program and device status information of each module can be coordinated within a certain time slot during the running of the distributed application, so as to realize and ensure the real-time interactive experience when the program is running.
  • the task execution module is a module that executes distributed applications. Referring to FIG. 12 and FIG.
  • the core software and hardware modules of the embodiment of the present application may include three modules: an information collection module 001 , a task synchronization module 002 , and a task scheduling module 003 .
  • the information collection module is used to collect status information of applications and devices, such as feature rules corresponding to each application, and managers can add, delete, modify, and view these rule files by themselves.
  • the newly added module is the identification and matching module 031, which is used to match the characteristics of the application and the device, and update the matching identification list in real time.
  • the real-time scheduling module 032 is used to read the current equipment and application tasks that need to be adjusted from the matching list, and determine whether the tasks of some equipment need to be terminated, whether tasks need to be migrated, and whether the task segmentation strategy needs to be adjusted.
  • the state synchronization module 021 synchronizes the data state to the slave device according to the synchronization policy according to the synchronization device and application object obtained by the real-time scheduling module and the memory arrangement information obtained from the data processing module.
  • the data processing module 022 stores the data in a way that can be quickly mapped according to the data arrangement and provides an operation interface.
  • the embodiment of the present application also provides that the present application provides an application running method, the method is applied to a target system, the target system includes a first terminal device and a second terminal device, the first terminal device and the second terminal device
  • the communication connection between two terminal devices, the method includes:
  • the first terminal device acquires device information and/or software execution environment of the second terminal device when running the target application;
  • the first terminal device meets the execution requirements of the first task of the target application based on the device information and/or software execution environment of the second terminal device, and transmits the task running information of the first task to the second terminal device. a terminal device, and the first terminal device does not perform the first task;
  • the second terminal device executes the first task according to task running information related to the first task.
  • the method before transmitting the task execution information related to the first task to the second terminal device, the method further includes: the first terminal device presents the first selection information, so The first selection information indicates at least one task of the target application, the device information and/or software execution environment of the second terminal device meet the execution requirements of the at least one task, and the user receives the information based on the first selection information the first input of the feedback;
  • the first terminal device includes the first task based on the first input, and enables the action of transmitting task execution information related to the first task to the second terminal device.
  • the method further includes:
  • the first terminal device transmits state data related to the first task to the second terminal device; the state data is state data required by the second terminal device to execute the first task.
  • the first terminal device and the second terminal device are connected through near field communication, Internet network, WIFI, Bluetooth, ultra-wideband, ZigBee, RFID, 4G, 5G, LoRa, At least one of SIGFOX, Z-Wave and NB-loT communicates.
  • the software execution environment includes:
  • Component information where the component information is used to indicate whether the second terminal device is deployed with an operating component of the target application, and the operating component is configured to have the ability to execute the first task;
  • the implementation requirements include:
  • the second terminal device is deployed with the running component.
  • the device information includes at least one of the following information:
  • Power supply information processor information, memory information, transmission bandwidth, input device information, output device information, network load.
  • the software execution environment includes: the device state of the terminal device and/or the running state of the running component;
  • the task running information related to the first task includes first indication information, and the first indication information uses To indicate at least one of the following information: the second terminal device is powered on, ends dormancy, ends screen off or unlocks the screen, instructs the second terminal device to run the running component, and executes the running component through the running component. first task; or,
  • the task running information related to the first task includes a second indication information, the second indication information is used to indicate that at least one of the second terminal devices runs the running component and executes the first task through the running component; or,
  • the task running information related to the first task includes a third indication information
  • the third indication information is used to instruct the second terminal device to execute the first task through the running component.
  • the synchronization frequency needs to be very high. For example, each frame of data needs to be synchronized once. If For a 60fps game, it takes about 16ms to perform a synchronization operation, so the delay requirement for synchronization is very high.
  • the first terminal device and the second terminal device are heterogeneous devices, the first terminal device needs to serialize the data to be sent to obtain binary data, and transfer the binary data To the second terminal device, the second terminal device needs to deserialize the binary data to obtain data that can be recognized by itself. The above sequence deserialization operation will cause a certain delay, which is required in games and other games that require high synchronization requirements. In the scene, it is not allowed.
  • the task running information related to the first task may include: execution data and index information of the execution data; wherein, the execution data is the data required to execute the first task, and the The index information includes a memory address, the memory address is an address of a storage space available on the second terminal device, and the index information is used to instruct the second terminal device to store the execution data to the memory address corresponding to in the storage space.
  • the available storage space on the second terminal device can be obtained first, and the task running information together with the address of the storage space selected by the first terminal device can be transmitted to the second terminal device, and the second terminal device can information, storing the execution data in the storage space corresponding to the memory address, and acquiring the execution data from the storage space corresponding to the memory address when executing the first task. Since the task component on the second terminal device can directly execute the first task, there is no need to perform format conversion on the execution data, and no sequence and deserialization operations are required, which reduces the processing delay.
  • the second terminal device does not execute other tasks of the target application based on completing the execution of the first task and not receiving an execution trigger for other tasks in the target application by the first terminal device.
  • a lockstep synchronization mechanism based on heterogeneous memory arrangement is adopted, a unified state synchronization data interface is designed, and the data addresses of different platforms are sequenced (for example, there is a connection relationship between multiple terminal devices and the first terminal device, Then the first terminal device can transfer the task running information to different terminal devices according to the preset order), pack and notify the opposite side, and at the same time perform a strictly limited synchronization method according to the lockstep method (as a terminal device receiving task running information, except for the received task running information, and will not perform other processing related to the target application).
  • the first task is one of a UI display task, an image rendering task, a logic state update task, an AI task, and a data processing or analysis task.
  • the target application task includes the first task and the second task; the first task and the second task are rendering tasks; the first task is used to draw the first application Interface, the second task is used to draw a second application interface, the first application interface and the second application interface are different interfaces; the first terminal device can execute the second task and trigger the display The device displays the second application interface; the second terminal device may trigger the display device to display the first application interface. It is equivalent to dividing the content originally displayed on the same display screen into multiple interfaces, which can be rendered and displayed on different terminal devices.
  • the second terminal device may execute the task according to the task execution information related to the first task, and the display parameters of the display device of the second terminal device (such as the resolution of the display device). rate, screen refresh rate, etc.), execute the first task.
  • the second terminal device may also feed back the execution result to the first terminal device.
  • the present application also provides an application running device, the application running device is applied to a first terminal device, and the first terminal device communicates with the second terminal device, referring to Fig. 14, which is provided by an embodiment of the present application
  • a schematic structural diagram of application running, as shown in FIG. 14 , the application running device 1400 includes:
  • An acquiring module 1401, configured to acquire device information and/or software execution environment of the second terminal device when running the target application
  • step 501 For the description of the obtaining module 1401, reference may be made to the description of step 501, which will not be repeated here.
  • the sending module 1402 is configured to, based on the device information and/or software execution environment of the second terminal device meeting the execution requirements of the first task of the target application, transmit the task running information related to the first task to the second terminal device. and two terminal devices, so that the second terminal device executes the first task according to the task running information, and the first terminal device does not execute the first task.
  • step 502 For the description of the sending module 1402, reference may be made to the description of step 502, which will not be repeated here.
  • the device also includes:
  • a presenting module configured to present first selection information indicating at least one task of the target application before transmitting the task execution information related to the first task to the second terminal device,
  • the device information and/or software execution environment of the second terminal device meet the execution requirements of the at least one task, and receive a first input fed back by the user according to the first selection information;
  • the sending module is further configured to, after the task running information related to the first task is transferred to the second terminal device, transfer the state data related to the first task to the The second terminal device; the state data is state data required by the second terminal device to execute the first task.
  • the first terminal device and the second terminal device are connected through near field communication, Internet network, WIFI, Bluetooth, ultra-wideband, ZigBee, RFID, 4G, 5G, LoRa, At least one of SIGFOX, Z-Wave and NB-loT communicates.
  • the software execution environment includes:
  • Component information where the component information is used to indicate whether the second terminal device is deployed with an operating component of the target application, and the operating component is configured to have the ability to execute the first task;
  • the implementation requirements include:
  • the second terminal device is deployed with the running component.
  • the device information includes at least one of the following information:
  • Power supply information processor information, memory information, transmission bandwidth, input device information, output device information, network load.
  • the software execution environment includes: the device state of the terminal device and/or the running state of the running component;
  • the task running information related to the first task includes first indication information, and the first indication information uses To indicate at least one of the following information: the second terminal device is powered on, ends dormancy, ends screen off or unlocks the screen, instructs the second terminal device to run the running component, and executes the running component through the running component. first task; or,
  • the task running information related to the first task includes a second indication information, the second indication information is used to indicate that at least one of the second terminal devices runs the running component and executes the first task through the running component; or,
  • the task running information related to the first task includes a third indication information
  • the third indication information is used to instruct the second terminal device to execute the first task through the running component.
  • the acquiring module is specifically used for:
  • the multiple terminal devices include the second terminal device
  • the device also includes:
  • a terminal selection module configured to enable selection of the second terminal device among the plurality of terminal devices before the task execution information of the first task is transmitted to the second terminal device.
  • the terminal selection module is specifically configured to:
  • Second selection information is used to instruct the user to select from the plurality of terminal devices, and receiving a second input fed back by the user according to the second selection information;
  • a selection for the second terminal device of the plurality of terminal devices is enabled based on the second input including the second terminal device.
  • the terminal selection module is specifically configured to:
  • the target application task includes the first task and the second task
  • the device also includes:
  • a task execution module configured to keep executing the second task when the device information of the second terminal device does not meet the execution requirements of the target application second task.
  • the task running information related to the first task includes:
  • Execution data and index information of the execution data wherein, the execution data is data required to execute the first task, and the index information includes a memory address, and the memory address is available on the second terminal device The address of the storage space, the index information is used to instruct the second terminal device to store the execution data in the storage space corresponding to the memory address.
  • the first task is one of a UI display task, an image rendering task, a logic state update task, an AI task, and a data processing or analysis task.
  • FIG. 15 is a schematic structural diagram of an application running device provided in an embodiment of the present application.
  • the first terminal device includes:
  • An obtaining module 1501 configured to obtain device information and/or software execution environment of the second terminal device when running the target application
  • step 501 For the description of the obtaining module 1501, reference may be made to the description of step 501, which will not be repeated here.
  • the sending module 1502 is configured to meet the execution requirements of the first task of the target application based on the device information and/or software execution environment of the second terminal device, and transmit the task running information of the first task to the second terminal device. a terminal device, and the first terminal device does not perform the first task;
  • the second terminal device includes:
  • the first task execution module 1503 is configured to execute the first task according to the task running information related to the first task.
  • the first terminal device further includes:
  • a presenting module configured for the first terminal device to present first selection information, the first selection information indicating at least one task of the target application, and the device information and/or software execution environment of the second terminal device satisfying the performing at least one task, and receiving a first input fed back by the user according to the first selection information;
  • the sending module is further configured to, after the task running information related to the first task is transmitted to the second terminal device, the first terminal device sends the first task Relevant state data is transmitted to the second terminal device; the state data is state data required by the second terminal device to execute the first task.
  • the first terminal device and the second terminal device are connected through near-field communication, WIFI, Bluetooth, ultra-wideband, ZigBee, RFID, 4G, 5G, LoRa, SIGFOX, Z -At least one of Wave and NB-loT communication.
  • the device information includes:
  • Component information where the component information is used to indicate whether the second terminal device is deployed with an operating component of the target application, and the operating component is configured to have the ability to execute the first task;
  • the implementation requirements include:
  • the second terminal device is deployed with the running component.
  • the device information includes at least one of the following information:
  • Power supply information processor information, memory information, transmission bandwidth, input device information, output device information, network load.
  • the software execution environment includes: the device state of the terminal device and/or the running state of the running component;
  • the task running information related to the first task includes first indication information, and the first indication information uses To indicate at least one of the following information: the second terminal device is powered on, ends dormancy, ends screen off or unlocks the screen, instructs the second terminal device to run the running component, and executes the running component through the running component. first task; or,
  • the task running information related to the first task includes a second indication information, the second indication information is used to indicate that at least one of the second terminal devices runs the running component and executes the first task through the running component; or,
  • the task running information related to the first task includes a third indication information
  • the third indication information is used to instruct the second terminal device to execute the first task through the running component.
  • the task running information related to the first task includes:
  • Execution data and index information of the execution data wherein, the execution data is data required to execute the first task, and the index information includes a memory address, and the memory address is available on the second terminal device the address of the storage space, the index information is used to instruct the second terminal device to store the execution data in the storage space corresponding to the memory address;
  • the first task execution module is specifically used for:
  • the execution data is acquired from the storage space corresponding to the memory address.
  • the second terminal device does not execute the first task based on completing execution of the first task and not receiving execution triggers for other tasks in the target application by the first terminal device. Other tasks for the target application described above.
  • the first task is one of a UI display task, an image rendering task, a logic state update task, an AI task, and a data processing or analysis task.
  • the target application task includes the first task and the second task; the first task and the second task are rendering tasks; the first task is used to draw the first application interface, the second task is used to draw a second application interface, the first application interface and the second application interface are different interfaces;
  • the first terminal device also includes:
  • a second task execution module configured to execute the second task, and trigger a display device to display the second application interface
  • the first task execution module is specifically used for:
  • the second terminal device triggers the display device to display the first application interface.
  • the first task is a rendering task
  • the first task execution module is specifically used for:
  • the terminal device can be the terminal device (such as the first terminal device and the second terminal device) described in the embodiment of the application.
  • the example provides a schematic structural diagram of a terminal device.
  • the terminal device 1600 can specifically be embodied as a virtual reality VR device, a mobile phone, a tablet, a notebook computer, a smart wearable device, etc., which are not limited here.
  • the terminal device 1600 includes: a receiver 1601, a transmitter 1602, a processor 1603, and a memory 1604 (the number of processors 1603 in the terminal device 1600 may be one or more, and one processor is taken as an example in FIG.
  • the processor 1603 may include an application processor 16031 and a communication processor 16032 .
  • the receiver 1601 , the transmitter 1602 , the processor 1603 and the memory 1604 may be connected through a bus or in other ways.
  • the memory 1604 may include read-only memory and random-access memory, and provides instructions and data to the processor 1603 .
  • a part of the memory 1604 may also include a non-volatile random access memory (non-volatile random access memory, NVRAM).
  • NVRAM non-volatile random access memory
  • the memory 1604 stores processors and operating instructions, executable modules or data structures, or their subsets, or their extended sets, wherein the operating instructions may include various operating instructions for implementing various operations.
  • the processor 1603 controls the operation of the terminal device.
  • various components of the terminal device are coupled together through a bus system, where the bus system may include a power bus, a control bus, and a status signal bus in addition to a data bus.
  • the various buses are referred to as bus systems in the figures.
  • the methods disclosed in the foregoing embodiments of the present application may be applied to the processor 1603 or implemented by the processor 1603 .
  • the processor 1603 may be an integrated circuit chip, which has a signal processing capability.
  • each step of the above method may be implemented by an integrated logic circuit of hardware in the processor 1603 or instructions in the form of software.
  • the above-mentioned processor 1603 can be a general-purpose processor, a digital signal processor (digital signal processing, DSP), a microprocessor or a microcontroller, and can further include an application-specific integrated circuit (application specific integrated circuit, ASIC), field programmable Field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
  • DSP digital signal processing
  • ASIC application specific integrated circuit
  • FPGA field programmable Field-programmable gate array
  • the processor 1603 may implement or execute various methods, steps, and logic block diagrams disclosed in the embodiments of the present application.
  • a general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like.
  • the steps of the method disclosed in the embodiments of this application can be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor.
  • the software module can be located in a mature storage medium in the field such as random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, register.
  • the storage medium is located in the memory 1604, and the processor 1603 reads the information in the memory 1604, and completes the steps of the above method in combination with its hardware.
  • the processor 1603 may read the information in the memory 1604, and combine its hardware to complete the steps from step 501 to step 503 in the foregoing embodiment.
  • the receiver 1601 can be used to receive input digital or character information, and generate signal input related to related settings and function control of the terminal device.
  • the transmitter 1602 can be used to output digital or character information through the first interface; the transmitter 1602 can also be used to send instructions to the disk group through the first interface to modify the data in the disk group; the transmitter 1602 can also include a display device such as a display screen .
  • An embodiment of the present application further provides a computer program product that, when running on a computer, causes the computer to execute the steps of the application running method described in the embodiment corresponding to FIG. 5 in the above embodiments.
  • An embodiment of the present application also provides a computer-readable storage medium, the computer-readable storage medium stores a program for signal processing, and when it is run on a computer, the computer executes the method described in the foregoing embodiments The steps of the image processing method.
  • the image display device provided in the embodiment of the present application may specifically be a chip, and the chip includes: a processing unit and a communication unit, the processing unit may be, for example, a processor, and the communication unit may be, for example, an input/output interface, a pin or a circuit, etc. .
  • the processing unit can execute the computer-executed instructions stored in the storage unit, so that the chips in the execution device execute the data processing methods described in the above embodiments, or make the chips in the training device execute the data processing methods described in the above embodiments.
  • the storage unit is a storage unit in the chip, such as a register, a cache, etc.
  • the storage unit may also be a storage unit located outside the chip in the wireless access device, such as only Read-only memory (ROM) or other types of static storage devices that can store static information and instructions, random access memory (random access memory, RAM), etc.
  • ROM Read-only memory
  • RAM random access memory
  • the device embodiments described above are only illustrative, and the units described as separate components may or may not be physically separated, and the components shown as units may or may not be A physical unit can be located in one place, or it can be distributed to multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • the connection relationship between the modules indicates that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines.
  • the essence of the technical solution of this application or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product is stored in a readable storage medium, such as a floppy disk of a computer , U disk, mobile hard disk, ROM, RAM, magnetic disk or optical disk, etc., including several instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the method described in each embodiment of the present application .
  • a computer device which can be a personal computer, a server, or a network device, etc.
  • all or part of them may be implemented by software, hardware, firmware or any combination thereof.
  • software When implemented using software, it may be implemented in whole or in part in the form of a computer program product.
  • the computer program product includes one or more computer instructions.
  • the computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable devices.
  • the computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website, computer, server, or data center Transmission to another website site, computer, server, or data center by wired (eg, coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (eg, infrared, wireless, microwave, etc.).
  • wired eg, coaxial cable, optical fiber, digital subscriber line (DSL)
  • wireless eg, infrared, wireless, microwave, etc.
  • the computer-readable storage medium may be any available medium that can be stored by a computer, or a data storage device such as a server or a data center integrated with one or more available media.
  • the available medium may be a magnetic medium (such as a floppy disk, a hard disk, or a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (Solid State Disk, SSD)), etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Stored Programmes (AREA)
  • Telephone Function (AREA)

Abstract

本申请提供了一种应用运行方法,方法包括:第一终端设备在运行目标应用时,获取第二终端设备的设备信息和/或软件运行环境;第一终端设备基于第二终端设备的设备信息和/或软件运行环境满足目标应用第一任务的执行要求,将第一任务的任务运行信息传递至第二终端设备,且第一终端设备不执行第一任务;第二终端设备根据第一任务相关的任务运行信息执行第一任务。本申请在第二终端设备可以执行目标应用的一部分任务的前提下,将目标应用的一部分任务转移到第二终端设备上执行,可以降低第一终端设备的运算负载。

Description

一种应用运行方法以及相关设备
本申请要求于2021年10月22日提交中国专利局、申请号为202111236221.5、发明名称为“一种应用运行方法以及相关设备”的中国专利申请的优先权、以及于2022年10月21日提交中国专利局、申请号为、发明名称为“一种应用运行方法以及相关设备”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。
技术领域
本申请涉及计算机领域,尤其涉及一种应用运行方法以及相关设备。
背景技术
当前应用发展越来越大型化、复杂化,比如视频应用从显示1080P的实时画面发展到2K、4K未来还会出现8K等更高画质,对于设备算力有更高需求。
近年来随着智能手机的普及,移动应用发展迅速,但是当前移动应用的开发与运行方式主要是基于单个设备设计的,单设备运行应用存在性能瓶颈与体验瓶颈,主要问题如下:由于受到CPU核心频率和GPU核心频率的限制,因此单个设备的计算能力有限,相比于大型服务器设计者需要在计算能力与体积、成本因素间做平衡,类似的其存储与通信能力也受到设备本身的设计限制,无法无限制满足上层应用对性能的要求。
发明内容
本申请实施例提供了一种应用的分布式运行方法,所述方法应用于第一终端设备,所述第一终端设备与所述第二终端设备通信连接,第二终端设备终端设备可以在本地也可以在远端,所述方法包括:在运行目标应用时,获取第二终端设备的设备信息和/或者软件执行环境;基于所述第二终端设备的设备信息和/或软件执行环境满足所述目标应用第一任务的执行要求,将所述第一任务相关的任务运行信息传递至所述第二终端设备,以便所述第二终端设备根据所述任务运行信息执行所述第一任务,且所述第一终端设备不执行所述第一任务。通过上述方法,在第二终端设备可以执行目标应用的一部分任务的前提下,将目标应用的一部分任务转移到第二终端设备上执行,可以降低第一终端设备的运算负载,同时如果目标应用所在的第一终端设备硬件性能,比如显示分辨率、CPU、GPU计算能力不如第二终端硬件设备的性能,则通过该方法转移后的任务可以在第二终端设备上获得更好的应用表现。
应理解,这里的“在运行目标应用时”,可以理解为启动目标应用时,或者是运行目标应用的过程中。
应理解,所述第一任务可以为UI显示任务、图像渲染任务、逻辑状态更新任务、AI任务、数据处理或分析任务中的一种。第一任务还可以是其他需要耗费较大运算资源和/或存储资源和或通信资源的任务。第一任务可以是第一终端设备当前可用的运算资源和/或存储资源或通信资源不足时待运行的任务,例如,第一终端设备能否正常运行第一任务(例如是否会卡顿、是否需要花费大量时间等),或者是第一终端设备运行第一任务时是否会对 其他任务造成影响,或者是运行第一终端设备运行第一任务时是否会发烫等等。
应理解,第一任务可以是预先划分好的任务类别,例如,游戏应用中预先划分为逻辑状态更新任务以及渲染任务,或者是将渲染任务拆分为多个子任务等。又例如:微信中文字聊天和视频电话是两个可拆分的子任务,在单机中一般一次只能执行一个任务,在本例中则可以拆开为两个子任务,将其中一个转移到第二终端设备运行。
应理解,第一终端设备还可以将多个任务转移到第二终端设备,或者是将多个任务转移到不同的终端设备上执行,这里并不限定。
应理解,第二终端设备可以不仅仅是根据第一任务的任务信息来执行第一任务,还可以考虑到第二终端设备的状态、应用执行效果及用户体验等信息之间平衡,例如显示效果与设备动态资源利用率之间的平衡,最后达成最佳显示,第二终端设备在执行第一任务的执行效果以及执行效率等可以相比第一设备有所增强。
应理解,设备信息和软件运行环境可以根据应用场景的变化而变化,但是也有不变量,比如设备的电量、CPU、内存占用情况、网络负载状态,当前设备上的应用进程的运行时状态;变化的部分包括但不限于,例如第一设备的目标应用分割后的子任务中,可能包含UI窗口信息,则第一设备也需要收集第二设备上的窗口信息。
在一种可能的实现中,所述第一终端设备和所述第二终端设备之间既可以通过近场通信也可以通过远场通信、即Internet网络、WIFI、蓝牙、超宽带、紫蜂ZigBee、RFID、4G、5G、LoRa、SIGFOX、Z-Wave以及NB-loT中的至少一种方式通信。例如,所述第一终端设备和所述第二终端设备之间通过Internet网络、WIFI、蓝牙、超宽带、紫蜂ZigBee、RFID、4G、5G、LoRa、SIGFOX、Z-Wave以及NB-loT中的多种方式连接。
应理解,第一终端设备和所述第二终端设备之间还可以通过其他时延较低的无线连接方式(例如3GPP、non-3GPP以及基于短距技术实现的连接方式)进行连接,这里并不限定。
应理解,在第一任务为渲染任务时,在一些场景中,第一终端设备的显示设备的显示参数(例如刷新率、屏幕大小、刷新率等)和第二终端设备的显示设备的显示参数不同,第二终端设备可以根据所述第一任务相关的任务运行信息,以及所述第二终端设备的显示设备的显示参数(例如显示设备的分辨率、屏幕刷新率等等),执行所述第一任务,进而可以显示和第二终端设备的显示设备相适配的显示内容。
在一种可能的实现中,所述将所述第一任务相关的任务运行信息传递至所述第二终端设备之前,所述方法还包括:呈现第一选择信息,所述第一选择信息指示所述目标应用至少一个任务,所述第二终端设备的设备信息和/或软件执行环境满足所述至少一个任务的执行要求,并接收到用户根据所述第一选择信息反馈的第一输入;基于所述第一输入包括所述第一任务,使能所述将所述第一任务相关的任务运行信息传递至所述第二终端设备的动作。目标任务可以包括多个子任务,第一终端设备可以基于至少一个任务和第二终端设备的设备信息和/或软件执行环境,从多个子任务中确定出适合转移到第二终端设备上的任务(至少一个任务),并呈现给用户,由用户进行选择。
在一种可能的实现中,所述将第一任务相关的任务运行信息传递至所述第二终端设备 之后,所述方法还包括:将所述第一任务相关的状态数据传递至所述第二终端设备;所述状态数据为所述第二终端设备执行所述第一任务时所需的状态数据。例如,在投屏场景中,状态数据可以为UI引擎的中间状态、或者是应用本身的数据状态信息。该状态数据可以为用户在第一终端设备上输入的,也可以是第一终端设备自身生成的。
在一种可能的实现中,第一终端设备可以获取到存在通信连接的终端设备的设备信息,并基于对设备信息的分析来确定该终端设备是否具备运行要转移的任务的能力,以及确定将哪部分任务转移到终端设备。
以转移对象为第一任务为例,在一种可能的实现中,所述软件运行环境信息包括组件信息,所述组件信息用于指示所述第二终端设备是否部署有所述目标应用的运行组件,所述运行组件被配置为具备执行所述第一任务的能力。
运行组件可以为AI引擎,AI引擎可以用于执行AI任务,运行组件可以为渲染引擎,渲染引擎可以用于执行渲染任务,运行组件可以为逻辑引擎和物理引擎,逻辑引擎和物理引擎可以用于执行逻辑状态更新任务,若第二终端设备未部署有具备执行所述第一任务的能力的运行组件,则不需要将第一任务转移到第二终端设备执行。
其中,所述第一任务可以为UI显示任务、图像渲染任务、逻辑状态更新任务、AI任务、数据处理或分析任务中的一种。其中,运行组件可以为AI引擎,AI引擎可以用于执行AI任务,运行组件可以为渲染引擎,渲染引擎可以用于执行渲染任务,运行组件可以为逻辑引擎和物理引擎,逻辑引擎和物理引擎可以用于执行逻辑状态更新任务,若第二终端设备未部署有具备执行所述第一任务的能力的运行组件,则不需要将第一任务转移到第二终端设备执行。
其中,软件运行环境可以包括运行应用的系统环境,需要两端设备同时具备一些必要的运行组件,例如如果是游戏可能需要显卡组件,如果是移动应用可能需要有Android4.0以上版本运行环境等。
应理解,第一终端设备可以部署有运行组件,在第一终端设备处于单机运行目标应用的情况下,第一终端设备可以通过运行组件来执行第一任务,且第一终端设备部署的运行组件和第二终端设备部署的运行组件一致(或者基本一致)。
以转移对象为第一任务为例,在一种可能的实现中,所述设备信息可以包括,电源信息,电源信息可以为第二终端设备的可用电量、充电模式(例如是有源输入还是电池供电),例如,在电源信息指示第二终端设备的可用电量较少时,则可以确定不需要将第一任务转移到第二终端设备执行。
以转移对象为第一任务为例,在一种可能的实现中,所述设备信息可以包括,处理器信息,处理器可以为CPU、GPU、TPU、NPU等,处理器信息可以但不限于为CPU型号/主频,GPU型号/主频,当前CPU/GPU负载等。由于一些任务对于处理器的类型、处理能力、当前可用的资源都有一定的要求,在不满足任务要求时,则可以确定不需要将第一任务转移到第二终端设备执行;或者当前设备处理器负载较高,而第二终端设备处理器资源充足也可以决定立即执行任务转移。
以转移对象为第一任务为例,在一种可能的实现中,所述设备信息可以包括,内存信 息,例如内存大小。在不满足第一任务对于内存大小的要求时,则可以确定不需要将第一任务转移到第二终端设备执行。
以转移对象为第一任务为例,在一种可能的实现中,所述设备信息可以包括,传输带宽,传输带宽可以为当前网络的可用带宽。在一些实现中,为了保证任务执行的同步,则需要保证在将第一任务传输到第二终端设备时,不会存在延迟(或者延迟很小),则需要保证第一终端设备和第二终端设备之间的传输带宽需要满足第一任务对应的要求。在不满足要求时,则可以确定不需要将第一任务转移到第二终端设备执行。
以转移对象为第一任务为例,在一种可能的实现中,所述设备信息可以包括,输入设备信息,输入设备信息可以包括输入设备类型,输入设备可以为音频输入、鼠标输入、键盘输入等。在一些任务中,例如涉及和用户交互的任务,第一任务还需要支持用户的输入,而由于第一任务转移到第二终端设备执行,则需要第二终端设备具备第一任务所要求的输入设备类型。在不满足要求时,则可以确定不需要将第一任务转移到第二终端设备执行。
以转移对象为第一任务为例,在一种可能的实现中,所述设备信息可以包括,输出设备信息,输出设备信息可以包括输出设备类型,输出设备可以为音频输出、图像/视频输出等,输出设备信息还可以为屏幕分辨率、刷新率等显示设备的显示属性。在一些任务中,例如涉及和用户交互的任务,第一任务还需要支持输出(例如渲染任务要求存在显示设备,以及显示设备的参数要求),而由于第一任务转移到第二终端设备执行,则需要第二终端设备具备第一任务所要求的输出设备类型以及输出设备的属性。在不满足要求时,则可以确定不需要将第一任务转移到第二终端设备执行。
以转移对象为第一任务为例,在一种可能的实现中,所述设备信息可以包括,网络负载信息,网络负载信息可以包括当前设备网卡工作状态,网络带宽占用率、端到端延迟时间等。由于一些任务对于网络处理能力、当前可用网络资源都有一定的要求,在不满足转移任务要求时,则可以确定不需要将第一任务转移到第二终端设备执行;或者当前设备网络负载较高,而第二终端设备网络资源充足也可以决定立即执行任务转移。
在一种可能的实现中,所述设备信息包括:所述终端设备的设备状态和/或所述运行组件的运行状态,设备状态是指设备已经上电,可能正在运行或没有运行某些应用,对应的未设备状态包括设备下电、设备休眠或者息屏等情况;
在所述设备状态指示所述第二终端设备未启动的情况下,所述第一任务相关的任务运行信息包括第一指示信息,所述第一指示信息用于触发所述第二终端设备启动、指示所述第二终端设备运行所述运行组件、以及通过所述运行组件执行所述第一任务;或者,
在所述设备状态指示所述第二终端设备启动、且所述运行状态指示所述第二终端设备未运行所述运行组件的情况下,所述第一任务相关的任务运行信息包括第二指示信息,所述第二指示信息用于指示所述第二终端设备运行所述运行组件、以及通过所述运行组件执行所述第一任务;或者,
在所述设备状态指示所述第二终端设备启动、且在所述运行状态指示所述第二终端设备运行所述运行组件的情况下,所述第一任务相关的任务运行信息包括第三指示信息,所述第三指示信息用于指示所述第二终端设备通过所述运行组件执行所述第一任务。
在一种可能的实现中,如果第一终端设备确定出多个满足要求的终端设备时,可以从多个终端设备中选择部分(或全部)终端设备作为第一任务的转移对象。
在一种可能的实现中,可以获取与所述第一终端设备连接的多个终端设备的设备信息,所述多个终端设备包括所述第二终端设备,并使能针对于所述多个终端设备中所述第二终端设备的选择。
在一种可能的实现中,可以呈现第二选择信息,所述第二选择信息用于指示用户从所述多个终端设备进行选择,并接收到用户根据所述第二选择信息反馈的第二输入;基于所述第二输入包括所述第二终端设备,使能针对于所述多个终端设备中所述第二终端设备的选择。
在一种可能的实现中,第一终端设备还可以基于所述第二终端设备的设备信息和或应用信息为所述多个终端设备中选择最符合所述第一任务的执行要求的设备信息和或应用信息,使能针对于所述多个终端设备中所述第二终端设备的选择。其中,所谓最符合执行要求,可以理解为在满足第一任务的执行要求的终端设备中选择性能最佳的终端设备。
在一种可能的实现中,所述目标应用任务包括所述第一任务和第二任务;所述方法还包括:执行所述第二任务。
在游戏应用时,为了保证应用的正常运行,需要保证第一终端设备和第二终端设备之间数据的实时同步,且同步频率需要很高,例如每一帧数据都需要进行一次状态同步,如果是60fps的游戏则约16ms进行一次同步操作,因此对同步的延时要求非常高。在现有的实现中,在第一终端设备和第二终端设备为异构设备的情况下,第一终端设备需要将待发送的数据进行序列化处理,以得到二进制数据,并将二进制数据传递至第二终端设备,第二终端设备需要对二进制数据进行反序列化,以得到自身能够识别出来的数据,上述序列反序列化操作会造成一定的时延,这在游戏等需要高同步要求的场景中,是不被允许的。
本申请实施例中,所述第一任务相关的任务运行信息可以包括:执行数据以及所述执行数据的索引信息;其中,所述执行数据为执行所述第一任务所需的数据,所述索引信息包括内存地址,所述内存地址为所述第二终端设备上可用的存储空间的地址,所述索引信息用于指示所述第二终端设备将所述执行数据存储至所述内存地址对应的存储空间中。
也就是说,可以先获取到第二终端设备上的可用存储空间,并将任务运行信息连同第一终端设备选择的存储空间的地址传递至第二终端设备,第二终端设备可以根据所述索引信息,将所述执行数据存储至所述内存地址对应的存储空间中,并在执行所述第一任务时,从所述内存地址对应的存储空间中获取所述执行数据。由于第二终端设备上的任务组件可以直接执行第一任务,并不需要对执行数据进行格式转换,且不需要序列和反序列化的操作,降低了处理时延。
在一种可能的实现中,所述第一任务为UI显示任务、图像渲染任务、逻辑状态更新任 务、AI任务、数据处理或分析任务中的一种。
第二方面,本申请提供了一种应用运行方法,所述方法应用于目标系统,所述目标系统包括第一终端设备和第二终端设备,所述第一终端设备与所述第二终端设备通信连接,所述方法包括:
所述第一终端设备在运行目标应用时,获取第二终端设备的设备信息和/或软件执行环境;
所述第一终端设备基于所述第二终端设备的设备信息和/或软件执行环境满足所述目标应用第一任务的执行要求,将所述第一任务的任务运行信息传递至所述第二终端设备,且所述第一终端设备不执行所述第一任务;
所述第二终端设备根据所述第一任务相关的任务运行信息执行所述第一任务。
在一种可能的实现中,所述将所述第一任务相关的任务运行信息传递至所述第二终端设备之前,所述方法还包括:所述第一终端设备呈现第一选择信息,所述第一选择信息指示所述目标应用至少一个任务,所述第二终端设备的设备信息和/或软件执行环境满足所述至少一个任务的执行要求,并接收到用户根据所述第一选择信息反馈的第一输入;
所述第一终端设备基于所述第一输入包括所述第一任务,使能所述将所述第一任务相关的任务运行信息传递至所述第二终端设备的动作。
在一种可能的实现中,所述将第一任务相关的任务运行信息传递至所述第二终端设备之后,所述方法还包括:
所述第一终端设备将所述第一任务相关的状态数据传递至所述第二终端设备;所述状态数据为所述第二终端设备执行所述第一任务时所需的状态数据。
在一种可能的实现中,所述第一终端设备和所述第二终端设备之间通过近场通信、Internet网络、WIFI、蓝牙、超宽带、紫蜂ZigBee、RFID、4G、5G、LoRa、SIGFOX、Z-Wave以及NB-loT中的至少一种方式通信。
在一种可能的实现中,所述软件执行环境包括:
组件信息,所述组件信息用于指示所述第二终端设备是否部署有所述目标应用的运行组件,所述运行组件被配置为具备执行所述第一任务的能力;
所述执行要求,包括:
所述第二终端设备部署有所述运行组件。
在一种可能的实现中,所述设备信息包括如下信息的至少一种:
电源信息、处理器信息、内存信息、传输带宽、输入设备信息、输出设备信息、网络负载。
在一种可能的实现中,所述软件执行环境包括:所述终端设备的设备状态和/或所述运行组件的运行状态;
在所述设备状态指示所述第二终端设备处于下电、休眠、息屏或者锁屏的情况下,所述第一任务相关的任务运行信息包括第一指示信息,所述第一指示信息用于指示如下信息的至少一个:所述第二终端设备上电、结束休眠、结束息屏或者解除锁屏、指示所述第二终端设备运行所述运行组件、以及通过所述运行组件执行所述第一任务;或者,
在所述设备状态指示所述第二终端设备启动、且所述运行状态指示所述第二终端设备未运行所述运行组件的情况下,所述第一任务相关的任务运行信息包括第二指示信息,所述第二指示信息用于指示如下信息的至少一个所述第二终端设备运行所述运行组件、以及通过所述运行组件执行所述第一任务;或者,
在所述设备状态指示所述第二终端设备启动、且在所述运行状态指示所述第二终端设备运行所述运行组件的情况下,所述第一任务相关的任务运行信息包括第三指示信息,所述第三指示信息用于指示所述第二终端设备通过所述运行组件执行所述第一任务。
在游戏应用时,为了保证应用的正常运行,需要保证第一终端设备和第二终端设备之间数据的实时同步,且同步频率需要很高,例如每一帧数据都需要进行一次状态同步,如果是60fps的游戏则约16ms进行一次同步操作,因此对同步的延时要求非常高。在现有的实现中,在第一终端设备和第二终端设备为异构设备的情况下,第一终端设备需要将待发送的数据进行序列化处理,以得到二进制数据,并将二进制数据传递至第二终端设备,第二终端设备需要对二进制数据进行反序列化,以得到自身能够识别出来的数据,上述序列反序列化操作会造成一定的时延,这在游戏等需要高同步要求的场景中,是不被允许的。
本申请实施例中,所述第一任务相关的任务运行信息可以包括:执行数据以及所述执行数据的索引信息;其中,所述执行数据为执行所述第一任务所需的数据,所述索引信息包括内存地址,所述内存地址为所述第二终端设备上可用的存储空间的地址,所述索引信息用于指示所述第二终端设备将所述执行数据存储至所述内存地址对应的存储空间中。
也就是说,可以先获取到第二终端设备上的可用存储空间,并将任务运行信息连同第一终端设备选择的存储空间的地址传递至第二终端设备,第二终端设备可以根据所述索引信息,将所述执行数据存储至所述内存地址对应的存储空间中,并在执行所述第一任务时,从所述内存地址对应的存储空间中获取所述执行数据。由于第二终端设备上的任务组件可以直接执行第一任务,并不需要对执行数据进行格式转换,且不需要序列和反序列化的操作,降低了处理时延。
所述第二终端设备基于完成所述第一任务的执行、且未接收到所述第一终端设备针对于所述目标应用中其他任务的执行触发,不执行所述目标应用的其他任务。本申请实施例中,采用了基于异构内存排布的lockstep同步机制,设计统一的状态同步数据接口,将不同平台的数据地址按顺序(例如多个终端设备和第一终端设备存在连接关系,则第一终端 设备可以按照预设的顺序将任务运行信息传递至不同的终端设备)打包通知对面,同时按照lockstep方式进行严格限定的同步方法(作为接收任务运行信息的终端设备,除了接收到的任务运行信息,不会执行和目标应用相关的其他处理)。
在一种可能的实现中,所述第一任务为UI显示任务、图像渲染任务、逻辑状态更新任务、AI任务、数据处理或分析任务中的一种。
在一种可能的实现中,所述目标应用任务包括所述第一任务和第二任务;所述第一任务和所述第二任务为渲染任务;所述第一任务用于绘制第一应用界面,所述第二任务用于绘制第二应用界面,所述第一应用界面和所述第二应用界面为不同的界面;所述第一终端设备可以执行所述第二任务,并触发显示设备显示所述第二应用界面;所述第二终端设备可以触发显示设备显示所述第一应用界面。相当于可以将原本显示在同一个显示屏上的内容分割为多个界面,在不同的终端设备上进行渲染和显示。
在一种可能的实现中,以渲染任务为例,第二终端设备可以根据所述第一任务相关的任务运行信息,以及所述第二终端设备的显示设备的显示参数(例如显示设备的分辨率、屏幕刷新率等等),执行所述第一任务。
在一种可能的实现中,第二终端设备在执行第一任务之后,还可以将执行结果反馈至第一终端设备。
本申请还提供了一种应用运行方法,所述方法应用于第一终端设备,所述第一终端设备与所述第二终端设备通信连接,所述方法包括:运行目标应用;所述目标应用包括第一任务;所述第一任务为所述目标应用中的部分任务;其中,所述目标应用为具有对话功能的应用;所述第一任务为部分或全部聊天对象的聊天框的UI显示、视频聊天的UI显示或者音频聊天的UI显示;或者,所述目标应用为绘画应用、文本编辑类应用、图像编辑类应用或者音视频编辑应用;所述第一任务为编辑工具界面的UI显示、或者编辑对象的UI显示;保持所述第二任务的执行,并将所述第一任务相关的任务运行信息传递至所述第二终端设备,以便所述第二终端设备根据所述任务运行信息执行所述第一任务,且所述第一终端设备不执行所述第一任务。
在一种可能的实现中,具有对话功能的应用可以包括即时聊天应用、社交类应用等,通过该应用,可以和其他联系人进行文本对话、音频对话、视频对话等。
例如,具有对话功能的应用当前的任务可以包括聊天界面的UI显示以及视频通话,第一终端设备可以将视频通话相关的任务运行信息转移到第二终端设备上执行。
例如,具有对话功能的应用当前的任务可以包括聊天界面的UI显示,聊天界面可以包括多个联系人(也就是聊天对象),第一终端设备可以将多个联系人中部分联系人的聊天框的UI显示转移到第二终端设备上执行。
在一种可能的实现中,所述目标应用还包括不同于第一任务的第二任务;
所述目标应用为具有对话功能的应用;所述第二任务为和所述第一任务中不同的聊天对象的UI显示、视频聊天的UI显示或者音频聊天的UI显示;
所述目标应用为绘画应用、文本编辑类应用、图像编辑类应用或者音视频编辑应用;所述第二任务为编辑工具界面的UI显示、或者编辑对象的UI显示;
所述方法还包括:
在所述第二终端设备的设备信息不满足所述目标应用所述第二任务的执行要求时,保持对所述第二任务的执行。
第三方面,本申请提供了一种应用运行装置,所述装置应用于第一终端设备,所述第一终端设备与所述第二终端设备通信连接,所述装置包括:
获取模块,用于在运行目标应用时,获取第二终端设备的设备信息和/或软件执行环境;
发送模块,用于在所述第二终端设备的设备信息和/或软件执行环境满足所述目标应用第一任务的执行要求时,将所述第一任务相关的任务运行信息传递至所述第二终端设备,以便所述第二终端设备根据所述任务运行信息执行所述第一任务,且所述第一终端设备不执行所述第一任务。
在一种可能的实现中,所述目标应用还包括第二任务;所述装置还包括:
任务执行模块,用于在所述第二终端设备根据所述任务运行信息执行所述第一任务时,在所述第一终端设备上保持所述第二任务的执行。
在一种可能的实现中,所述装置还包括:
接收模块,用于在所述将所述第一任务相关的任务运行信息传递至所述第二终端设备之前,接收到用户的第一输入,所述第一输入指示将所述第一任务相关的任务运行信息传递至所述第二终端设备。
在一种可能的实现中,所述装置还包括:
呈现模块,用于在所述将所述第一任务相关的任务运行信息传递至所述第二终端设备之前,呈现第一选择信息,所述第一选择信息指示所述目标应用至少一个任务,所述第二终端设备的设备信息和/或软件执行环境满足所述至少一个任务的执行要求;
所述接收模块,具体用于:
接收到用户根据所述第一选择信息反馈的第一输入;所述第一输入包括所述第一任务。
在一种可能的实现中,所述发送模块,还用于在所述将第一任务相关的任务运行信息传递至所述第二终端设备之后,将所述第一任务相关的状态数据传递至所述第二终端设备;所述状态数据为所述第二终端设备执行所述第一任务时所需的状态数据。
在一种可能的实现中,所述第一终端设备和所述第二终端设备之间通过近场通信、Internet网络、WIFI、蓝牙、超宽带、紫蜂ZigBee、RFID、4G、5G、LoRa、SIGFOX、Z-Wave以及NB-loT中的至少一种方式通信。
在一种可能的实现中,所述软件执行环境包括:
组件信息,所述组件信息用于指示所述第二终端设备是否部署有所述目标应用的运行组件,所述运行组件被配置为具备执行所述第一任务的能力;
所述执行要求,包括:
所述第二终端设备部署有所述运行组件。
在一种可能的实现中,所述设备信息包括如下信息的至少一种:
电源信息、处理器信息、内存信息、传输带宽、输入设备信息、输出设备信息、网络负载。
在一种可能的实现中,所述软件执行环境包括:所述终端设备的设备状态和/或所述运行组件的运行状态;
在所述设备状态指示所述第二终端设备处于下电、休眠、息屏或者锁屏的情况下,所述第一任务相关的任务运行信息包括第一指示信息,所述第一指示信息用于指示如下信息的至少一个:所述第二终端设备上电、结束休眠、结束息屏或者解除锁屏、指示所述第二终端设备运行所述运行组件、以及通过所述运行组件执行所述第一任务;或者,
在所述设备状态指示所述第二终端设备启动、且所述运行状态指示所述第二终端设备未运行所述运行组件的情况下,所述第一任务相关的任务运行信息包括第二指示信息,所述第二指示信息用于指示如下信息的至少一个所述第二终端设备运行所述运行组件、以及通过所述运行组件执行所述第一任务;或者,
在所述设备状态指示所述第二终端设备启动、且在所述运行状态指示所述第二终端设备运行所述运行组件的情况下,所述第一任务相关的任务运行信息包括第三指示信息,所述第三指示信息用于指示所述第二终端设备通过所述运行组件执行所述第一任务。
在一种可能的实现中,所述获取模块,具体用于:
获取与所述第一终端设备连接的多个终端设备的设备信息,所述多个终端设备包括所述第二终端设备;
所述装置还包括:
终端选择模块,用于在所述将所述第一任务的任务运行信息传递至所述第二终端设备之前,使能针对于所述多个终端设备中所述第二终端设备的选择。
在一种可能的实现中,所述终端选择模块,具体用于:
呈现第二选择信息,所述第二选择信息用于指示用户从所述多个终端设备进行选择, 并接收到用户根据所述第二选择信息反馈的第二输入;
基于所述第二输入包括所述第二终端设备,使能针对于所述多个终端设备中所述第二终端设备的选择。
在一种可能的实现中,所述终端选择模块,具体用于:
接收到用户的第二输入,所述第二输入指示从所述多个终端设备中选择所述第二设备。
在一种可能的实现中,所述终端选择模块,具体用于:
呈现第二选择信息,所述第二选择信息用于指示用户从所述多个终端设备进行选择;
接收到用户根据所述第二选择信息反馈的第二输入;所述第二输入包括所述第二终端设备。
在一种可能的实现中,所述终端选择模块,具体用于:
基于所述第二终端设备的设备信息为所述多个终端设备中选择最符合所述第一任务的执行要求的设备信息,使能针对于所述多个终端设备中所述第二终端设备的选择。
在一种可能的实现中,所述目标应用任务包括所述第一任务和第二任务;
所述装置还包括:
任务执行模块,用于在所述第二终端设备的设备信息不满足所述目标应用第二任务的执行要求时,保持对所述第二任务的执行。
在一种可能的实现中,所述第一任务相关的任务运行信息,包括:
执行数据以及所述执行数据的索引信息;其中,所述执行数据为执行所述第一任务所需的数据,所述索引信息包括内存地址,所述内存地址为所述第二终端设备上可用的存储空间的地址,所述索引信息用于指示所述第二终端设备将所述执行数据存储至所述内存地址对应的存储空间中。
在一种可能的实现中,所述第一任务为UI显示任务、图像渲染任务、逻辑状态更新任务、AI任务、数据处理或分析任务中的一种。
第四方面,本申请提供了一种应用运行装置,所述装置应用于目标系统,所述目标系统包括第一终端设备和第二终端设备,所述第一终端设备与所述第二终端设备通信连接,所述第一终端设备,包括:
获取模块,用于在运行目标应用时,获取第二终端设备的设备信息和/或软件执行环境;
发送模块,用于基于所述第二终端设备的设备信息和/或软件执行环境满足所述目标应用第一任务的执行要求,将所述第一任务的任务运行信息传递至所述第二终端设备,且所述第一终端设备不执行所述第一任务;
所述第二终端设备,包括:
第一任务执行模块,用于根据所述第一任务相关的任务运行信息执行所述第一任务。
在一种可能的实现中,所述目标应用还包括第二任务;所述第二终端设备,包括:
第二任务执行模块,用于在所述第二终端设备根据所述任务运行信息执行所述第一任务时,所述第一终端设备保持所述第二任务的执行。
在一种可能的实现中,所述第一终端设备还包括:
接收模块,用于在所述将所述第一任务相关的任务运行信息传递至所述第二终端设备之前,接收到用户的第一输入,所述第一输入指示将所述第一任务相关的任务运行信息传递至所述第二终端设备。
在一种可能的实现中,所述第一终端设备还包括:
呈现模块,用于所述第一终端设备呈现第一选择信息,所述第一选择信息指示所述目标应用至少一个任务,所述第二终端设备的设备信息和/或软件执行环境满足所述至少一个任务的执行要求;
所述接收模块,具体用于:
接收到用户根据所述第一选择信息反馈的第一输入;所述第一输入包括所述第一任务。
在一种可能的实现中,所述发送模块,还用于在所述将第一任务相关的任务运行信息传递至所述第二终端设备之后,所述第一终端设备将所述第一任务相关的状态数据传递至所述第二终端设备;所述状态数据为所述第二终端设备执行所述第一任务时所需的状态数据。
在一种可能的实现中,所述发送模块,还用于在所述将第一任务相关的任务运行信息传递至所述第二终端设备之后,所述第一终端设备将所述第一任务相关的状态数据传递至所述第二终端设备;所述状态数据为所述第二终端设备执行所述第一任务时所需的状态数据。
在一种可能的实现中,所述第一终端设备和所述第二终端设备之间通过近场通信、WIFI、蓝牙、超宽带、紫蜂ZigBee、RFID、4G、5G、LoRa、SIGFOX、Z-Wave以及NB-loT中的至少一种方式通信。
在一种可能的实现中,所述设备信息包括:
组件信息,所述组件信息用于指示所述第二终端设备是否部署有所述目标应用的运行组件,所述运行组件被配置为具备执行所述第一任务的能力;
所述执行要求,包括:
所述第二终端设备部署有所述运行组件。
在一种可能的实现中,所述设备信息包括如下信息的至少一种:
电源信息、处理器信息、内存信息、传输带宽、输入设备信息、输出设备信息、网络负载。
在一种可能的实现中,所述软件执行环境包括:所述终端设备的设备状态和/或所述运行组件的运行状态;
在所述设备状态指示所述第二终端设备处于下电、休眠、息屏或者锁屏的情况下,所述第一任务相关的任务运行信息包括第一指示信息,所述第一指示信息用于指示如下信息的至少一个:所述第二终端设备上电、结束休眠、结束息屏或者解除锁屏、指示所述第二终端设备运行所述运行组件、以及通过所述运行组件执行所述第一任务;或者,
在所述设备状态指示所述第二终端设备启动、且所述运行状态指示所述第二终端设备未运行所述运行组件的情况下,所述第一任务相关的任务运行信息包括第二指示信息,所述第二指示信息用于指示如下信息的至少一个所述第二终端设备运行所述运行组件、以及通过所述运行组件执行所述第一任务;或者,
在所述设备状态指示所述第二终端设备启动、且在所述运行状态指示所述第二终端设备运行所述运行组件的情况下,所述第一任务相关的任务运行信息包括第三指示信息,所述第三指示信息用于指示所述第二终端设备通过所述运行组件执行所述第一任务。
在一种可能的实现中,所述第一任务相关的任务运行信息,包括:
执行数据以及所述执行数据的索引信息;其中,所述执行数据为执行所述第一任务所需的数据,所述索引信息包括内存地址,所述内存地址为所述第二终端设备上可用的存储空间的地址,所述索引信息用于指示所述第二终端设备将所述执行数据存储至所述内存地址对应的存储空间中;
所述第一任务执行模块,具体用于:
根据所述索引信息,将所述执行数据存储至所述内存地址对应的存储空间中;
在执行所述第一任务时,从所述内存地址对应的存储空间中获取所述执行数据。
在一种可能的实现中,所述第二终端设备基于完成所述第一任务的执行、且未接收到所述第一终端设备针对于所述目标应用中其他任务的执行触发,不执行所述目标应用的其他任务。
在一种可能的实现中,所述第一任务为UI显示任务、图像渲染任务、逻辑状态更新任务、AI任务、数据处理或分析任务中的一种。
在一种可能的实现中,所述目标应用任务包括所述第一任务和第二任务;所述第一任 务和所述第二任务为渲染任务;所述第一任务用于绘制第一应用界面,所述第二任务用于绘制第二应用界面,所述第一应用界面和所述第二应用界面为不同的界面;
所述第一终端设备还包括:
第二任务执行模块,用于执行所述第二任务,并触发显示设备显示所述第二应用界面;
所述第一任务执行模块,具体用于:
所述第二终端设备触发显示设备显示所述第一应用界面。
在一种可能的实现中,所述第一任务为渲染任务;
所述第一任务执行模块,具体用于:
根据所述第一任务相关的任务运行信息,以及所述第二终端设备的显示设备的显示参数,执行所述第一任务。
第五方面,本申请提供了一种终端设备,所述设备包括处理器、存储器和总线,其中:
所述处理器、所述存储器通过所述总线连接;
所述存储器,用于存放计算机程序或指令;
所述处理器,用于调用或执行所述存储器上所存放的程序或指令以实现上述第一方面及第一方面中任一项可能实现方式所述的步骤、以及第二方面及第二方面中任一项可能实现方式所述的步骤。
第六方面,本申请提供了一种计算机存储介质,包括计算机指令,当计算机指令在电子设备或服务器上运行时,执行上述第一方面及第一方面中任一项可能实现方式所述的步骤、以及第二方面及第二方面中任一项可能实现方式所述的步骤。
第七方面,本申请提供了一种计算机程序产品,当计算机程序产品在电子设备或服务器上运行时,执行上述第一方面及第一方面中任一项可能实现方式所述的步骤、以及第二方面及第二方面中任一项可能实现方式所述的步骤。
第八方面,本申请提供了一种芯片系统,该芯片系统包括处理器,用于支持终端设备实现上述方面中所涉及的功能,例如,发送或处理上述方法中所涉及的数据;或,信息。在一种可能的设计中,所述芯片系统还包括存储器,所述存储器,用于保存执行设备或训练设备必要的程序指令和数据。该芯片系统,可以由芯片构成,也可以包括芯片和其他分立器件。
本申请实施例提供了一种应用运行方法,所述方法应用于目标系统,所述目标系统包括第一终端设备和第二终端设备,所述第一终端设备与所述第二终端设备通信连接,所述方法包括:所述第一终端设备在运行目标应用时,获取第二终端设备的设备信息;所述第一终端设备基于所述第二终端设备的设备信息满足所述目标应用第一任务的执行要求,将所述第一任务的任务运行信息传递至所述第二终端设备,且所述第一终端设备不执行所述第一任务;所述第二终端设备根据所述第一任务相关的任务运行信息执行所述第一任务。通过上述方法,在第二终端设备可以执行目标应用的一部分任务的前提下,将目标应用的一部分任务转移到第二终端设备上执行,可以降低第一终端设备的运算负载。
附图说明
图1为本申请实施例提供的应用架构示意图;
图2为本申请实施例提供的应用架构示意图;
图3为本申请实施例提供的终端设备的结构示意图;
图4为本申请实施例的终端设备的软件结构框图;
图5为本申请实施例提供的一种应用运行方法的实施例示意图;
图6为本申请实施例提供的一种界面示意图;
图7为本申请实施例提供的一种界面示意图;
图8为本申请实施例提供的一种索引信息示意图;
图9为本申请实施例提供的一种同步方法的实施例示意图;
图10为本申请实施例提供的一种应用运行方法的实施例示意图;
图11为本申请实施例提供的一种应用运行方法的实施例示意图;
图12为本申请实施例提供的一种应用运行装置的实施例示意图;
图13为本申请实施例提供的一种应用运行装置的实施例示意图;
图14为本申请实施例提供的一种应用运行装置的实施例示意图;
图15为本申请实施例提供的一种应用运行装置的实施例示意图;
图16为本申请实施例提供的一种终端设备的实施例示意图;
图17为一种应用任务的转移示意图。
具体实施方式
下面结合本申请实施例中的附图对本申请实施例进行描述。本申请实施例的实施方式部分使用的术语仅用于对本申请实施例的具体实施例进行解释,而非旨在限定本申请。
下面结合附图,对本申请的实施例进行描述。本领域普通技术人员可知,随着技术的发展和新场景的出现,本申请实施例提供的技术方案对于类似的技术问题,同样适用。
本申请的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别类似的对象,而不必用于描述特定的顺序或先后次序。应该理解这样使用的术语在适当情况下可以互换,这仅仅是描述本申请的实施例中对相同属性的对象在描述时所采用的区分方式。此外,术语“包括”和“具有”以及他们的任何变形,意图在于覆盖不排他的包含,以便包含一系列单元的过程、方法、系统、产品或设备不必限于那些单元,而是可包括没有清楚地列出的或对于这些过程、方法、产品或设备固有的其它单元。
为了便于理解本申请实施例,这里先说明本申请实施例涉及的应用架构。
参照图1,图1为本申请实施例提供的一种设备组网关系架构图的示意,其中,系统可以包括终端设备1和终端设备2。其中,终端设备1可以通过有线或者无线的方式与终端设备2进行连接。其中,其中,无线的方式包括通用分组无线服务技术(General Packet Radio Service,GPRS)、无线局域网(Wireless Local Area Networks,WLAN)、紫蜂(ZigBee)、蓝牙(Bluetooth)、近场通信(Near Field Communication,NFC)等等。应理解,上述通讯方式的举例 仅仅是用于进行说明,不应构成具体限定。
终端设备1以及终端设备2可以是移动终端。移动终端可以是智能手机、掌上处理设备、平板电脑、移动笔记本、虚拟现实设备、一体化掌机等等。移动终端还可以是可穿戴设备,可穿戴设备也可以称为穿戴式智能设备,是应用穿戴式技术对日常穿戴进行智能化设计、开发出可以穿戴的设备的总称,如眼镜、手套、手表、服饰及鞋等。可穿戴设备可以是直接穿在身上,或是整合到用户的衣服或配件的一种便携式设备。可穿戴设备可以通过软件支持以及数据交互、云端交互来实现强大的功能。广义穿戴式智能设备包括功能全、尺寸大、可不依赖智能手机实现完整或者部分的功能,例如:智能手表或智能眼镜等,以及只专注于某一类应用功能,需要和其它设备如智能手机配合使用,如各类进行体征监测的智能手环、智能首饰等。
终端设备1以及终端设备2还可以是是车载设备、智能会议设备、智能广告设备、智能家电等等。智能家电可以是自带音效设备的家电,例如,智能电视机等等,也可以是不带音效设备的家电,例如,智能投影仪,智能显示屏等等,此处不作具体限定。
在一应用场景中,终端设备1可以是显示屏体积比较小的人机交互设备,终端设备2可以是显示屏体积比较大的设备,例如,终端设备1可以是手机,终端设备2可以是电视机或者PC。应理解,上述举例仅用于说明,不能构成具体限定。
应理解,系统架构还可以包括更多数量的终端,参照图2,终端设备1可以和多个终端设备通信连接,此外,尽管图2中未示出,上述多个终端设备之间也可以通讯连接。
为便于理解,下面以终端设备为手机为例,对本申请实施例提供的终端设备100的结构进行示例说明。参见图3,图3是本申请实施例提供的终端设备的结构示意图。
如图3所示,终端设备100可以包括处理器110,外部存储器接口120,内部存储器121,通用串行总线(universal serial bus,USB)接口130,充电管理模块140,电源管理模块141,电池142,天线1,天线2,移动通信模块150,无线通信模块160,音频模块170,扬声器170A,受话器170B,麦克风170C,耳机接口170D,传感器模块180,按键190,马达191,指示器192,摄像头193,显示屏194,以及用户标识模块(subscriber identification module,SIM)卡接口195等。其中传感器模块180可以包括压力传感器180A,陀螺仪传感器180B,气压传感器180C,磁传感器180D,加速度传感器180E,距离传感器180F,接近光传感器180G,指纹传感器180H,温度传感器180J,触摸传感器180K,环境光传感器180L,骨传导传感器180M等。
可以理解的是,本申请实施例示意的结构并不构成对终端设备100的具体限定。在本申请另一些实施例中,终端设备100可以包括比图示更多或更少的部件,或者组合某些部件,或者拆分某些部件,或者不同的部件布置。图示的部件可以以硬件,软件或软件和硬件的组合实现。
处理器110可以包括一个或多个处理单元,例如:处理器110可以包括应用处理器(application processor,AP),调制解调处理器,图形处理器(graphics processing unit,GPU),图像信号处理器(image signal processor,ISP),控制器,视频编解码器,数字信号处理器(digital  signal processor,DSP),基带处理器,和/或神经网络处理器(neural-network processing unit,NPU)等。其中,不同的处理单元可以是独立的器件,也可以集成在一个或多个处理器中。
控制器可以根据指令操作码和时序信号,产生操作控制信号,完成取指令和执行指令的控制。
处理器110中还可以设置存储器,用于存储指令和数据。在一些实施例中,处理器110中的存储器为高速缓冲存储器。该存储器可以保存处理器110刚用过或循环使用的指令或数据。如果处理器110需要再次使用该指令或数据,可从所述存储器中直接调用。避免了重复存取,减少了处理器110的等待时间,因而提高了系统的效率。
在一些实施例中,处理器110可以包括一个或多个接口。接口可以包括集成电路(inter-integrated circuit,I2C)接口,集成电路内置音频(inter-integrated circuit sound,I2S)接口,脉冲编码调制(pulse code modulation,PCM)接口,通用异步收发传输器(universal asynchronous receiver/transmitter,UART)接口,移动产业处理器接口(mobile industry processor interface,MIPI),通用输入输出(general-purpose input/output,GPIO)接口,用户标识模块(subscriber identity module,SIM)接口,和/或通用串行总线(universal serial bus,USB)接口等。
I2C接口是一种双向同步串行总线,包括一根串行数据线(serial data line,SDA)和一根串行时钟线(derail clock line,SCL)。在一些实施例中,处理器110可以包含多组I2C总线。处理器110可以通过不同的I2C总线接口分别耦合触摸传感器180K,充电器,闪光灯,摄像头193等。例如:处理器110可以通过I2C接口耦合触摸传感器180K,使处理器110与触摸传感器180K通过I2C总线接口通信,实现终端设备100的触摸功能。
I2S接口可以用于音频通信。在一些实施例中,处理器110可以包含多组I2S总线。处理器110可以通过I2S总线与音频模块170耦合,实现处理器110与音频模块170之间的通信。在一些实施例中,音频模块170可以通过I2S接口向无线通信模块160传递音频信号,实现通过蓝牙耳机接听电话的功能。
PCM接口也可以用于音频通信,将模拟信号抽样,量化和编码。在一些实施例中,音频模块170与无线通信模块160可以通过PCM总线接口耦合。在一些实施例中,音频模块170也可以通过PCM接口向无线通信模块160传递音频信号,实现通过蓝牙耳机接听电话的功能。所述I2S接口和所述PCM接口都可以用于音频通信。
UART接口是一种通用串行数据总线,用于异步通信。该总线可以为双向通信总线。它将要传输的数据在串行通信与并行通信之间转换。在一些实施例中,UART接口通常被用于连接处理器110与无线通信模块160。例如:处理器110通过UART接口与无线通信模块160中的蓝牙模块通信,实现蓝牙功能。在一些实施例中,音频模块170可以通过UART接口向无线通信模块160传递音频信号,实现通过蓝牙耳机播放音乐的功能。
MIPI接口可以被用于连接处理器110与显示屏194,摄像头193等外围器件。MIPI接口包括摄像头串行接口(camera serial interface,CSI),显示屏串行接口(display serial interface,DSI)等。在一些实施例中,处理器110和摄像头193通过CSI接口通信,实现终端设备100的拍摄功能。处理器110和显示屏194通过DSI接口通信,实现终端设备100的显示功能。
GPIO接口可以通过软件配置。GPIO接口可以被配置为控制信号,也可被配置为数据信号。在一些实施例中,GPIO接口可以用于连接处理器110与摄像头193,显示屏194,无线通信模块160,音频模块170,传感器模块180等。GPIO接口还可以被配置为I2C接口,I2S接口,UART接口,MIPI接口等。
USB接口130是符合USB标准规范的接口,具体可以是Mini USB接口,Micro USB接口,USB Type C接口等。USB接口130可以用于连接充电器为终端设备100充电,也可以用于终端设备100与外围设备之间传输数据。也可以用于连接耳机,通过耳机播放音频。该接口还可以用于连接其他电子设备,例如AR设备等。
可以理解的是,本申请实施例示意的各模块间的接口连接关系,只是示意性说明,并不构成对终端设备100的结构限定。在本申请另一些实施例中,终端设备100也可以采用上述实施例中不同的接口连接方式,或多种接口连接方式的组合。
充电管理模块140用于从充电器接收充电输入。其中,充电器可以是无线充电器,也可以是有线充电器。在一些有线充电的实施例中,充电管理模块140可以通过USB接口130接收有线充电器的充电输入。在一些无线充电的实施例中,充电管理模块140可以通过终端设备100的无线充电线圈接收无线充电输入。充电管理模块140为电池142充电的同时,还可以通过电源管理模块141为电子设备供电。
电源管理模块141用于连接电池142,充电管理模块140与处理器110。电源管理模块141接收电池142和/或充电管理模块140的输入,为处理器110,内部存储器121,显示屏194,摄像头193,和无线通信模块160等供电。电源管理模块141还可以用于监测电池容量,电池循环次数,电池健康状态(漏电,阻抗)等参数。在其他一些实施例中,电源管理模块141也可以设置于处理器110中。在另一些实施例中,电源管理模块141和充电管理模块140也可以设置于同一个器件中。
终端设备100的无线通信功能可以通过天线1,天线2,移动通信模块150,无线通信模块160,调制解调处理器以及基带处理器等实现。
天线1和天线2用于发射和接收电磁波信号。终端设备100中的每个天线可用于覆盖单个或多个通信频带。不同的天线还可以复用,以提高天线的利用率。例如:可以将天线1复用为无线局域网的分集天线。在另外一些实施例中,天线可以和调谐开关结合使用。
移动通信模块150可以提供应用在终端设备100上的包括2G/3G/4G/5G等无线通信的解决方案。移动通信模块150可以包括至少一个滤波器,开关,功率放大器,低噪声放大器(low noise amplifier,LNA)等。移动通信模块150可以由天线1接收电磁波,并对接收的电磁波进行滤波,放大等处理,传送至调制解调处理器进行解调。移动通信模块150还可以对经调制解调处理器调制后的信号放大,经天线1转为电磁波辐射出去。在一些实施例中,移动通信模块150的至少部分功能模块可以被设置于处理器110中。在一些实施例中,移动通信模块150的至少部分功能模块可以与处理器110的至少部分模块被设置在同一个器件中。
调制解调处理器可以包括调制器和解调器。其中,调制器用于将待发送的低频基带信号调制成中高频信号。解调器用于将接收的电磁波信号解调为低频基带信号。随后解调器 将解调得到的低频基带信号传送至基带处理器处理。低频基带信号经基带处理器处理后,被传递给应用处理器。应用处理器通过音频设备(不限于扬声器170A,受话器170B等)输出声音信号,或通过显示屏194显示图像或视频。在一些实施例中,调制解调处理器可以是独立的器件。在另一些实施例中,调制解调处理器可以独立于处理器110,与移动通信模块150或其他功能模块设置在同一个器件中。
无线通信模块160可以提供应用在终端设备100上的包括无线局域网(wireless local area networks,WLAN)(如无线保真(wireless fidelity,Wi-Fi)网络),蓝牙(bluetooth,BT),全球导航卫星系统(global navigation satellite system,GNSS),调频(frequency modulation,FM),近距离无线通信技术(near field communication,NFC),红外技术(infrared,IR)等无线通信的解决方案。无线通信模块160可以是集成至少一个通信处理模块的一个或多个器件。无线通信模块160经由天线2接收电磁波,将电磁波信号调频以及滤波处理,将处理后的信号发送到处理器110。无线通信模块160还可以从处理器110接收待发送的信号,对其进行调频,放大,经天线2转为电磁波辐射出去。
在一些实施例中,终端设备100的天线1和移动通信模块150耦合,天线2和无线通信模块160耦合,使得终端设备100可以通过无线通信技术与网络以及其他设备通信。所述无线通信技术可以包括全球移动通讯系统(global system for mobile communications,GSM),通用分组无线服务(general packet radio service,GPRS),码分多址接入(code division multiple access,CDMA),宽带码分多址(wideband code division multiple access,WCDMA),时分码分多址(time-division code division multiple access,TD-SCDMA),长期演进(long term evolution,LTE),BT,GNSS,WLAN,NFC,FM,和/或IR技术等。所述GNSS可以包括全球卫星定位系统(global positioning system,GPS),全球导航卫星系统(global navigation satellite system,GLONASS),北斗卫星导航系统(beidou navigation satellite system,BDS),准天顶卫星系统(quasi-zenith satellite system,QZSS)和/或星基增强系统(satellite based augmentation systems,SBAS)。
终端设备100通过GPU,显示屏194,以及应用处理器等实现显示功能。GPU为图像处理的微处理器,连接显示屏194和应用处理器。GPU用于执行数学和几何计算,用于图形渲染。处理器110可包括一个或多个GPU,其执行程序指令以生成或改变显示信息。具体的,处理器110中的一个或多个GPU可以实现图像的渲染任务。
显示屏194用于显示图像,视频等。显示屏194包括显示面板。显示面板可以采用液晶显示屏(liquid crystal display,LCD),有机发光二极管(organic light-emitting diode,OLED),有源矩阵有机发光二极体或主动矩阵有机发光二极体(active-matrix organic light emitting diode的,AMOLED),柔性发光二极管(flex light-emitting diode,FLED),Miniled,MicroLed,Micro-oLed,量子点发光二极管(quantum dot light emitting diodes,QLED)等。在一些实施例中,终端设备100可以包括1个或N个显示屏194,N为大于1的正整数。
终端设备100可以通过ISP,摄像头193,视频编解码器,GPU,显示屏194以及应用处理器等实现拍摄功能。
ISP用于处理摄像头193反馈的数据。例如,拍照时,打开快门,光线通过镜头被传 递到摄像头感光元件上,光信号转换为电信号,摄像头感光元件将所述电信号传递给ISP处理,转化为肉眼可见的图像。ISP还可以对图像的噪点,亮度,肤色进行算法优化。ISP还可以对拍摄场景的曝光,色温等参数优化。在一些实施例中,ISP可以设置在摄像头193中。
摄像头193用于捕获静态图像或视频。物体通过镜头生成光学图像投射到感光元件。感光元件可以是电荷耦合器件(charge coupled device,CCD)或互补金属氧化物半导体(complementary metal-oxide-semiconductor,CMOS)光电晶体管。感光元件把光信号转换成电信号,之后将电信号传递给ISP转换成数字图像信号。ISP将数字图像信号输出到DSP加工处理。DSP将数字图像信号转换成标准的RGB,YUV等格式的图像信号。在一些实施例中,终端设备100可以包括1个或N个摄像头193,N为大于1的正整数。
视频编解码器用于对数字视频压缩或解压缩。终端设备100可以支持一种或多种视频编解码器。这样,终端设备100可以播放或录制多种编码格式的视频,例如:动态图像专家组(moving picture experts group,MPEG)1,MPEG2,MPEG3,MPEG4等。
NPU为神经网络(neural-network,NN)计算处理器,通过借鉴生物神经网络结构,例如借鉴人脑神经元之间传递模式,对输入信息快速处理,还可以不断的自学习。通过NPU可以实现终端设备100的智能认知等应用,例如:图像识别,人脸识别,语音识别,文本理解等。
外部存储器接口120可以用于连接外部存储卡,例如Micro SD卡,实现扩展终端设备100的存储能力。外部存储卡通过外部存储器接口120与处理器110通信,实现数据存储功能。例如将音乐,视频等文件保存在外部存储卡中。
内部存储器121可以用于存储计算机可执行程序代码,所述可执行程序代码包括指令。内部存储器121可以包括存储程序区和存储数据区。其中,存储程序区可存储操作系统,至少一个功能所需的应用程序(比如声音播放功能,图像播放功能等)等。存储数据区可存储终端设备100使用过程中所创建的数据(比如音频数据,电话本等)等。此外,内部存储器121可以包括高速随机存取存储器,还可以包括非易失性存储器,例如至少一个磁盘存储器件,闪存器件,通用闪存存储器(universal flash storage,UFS)等。处理器110通过运行存储在内部存储器121的指令,和/或存储在设置于处理器中的存储器的指令,执行终端设备100的各种功能应用以及数据处理。
终端设备100可以通过音频模块170,扬声器170A,受话器170B,麦克风170C,耳机接口170D,以及应用处理器等实现音频功能。例如音乐播放,录音等。
音频模块170用于将数字音频信息转换成模拟音频信号输出,也用于将模拟音频输入转换为数字音频信号。音频模块170还可以用于对音频信号编码和解码。在一些实施例中,音频模块170可以设置于处理器110中,或将音频模块170的部分功能模块设置于处理器110中。
扬声器170A,也称“喇叭”,用于将音频电信号转换为声音信号。终端设备100可以通过扬声器170A收听音乐,或收听免提通话。
受话器170B,也称“听筒”,用于将音频电信号转换成声音信号。当终端设备100接 听电话或语音信息时,可以通过将受话器170B靠近人耳接听语音。
麦克风170C,也称“话筒”,“传声器”,用于将声音信号转换为电信号。当拨打电话或发送语音信息时,用户可以通过人嘴靠近麦克风170C发声,将声音信号输入到麦克风170C。终端设备100可以设置至少一个麦克风170C。在另一些实施例中,终端设备100可以设置两个麦克风170C,除了采集声音信号,还可以实现降噪功能。在另一些实施例中,终端设备100还可以设置三个,四个或更多麦克风170C,实现采集声音信号,降噪,还可以识别声音来源,实现定向录音功能等。
耳机接口170D用于连接有线耳机。耳机接口170D可以是USB接口130,也可以是3.5mm的开放移动电子设备平台(open mobile terminal platform,OMTP)标准接口,美国蜂窝电信工业协会(cellular telecommunications industry association of the USA,CTIA)标准接口。
压力传感器180A用于感受压力信号,可以将压力信号转换成电信号。在一些实施例中,压力传感器180A可以设置于显示屏194。压力传感器180A的种类很多,如电阻式压力传感器,电感式压力传感器,电容式压力传感器等。电容式压力传感器可以是包括至少两个具有导电材料的平行板。当有力作用于压力传感器180A,电极之间的电容改变。终端设备100根据电容的变化确定压力的强度。当有触摸操作作用于显示屏194,终端设备100根据压力传感器180A检测所述触摸操作强度。终端设备100也可以根据压力传感器180A的检测信号计算触摸的位置。在一些实施例中,作用于相同触摸位置,但不同触摸操作强度的触摸操作,可以对应不同的操作指令。例如:当有触摸操作强度小于第一压力阈值的触摸操作作用于短消息应用图标时,执行查看短消息的指令。当有触摸操作强度大于或等于第一压力阈值的触摸操作作用于短消息应用图标时,执行新建短消息的指令。
陀螺仪传感器180B可以用于确定终端设备100的运动姿态。在一些实施例中,可以通过陀螺仪传感器180B确定终端设备100围绕三个轴(即,x,y和z轴)的角速度。陀螺仪传感器180B可以用于拍摄防抖。示例性的,当按下快门,陀螺仪传感器180B检测终端设备100抖动的角度,根据角度计算出镜头模组需要补偿的距离,让镜头通过反向运动抵消终端设备100的抖动,实现防抖。陀螺仪传感器180B还可以用于导航,体感游戏场景。
气压传感器180C用于测量气压。在一些实施例中,终端设备100通过气压传感器180C测得的气压值计算海拔高度,辅助定位和导航。
磁传感器180D包括霍尔传感器。终端设备100可以利用磁传感器180D检测翻盖皮套的开合。在一些实施例中,当终端设备100是翻盖机时,终端设备100可以根据磁传感器180D检测翻盖的开合。进而根据检测到的皮套的开合状态或翻盖的开合状态,设置翻盖自动解锁等特性。
加速度传感器180E可检测终端设备100在各个方向上(一般为三轴)加速度的大小。当终端设备100静止时可检测出重力的大小及方向。还可以用于识别电子设备姿态,应用于横竖屏切换,计步器等应用。
距离传感器180F,用于测量距离。终端设备100可以通过红外或激光测量距离。在一些实施例中,拍摄场景,终端设备100可以利用距离传感器180F测距以实现快速对焦。
接近光传感器180G可以包括例如发光二极管(LED)和光检测器,例如光电二极管。发 光二极管可以是红外发光二极管。终端设备100通过发光二极管向外发射红外光。终端设备100使用光电二极管检测来自附近物体的红外反射光。当检测到充分的反射光时,可以确定终端设备100附近有物体。当检测到不充分的反射光时,终端设备100可以确定终端设备100附近没有物体。终端设备100可以利用接近光传感器180G检测用户手持终端设备100贴近耳朵通话,以便自动熄灭屏幕达到省电的目的。接近光传感器180G也可用于皮套模式,口袋模式自动解锁与锁屏。
环境光传感器180L用于感知环境光亮度。终端设备100可以根据感知的环境光亮度自适应调节显示屏194亮度。环境光传感器180L也可用于拍照时自动调节白平衡。环境光传感器180L还可以与接近光传感器180G配合,检测终端设备100是否在口袋里,以防误触。
指纹传感器180H用于采集指纹。终端设备100可以利用采集的指纹特性实现指纹解锁,访问应用锁,指纹拍照,指纹接听来电等。
温度传感器180J用于检测温度。在一些实施例中,终端设备100利用温度传感器180J检测的温度,执行温度处理策略。例如,当温度传感器180J上报的温度超过阈值,终端设备100执行降低位于温度传感器180J附近的处理器的性能,以便降低功耗实施热保护。在另一些实施例中,当温度低于另一阈值时,终端设备100对电池142加热,以避免低温导致终端设备100异常关机。在其他一些实施例中,当温度低于又一阈值时,终端设备100对电池142的输出电压执行升压,以避免低温导致的异常关机。
触摸传感器180K,也称“触控器件”。触摸传感器180K可以设置于显示屏194,由触摸传感器180K与显示屏194组成触摸屏,也称“触控屏”。触摸传感器180K用于检测作用于其上或附近的触摸操作。触摸传感器可以将检测到的触摸操作传递给应用处理器,以确定触摸事件类型。可以通过显示屏194提供与触摸操作相关的视觉输出。在另一些实施例中,触摸传感器180K也可以设置于终端设备100的表面,与显示屏194所处的位置不同。
骨传导传感器180M可以获取振动信号。在一些实施例中,骨传导传感器180M可以获取人体声部振动骨块的振动信号。骨传导传感器180M也可以接触人体脉搏,接收血压跳动信号。在一些实施例中,骨传导传感器180M也可以设置于耳机中,结合成骨传导耳机。音频模块170可以基于所述骨传导传感器180M获取的声部振动骨块的振动信号,解析出语音信号,实现语音功能。应用处理器可以基于所述骨传导传感器180M获取的血压跳动信号解析心率信息,实现心率检测功能。
按键190包括开机键,音量键等。按键190可以是机械按键。也可以是触摸式按键。终端设备100可以接收按键输入,产生与终端设备100的用户设置以及功能控制有关的键信号输入。
马达191可以产生振动提示。马达191可以用于来电振动提示,也可以用于触摸振动反馈。例如,作用于不同应用(例如拍照,音频播放等)的触摸操作,可以对应不同的振动反馈效果。作用于显示屏194不同区域的触摸操作,马达191也可对应不同的振动反馈效果。不同的应用场景(例如:时间提醒,接收信息,闹钟,游戏等)也可以对应不同的振动反馈效果。触摸振动反馈效果还可以支持自定义。
指示器192可以是指示灯,可以用于指示充电状态,电量变化,也可以用于指示消息,未接来电,通知等。
SIM卡接口195用于连接SIM卡。SIM卡可以通过插入SIM卡接口195,或从SIM卡接口195拔出,实现和终端设备100的接触和分离。终端设备100可以支持1个或N个SIM卡接口,N为大于1的正整数。SIM卡接口195可以支持Nano SIM卡,Micro SIM卡,SIM卡等。同一个SIM卡接口195可以同时插入多张卡。所述多张卡的类型可以相同,也可以不同。SIM卡接口195也可以兼容不同类型的SIM卡。SIM卡接口195也可以兼容外部存储卡。终端设备100通过SIM卡和网络交互,实现通话以及数据通信等功能。在一些实施例中,终端设备100采用eSIM,即:嵌入式SIM卡。eSIM卡可以嵌在终端设备100中,不能和终端设备100分离。
终端设备100的软件系统可以采用分层架构,事件驱动架构,微核架构,微服务架构,或云架构。本申请实施例以分层架构的Android系统为例,示例性说明终端设备100的软件结构。
图4是本公开实施例的终端设备100的软件结构框图。
分层架构将软件分成若干个层,每一层都有清晰的角色和分工。层与层之间通过软件接口通信。在一些实施例中,将Android系统分为四层,从上至下分别为应用程序层,应用程序框架层,安卓运行时(Android runtime)和系统库,以及内核层。
应用程序层可以包括一系列应用程序包。
如图4所示,应用程序包可以包括相机,图库,日历,通话,地图,导航,WLAN,蓝牙,音乐,视频,短信息等应用程序。
应用程序框架层为应用程序层的应用程序提供应用编程接口(application programming interface,API)和编程框架。应用程序框架层包括一些预先定义的函数。
如图4所示,应用程序框架层可以包括窗口管理器,内容提供器,视图系统,电话管理器,资源管理器,通知管理器等。
窗口管理器用于管理窗口程序。窗口管理器可以获取显示屏大小,判断是否有状态栏,锁定屏幕,截取屏幕等。
内容提供器用来存放和获取数据,并使这些数据可以被应用程序访问。所述数据可以包括视频,图像,音频,拨打和接听的电话,浏览历史和书签,电话簿等。
视图系统包括可视控件,例如显示文字的控件,显示图片的控件等。视图系统可用于构建应用程序。显示界面可以由一个或多个视图组成的。例如,包括短信通知图标的显示界面,可以包括显示文字的视图以及显示图片的视图。
电话管理器用于提供终端设备100的通信功能。例如通话状态的管理(包括接通,挂断等)。
资源管理器为应用程序提供各种资源,比如本地化字符串,图标,图片,布局文件,视频文件等等。
通知管理器使应用程序可以在状态栏中显示通知信息,可以用于传达告知类型的消息,可以短暂停留后自动消失,无需用户交互。比如通知管理器被用于告知下载完成,消息提 醒等。通知管理器还可以是以图表或者滚动条文本形式出现在系统顶部状态栏的通知,例如后台运行的应用程序的通知,还可以是以对话窗口形式出现在屏幕上的通知。例如在状态栏提示文本信息,发出提示音,电子设备振动,指示灯闪烁等。
Android Runtime包括核心库和虚拟机。Android runtime负责安卓系统的调度和管理。
核心库包含两部分:一部分是java语言需要调用的功能函数,另一部分是安卓的核心库。
应用程序层和应用程序框架层运行在虚拟机中。虚拟机将应用程序层和应用程序框架层的java文件执行为二进制文件。虚拟机用于执行对象生命周期的管理,堆栈管理,线程管理,安全和异常的管理,以及垃圾回收等功能。
系统库可以包括多个功能模块。例如:表面管理器(surface manager),媒体库(Media Libraries),三维图形处理库(例如:OpenGL ES),2D图形引擎(例如:SGL)等。
表面管理器用于对显示子系统进行管理,并且为多个应用程序提供了2D和3D图层的融合。
媒体库支持多种常用的音频,视频格式回放和录制,以及静态图像文件等。媒体库可以支持多种音视频编码格式,例如:MPEG4,H.264,MP3,AAC,AMR,JPG,PNG等。
三维图形处理库用于实现三维图形绘图,图像渲染,合成,和图层处理等。
2D图形引擎是2D绘图的绘图引擎。
内核层是硬件和软件之间的层。内核层至少包含显示驱动,摄像头驱动,音频驱动,传感器驱动。
参照图5,图5为本申请实施例提供的一种应用运行方法的流程示意,所述方法应用于目标系统,所述目标系统包括第一终端设备和第二终端设备,所述第一终端设备与所述第二终端设备通信连接,如图5所述,本申请实施例提供的一种应用运行方法,包括:
501、所述第一终端设备在运行目标应用时,获取第二终端设备的设备信息和/或软件执行环境。
本申请实施例中,用户可以在第一终端设备上运行目标应用,目标应用可以是游戏应用、具有对话功能的应用、数据分析应用、音视频播放应用等。在一种可能的实现中,目标应用为算力开销、存储开销较大的应用。
在一种可能的实现中,由于第一终端设备受到CPU核心频率和GPU核心频率等的限制,第一终端设备的计算能力有限,会使得第一终端设备在运行目标应用时的计算资源占用过高,为了降低第一终端设备在运行目标应用时的资源负载,可以将目标应用的一部分计算任务转移到其他终端设备上,并由其他终端设备执行。
在一种可能的实现中,第一终端设备在运行目标应用时,可以呈现供用户选择的选项,来确定用户是否需要或者允许第一终端设备将目标应用的一部分计算任务转移到其他终端设备上。
示例性的,可以参照图6,用户在打开第一终端设备上的目标应用之后,第一终端设备可以显示如图所示的界面,该界面可以包括两个选项(“单机模式”选项以及“分布式模 式”选项),用户可以通过在界面的选择来确定是进入单机模式还是分布式模式,若用户选择单机模式,则第一终端设备可以运行目标应用,且不将目标应用的一部分计算任务转移到其他终端设备上,若用户选择分布式模式,则第一终端设备可以运行目标应用,且将目标应用的一部分计算任务转移到其他终端设备上。
接下来描述如何选择可以作为任务转移对象的终端设备。
在一种可能的实现中,第一终端设备可以获取到和第一终端设备存在通信连接的终端设备,并从存在通信连接的终端设备中选择可以作为任务转移对象的终端设备。
其中,通信连接可以是Internet网络、WIFI、蓝牙、超宽带、紫蜂ZigBee、(radio frequency identification,RFID)、4G、5G、LoRa、SIGFOX、Z-Wave以及NB-loT中的至少一种方式通信。例如,所述第一终端设备和所述第二终端设备之间通过近场通信、WIFI、蓝牙、超宽带、紫蜂ZigBee、RFID、4G、5G、LoRa、SIGFOX、Z-Wave以及NB-loT中的多种方式连接。
应理解,第一终端设备和所述第二终端设备之间还可以通过其他时延较低的无线连接方式(例如3GPP、non-3GPP以及基于短距技术实现的连接方式)进行连接,这里并不限定。
示例性的,第一终端设备可以为手机,第二终端设备可以为PC,其中phone与desktop与通过无线方式进行交互,如WiFi Direct或者WiFi热点的形式进行连接,Desktop与Monitor之间通过专用的线缆进行连接,如GMSL。通过将手机游戏部分任务分布式运行在desktop上,实能够带来大屏沉浸体验,同时将重负载的渲染运行在desktop中能够使能一些在phone中运行不畅(帧率低,卡顿)的游戏应用,带来更好的游戏体验。该方法的特点还在于两个设备可以同时独立的运行一个应用程序的不同部分,因此也扩展了应用的使用方法,是一种真正的分布式应用形态,与当前的几种分布式应用存在本质区别,当前的分布式应用主要可以归纳为三类,第一类Mapreduce类分布式多任务框架,此类框架多应用于同构系统中,特别是服务器集群中,用于多任务并发执行,与本例中动态的分割应用任务存在本质区别,同时较少考虑到异构环境的任务执行与同步,无法实现本例的最终目标即分担第一设备的负载压力;第二类图像的流式计算,其主要技术点在于分割图像任务到第二终端设备,再返回给第一设备显示。该过程可以实现部分算力转移,但是适应场景受限,特别是图像数据本身数据量较大,无法支持高度实时性的应用场景,而本例能够处理分割后存在的通信瓶颈问题。第三类云游戏类应用,该类应用通过云端渲染图像,回传本地实现了部分算力转移,相比前一类更加对实时性支持较差,同时也不具备切分应用任务的能力,只是简单的图像任务渲染回传。
接下来描述如何从存在通信连接的终端设备中选择可以作为任务转移对象的终端设备。
在一种可能的实现中,第一终端设备可以获取到存在通信连接的终端设备的设备信息,并基于对设备信息的分析来确定该终端设备是否具备运行要转移的任务的能力,以及确定将哪部分任务转移到终端设备。
以转移对象为第一任务为例,在一种可能的实现中,所述设备信息和或应用信息可以包括第二终端设备当前的设备运行状态以及可能有的应用运行状态信息,该类信息在转移 完成后也需要有选择性的同步,在这一阶段可以称为应用运行时信息,即包含了设备信息,已有应用信息及转移过来的应用运行信息。
其中,所述第一任务可以为UI显示任务、图像渲染任务、逻辑状态更新任务、AI任务、数据处理或分析任务中的一种渲。AI引擎可以用于执行AI任务,渲染引擎可以用于执行渲染任务,逻辑引擎和物理引擎可以用于执行逻辑状态更新任务,若第二终端设备未部署有具备执行所述第一任务的能力的运行组件,则不需要将第一任务转移到第二终端设备执行。
以游戏应用为例,运行组件可以包括逻辑状态更新引擎、物理引擎、AI引擎、渲染引擎等等,其中,逻辑状态更新引擎和物理引擎可以称之为模拟引擎,模拟引擎可以执行游戏逻辑并且控制游戏设置模拟的执行。渲染引擎可以称之为呈现引擎,呈现引擎可以控制游戏设置帧的渲染的执行以及帧的呈现的输出。模拟引擎可以读入游戏规则并且基于从一个或多个用户接收的输入来生成游戏状态。模拟引擎可以控制游戏应用程序内的个体对象(诸如虚拟部件、虚拟效果和/或虚拟角色)的执行。模拟引擎可以管理和确定角色移动、角色状态、碰撞检测,可以基于碰撞得出角色所期望的运动。模拟引擎接收用户输入并且确定角色事件,诸如动作、碰撞、奔跑、投掷、攻击和适合于游戏的其他事件。角色事件可以通过角色移动规则来控制,所述角色移动规则确定角色响应于事件应作出的适当运动。模拟引擎可以包括可以确定角色的新姿势的物理引擎。物理引擎可以将以下项作为其输入:各种角色的骨架模型、环境设置、角色状态诸如当前姿势(例如,被表达为位置、关节角度或其他规格的身体部位的位置)以及通过角色移动模块提供的身体部位和运动的速度(线速度和/或角速度),它们可以呈一些或全部身体部位的力/转矩矢量集合的形式。根据此信息,物理引擎使用物理规则生成角色的新姿势,并且这些新姿势可以被用来更新角色状态。模拟引擎根据定义的游戏规则提供用户输入,以控制游戏应用程序的各个方面。游戏规则的实施例包括得分规则、可能的输入、动作/事件、响应于输入的移动等。其他部件可以控制接受什么输入以及游戏如何进行以及游戏设置的其他方面。
模拟引擎可以输出图形状态数据,所述图形状态数据由呈现引擎使用以在游戏应用程序内生成并且渲染帧。每个虚拟对象可以被配置为通过模拟引擎处理的状态流过程。每个状态流过程可以生成用于呈现引擎的图形状态数据。例如,状态流过程可以包括各种虚拟对象,诸如发射器、灯、模型、封堵器(occluder)、地形、视觉环境以及与游戏应用程序一起影响游戏的状态的其他虚拟对象。本文进一步详细描述了模拟引擎的执行。
呈现引擎可以在游戏应用程序内使用图形状态数据生成并且渲染帧以便输出到显示器。呈现引擎可以组合虚拟对象,诸如虚拟角色、动画对象、非动画对象、背景对象、照明、反射等,以生成用于显示的完整场景和新帧。呈现引擎在渲染过程期间考虑了表面、颜色纹理以及其他参数。呈现引擎可以将虚拟对象(例如,虚拟环境中的照明和虚拟角色图像与非动画对象和背景对象)组合以生成并且渲染帧。本文进一步详细描述了呈现引擎的执行。
在一种可能的实现中,可以将目标应用拆分为多个任务,以游戏应用为例,游戏应用可以动态的拆分为2个任务,游戏逻辑状态更新任务和渲染任务。游戏逻辑更新任务对设备的需求可以是需要CPU设备、CPU主频在2.1Ghz以上,内存大于2GB,Android系统或者 Windows系统,音频输入输出,需要触屏输入,发送带宽50Mbps以上;渲染任务需要CPU设备,CPU主频在2.1Ghz以上,GPU设备需求度高,显示设备1080P以上,60Hz以上,接收带宽要求等。
以转移对象为第一任务为例,在一种可能的实现中,所述设备信息可以包括,电源信息,电源信息可以为第二终端设备的可用电量、充电模式(例如是有源输入还是电池供电),例如,在电源信息指示第二终端设备的可用电量较少时,则可以确定不需要将第一任务转移到第二终端设备执行。
以转移对象为第一任务为例,在一种可能的实现中,所述设备信息可以包括,处理器信息,处理器可以为CPU、GPU、TPU、NPU等,处理器信息可以但不限于为CPU型号/主频,GPU型号/主频,当前CPU/GPU负载等。由于一些任务对于处理器的类型、处理能力、当前可用的资源都有一定的要求,在不满足任务要求时,则可以确定不需要将第一任务转移到第二终端设备执行。
以转移对象为第一任务为例,在一种可能的实现中,所述设备信息可以包括,内存信息,例如内存大小。在不满足第一任务对于内存大小的要求时,则可以确定不需要将第一任务转移到第二终端设备执行。
以转移对象为第一任务为例,在一种可能的实现中,所述设备信息可以包括,传输带宽,传输带宽可以为当前网络的可用带宽。在一些实现中,为了保证任务执行的同步,则需要保证在将第一任务传输到第二终端设备时,不会存在延迟(或者延迟很小),则需要保证第一终端设备和第二终端设备之间的传输带宽需要满足第一任务对应的要求。在不满足要求时,则可以确定不需要将第一任务转移到第二终端设备执行。
以转移对象为第一任务为例,在一种可能的实现中,所述设备信息可以包括,输入设备信息,输入设备信息可以包括输入设备类型,输入设备可以为音频输入、鼠标输入、键盘输入等。在一些任务中,例如涉及和用户交互的任务,第一任务还需要支持用户的输入,而由于第一任务转移到第二终端设备执行,则需要第二终端设备具备第一任务所要求的输入设备类型。在不满足要求时,则可以确定不需要将第一任务转移到第二终端设备执行。
以转移对象为第一任务为例,在一种可能的实现中,所述设备信息可以包括,输出设备信息,输出设备信息可以包括输出设备类型,输出设备可以为音频输出、图像/视频输出等,输出设备信息还可以为屏幕分辨率、刷新率等显示设备的显示属性。在一些任务中,例如涉及和用户交互的任务,第一任务还需要支持输出(例如渲染任务要求存在显示设备,以及显示设备的参数要求),而由于第一任务转移到第二终端设备执行,则需要第二终端设备具备第一任务所要求的输出设备类型以及输出设备的属性。在不满足要求时,则可以确定不需要将第一任务转移到第二终端设备执行。
以转移对象为第一任务为例,在一种可能的实现中,所述设备信息可以包括,网络负载信息,网络负载信息可以包括当前设备网卡工作状态,网络带宽占用率、端到端延迟时间等。由于一些任务对于网络处理能力、当前可用网络资源都有一定的要求,在不满足转移任务要求时,则可以确定不需要将第一任务转移到第二终端设备执行;或者当前设备网络负载较高,而第二终端设备网络资源充足也可以决定立即执行任务转移。
在一种可能的实现中,第一终端设备可以向第二终端设备发起获取设备信息的请求,示例性的,第一终端设备为手机,手机的处理器通过信息采集模块发起获取设备规格和状态信息的请求,来获取已连接设备(如PC),PC返回设备信息,设备信息可以包括[CPU型号/主频,GPU型号/主频,内存大小,屏幕分辨率/刷新率,输入设备类型,音频输出,网络接口/能力,是否连接Internet,电量,当前CPU/GPU负载,当前网络可用带宽,当前剩余内存],例如,PC返回的信息可以是[Inteli711260/3.2Ghz,GTX3060,16GB,4K/120,[音频输入、鼠标输入、键盘输入],音频输出,WIFI802.11ax,连接Internet,有源,CPU负载30%/GPU负载5%,当前网络可用带宽200Mbps,当前剩余内存14GB],手机返回的设备信息可以是[ARMv8/2.6Ghz,Mali65,4GB,1080P/60Hz,[音频输入、触屏输入],音频输出,WIFI802.11ax,连接Internet,80%电量,CPU负载20%/GPU负载5%,当前网络可用带宽200Mbps,当前剩余内存3GB];获取本机及已连接设备的应用程序(即该实施例的游戏)版本与状态信息,PC返回程序信息[程序是否存在,程序版本号,当前是否存在该程序正在运行]。
在一种可能的实现中,如果第一终端设备确定出多个满足要求的终端设备时,可以从多个终端设备中选择部分(或全部)终端设备作为第一任务的转移对象。
在一种可能的实现中,可以获取与所述第一终端设备连接的多个终端设备的设备信息,所述多个终端设备包括所述第二终端设备,并使能针对于所述多个终端设备中所述第二终端设备的选择。
在一种可能的实现中,可以呈现第二选择信息,所述第二选择信息用于指示用户从所述多个终端设备进行选择,并接收到用户根据所述第二选择信息反馈的第二输入;基于所述第二输入包括所述第二终端设备,使能针对于所述多个终端设备中所述第二终端设备的选择。
参照图7,图7为一种第二选择信息的呈现方式的示意,图7示出了第一终端设备的显示界面,可以包括多个终端设备的选项,用户可以从多个终端设备的选项中选择第一任务的转移对象。
在一种可能的实现中,第一终端设备还可以基于所述第二终端设备的设备信息为所述多个终端设备中选择最符合所述第一任务的执行要求的设备信息,使能针对于所述多个终端设备中所述第二终端设备的选择。其中,所谓最符合执行要求,可以理解为在满足第一任务的执行要求的终端设备中选择性能最佳的终端设备。
以目标应用为游戏应用、第一终端设备为手机,第二终端设备为PC为例,第一终端设备可以根据逻辑状态更新任务和渲染显示任务的需求,与设备列表进行匹配,输出最佳调度结果。利用KM匹配算法,寻找最佳匹配。例如,渲染显示任务要求对屏幕分辨率,刷新率,以及GPU的规格越高,周边设备中PC的GPU规格,屏幕分辨率,刷新率的规格更高,且游戏逻辑更新任务需要触屏输入需要手机设备才能满足,因此本案例中最佳的任务调度结果是逻辑更新任务调度在手机本地,渲染任务调度在PC。
通过上述方法,第一终端设备可以确定第二终端设备为第一任务的转移对象,应理解,除了第二终端设备之外,第一终端设备还可以确定其他终端设备为第一任务的转移对象, 这里并不限定。
502、所述第一终端设备基于所述第二终端设备的设备信息满足所述目标应用第一任务的执行要求,将所述第一任务的任务运行信息传递至所述第二终端设备,且所述第一终端设备不执行所述第一任务。
本申请实施例中,第一终端设备在进行第一任务的转移时,需要将和执行第一任务相关的数据(或者称之为任务运行信息)传递至第二终端设备。
在一种可能的实现中,所述设备信息和或应用信息还可以包括:所述终端设备的设备状态和/或所述运行组件的运行状态;在所述设备状态指示所述第二终端设备未启动的情况下,所述第一任务相关的任务运行信息包括第一指示信息,所述第一指示信息用于触发所述第二终端设备启动、指示所述第二终端设备运行所述运行组件、以及通过所述运行组件执行所述第一任务;或者,在所述设备状态指示所述第二终端设备启动、且所述运行状态指示所述第二终端设备未运行所述运行组件的情况下,所述第一任务相关的任务运行信息包括第二指示信息,所述第二指示信息用于指示所述第二终端设备运行所述运行组件、以及通过所述运行组件执行所述第一任务;或者,在所述设备状态指示所述第二终端设备启动、且在所述运行状态指示所述第二终端设备运行所述运行组件的情况下,所述第一任务相关的任务运行信息包括第三指示信息,所述第三指示信息用于指示所述第二终端设备通过所述运行组件执行所述第一任务。
以游戏应用为例,在一种可能的实现中,任务运行信息可以包括任务列表及分布式游戏的启动参数,例如,game.exe–task render(还有其他和渲染相关的信息),通过网络发送给第二终端设备,并启动任务。第一终端设备可以本地启动游戏,启动游戏逻辑状态更新任务。
在一些可能的实现中,任务运行信息可以包括执行数据。
以游戏应用为例,执行数据可以包括游戏数据、游戏状态数据等等。
其中,游戏数据可以包括游戏规则,预先记录的运动捕捉姿势/路径、环境设置、环境对象、约束条件、骨架模型、路线信息和/或其他游戏应用程序信息。游戏数据的至少一部分可以存储在应用程序数据存储器中。在一些实施方案中,游戏数据的一部分可以被远程地接收和/或存储,诸如在数据存储器中,在这样的实施方案中,可以在游戏应用程序的运行时间期间接收游戏数据。
其中,游戏状态数据可以包括游戏状态、角色状态、环境状态、场景对象存储、路线信息和/或与游戏应用程序的运行时间状态相关联的其他信息。例如,游戏状态数据可以识别游戏应用程序在特定时间点的状态,诸如角色位置、角色定向、角色动作、游戏等级属性以及对游戏应用程序的状态有贡献的其他信息。游戏状态数据可以包括模拟状态数据和图形状态数据。模拟状态数据包括由模拟引擎使用以执行游戏应用程序的模拟的游戏状态数据。图形状态数据包括基于模拟状态数据生成的并且由呈现引擎使用以生成并且渲染帧以便输出诸如输出到用户计算系统的显示器的游戏状态数据。
第一终端设备可以负责游戏应用中的逻辑任务(可以称之为第二任务),逻辑任务主要为游戏应用自身的处理逻辑,如处理游戏输入,移动对应的游戏对象位置。并基于逻辑任 务生成状态集(即第一任务的任务运行信息),第一终端设备可以通过自身的同步模块来将状态集同步至第二终端设备,状态集可以为逻辑实体:真实数据存储在分散的具体的对象中,也可以为物理实体:为一段连续的存储空间,保存具体的状态数据。状态数据可包括,位置,旋转,动画状态等信息。在本实施例中主要描述物理实体的状态集。使用物理实体的优势主要有:只需要和状态集交互,如果是逻辑实体,则需要感知状态的具体位置(通常在逻辑任务模块中),需要和应用逻辑产生交互。如此可减少同步模块和应用的耦合,使同步模块成为通用的独立的模块。
其中,同步模块可以通过同步物理实体,而不感知具体的状态,则在同步过程中可减少序列化反序列化操作,将一系列状态一次同步,而不需要对每个状态分别进行序列化反序列化工作,在复杂的游戏场景,状态可数以千计,可以节省大量开销。这要求物理实体的设计可以跨异构设备。
在游戏应用时,为了保证应用的正常运行,需要保证第一终端设备和第二终端设备之间数据的实时同步,且同步频率需要很高,例如每一帧数据都需要进行一次状态同步,如果是60fps的游戏则约16ms进行一次同步操作,因此对同步的延时要求非常高。在现有的实现中,在第一终端设备和第二终端设备为异构设备的情况下,第一终端设备需要将待发送的数据进行序列化处理,以得到二进制数据,并将二进制数据传递至第二终端设备,第二终端设备需要对二进制数据进行反序列化,以得到自身能够识别出来的数据,上述序列反序列化操作会造成一定的时延,这在游戏等需要高同步要求的场景中,是不被允许的。
本申请实施例中,所述第一任务相关的任务运行信息可以包括:执行数据以及所述执行数据的索引信息;其中,所述执行数据为执行所述第一任务所需的数据,所述索引信息包括内存地址,所述内存地址为所述第二终端设备上可用的存储空间的地址,所述索引信息用于指示所述第二终端设备将所述执行数据存储至所述内存地址对应的存储空间中。
也就是说,可以先获取到第二终端设备上的可用存储空间,并将任务运行信息连同第一终端设备选择的存储空间的地址传递至第二终端设备,第二终端设备可以根据所述索引信息,将所述执行数据存储至所述内存地址对应的存储空间中,并在执行所述第一任务时,从所述内存地址对应的存储空间中获取所述执行数据。由于第二终端设备上的任务组件可以直接执行第一任务,并不需要对执行数据进行格式转换,且不需要序列和反序列化的操作,降低了处理时延。
示例性的,一种状态集的布局模块如下图8所示,其中索引区保存一系列索引信息,它可能包括:同步对象指针,指针在游戏逻辑中具体物理同步对象的指针,通过此指针可获取真实同步对象。类型索引,表明此指针的类型。具体类型信息是静态的,第一终端设备和第二终端设备部署时全局同步。偏移表示同步对象在状态集中的偏移。
基于此种状态集布局,以第一终端设备为手机和第二终端设备为desktop为例,phone端的同步流程的主要步骤有:游戏初始阶段,在Go1对象生成时,当其存在需要同步属性(可通过注解等方式标识),向同步模块申请状态集同步信息。包括索引信息{position指针,postion的类型索引,0(状态集中偏移)},及具体保存postion rawdata的空间。并将信息同步至对端,游戏逻辑执行,其间会修改对象位置信息。执行同步模块,同步模块执行同 步信息收集,依次对索引信息做如下处理:根据类型索引,获得具体类型。将指针转换为此类型的指针,并将指针指向内容保存到position rawdata中。此position rawdata的内存可布局可以直接对原始position进行copy,也可以采用其它形式来避免不同设备的layout,实现对异构设备的兼容。对rawdata部分直接封装,并将数据发送给对端。相应的,desktop端的同步流程的主要步骤可以有:同步模块执行接收工作,获得对端的状态数据,同步模块将状态数据覆盖本端状态集的rawdata部分;同步模块执行状态同步操作,依次对索引信息做如下处理:根据类型索引,获得具体类型,将指针转换为此类型的指针,并将偏移量对应的rawdata数据更新至指针指向的具体内容中,渲染模块刷新游戏对象GO时,其位置信息已获得更新,将GO在对应位置上显示。
503、所述第二终端设备根据所述第一任务相关的任务运行信息执行所述第一任务。
所述第二终端设备基于完成所述第一任务的执行、且未接收到所述第一终端设备针对于所述目标应用中其他任务的执行触发,不执行所述目标应用的其他任务。本申请实施例中,采用了基于异构内存排布的锁步lockstep同步机制,设计统一的状态同步数据接口,将不同平台的数据地址按顺序(例如多个终端设备和第一终端设备存在连接关系,则第一终端设备可以按照预设的顺序将任务运行信息传递至不同的终端设备)打包通知对面,同时按照lockstep方式进行严格限定的同步方法(作为接收任务运行信息的终端设备,除了接收到的任务运行信息,不会执行和目标应用相关的其他处理)。
以渲染任务为例,在一种可能的实现中,第二终端设备可以根据所述第一任务相关的任务运行信息,以及所述第二终端设备的显示设备的显示参数(例如显示设备的分辨率、屏幕刷新率等等),执行所述第一任务。
示例性的,参照图9,状态同步时序流程图可以如图9所示,可以包括以下步骤。
设备A执行物理任务,AI任务,逻辑任务后,游戏状态获得了相应的修改得到状态a。
设备A同步模块将状态a同步给设备B的同步模块。此步骤具体参与后续实施例。
设备B直接利用状态a进行渲染任务,并将画面传递给显示设备。
设备B渲染任务完成,不进行其它运算,等待设备A的下一个状态。
在一种可能的实现中,所述目标应用任务包括所述第一任务和第二任务;所述第一任务和所述第二任务为渲染任务;所述第一任务用于绘制第一应用界面,所述第二任务用于绘制第二应用界面,所述第一应用界面和所述第二应用界面为不同的界面;所述第一终端设备可以执行所述第二任务,并触发显示设备显示所述第二应用界面;所述第二终端设备可以触发显示设备显示所述第一应用界面。相当于可以将原本显示在同一个显示屏上的内容分割为多个界面,在不同的终端设备上进行渲染和显示。
例如,游戏的主界面可以转移到大屏显示,小地图在PAD端显示,而任务界面在笔记本显示,同时手机端显示控制界面,进而可以实现多屏多视角的显示模式,为消费者带来更好的游戏应用体验。
在一种可能的实现中,第二终端设备在执行第一任务之后,还可以将执行结果反馈至第一终端设备。
以游戏应用为例,在游戏过程中,可以定时(例如每个1s)执行上述步骤501至步骤503, 重新获取设备信息,查看最优调度是否有变化,如果有则进行重新调度。例如,当手机(第一终端设备)离开PC(第二终端设备)时,手机和PC间的网络变差带宽无法满足要求,或者无网络连接,则在第四步时获取PC设备的信息时,可用带宽为10Mbps或者无法获取PC设备信息,则进第五步和第六步时,PC端无法满足渲染任务要求,调度算法将两个任务都调度回手机,此时,手机同时进行逻辑状态更新及渲染显示,退回单机游戏模式;当手机距离PC较近时,PC又重新可以满足渲染任务需求,则提示用户是否进入分布式模式,重新进行任务调度,进入分布式模式。
本申请实施例提供了一种应用运行方法,所述方法应用于目标系统,所述目标系统包括第一终端设备和第二终端设备,所述第一终端设备与所述第二终端设备通信连接,所述方法包括:所述第一终端设备在运行目标应用时,获取第二终端设备的设备信息;所述第一终端设备基于所述第二终端设备的设备信息满足所述目标应用第一任务的执行要求,将所述第一任务的任务运行信息传递至所述第二终端设备,且所述第一终端设备不执行所述第一任务;所述第二终端设备根据所述第一任务相关的任务运行信息执行所述第一任务。通过上述方法,在第二终端设备可以执行目标应用的一部分任务的前提下,将目标应用的一部分任务转移到第二终端设备上执行,可以降低第一终端设备的运算负载。
参照表1,表1为单机与分布式版本手机功耗与性能测试效果对比:
表1
Figure PCTCN2022126849-appb-000001
上表为本申请实施例在分布式环境下的效果,可以看到单端游戏突破了性能瓶颈上限,实现了显示分辨率,实时帧率的提升。同时峰值温度下降,表明了本技术可以有效降低单端CPU负载,节约能耗。
参照图10和图11,图10和图11为以游戏应用为例,对本申请实施例提供的应用运行方法的流程描述。
另以一个具备聊天、视频功能的简单应用为例说明,如图17所示,在转移前阶段,左边设备在运行应用时一般只能与用户在一个功能上进行交互,可能的情况是用户当前正在使用聊天功能,则视频功能要么处于关闭状态,要么按照当前的一般实现,为了两个功能能够同时使用,那么视频功能只能以小窗口模式显示在前台,确认好对端设备具备转移条件后,则可以讲视频功能在右边应用中运行起来。如果之前在左边设备视频功能已经以小窗口模式运行中,则可以实现右边无缝切换为运行中的视频功能,退出小窗口模式。这样用户可以在两台设备中并行的实现聊天功能与视频功能,不受单台设备限制。
接下来结合软件模块,介绍本申请实施例提供的应用运行装置。应用运行装置的基础模块可以包含规则信息采集、任务调度、任务同步、任务执行等模块。信息采集模块用来负责收集分布式设备本身的状态、分布式应用程序的执行现场及状态;任务调度模块是负责根据采集信息决定任务调度的具体措施和任务切分策略及调度对象;任务同步模块是在分布式应用运行过程中保证各个模块的程序和设备状态信息能够保证一定时隙内的协同, 以实现和保证程序运行时的实时交互体验的模块。任务执行模块是执行分布式应用程序的模块。参照图12和图13,本申请实施例的核心软硬件模块可以包含信息采集模块001、任务同步模块002、任务调度模块003、三个模块。其中信息采集模块用来收集应用与设备的状态信息,例如每一个应用与之相对应的特征规则,管理者可以自己增加、删除、修改、查看这些规则文件。新增模块为识别匹配模块031,用于匹配应用与设备特征,实时更新匹配识别列表。实时调度模块032用来从匹配列表中读取当前需要调整的设备与应用任务,判断是否需要终止某些设备的任务,是否需要迁移任务,以及是否需要调整任务切分策略。状态同步模块021根据实时调度模块得到的同步设备和应用对象,以及从数据处理模块中得到的内存排布信息讲数据状态按照同步策略同步给从设备。数据处理模块022根据数据排布将数据按照可以快速映射的方式存储并提供操作接口。
本申请实施例还提供了本申请提供了一种应用运行方法,所述方法应用于目标系统,所述目标系统包括第一终端设备和第二终端设备,所述第一终端设备与所述第二终端设备通信连接,所述方法包括:
所述第一终端设备在运行目标应用时,获取第二终端设备的设备信息和/或软件执行环境;
所述第一终端设备基于所述第二终端设备的设备信息和/或软件执行环境满足所述目标应用第一任务的执行要求,将所述第一任务的任务运行信息传递至所述第二终端设备,且所述第一终端设备不执行所述第一任务;
所述第二终端设备根据所述第一任务相关的任务运行信息执行所述第一任务。
在一种可能的实现中,所述将所述第一任务相关的任务运行信息传递至所述第二终端设备之前,所述方法还包括:所述第一终端设备呈现第一选择信息,所述第一选择信息指示所述目标应用至少一个任务,所述第二终端设备的设备信息和/或软件执行环境满足所述至少一个任务的执行要求,并接收到用户根据所述第一选择信息反馈的第一输入;
所述第一终端设备基于所述第一输入包括所述第一任务,使能所述将所述第一任务相关的任务运行信息传递至所述第二终端设备的动作。
在一种可能的实现中,所述将第一任务相关的任务运行信息传递至所述第二终端设备之后,所述方法还包括:
所述第一终端设备将所述第一任务相关的状态数据传递至所述第二终端设备;所述状态数据为所述第二终端设备执行所述第一任务时所需的状态数据。
在一种可能的实现中,所述第一终端设备和所述第二终端设备之间通过近场通信、Internet网络、WIFI、蓝牙、超宽带、紫蜂ZigBee、RFID、4G、5G、LoRa、SIGFOX、Z-Wave以及NB-loT中的至少一种方式通信。
在一种可能的实现中,所述软件执行环境包括:
组件信息,所述组件信息用于指示所述第二终端设备是否部署有所述目标应用的运行组件,所述运行组件被配置为具备执行所述第一任务的能力;
所述执行要求,包括:
所述第二终端设备部署有所述运行组件。
在一种可能的实现中,所述设备信息包括如下信息的至少一种:
电源信息、处理器信息、内存信息、传输带宽、输入设备信息、输出设备信息、网络负载。
在一种可能的实现中,所述软件执行环境包括:所述终端设备的设备状态和/或所述运行组件的运行状态;
在所述设备状态指示所述第二终端设备处于下电、休眠、息屏或者锁屏的情况下,所述第一任务相关的任务运行信息包括第一指示信息,所述第一指示信息用于指示如下信息的至少一个:所述第二终端设备上电、结束休眠、结束息屏或者解除锁屏、指示所述第二终端设备运行所述运行组件、以及通过所述运行组件执行所述第一任务;或者,
在所述设备状态指示所述第二终端设备启动、且所述运行状态指示所述第二终端设备未运行所述运行组件的情况下,所述第一任务相关的任务运行信息包括第二指示信息,所述第二指示信息用于指示如下信息的至少一个所述第二终端设备运行所述运行组件、以及通过所述运行组件执行所述第一任务;或者,
在所述设备状态指示所述第二终端设备启动、且在所述运行状态指示所述第二终端设备运行所述运行组件的情况下,所述第一任务相关的任务运行信息包括第三指示信息,所述第三指示信息用于指示所述第二终端设备通过所述运行组件执行所述第一任务。
在游戏应用时,为了保证应用的正常运行,需要保证第一终端设备和第二终端设备之间数据的实时同步,且同步频率需要很高,例如每一帧数据都需要进行一次状态同步,如果是60fps的游戏则约16ms进行一次同步操作,因此对同步的延时要求非常高。在现有的实现中,在第一终端设备和第二终端设备为异构设备的情况下,第一终端设备需要将待发送的数据进行序列化处理,以得到二进制数据,并将二进制数据传递至第二终端设备,第二终端设备需要对二进制数据进行反序列化,以得到自身能够识别出来的数据,上述序列反序列化操作会造成一定的时延,这在游戏等需要高同步要求的场景中,是不被允许的。
本申请实施例中,所述第一任务相关的任务运行信息可以包括:执行数据以及所述执行数据的索引信息;其中,所述执行数据为执行所述第一任务所需的数据,所述索引信息包括内存地址,所述内存地址为所述第二终端设备上可用的存储空间的地址,所述索引信息用于指示所述第二终端设备将所述执行数据存储至所述内存地址对应的存储空间中。
也就是说,可以先获取到第二终端设备上的可用存储空间,并将任务运行信息连同第一终端设备选择的存储空间的地址传递至第二终端设备,第二终端设备可以根据所述索引信息,将所述执行数据存储至所述内存地址对应的存储空间中,并在执行所述第一任务时,从所述内存地址对应的存储空间中获取所述执行数据。由于第二终端设备上的任务组件可以直接执行第一任务,并不需要对执行数据进行格式转换,且不需要序列和反序列化的操作,降低了处理时延。
所述第二终端设备基于完成所述第一任务的执行、且未接收到所述第一终端设备针对于所述目标应用中其他任务的执行触发,不执行所述目标应用的其他任务。本申请实施例中,采用了基于异构内存排布的lockstep同步机制,设计统一的状态同步数据接口,将不同平台的数据地址按顺序(例如多个终端设备和第一终端设备存在连接关系,则第一终端 设备可以按照预设的顺序将任务运行信息传递至不同的终端设备)打包通知对面,同时按照lockstep方式进行严格限定的同步方法(作为接收任务运行信息的终端设备,除了接收到的任务运行信息,不会执行和目标应用相关的其他处理)。
在一种可能的实现中,所述第一任务为UI显示任务、图像渲染任务、逻辑状态更新任务、AI任务、数据处理或分析任务中的一种。
在一种可能的实现中,所述目标应用任务包括所述第一任务和第二任务;所述第一任务和所述第二任务为渲染任务;所述第一任务用于绘制第一应用界面,所述第二任务用于绘制第二应用界面,所述第一应用界面和所述第二应用界面为不同的界面;所述第一终端设备可以执行所述第二任务,并触发显示设备显示所述第二应用界面;所述第二终端设备可以触发显示设备显示所述第一应用界面。相当于可以将原本显示在同一个显示屏上的内容分割为多个界面,在不同的终端设备上进行渲染和显示。
在一种可能的实现中,以渲染任务为例,第二终端设备可以根据所述第一任务相关的任务运行信息,以及所述第二终端设备的显示设备的显示参数(例如显示设备的分辨率、屏幕刷新率等等),执行所述第一任务。
在一种可能的实现中,第二终端设备在执行第一任务之后,还可以将执行结果反馈至第一终端设备。
本申请还提供了一种应用运行装置,应用运行装置应用于第一终端设备,所述第一终端设备与所述第二终端设备通信连接,参照图14,图14为本申请实施例提供的一种应用运行的结构示意,如图14中示出的那样,所述应用运行装置1400包括:
获取模块1401,用于在运行目标应用时,获取第二终端设备的设备信息和/或软件执行环境;
关于获取模块1401的描述,可以参照步骤501的描述,这里不再赘述。
发送模块1402,用于基于所述第二终端设备的设备信息和/或软件执行环境满足所述目标应用第一任务的执行要求,将所述第一任务相关的任务运行信息传递至所述第二终端设备,以便所述第二终端设备根据所述任务运行信息执行所述第一任务,且所述第一终端设备不执行所述第一任务。
关于发送模块1402的描述,可以参照步骤502的描述,这里不再赘述。
在一种可能的实现中,所述装置还包括:
呈现模块,用于在所述将所述第一任务相关的任务运行信息传递至所述第二终端设备之前,呈现第一选择信息,所述第一选择信息指示所述目标应用至少一个任务,所述第二终端设备的设备信息和/或软件执行环境满足所述至少一个任务的执行要求,并接收到用户根据所述第一选择信息反馈的第一输入;
基于所述第一输入包括所述第一任务,使能所述将所述第一任务相关的任务运行信息传递至所述第二终端设备的动作。
在一种可能的实现中,所述发送模块,还用于在所述将第一任务相关的任务运行信息传递至所述第二终端设备之后,将所述第一任务相关的状态数据传递至所述第二终端设备;所述状态数据为所述第二终端设备执行所述第一任务时所需的状态数据。
在一种可能的实现中,所述第一终端设备和所述第二终端设备之间通过近场通信、Internet网络、WIFI、蓝牙、超宽带、紫蜂ZigBee、RFID、4G、5G、LoRa、SIGFOX、Z-Wave以及NB-loT中的至少一种方式通信。
在一种可能的实现中,所述软件执行环境包括:
组件信息,所述组件信息用于指示所述第二终端设备是否部署有所述目标应用的运行组件,所述运行组件被配置为具备执行所述第一任务的能力;
所述执行要求,包括:
所述第二终端设备部署有所述运行组件。
在一种可能的实现中,所述设备信息包括如下信息的至少一种:
电源信息、处理器信息、内存信息、传输带宽、输入设备信息、输出设备信息、网络负载。
在一种可能的实现中,所述软件执行环境包括:所述终端设备的设备状态和/或所述运行组件的运行状态;
在所述设备状态指示所述第二终端设备处于下电、休眠、息屏或者锁屏的情况下,所述第一任务相关的任务运行信息包括第一指示信息,所述第一指示信息用于指示如下信息的至少一个:所述第二终端设备上电、结束休眠、结束息屏或者解除锁屏、指示所述第二终端设备运行所述运行组件、以及通过所述运行组件执行所述第一任务;或者,
在所述设备状态指示所述第二终端设备启动、且所述运行状态指示所述第二终端设备未运行所述运行组件的情况下,所述第一任务相关的任务运行信息包括第二指示信息,所述第二指示信息用于指示如下信息的至少一个所述第二终端设备运行所述运行组件、以及通过所述运行组件执行所述第一任务;或者,
在所述设备状态指示所述第二终端设备启动、且在所述运行状态指示所述第二终端设备运行所述运行组件的情况下,所述第一任务相关的任务运行信息包括第三指示信息,所述第三指示信息用于指示所述第二终端设备通过所述运行组件执行所述第一任务。
在一种可能的实现中,所述获取模块,具体用于:
获取与所述第一终端设备连接的多个终端设备的设备信息,所述多个终端设备包括所述第二终端设备;
所述装置还包括:
终端选择模块,用于在所述将所述第一任务的任务运行信息传递至所述第二终端设备之前,使能针对于所述多个终端设备中所述第二终端设备的选择。
在一种可能的实现中,所述终端选择模块,具体用于:
呈现第二选择信息,所述第二选择信息用于指示用户从所述多个终端设备进行选择,并接收到用户根据所述第二选择信息反馈的第二输入;
基于所述第二输入包括所述第二终端设备,使能针对于所述多个终端设备中所述第二终端设备的选择。
在一种可能的实现中,所述终端选择模块,具体用于:
基于所述第二终端设备的设备信息为所述多个终端设备中选择最符合所述第一任务的 执行要求的设备信息,使能针对于所述多个终端设备中所述第二终端设备的选择。
在一种可能的实现中,所述目标应用任务包括所述第一任务和第二任务;
所述装置还包括:
任务执行模块,用于在所述第二终端设备的设备信息不满足所述目标应用第二任务的执行要求时,保持对所述第二任务的执行。
在一种可能的实现中,所述第一任务相关的任务运行信息,包括:
执行数据以及所述执行数据的索引信息;其中,所述执行数据为执行所述第一任务所需的数据,所述索引信息包括内存地址,所述内存地址为所述第二终端设备上可用的存储空间的地址,所述索引信息用于指示所述第二终端设备将所述执行数据存储至所述内存地址对应的存储空间中。
在一种可能的实现中,所述第一任务为UI显示任务、图像渲染任务、逻辑状态更新任务、AI任务、数据处理或分析任务中的一种。
本申请还提供了一种应用运行装置,应用运行装置应用于目标系统,所述目标系统包括第一终端设备和第二终端设备,所述第一终端设备与所述第二终端设备通信连接,参照图15,图15为本申请实施例提供的一种应用运行装置的结构示意,如图15中示出的那样,所述第一终端设备,包括:
获取模块1501,用于在运行目标应用时,获取第二终端设备的设备信息和/或软件执行环境;
关于获取模块1501的描述,可以参照步骤501的描述,这里不再赘述。
发送模块1502,用于基于所述第二终端设备的设备信息和/或软件执行环境满足所述目标应用第一任务的执行要求,将所述第一任务的任务运行信息传递至所述第二终端设备,且所述第一终端设备不执行所述第一任务;
关于发送模块1502的描述,可以参照步骤502的描述,这里不再赘述。
所述第二终端设备,包括:
第一任务执行模块1503,用于根据所述第一任务相关的任务运行信息执行所述第一任务。
关于第一任务执行模块1503的描述,可以参照步骤503的描述,这里不再赘述。
在一种可能的实现中,所述第一终端设备还包括:
呈现模块,用于所述第一终端设备呈现第一选择信息,所述第一选择信息指示所述目标应用至少一个任务,所述第二终端设备的设备信息和/或软件执行环境满足所述至少一个任务的执行要求,并接收到用户根据所述第一选择信息反馈的第一输入;
基于所述第一输入包括所述第一任务,使能所述将所述第一任务相关的任务运行信息传递至所述第二终端设备的动作。
在一种可能的实现中,所述发送模块,还用于在所述将第一任务相关的任务运行信息传递至所述第二终端设备之后,所述第一终端设备将所述第一任务相关的状态数据传递至所述第二终端设备;所述状态数据为所述第二终端设备执行所述第一任务时所需的状态数据。
在一种可能的实现中,所述第一终端设备和所述第二终端设备之间通过近场通信、WIFI、蓝牙、超宽带、紫蜂ZigBee、RFID、4G、5G、LoRa、SIGFOX、Z-Wave以及NB-loT中的至少一种方式通信。
在一种可能的实现中,所述设备信息包括:
组件信息,所述组件信息用于指示所述第二终端设备是否部署有所述目标应用的运行组件,所述运行组件被配置为具备执行所述第一任务的能力;
所述执行要求,包括:
所述第二终端设备部署有所述运行组件。
在一种可能的实现中,所述设备信息包括如下信息的至少一种:
电源信息、处理器信息、内存信息、传输带宽、输入设备信息、输出设备信息、网络负载。
在一种可能的实现中,所述软件执行环境包括:所述终端设备的设备状态和/或所述运行组件的运行状态;
在所述设备状态指示所述第二终端设备处于下电、休眠、息屏或者锁屏的情况下,所述第一任务相关的任务运行信息包括第一指示信息,所述第一指示信息用于指示如下信息的至少一个:所述第二终端设备上电、结束休眠、结束息屏或者解除锁屏、指示所述第二终端设备运行所述运行组件、以及通过所述运行组件执行所述第一任务;或者,
在所述设备状态指示所述第二终端设备启动、且所述运行状态指示所述第二终端设备未运行所述运行组件的情况下,所述第一任务相关的任务运行信息包括第二指示信息,所述第二指示信息用于指示如下信息的至少一个所述第二终端设备运行所述运行组件、以及通过所述运行组件执行所述第一任务;或者,
在所述设备状态指示所述第二终端设备启动、且在所述运行状态指示所述第二终端设备运行所述运行组件的情况下,所述第一任务相关的任务运行信息包括第三指示信息,所述第三指示信息用于指示所述第二终端设备通过所述运行组件执行所述第一任务。
在一种可能的实现中,所述第一任务相关的任务运行信息,包括:
执行数据以及所述执行数据的索引信息;其中,所述执行数据为执行所述第一任务所需的数据,所述索引信息包括内存地址,所述内存地址为所述第二终端设备上可用的存储空间的地址,所述索引信息用于指示所述第二终端设备将所述执行数据存储至所述内存地址对应的存储空间中;
所述第一任务执行模块,具体用于:
根据所述索引信息,将所述执行数据存储至所述内存地址对应的存储空间中;
在执行所述第一任务时,从所述内存地址对应的存储空间中获取所述执行数据。
在一种可能的实现中,所述第二终端设备基于完成所述第一任务的执行、且未接收到所述第一终端设备针对于所述目标应用中其他任务的执行触发,不执行所述目标应用的其他任务。
在一种可能的实现中,所述第一任务为UI显示任务、图像渲染任务、逻辑状态更新任务、AI任务、数据处理或分析任务中的一种。
在一种可能的实现中,所述目标应用任务包括所述第一任务和第二任务;所述第一任务和所述第二任务为渲染任务;所述第一任务用于绘制第一应用界面,所述第二任务用于绘制第二应用界面,所述第一应用界面和所述第二应用界面为不同的界面;
所述第一终端设备还包括:
第二任务执行模块,用于执行所述第二任务,并触发显示设备显示所述第二应用界面;
所述第一任务执行模块,具体用于:
所述第二终端设备触发显示设备显示所述第一应用界面。
在一种可能的实现中,所述第一任务为渲染任务;
所述第一任务执行模块,具体用于:
根据所述第一任务相关的任务运行信息,以及所述第二终端设备的显示设备的显示参数,执行所述第一任务。
接下来介绍本申请实施例提供的一种终端设备,终端设备可以为本申请实施例描述的终端设备(例如第一终端设备、第二终端设备),请参阅图16,图16为本申请实施例提供的终端设备的一种结构示意图,终端设备1600具体可以表现为虚拟现实VR设备、手机、平板、笔记本电脑、智能穿戴设备等,此处不做限定。具体的,终端设备1600包括:接收器1601、发射器1602、处理器1603和存储器1604(其中终端设备1600中的处理器1603的数量可以一个或多个,图16中以一个处理器为例),其中,处理器1603可以包括应用处理器16031和通信处理器16032。在本申请的一些实施例中,接收器1601、发射器1602、处理器1603和存储器1604可通过总线或其它方式连接。
存储器1604可以包括只读存储器和随机存取存储器,并向处理器1603提供指令和数据。存储器1604的一部分还可以包括非易失性随机存取存储器(non-volatile random access memory,NVRAM)。存储器1604存储有处理器和操作指令、可执行模块或者数据结构,或者它们的子集,或者它们的扩展集,其中,操作指令可包括各种操作指令,用于实现各种操作。
处理器1603控制终端设备的操作。具体的应用中,终端设备的各个组件通过总线系统耦合在一起,其中总线系统除包括数据总线之外,还可以包括电源总线、控制总线和状态信号总线等。但是为了清楚说明起见,在图中将各种总线都称为总线系统。
上述本申请实施例揭示的方法可以应用于处理器1603中,或者由处理器1603实现。处理器1603可以是一种集成电路芯片,具有信号的处理能力。在实现过程中,上述方法的各步骤可以通过处理器1603中的硬件的集成逻辑电路或者软件形式的指令完成。上述的处理器1603可以是通用处理器、数字信号处理器(digital signal processing,DSP)、微处理器或微控制器,还可进一步包括专用集成电路(application specific integrated circuit,ASIC)、现场可编程门阵列(field-programmable gate array,FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。该处理器1603可以实现或者执行本申请实施例中的公开的各方法、步骤及逻辑框图。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。结合本申请实施例所公开的方法的步骤可以直接体现为硬件译码处 理器执行完成,或者用译码处理器中的硬件及软件模块组合执行完成。软件模块可以位于随机存储器,闪存、只读存储器,可编程只读存储器或者电可擦写可编程存储器、寄存器等本领域成熟的存储介质中。该存储介质位于存储器1604,处理器1603读取存储器1604中的信息,结合其硬件完成上述方法的步骤。具体的,处理器1603可以读取存储器1604中的信息,结合其硬件完成上述实施例中步骤501至步骤503的步骤。
接收器1601可用于接收输入的数字或字符信息,以及产生与终端设备的相关设置以及功能控制有关的信号输入。发射器1602可用于通过第一接口输出数字或字符信息;发射器1602还可用于通过第一接口向磁盘组发送指令,以修改磁盘组中的数据;发射器1602还可以包括显示屏等显示设备。
本申请实施例中还提供一种包括计算机程序产品,当其在计算机上运行时,使得计算机执行上述实施例中图5对应的实施例中描述的应用运行方法的步骤。
本申请实施例中还提供一种计算机可读存储介质,该计算机可读存储介质中存储有用于进行信号处理的程序,当其在计算机上运行时,使得计算机执行如前述实施例描述的方法中的图像处理方法的步骤。
本申请实施例提供的图像显示装置具体可以为芯片,芯片包括:处理单元和通信单元,所述处理单元例如可以是处理器,所述通信单元例如可以是输入/输出接口、管脚或电路等。该处理单元可执行存储单元存储的计算机执行指令,以使执行设备内的芯片执行上述实施例描述的数据处理方法,或者,以使训练设备内的芯片执行上述实施例描述的数据处理方法。可选地,所述存储单元为所述芯片内的存储单元,如寄存器、缓存等,所述存储单元还可以是所述无线接入设备端内的位于所述芯片外部的存储单元,如只读存储器(read-only memory,ROM)或可存储静态信息和指令的其他类型的静态存储设备,随机存取存储器(random access memory,RAM)等。
另外需说明的是,以上所描述的装置实施例仅仅是示意性的,其中所述作为分离部件说明的单元可以是或者也可以不是物理上分开的,作为单元显示的部件可以是或者也可以不是物理单元,即可以位于一个地方,或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的。另外,本申请提供的装置实施例附图中,模块之间的连接关系表示它们之间具有通信连接,具体可以实现为一条或多条通信总线或信号线。
通过以上的实施方式的描述,所属领域的技术人员可以清楚地了解到本申请可借助软件加必需的通用硬件的方式来实现,当然也可以通过专用硬件包括专用集成电路、专用CPU、专用存储器、专用元器件等来实现。一般情况下,凡由计算机程序完成的功能都可以很容易地用相应的硬件来实现,而且,用来实现同一功能的具体硬件结构也可以是多种多样的,例如模拟电路、数字电路或专用电路等。但是,对本申请而言更多情况下软件程序实现是更佳的实施方式。基于这样的理解,本申请的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来,该计算机软件产品存储在可读取的存储介质中,如计算机的软盘、U盘、移动硬盘、ROM、RAM、磁碟或者光盘等,包括若干指令用以使得一台计算机设备(可以是个人计算机,服务器,或者网络设备等)执行本申请各个实施 例所述的方法。
在上述实施例中,可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件实现时,可以全部或部分地以计算机程序产品的形式实现。
所述计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行所述计算机程序指令时,全部或部分地产生按照本申请实施例所述的流程或功能。所述计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。所述计算机指令可以存储在计算机可读存储介质中,或者从一个计算机可读存储介质向另一计算机可读存储介质传输,例如,所述计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线(DSL))或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。所述计算机可读存储介质可以是计算机能够存储的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。所述可用介质可以是磁性介质,(例如,软盘、硬盘、磁带)、光介质(例如,DVD)、或者半导体介质(例如固态硬盘(Solid State Disk,SSD))等。

Claims (61)

  1. 一种应用运行方法,其特征在于,所述方法应用于第一终端设备,所述第一终端设备与所述第二终端设备通信连接,所述方法包括:
    在运行目标应用时,获取第二终端设备的设备信息和/或软件执行环境;
    在所述第二终端设备的设备信息和/或软件执行环境满足所述目标应用第一任务的执行要求时,将所述第一任务相关的任务运行信息传递至所述第二终端设备,以便所述第二终端设备根据所述任务运行信息执行所述第一任务,且所述第一终端设备不执行所述第一任务。
  2. 根据权利要求1所述的方法,其特征在于,所述目标应用还包括第二任务;所述方法还包括:
    在所述第二终端设备根据所述任务运行信息执行所述第一任务时,在所述第一终端设备上保持所述第二任务的执行。
  3. 根据权利要求1或2所述的方法,其特征在于,所述将所述第一任务相关的任务运行信息传递至所述第二终端设备之前,所述方法还包括:
    接收到用户的第一输入,所述第一输入指示将所述第一任务相关的任务运行信息传递至所述第二终端设备。
  4. 根据权利要求3所述的方法,其特征在于,所述将所述第一任务相关的任务运行信息传递至所述第二终端设备之前,所述方法还包括:
    呈现第一选择信息,所述第一选择信息指示所述目标应用至少一个任务,所述第二终端设备的设备信息和/或软件执行环境满足所述至少一个任务的执行要求;
    所述接收到用户的第一输入,包括:
    接收到用户根据所述第一选择信息反馈的第一输入;所述第一输入包括所述第一任务。
  5. 根据权利要求1至4任一所述的方法,其特征在于,所述将第一任务相关的任务运行信息传递至所述第二终端设备之后,所述方法还包括:
    将所述第一任务相关的状态数据传递至所述第二终端设备;所述状态数据为所述第二终端设备执行所述第一任务时所需的状态数据。
  6. 根据权利要求1至5任一所述的方法,其特征在于,所述第一终端设备和所述第二终端设备之间通过近场通信、Internet网络、WIFI、蓝牙、超宽带、紫蜂ZigBee、RFID、4G、5G、LoRa、SIGFOX、Z-Wave以及NB-loT中的至少一种方式通信。
  7. 根据权利要求1至6任一所述的方法,其特征在于,所述软件执行环境包括:
    组件信息,所述组件信息用于指示所述第二终端设备是否部署有所述目标应用的运行组件,所述运行组件被配置为具备执行所述第一任务的能力;
    所述执行要求,包括:
    所述第二终端设备部署有所述运行组件。
  8. 根据权利要求1至7任一所述的方法,其特征在于,所述设备信息包括如下信息的至少一种:
    电源信息、处理器信息、内存信息、传输带宽、输入设备信息、输出设备信息、网络负载。
  9. 根据权利要求7或8所述的方法,其特征在于,所述软件执行环境包括:所述终端设备的设备状态和/或所述运行组件的运行状态;
    在所述设备状态指示所述第二终端设备处于下电、休眠、息屏或者锁屏的情况下,所述第一任务相关的任务运行信息包括第一指示信息,所述第一指示信息用于指示如下信息的至少一个:所述第二终端设备上电、结束休眠、结束息屏或者解除锁屏、指示所述第二终端设备运行所述运行组件、以及通过所述运行组件执行所述第一任务;或者,
    在所述设备状态指示所述第二终端设备启动、且所述运行状态指示所述第二终端设备未运行所述运行组件的情况下,所述第一任务相关的任务运行信息包括第二指示信息,所述第二指示信息用于指示如下信息的至少一个所述第二终端设备运行所述运行组件、以及通过所述运行组件执行所述第一任务;或者,
    在所述设备状态指示所述第二终端设备启动、且在所述运行状态指示所述第二终端设备运行所述运行组件的情况下,所述第一任务相关的任务运行信息包括第三指示信息,所述第三指示信息用于指示所述第二终端设备通过所述运行组件执行所述第一任务。
  10. 根据权利要求1至9任一所述的方法,其特征在于,所述获取第二终端设备的设备信息,包括:获取与所述第一终端设备连接的多个终端设备的设备信息,所述多个终端设备包括所述第二终端设备;
    所述将所述第一任务的任务运行信息传递至所述第二终端设备之前,所述方法还包括:
    使能针对于所述多个终端设备中所述第二设备的选择。
  11. 根据权利要求10所述的方法,其特征在于,所述使能针对于所述多个终端设备中所述第二设备的选择,包括:
    接收到用户的第二输入,所述第二输入指示从所述多个终端设备中选择所述第二设备。
  12. 根据权利要求11所述的方法,其特征在于,所述将所述第一任务的任务运行信息传递至所述第二终端设备之前,所述方法还包括:
    呈现第二选择信息,所述第二选择信息用于指示用户从所述多个终端设备进行选择;
    所述接收到用户的第二输入,包括:
    接收到用户根据所述第二选择信息反馈的第二输入;所述第二输入包括所述第二终端设备。
  13. 根据权利要求10所述的方法,其特征在于,所述使能针对于所述多个终端设备中所述第二设备的选择,包括:
    基于所述第二终端设备的设备信息为所述多个终端设备中选择最符合所述第一任务的执行要求的设备信息,使能针对于所述多个终端设备中所述第二设备的选择。
  14. 根据权利要求1至13任一所述的方法,其特征在于,所述第一任务相关的任务运行信息,包括:
    执行数据以及所述执行数据的索引信息;其中,所述执行数据为执行所述第一任务所需的数据,所述索引信息包括内存地址,所述内存地址为所述第二终端设备上可用的存储空间的地址,所述索引信息用于指示所述第二终端设备将所述执行数据存储至所述内存内存地址对应的内存空间中。
  15. 根据权利要求1至14任一所述的方法,其特征在于,所述第一任务为UI显示任务、图像渲染任务、逻辑状态更新任务、AI任务、数据处理或分析任务中的一种。
  16. 一种应用运行方法,其特征在于,所述方法应用于目标系统,所述目标系统包括第一终端设备和第二终端设备,所述第一终端设备与所述第二终端设备通信连接,所述方法包括:
    所述第一终端设备在运行目标应用时,获取第二终端设备的设备信息和/或软件执行环境;
    所述第一终端设备基于所述第二终端设备的设备信息和/或软件执行环境满足所述目标应用第一任务的执行要求,将所述第一任务的任务运行信息传递至所述第二终端设备,且所述第一终端设备不执行所述第一任务;
    所述第二终端设备根据所述第一任务相关的任务运行信息执行所述第一任务。
  17. 根据权利要求16所述的方法,其特征在于,所述目标应用还包括第二任务;所述方法还包括:
    在所述第二终端设备根据所述任务运行信息执行所述第一任务时,所述第一终端设备保持所述第二任务的执行。
  18. 根据权利要求17所述的方法,其特征在于,所述将所述第一任务相关的任务运行信息传递至所述第二终端设备之前,所述方法还包括:所述第一终端设备接收到用户的第一输入,所述第一输入指示将所述第一任务相关的任务运行信息传递至所述第二终端设备。
  19. 根据权利要求18所述的方法,其特征在于,所述将所述第一任务相关的任务运行信息传递至所述第二终端设备之前,所述方法还包括:
    所述第一终端设备呈现第一选择信息,所述第一选择信息指示所述目标应用至少一个任务,所述第二终端设备的设备信息和/或软件执行环境满足所述至少一个任务的执行要求;
    所述接收到用户的第一输入,包括:
    接收到用户根据所述第一选择信息反馈的第一输入;所述第一输入包括所述第一任务。
  20. 根据权利要求16至19任一所述的方法,其特征在于,
    所述将第一任务相关的任务运行信息传递至所述第二终端设备之后,所述方法还包括:
    所述第一终端设备将所述第一任务相关的状态数据传递至所述第二终端设备;所述状态数据为所述第二终端设备执行所述第一任务时所需的状态数据。
  21. 根据权利要求16至20任一所述的方法,其特征在于,所述第一终端设备和所述第二终端设备之间通过近场通信、Internet网络、WIFI、蓝牙、超宽带、紫蜂ZigBee、RFID、4G、5G、LoRa、SIGFOX、Z-Wave以及NB-loT中的至少一种方式通信。
  22. 根据权利要求16至21任一所述的方法,其特征在于,所述软件执行环境包括:
    组件信息,所述组件信息用于指示所述第二终端设备是否部署有所述目标应用的运行组件,所述运行组件被配置为具备执行所述第一任务的能力;
    所述执行要求,包括:
    所述第二终端设备部署有所述运行组件。
  23. 根据权利要求16至22任一所述的方法,其特征在于,所述设备信息包括如下信息的至少一种:
    电源信息、处理器信息、内存信息、传输带宽、输入设备信息、输出设备信息、网络负载。
  24. 根据权利要求22或23所述的方法,其特征在于,所述软件执行环境包括:所述终端设备的设备状态和/或所述运行组件的运行状态;
    在所述设备状态指示所述第二终端设备处于下电、休眠、息屏或者锁屏的情况下,所述第一任务相关的任务运行信息包括第一指示信息,所述第一指示信息用于指示如下信息的至少一个:所述第二终端设备上电、结束休眠、结束息屏或者解除锁屏、指示所述第二终端设备运行所述运行组件、以及通过所述运行组件执行所述第一任务;或者,
    在所述设备状态指示所述第二终端设备启动、且所述运行状态指示所述第二终端设备未运行所述运行组件的情况下,所述第一任务相关的任务运行信息包括第二指示信息,所述第二指示信息用于指示如下信息的至少一个所述第二终端设备运行所述运行组件、以及 通过所述运行组件执行所述第一任务;或者,
    在所述设备状态指示所述第二终端设备启动、且在所述运行状态指示所述第二终端设备运行所述运行组件的情况下,所述第一任务相关的任务运行信息包括第三指示信息,所述第三指示信息用于指示所述第二终端设备通过所述运行组件执行所述第一任务。
  25. 根据权利要求16至24任一所述的方法,其特征在于,所述第一任务相关的任务运行信息,包括:
    执行数据以及所述执行数据的索引信息;其中,所述执行数据为执行所述第一任务所需的数据,所述索引信息包括内存地址,所述内存地址为所述第二终端设备上可用的存储空间的地址,所述索引信息用于指示所述第二终端设备将所述执行数据存储至所述内存内存地址对应的内存空间中;
    所述第二终端设备根据所述第一任务相关的任务运行信息执行所述第一任务,包括:
    所述第二终端设备根据所述索引信息,将所述执行数据存储至所述内存地址对应的存储空间中;
    所述第二终端设备在执行所述第一任务时,从所述内存地址对应的存储空间中获取所述执行数据。
  26. 根据权利要求16至25任一所述的方法,其特征在于,
    所述第二终端设备基于完成所述第一任务的执行、且未接收到所述第一终端设备针对于所述目标应用中其他任务的执行触发,不执行所述目标应用的其他任务。
  27. 根据权利要求16至26任一所述的方法,其特征在于,所述目标应用任务包括所述第一任务和第二任务;所述第一任务和所述第二任务为渲染任务;所述第一任务用于绘制第一应用界面,所述第二任务用于绘制第二应用界面,所述第一应用界面和所述第二应用界面为不同的界面;
    所述方法还包括:
    所述第一终端设备执行所述第二任务,并触发显示设备显示所述第二应用界面;
    所述第二终端设备触发显示设备显示所述第一应用界面。
  28. 根据权利要求16至27任一所述的方法,其特征在于,所述第一任务为渲染任务;
    所述第二终端设备根据所述第一任务相关的任务运行信息执行所述第一任务,包括:
    所述第二终端设备根据所述第一任务相关的任务运行信息,以及所述第二终端设备的显示设备的显示参数,执行所述第一任务。
  29. 一种应用运行方法,其特征在于,所述方法应用于第一终端设备,所述第一终端设备与所述第二终端设备通信连接,所述方法包括:
    运行目标应用;所述目标应用包括第一任务;所述第一任务为所述目标应用中的部分 任务;其中,
    所述目标应用为具有对话功能的应用;所述第一任务为部分或全部聊天对象的聊天框的UI显示、视频聊天的UI显示或者音频聊天的UI显示;或者,
    所述目标应用为绘画应用、文本编辑类应用、图像编辑类应用或者音视频编辑应用;所述第一任务为编辑工具界面的UI显示、或者编辑对象的UI显示;
    保持所述第二任务的执行,并将所述第一任务相关的任务运行信息传递至所述第二终端设备,以便所述第二终端设备根据所述任务运行信息执行所述第一任务,且所述第一终端设备不执行所述第一任务。
  30. 根据权利要求29所述的方法,其特征在于,所述目标应用还包括不同于第一任务的第二任务;
    所述目标应用为具有对话功能的应用;所述第二任务为和所述第一任务中不同的聊天对象的UI显示、视频聊天的UI显示或者音频聊天的UI显示;
    所述目标应用为绘画应用、文本编辑类应用、图像编辑类应用或者音视频编辑应用;所述第二任务为编辑工具界面的UI显示、或者编辑对象的UI显示;
    所述方法还包括:
    在所述第二终端设备的设备信息不满足所述目标应用所述第二任务的执行要求时,保持对所述第二任务的执行。
  31. 一种应用运行装置,其特征在于,所述装置应用于第一终端设备,所述第一终端设备与所述第二终端设备通信连接,所述装置包括:
    获取模块,用于在运行目标应用时,获取第二终端设备的设备信息和/或软件执行环境;
    发送模块,用于在所述第二终端设备的设备信息和/或软件执行环境满足所述目标应用第一任务的执行要求时,将所述第一任务相关的任务运行信息传递至所述第二终端设备,以便所述第二终端设备根据所述任务运行信息执行所述第一任务,且所述第一终端设备不执行所述第一任务。
  32. 根据权利要求31所述的装置,其特征在于,所述目标应用还包括第二任务;所述装置还包括:
    任务执行模块,用于在所述第二终端设备根据所述任务运行信息执行所述第一任务时,在所述第一终端设备上保持所述第二任务的执行。
  33. 根据权利要求31或32所述的装置,其特征在于,所述装置还包括:
    接收模块,用于在所述将所述第一任务相关的任务运行信息传递至所述第二终端设备之前,接收到用户的第一输入,所述第一输入指示将所述第一任务相关的任务运行信息传递至所述第二终端设备。
  34. 根据权利要求33所述的装置,其特征在于,所述装置还包括:
    呈现模块,用于在所述将所述第一任务相关的任务运行信息传递至所述第二终端设备之前,呈现第一选择信息,所述第一选择信息指示所述目标应用至少一个任务,所述第二终端设备的设备信息和/或软件执行环境满足所述至少一个任务的执行要求;
    所述接收模块,具体用于:
    接收到用户根据所述第一选择信息反馈的第一输入;所述第一输入包括所述第一任务。
  35. 根据权利要求31至34任一所述的装置,其特征在于,所述发送模块,还用于在所述将第一任务相关的任务运行信息传递至所述第二终端设备之后,将所述第一任务相关的状态数据传递至所述第二终端设备;所述状态数据为所述第二终端设备执行所述第一任务时所需的状态数据。
  36. 根据权利要求31至35任一所述的装置,其特征在于,所述第一终端设备和所述第二终端设备之间通过近场通信、Internet网络、WIFI、蓝牙、超宽带、紫蜂ZigBee、RFID、4G、5G、LoRa、SIGFOX、Z-Wave以及NB-loT中的至少一种方式通信。
  37. 根据权利要求31至36任一所述的装置,其特征在于,所述软件执行环境包括:
    组件信息,所述组件信息用于指示所述第二终端设备是否部署有所述目标应用的运行组件,所述运行组件被配置为具备执行所述第一任务的能力;
    所述执行要求,包括:
    所述第二终端设备部署有所述运行组件。
  38. 根据权利要求31至37任一所述的装置,其特征在于,所述设备信息包括如下信息的至少一种:
    电源信息、处理器信息、内存信息、传输带宽、输入设备信息、输出设备信息、网络负载。
  39. 根据权利要求37或38所述的装置,其特征在于,所述软件执行环境包括:所述终端设备的设备状态和/或所述运行组件的运行状态;
    在所述设备状态指示所述第二终端设备处于下电、休眠、息屏或者锁屏的情况下,所述第一任务相关的任务运行信息包括第一指示信息,所述第一指示信息用于指示如下信息的至少一个:所述第二终端设备上电、结束休眠、结束息屏或者解除锁屏、指示所述第二终端设备运行所述运行组件、以及通过所述运行组件执行所述第一任务;或者,
    在所述设备状态指示所述第二终端设备启动、且所述运行状态指示所述第二终端设备未运行所述运行组件的情况下,所述第一任务相关的任务运行信息包括第二指示信息,所述第二指示信息用于指示如下信息的至少一个所述第二终端设备运行所述运行组件、以及通过所述运行组件执行所述第一任务;或者,
    在所述设备状态指示所述第二终端设备启动、且在所述运行状态指示所述第二终端设备运行所述运行组件的情况下,所述第一任务相关的任务运行信息包括第三指示信息,所述第三指示信息用于指示所述第二终端设备通过所述运行组件执行所述第一任务。
  40. 根据权利要求31至39任一所述的装置,其特征在于,所述获取模块,具体用于:
    获取与所述第一终端设备连接的多个终端设备的设备信息,所述多个终端设备包括所述第二终端设备;
    所述装置还包括:
    终端选择模块,用于在所述将所述第一任务的任务运行信息传递至所述第二终端设备之前,使能针对于所述多个终端设备中所述第二终端设备的选择。
  41. 根据权利要求40所述的装置,其特征在于,所述终端选择模块,具体用于:
    接收到用户的第二输入,所述第二输入指示从所述多个终端设备中选择所述第二设备。
  42. 根据权利要求41所述的装置,其特征在于,所述终端选择模块,具体用于:
    呈现第二选择信息,所述第二选择信息用于指示用户从所述多个终端设备进行选择;
    接收到用户根据所述第二选择信息反馈的第二输入;所述第二输入包括所述第二终端设备。
  43. 根据权利要求40所述的装置,其特征在于,所述终端选择模块,具体用于:
    基于所述第二终端设备的设备信息为所述多个终端设备中选择最符合所述第一任务的执行要求的设备信息,使能针对于所述多个终端设备中所述第二终端设备的选择。
  44. 根据权利要求41至43任一所述的装置,其特征在于,所述第一任务相关的任务运行信息,包括:
    执行数据以及所述执行数据的索引信息;其中,所述执行数据为执行所述第一任务所需的数据,所述索引信息包括内存地址,所述内存地址为所述第二终端设备上可用的存储空间的地址,所述索引信息用于指示所述第二终端设备将所述执行数据存储至所述内存内存地址对应的内存空间中。
  45. 根据权利要求31至44任一所述的装置,其特征在于,所述第一任务为UI显示任务、图像渲染任务、逻辑状态更新任务、AI任务、数据处理或分析任务中的一种。
  46. 一种应用运行装置,其特征在于,所述装置应用于目标系统,所述目标系统包括第一终端设备和第二终端设备,所述第一终端设备与所述第二终端设备通信连接,所述第一终端设备,包括:
    获取模块,用于在运行目标应用时,获取第二终端设备的设备信息和/或软件执行环境;
    发送模块,用于基于所述第二终端设备的设备信息和/或软件执行环境满足所述目标应 用第一任务的执行要求,将所述第一任务的任务运行信息传递至所述第二终端设备,且所述第一终端设备不执行所述第一任务;
    所述第二终端设备,包括:
    第一任务执行模块,用于根据所述第一任务相关的任务运行信息执行所述第一任务。
  47. 根据权利要求46所述的装置,其特征在于,所述目标应用还包括第二任务;所述第二终端设备,包括:
    第二任务执行模块,用于在所述第二终端设备根据所述任务运行信息执行所述第一任务时,所述第一终端设备保持所述第二任务的执行。
  48. 根据权利要求47所述的装置,其特征在于,所述第一终端设备还包括:
    接收模块,用于在所述将所述第一任务相关的任务运行信息传递至所述第二终端设备之前,接收到用户的第一输入,所述第一输入指示将所述第一任务相关的任务运行信息传递至所述第二终端设备。
  49. 根据权利要求48所述的装置,其特征在于,所述第一终端设备还包括:
    呈现模块,用于所述第一终端设备呈现第一选择信息,所述第一选择信息指示所述目标应用至少一个任务,所述第二终端设备的设备信息和/或软件执行环境满足所述至少一个任务的执行要求;
    所述接收模块,具体用于:
    接收到用户根据所述第一选择信息反馈的第一输入;所述第一输入包括所述第一任务。
  50. 根据权利要求46至49任一所述的装置,其特征在于,所述发送模块,还用于在所述将第一任务相关的任务运行信息传递至所述第二终端设备之后,所述第一终端设备将所述第一任务相关的状态数据传递至所述第二终端设备;所述状态数据为所述第二终端设备执行所述第一任务时所需的状态数据。
  51. 根据权利要求46至50任一所述的装置,其特征在于,所述第一终端设备和所述第二终端设备之间通过近场通信、WIFI、蓝牙、超宽带、紫蜂ZigBee、RFID、4G、5G、LoRa、SIGFOX、Z-Wave以及NB-loT中的至少一种方式通信。
  52. 根据权利要求46至51任一所述的装置,其特征在于,所述设备信息包括:
    组件信息,所述组件信息用于指示所述第二终端设备是否部署有所述目标应用的运行组件,所述运行组件被配置为具备执行所述第一任务的能力;
    所述执行要求,包括:
    所述第二终端设备部署有所述运行组件。
  53. 根据权利要求46至52任一所述的装置,其特征在于,所述设备信息包括如下信息的至少一种:
    电源信息、处理器信息、内存信息、传输带宽、输入设备信息、输出设备信息、网络负载。
  54. 根据权利要求52或53所述的装置,其特征在于,所述软件执行环境包括:所述终端设备的设备状态和/或所述运行组件的运行状态;
    在所述设备状态指示所述第二终端设备处于下电、休眠、息屏或者锁屏的情况下,所述第一任务相关的任务运行信息包括第一指示信息,所述第一指示信息用于指示如下信息的至少一个:所述第二终端设备上电、结束休眠、结束息屏或者解除锁屏、指示所述第二终端设备运行所述运行组件、以及通过所述运行组件执行所述第一任务;或者,
    在所述设备状态指示所述第二终端设备启动、且所述运行状态指示所述第二终端设备未运行所述运行组件的情况下,所述第一任务相关的任务运行信息包括第二指示信息,所述第二指示信息用于指示如下信息的至少一个所述第二终端设备运行所述运行组件、以及通过所述运行组件执行所述第一任务;或者,
    在所述设备状态指示所述第二终端设备启动、且在所述运行状态指示所述第二终端设备运行所述运行组件的情况下,所述第一任务相关的任务运行信息包括第三指示信息,所述第三指示信息用于指示所述第二终端设备通过所述运行组件执行所述第一任务。
  55. 根据权利要求46至54任一所述的装置,其特征在于,所述第一任务相关的任务运行信息,包括:
    执行数据以及所述执行数据的索引信息;其中,所述执行数据为执行所述第一任务所需的数据,所述索引信息包括内存地址,所述内存地址为所述第二终端设备上可用的存储空间的地址,所述索引信息用于指示所述第二终端设备将所述执行数据存储至所述内存内存地址对应的内存空间中;
    所述第一任务执行模块,具体用于:
    根据所述索引信息,将所述执行数据存储至所述内存地址对应的存储空间中;
    在执行所述第一任务时,从所述内存地址对应的存储空间中获取所述执行数据。
  56. 根据权利要求46至55任一所述的装置,其特征在于,
    所述第二终端设备基于完成所述第一任务的执行、且未接收到所述第一终端设备针对于所述目标应用中其他任务的执行触发,不执行所述目标应用的其他任务。
  57. 根据权利要求46至56任一所述的装置,其特征在于,所述目标应用任务包括所述第一任务和第二任务;所述第一任务和所述第二任务为渲染任务;所述第一任务用于绘制第一应用界面,所述第二任务用于绘制第二应用界面,所述第一应用界面和所述第二应用界面为不同的界面;
    所述第一终端设备还包括:
    第二任务执行模块,用于执行所述第二任务,并触发显示设备显示所述第二应用界面;
    所述第一任务执行模块,具体用于:
    所述第二终端设备触发显示设备显示所述第一应用界面。
  58. 根据权利要求46至57任一所述的装置,其特征在于,所述第一任务为渲染任务;
    所述第一任务执行模块,具体用于:
    根据所述第一任务相关的任务运行信息,以及所述第二终端设备的显示设备的显示参数,执行所述第一任务。
  59. 一种终端设备,其特征在于,所述设备包括处理器、存储器和总线,其中:
    所述处理器、所述存储器通过所述总线连接;
    所述存储器,用于存放计算机程序或指令;
    所述处理器,用于调用或执行所述存储器上所存放的程序或指令以实现权利要求1-15、29以及30任一所述的方法步骤。
  60. 一种计算机可读存储介质,包括程序,当其在计算机上运行时,使得计算机执行如权利要求1至30中任一项所述的方法。
  61. 一种包含指令的计算机程序产品,其特征在于,当所述计算机程序产品在终端上运行时,使得所述终端执行所述权利要求1-30中任一项所述的方法。
PCT/CN2022/126849 2021-10-22 2022-10-22 一种应用运行方法以及相关设备 WO2023066395A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111236221.5A CN116010076A (zh) 2021-10-22 2021-10-22 一种应用运行方法以及相关设备
CN202111236221.5 2021-10-22

Publications (1)

Publication Number Publication Date
WO2023066395A1 true WO2023066395A1 (zh) 2023-04-27

Family

ID=85874006

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/126849 WO2023066395A1 (zh) 2021-10-22 2022-10-22 一种应用运行方法以及相关设备

Country Status (2)

Country Link
CN (2) CN115934314A (zh)
WO (1) WO2023066395A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116414541A (zh) * 2023-05-26 2023-07-11 摩尔线程智能科技(北京)有限责任公司 兼容多种任务工作模式的任务执行方法和装置
CN116739090A (zh) * 2023-05-12 2023-09-12 北京大学 基于Web浏览器的深度神经网络推理度量方法和装置

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116521260A (zh) * 2023-04-21 2023-08-01 启朔(深圳)科技有限公司 应用启动方法、应用运行方法、移动端、云端和存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101009583A (zh) * 2006-01-24 2007-08-01 华为技术有限公司 预定任务执行方法和管理任务执行方法、及其终端设备
CN102546649A (zh) * 2012-01-19 2012-07-04 华为技术有限公司 一种应用的转移方法、装置及其系统
US9002930B1 (en) * 2012-04-20 2015-04-07 Google Inc. Activity distribution between multiple devices
CN104731654A (zh) * 2015-04-03 2015-06-24 南京大学 一种针对安卓应用的迁移重构及支持系统
CN109274743A (zh) * 2018-09-27 2019-01-25 四川长虹电器股份有限公司 基于多设备协同配合的流媒体播放方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101009583A (zh) * 2006-01-24 2007-08-01 华为技术有限公司 预定任务执行方法和管理任务执行方法、及其终端设备
CN102546649A (zh) * 2012-01-19 2012-07-04 华为技术有限公司 一种应用的转移方法、装置及其系统
US9002930B1 (en) * 2012-04-20 2015-04-07 Google Inc. Activity distribution between multiple devices
CN104731654A (zh) * 2015-04-03 2015-06-24 南京大学 一种针对安卓应用的迁移重构及支持系统
CN109274743A (zh) * 2018-09-27 2019-01-25 四川长虹电器股份有限公司 基于多设备协同配合的流媒体播放方法

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116739090A (zh) * 2023-05-12 2023-09-12 北京大学 基于Web浏览器的深度神经网络推理度量方法和装置
CN116739090B (zh) * 2023-05-12 2023-11-28 北京大学 基于Web浏览器的深度神经网络推理度量方法和装置
CN116414541A (zh) * 2023-05-26 2023-07-11 摩尔线程智能科技(北京)有限责任公司 兼容多种任务工作模式的任务执行方法和装置
CN116414541B (zh) * 2023-05-26 2023-09-05 摩尔线程智能科技(北京)有限责任公司 兼容多种任务工作模式的任务执行方法和装置

Also Published As

Publication number Publication date
CN115934314A (zh) 2023-04-07
CN116010076A (zh) 2023-04-25

Similar Documents

Publication Publication Date Title
WO2021027747A1 (zh) 一种界面显示方法及设备
CN109814766B (zh) 一种应用显示方法及电子设备
WO2020259452A1 (zh) 一种移动终端的全屏显示方法及设备
WO2021013158A1 (zh) 显示方法及相关装置
WO2021052263A1 (zh) 语音助手显示方法及装置
WO2021063343A1 (zh) 语音交互方法及装置
WO2021000807A1 (zh) 一种应用程序中等待场景的处理方法和装置
WO2021000881A1 (zh) 一种分屏方法及电子设备
WO2020253758A1 (zh) 一种用户界面布局方法及电子设备
CN113254120B (zh) 数据处理方法和相关装置
WO2023066395A1 (zh) 一种应用运行方法以及相关设备
WO2020093988A1 (zh) 一种图像处理方法及电子设备
WO2021190344A1 (zh) 多屏幕显示电子设备和电子设备的多屏幕显示方法
WO2021036770A1 (zh) 一种分屏处理方法及终端设备
WO2022033320A1 (zh) 蓝牙通信方法、终端设备及计算机可读存储介质
CN113986002B (zh) 帧处理方法、装置及存储介质
WO2022037726A1 (zh) 分屏显示方法和电子设备
WO2022017393A1 (zh) 显示交互系统、显示方法及设备
WO2023030099A1 (zh) 跨设备交互的方法、装置、投屏系统及终端
WO2022179275A1 (zh) 终端应用控制的方法、终端设备及芯片系统
WO2020155875A1 (zh) 电子设备的显示方法、图形用户界面及电子设备
WO2022007707A1 (zh) 家居设备控制方法、终端设备及计算机可读存储介质
WO2021190524A1 (zh) 截屏处理的方法、图形用户接口及终端
WO2022135157A1 (zh) 页面显示的方法、装置、电子设备以及可读存储介质
WO2022152174A9 (zh) 一种投屏的方法和电子设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22882999

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2022882999

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 2022882999

Country of ref document: EP

Effective date: 20240423

NENP Non-entry into the national phase

Ref country code: DE