WO2021078161A1

WO2021078161A1 - Method for processing rendering instruction in simulator and mobile terminal

Info

Publication number: WO2021078161A1
Application number: PCT/CN2020/122528
Authority: WO
Inventors: 何书杰; 罗朴良; 李斌飞
Original assignee: 华为技术有限公司
Priority date: 2019-10-23
Filing date: 2020-10-21
Publication date: 2021-04-29
Also published as: CN110968395B; CN110968395A

Abstract

A method for processing a rendering instruction in a simulator. The method comprises: traversing a rendering instruction to be sent (301), the rendering instruction comprising a synchronous execution rendering instruction and an asynchronous execution rendering instruction; compressing the asynchronous executive rendering instruction obtained by traversing, so as to obtain a first instruction compressed packet (302); sending the first instruction compressed packet (303); and decompressing the first instruction compressed packet and executing the decompressed asynchronous executive rendering instruction (304). By means of the method above, the capability of data transmission of a simulator may be improved.

Description

Method for processing rendering instructions in simulator and mobile terminal

This application claims the priority of a Chinese patent application filed with the State Intellectual Property Office of China with an application number of 201911012928.0 on October 23, 2019. The title of the invention is "A method for processing rendering instructions in a simulator and a mobile terminal" in China The priority of the patent application, the entire content of which is incorporated in this application by reference.

Technical field

The embodiments of the present application relate to the field of electronic technology, and in particular, to a method for processing rendering instructions in an emulator and a mobile terminal.

Background technique

The Android emulator can simulate the running environment of the Android mobile phone on the personal computer, so that the user can run the application (such as games, etc.) of the Android phone on the personal computer. Compared with running Android applications on Android phones, running Android applications through the Android emulator has the following advantages: The screen of a personal computer is usually larger than the screen of a mobile phone, and a large screen can provide a better visual experience; there is no power consumption or network Concerns about traffic; using the keyboard and mouse of a personal computer, the control performance is smoother; multiple Android applications can be run at the same time; the computing performance of a personal computer is usually stronger than that of a mobile phone, which can break through the limitations of mobile phone performance (such as CPU Central Processing Unit) wait wait wait. During the operation of the simulator, the bandwidth of the transmission pipeline affects the speed of the rendering instruction transmission. If the pipeline bandwidth is small, the rendering instruction transmission may be slow, and the screen may drop frames and freezes, which will affect the user experience. Therefore, there is an urgent need to improve the transmission capability of the simulator.

Summary of the invention

The embodiment of the present application provides a method for processing rendering instructions in the simulator, which can improve the transmission capacity of the transmission pipeline of the simulator and avoid phenomena such as frame drop and freeze on the screen.

To achieve the foregoing objectives, the following technical solutions are adopted in the embodiments of the present application:

In the first aspect, an embodiment of the present application provides a method for processing rendering instructions in a simulator, including: traversing the rendering instructions to be sent, and the rendering instructions to be sent include synchronously executing rendering instructions and asynchronously executing rendering instructions; The obtained asynchronous execution rendering instruction is compressed to obtain a first instruction compressed package; the first instruction compressed package is sent; the first instruction compressed package is decompressed, and the decompressed asynchronous execution rendering instruction is executed. By compressing asynchronously executed rendering instructions, the transmission efficiency is improved.

In a possible implementation manner, the traversing the rendering instructions to be sent includes: detecting the rendering instructions to be sent one by one according to the order in which they are to be sent.

In a possible implementation manner, the asynchronously executed rendering instruction obtained by executing the decompression includes: sending the asynchronously executed rendering instruction to a GPU, and the GPU generates a rendering image.

In a possible implementation manner, the method further includes: sending the synchronous execution rendering instruction obtained by the traversal.

In a possible implementation manner, before the traversal of the rendering instructions to be sent, the method further includes: detecting the type of the rendering instructions and marking them.

In a possible implementation manner, the detection of the type of rendering instruction includes: for a rendering instruction with a return value of void, marking it as an asynchronous execution rendering instruction; for a rendering instruction with a return value of non-void, marking it as a synchronous execution rendering instruction.

In a possible implementation manner, the detecting the type of the rendering instruction includes: using an instruction function symbol table to detect the type of the rendering instruction.

In a possible implementation manner, before the traversing the rendering instructions to be sent, the method further includes: detecting a first bandwidth required by an application currently running in the simulator; and determining whether the first bandwidth is Exceeds the maximum bandwidth capability supported by the simulator.

In a possible implementation manner, the traversing the rendering instruction to be sent includes: if the traversal obtains the synchronous execution rendering instruction, then suspend the traversal and send the synchronous execution rendering instruction; if the traversal obtains the asynchronous execution rendering instruction, continue Traverse until it encounters a synchronous execution instruction and pause the traversal.

In a second aspect, an embodiment of the present application provides a mobile terminal, which includes one or more processors, a memory, and one or more programs; wherein, one or more programs are stored in the memory and configured to be One or more processors are executed, and one or more programs include instructions for executing the method according to the first aspect.

In a third aspect, an embodiment of the present application provides a storage medium for storing computer software instructions, and the computer software instructions are used to execute the method described in the first aspect.

In a fourth aspect, embodiments of the present application provide a computer program product, which when the computer program product is run, causes a computer to execute the method described in the first aspect.

Through the above scheme, the transmission capacity of the simulator can be improved, and the user experience can be improved.

Description of the drawings

FIG. 1 is a schematic diagram of a simulator transmission instruction provided by an embodiment of the application;

FIG. 2 is a schematic structural diagram of an operating system provided by an embodiment of the application;

3 is a flowchart of a method for processing rendering instructions in a simulator provided by an embodiment of the application;

FIG. 4 is a schematic diagram of another simulator transmission instruction provided by an embodiment of the application;

FIG. 5 is a flowchart of another method for processing rendering instructions in a simulator according to an embodiment of the application;

6 is a flowchart of another method for processing rendering instructions in a simulator provided by an embodiment of the application;

FIG. 7 is a schematic diagram of another simulator transmission instruction provided by an embodiment of the application.

Detailed ways

The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application. Among them, in the description of the embodiments of the present application, unless otherwise specified, "/" means or, for example, A/B can mean A or B; "and/or" in this document is only a description of related objects The association relationship of indicates that there can be three kinds of relationships, for example, A and/or B, which can indicate: A alone exists, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "plurality" refers to two or more than two.

The mobile terminals described in this application may include mobile phones, tablet computers, wearable devices (for example, watches, bracelets, smart helmets, etc.), personal computers (PC), vehicle-mounted devices, smart home devices, and augmented reality ( augmented reality (AR)/virtual reality (VR) equipment, laptops, ultra-mobile personal computers (UMPC), netbooks, personal digital assistants (PDAs), etc. Some embodiments of the present application take a personal computer as an example. It should be understood that these embodiments can also be applied to other types of mobile terminals.

The application program (application, app) involved in this application may be referred to as an application, which is a software program that can realize one or more specific functions. Generally, multiple applications can be installed in a mobile terminal, for example, sports applications, music applications, dictionary applications, and so on. It is understandable that the application may be an application that has been installed when the mobile terminal is shipped from the factory, or an application that the user downloads from the network or obtains from other electronic devices in the process of using the mobile terminal. Third-party applications usually refer to applications developed by non-mobile terminal manufacturers. For example, for Huawei

For mobile phones, third-party applications can be applications developed by developers other than Huawei.

The simulator can simulate the system or operating environment of the B terminal on the A terminal, and run the application on the B terminal on the A terminal. A common one, such as an Android emulator, can simulate the operating environment of the Android mobile phone on the personal computer, so that the user can run the application (such as games, etc.) of the Android phone on the personal computer. To install Microsoft’s

Take the personal computer of the system as an example. Android emulator software can be installed on the personal computer; the user can run Android applications (such as apk files) in the Android emulator software, and these Android applications can be downloaded by the user through the Android emulator software , It can also be downloaded by users through channels other than Android emulator software (such as browsers, etc.). Compared with running Android applications on Android phones, running Android applications through the Android emulator has the following advantages: The screen of a personal computer is usually larger than the screen of a mobile phone, and a large screen can provide a better visual experience; there is no power consumption or network Traffic concerns; using a personal computer’s keyboard and mouse, the control performance is smoother; multiple Android applications can be run at the same time; the computing performance of a personal computer is usually stronger than that of a mobile phone, which can break through the limitations of mobile phone performance (such as CPU Central Processing Unit Central Processing Unit) wait wait wait.

At present, when using the emulator to play games, there will be frame dropping, resulting in unsmooth images and poor user experience. As shown in Figure 1, the user installs the simulator client (guest) on the personal computer (host), the simulator client transmits instructions and other data to the personal computer through a transmission pipe (pipe), and the personal computer uses its own The GPU (Graphic Processing Unit) processes these instructions (for example, renders the rendering instructions). Rendering refers to the process of generating images from the model in the computer drawing process, which is generally performed by the graphics card GPU. For example, when an Android game application is run on the Android emulator software, the Android emulator software sends screen rendering instructions to the personal computing operating system, and the personal computer processes these rendering instructions through its own GPU and finally generates the game screen. Common rendering modes include DirectX engine rendering and OpenGL engine rendering. Due to the strong compatibility of OpenGL engine, this application uses OpenGL rendering mode as an example to introduce the rendering process of the simulator. It is understandable that these examples can also be applied to DirectX engine rendering. . OpenGL (Open Graphics Library) is a cross-language and cross-platform application programming interface (API) for rendering 2D and 3D vector graphics.

This application takes the Android emulator as an example, and Figure 2 shows a schematic diagram of the architecture of the Android Android system. The Android operating system architecture is divided into four layers, from high-level to bottom-level divided into application layer, application framework layer, function library layer and Linux kernel layer.

1. Application layer:

The application layer (Applications) is the top layer of the Android operating system architecture, including the core application software assembled by the Android operating system, such as email clients, text messages, calls, calendars, maps, browsers, contacts, etc. Of course, for developers, developers can write application software and install it to this layer. Generally speaking, application programs are developed in Java language, which is completed by calling API (Application Programming Interface) provided by the application framework layer.

2. Application framework layer:

The application framework layer (Application Framework) mainly provides developers with access to various APIs used by the application. Developers use the application framework layer to communicate with the underlying Android (such as the function library layer, the Linux kernel layer, etc.) Interactive and develop your own applications. The application framework layer is mainly a series of services and management systems of the Android operating system. The application framework layer mainly includes the following key services:

Activity Manager (Activity Manager) is used to manage the application life cycle and provide common navigation rollback functions;

Content Providers are used to manage data sharing and access between different applications;

The Notification Manager is used to control the application to display prompt information (such as Alerts, Notifications, etc.) to the user in the status bar;

Resource Manager: Provides non-code resources (such as strings, graphics, and layout files, etc.) for use by application programs;

Package Manager (Package Manager) is mainly used to manage the applications of the Android operating system;

View, with a rich and extensible collection of views, can be used to build an application, which specifically includes a list (List), a grid (Grid), a text box (TextBox), a button (Button), and embeddable Web browser;

The location manager (Location Manager) mainly allows applications to access the current geographic location of the mobile phone.

3. Function library layer:

The library layer (Libraries) is the support of the application framework and an important link between the application framework layer and the Linux kernel layer. The function library layer includes some function libraries compiled by the computer program C language or C++ language, these function libraries can be used by different components in the Android operating system, and they provide services for developers through the Android application framework. Specifically, the function library includes the libc function library, which is specially customized for embedded Linux-based devices; the function library also includes the Android operating system multimedia library (Media Framework), which supports the playback and playback of audio/video in multiple encoding formats. Recording, while supporting still image files, as well as common audio/video encoding formats. The function library also includes the interface management library (Surface Manager), which is mainly responsible for managing access to the display system. It is specifically used to manage the interaction between display and access operations when multiple applications are executed. It is also responsible for 2D drawing and 3D. The drawing is displayed and synthesized.

The function library layer also includes other function libraries for implementing various functions of Android operating system phones, such as: SGL (Scalable Graphics Library): 2D graphics and image processing engine based on XML (Extensible Markup Language) files; SSL (Secure Sockets Layer) ): Located between TVP/IP protocol (TransmissionControlProtocol/InternetProtocol, Transmission Control Protocol/Internet Protocol) and various application layer protocols, providing support for data communication; OpenGL/ES: 3D effect support; SQLite: relational database engine ; Webkit: Web browser engine; FreeType: Bitmap and Vector font support; etc.

Android Runtime is a running environment on the Android operating system, and a new virtual machine used by the Android operating system. In Android Runtime, using AOT (Ahead-Of-Time) technology, when an application is installed for the first time, the bytecode of the application will be pre-compiled into machine code, making the program a real local application. After running it again, the compilation step is omitted, and the startup and execution will become faster.

In some other examples of this application, Android Runtime can also be replaced by Core Libraries and Dalvik Virtual Machine. The core function library provides most of the functions in the Java language API (Application Programming Interface), and provides an interface to the application framework layer to call the underlying library through the JNI (Java Native Interface) method. It also contains some core APIs of Android, such as android.os, android.net, android.media and so on. The Dalvik virtual machine uses a JIT (Just-in-Time) runtime compilation mechanism. Every time a process is started, the virtual machine needs to recompile the bytecode in the background, which will have a certain impact on the startup speed. Every Android application runs in an instance in a Dalvik virtual machine, and each Dalvik virtual machine instance is an independent process space. The Dalvik virtual machine is designed to efficiently run multiple virtual machines on one device. The executable file format of the Dalvik virtual machine is .dex. The dex format is a compression format designed for Dalvik, suitable for systems with limited memory and processor speed. What needs to be mentioned is that the Dalvik virtual machine relies on the Linux kernel to provide basic functions (threads, low-level memory management). It is understandable that Android Runtime and Dalvik belong to different types of virtual machines, and those skilled in the art can choose different forms of virtual machines in different situations.

4. Linux kernel layer:

Android's core system services, such as security, memory management, process management, network protocol stack, and driver model, are all based on the Linux kernel. The Linux kernel also serves as an abstraction layer between the hardware and software stacks. This layer has many drivers related to mobile devices. The main drivers are: Display Driver: Linux-based Frame Buffer driver. Keyboard Driver (KeyBoard Driver): As the keyboard driver of the input device. Flash driver (Flash Memory Driver): Flash driver based on MTD (memory technology device). Camera Driver: A commonly used Linux-based v4l2 (Video for Linux) driver. Audio Driver: A commonly used advanced Linux sound system driver based on ALSA (Advanced Linux Sound Architecture). Bluetooth Driver: Wireless transmission technology based on IEEE 802.15.1 standard. WiFi Drive: A driver based on the IEEE 802.11 standard. Binder (IPC) driver: A special driver for Android that has a separate device node and provides the function of inter-process communication. Power Management: For example, battery power, etc.

During the operation of the simulator, the bandwidth of the transmission pipeline affects the speed of the rendering instruction transmission. If the pipeline bandwidth is small, it may cause the rendering instruction transmission to be slow, resulting in low GPU utilization of the personal computer (also known as GPU starvation) , Making the simulator performance degraded. At present, the maximum bandwidth capacity supported by the emulators on the market is about 1GB/s-2GB/s, which can render ordinary 1080P game screens (the bandwidth of 1080P screens is about 500MB/s); but with more and more high-definition game screens With the advent of, the rendering of simulators is becoming more and more inadequate. For example, the bandwidth required for 4K game screens is about 2GB/s, which has exceeded the maximum bandwidth capacity that ordinary simulators can support, so there will be screen drop frames, freezes, etc. Phenomenon that affects user experience. Therefore, there is an urgent need to improve the transmission capability of the simulator.

In view of the above problems, this application proposes a solution that can compress rendering instructions. By compressing rendering instructions, the amount of transmission data of rendering instructions can be reduced, and it can be reduced to the bandwidth capability supported by the simulator's transmission pipeline. In this way, rendering instructions can be transmitted smoothly and in time, avoiding frame dropping and freezing, etc., improving rendering fluency, and improving user experience.

However, not all rendering instructions support compression. Rendering instructions can be divided into synchronously executing rendering instructions and asynchronously executing rendering instructions. Synchronous execution of rendering instructions needs to be executed synchronously and waiting for the return value, and asynchronous execution of rendering instructions needs to be executed asynchronously and does not need to wait for the return value. To execute an instruction synchronously, it is necessary to wait for the execution of this instruction to complete and obtain the return value before executing other instructions, while asynchronous execution can be executed concurrently without waiting for the return value. For multiple synchronously executed rendering instructions, it is necessary to wait for one instruction to be executed and get the return value result before the next instruction can be executed, so multiple synchronously executed rendering instructions cannot be compressed and sent together.

Figure 3 provides a method for processing rendering instructions in the simulator, which includes the following steps:

301: Traverse rendering instructions to be sent, where the rendering instructions to be sent include synchronously executing rendering instructions and asynchronously executing rendering instructions.

During the running of the emulator client, for example, when the game in the emulator is opened to start running, it needs to send a rendering instruction to the mobile terminal for processing. Before sending the rendering instructions, the simulator client can traverse the rendering instructions to be sent, and the rendering instructions to be sent may include synchronously executing rendering instructions and asynchronously executing rendering instructions. The traversal can be detected one by one according to the sequence of the rendering instructions to be sent.

302: Compress the asynchronously executed rendering instructions obtained by the traversal to obtain the first instruction compression package.

For the asynchronously executed rendering instructions obtained by the traversal (also referred to as the first asynchronously executed rendering instruction set), the simulator client may compress them to reduce the bandwidth required for transmission. Asynchronous execution of the rendering instruction does not need to be executed synchronously, nor does it need to wait for the return value, so it can be compressed to obtain the first instruction compressed package. For the synchronous execution of rendering instructions, it may be transmitted without compression.

303: Send the first command compressed package.

The emulator client can send rendering instructions to the mobile terminal for processing. Since step 301 compresses the asynchronously executed rendering instructions, the bandwidth requirement of the transmission pipeline can be met. The mobile terminal introduces the first command compressed package through the transmission pipe.

304: Decompress the first instruction compression package, and execute the asynchronous execution rendering instruction obtained by the decompression.

The mobile terminal decompresses the received first instruction compression package, and obtains the asynchronous execution rendering instruction that needs to be rendered. These rendering instructions can be sent to the GPU for rendering, and corresponding rendering images are generated.

As shown in Figure 4, for example, using an Android emulator to run a game application, the emulator client can detect three rendering instructions to be sent-A rendering instruction, B rendering instruction and C rendering instruction, where A rendering instruction, B The rendering instruction is to execute the rendering instruction asynchronously, and the C rendering instruction is to execute the rendering instruction synchronously.

The simulator client traverses the above three rendering instructions to be sent, and obtains the two asynchronous rendering instructions of A rendering instruction and B rendering instruction in turn. The simulator client can temporarily save (such as stored in the system cache), and not Send it to the host immediately, but compress the two asynchronously executed rendering instructions to obtain the first instruction compression package. Then, the simulator client sends the first instruction compressed package to the host for decompression, and the host obtains the decompressed A rendering instruction and B rendering instruction. These two rendering instructions can be sent to the GPU for execution to obtain a rendered image. For the C rendering instruction, because the rendering instruction is executed synchronously, it needs to be executed synchronously and the return value is obtained, so it cannot be compressed into the first instruction compression package, but is sent to the host separately for execution by the GPU.

Optionally, for the rendering instructions traversed by the simulator, the type of the rendering instruction can be detected and marked in advance. For example, for a rendering instruction with a return value of void, it can be marked as an asynchronous execution rendering instruction; for a rendering with a return value of non-void Instructions can be marked as synchronous execution of rendering instructions. After marking, classify and store the asynchronous execution rendering instructions (compressible) and synchronous execution rendering instructions (uncompressible). When the application is run next time, the stored marked rendering instruction classification can be directly called without re-checking, thereby saving running time. Optionally, unless the simulator is reinstalled or the user deletes the application data, the classification needs to be rechecked.

Optionally, the instruction function symbol table can also be used to detect and mark the type of rendering instruction in advance. The instruction function symbol table is a preset symbol table, and you can know whether a certain instruction executes the rendering instruction synchronously or asynchronously by looking up the table. For example, for the following rendering instruction: syn-h-wb-4b-whnc-10y-whenrub-return(1), looking up the instruction function symbol table shows that this is a synchronous execution instruction.

Of course, since compressing rendering instructions also consumes time and CPU resources, there is also a method for the simulator to process rendering instructions, as shown in Figure 5, which includes the following steps:

501: Detect the first bandwidth required by the application currently running in the simulator;

The currently running application in the simulator can be an already started application or an application that the user will start. For different applications (such as ordinary games and high-definition games), the required bandwidth is different. Optionally, since some applications can select different display modes (such as normal and high-definition) after starting, the required bandwidth can be detected after the display mode is detected.

502: Determine whether the first bandwidth exceeds the maximum bandwidth capability supported by the simulator.

Different simulators have different maximum bandwidth capabilities. If the first bandwidth does not exceed the maximum bandwidth capability supported by the simulator, there is no need to compress rendering instructions.

503: If the first bandwidth exceeds the maximum bandwidth capability supported by the simulator, traverse the rendering instructions to be sent.

If the first bandwidth exceeds the maximum bandwidth capability supported by the emulator, the rendering instructions need to be compressed to meet the bandwidth requirements of the transmission pipeline, otherwise the picture freezes and frame drops may occur.

504: Compress the asynchronously executed rendering instructions obtained by the traversal to obtain a first instruction compression package.

505: Send the first command compressed package.

506: Decompress the first instruction compression package, and execute the asynchronous execution rendering instruction obtained by the decompression.

With the method shown in Figure 5, the maximum bandwidth capability supported by the simulator can be judged in advance. If the maximum bandwidth capability supports the current application, there is no need to compress rendering instructions.

In the actual running process, the simulator may have to send a lot of rendering instructions, which include both synchronous execution of rendering instructions and asynchronous execution of rendering instructions. Figure 6 provides another method of processing rendering instructions in the simulator, including the following steps:

601: Traverse rendering instructions to be sent, where the rendering instructions to be sent include synchronously executing rendering instructions and asynchronously executing rendering instructions.

602: If the traversal obtains the synchronous execution rendering instruction, pause the traversal and execute step 606; if the traverse obtains the asynchronous execution rendering instruction, continue the traversal, and pause the traversal when the synchronous execution instruction is encountered.

Since only the asynchronously executed rendering instructions can be compressed, as long as the traversal obtains the synchronously executed rendering instructions, the traversal needs to be suspended, and the synchronously executed rendering instructions are sent first (step 606). If the traversal result is an asynchronous execution rendering instruction, it will not be sent temporarily. Instead, it will continue to traverse the remaining asynchronous execution rendering instructions until the next synchronous execution rendering instruction is encountered. Then, the asynchronous execution rendering instructions obtained can be compressed. Send, and then send the synchronous execution rendering command.

603: Compress the asynchronously executed rendering instruction obtained by traversing in step 602 to obtain a first instruction compression package.

604: Send the first command compressed package.

605: Decompress the first command compression package, and execute the asynchronous execution rendering command obtained by the decompression.

606: Send the synchronous execution rendering instruction obtained by traversal in step 602, and execute the synchronous execution rendering instruction.

607: Continue to traverse the rendering instructions to be sent, and repeat steps 602-606.

Through the method of FIG. 6, it is possible to deal with a scene where there are multiple rendering instructions, and to improve transmission efficiency.

As shown in Figure 7, for example, using the Android emulator to run a game application, the emulator client can detect the 4 rendering instructions to be sent-A rendering instruction, B rendering instruction, C rendering instruction and D rendering instruction, where A Rendering instructions, B rendering instructions, and D rendering instructions are rendering instructions that are executed asynchronously, and C rendering instructions are rendering instructions that are executed synchronously.

The emulator client traverses the above 4 rendering instructions to be sent, and obtains two asynchronous rendering instructions A rendering instruction and B rendering instruction in turn. The emulator client can temporarily save it instead of sending it to the host immediately. Continue the traversal, and then encounter the synchronous execution rendering instruction-the C rendering instruction, then pause the traversal, compress the obtained two asynchronous execution rendering instructions, and obtain the first instruction compression package. Then, the simulator client sends the first instruction compressed package to the host for decompression, and the host obtains the decompressed A rendering instruction and B rendering instruction. These two rendering instructions can be sent to the GPU for execution to obtain a rendered image. For the C rendering instruction, because the rendering instruction is executed synchronously, it needs to be executed synchronously and the return value is obtained, so it cannot be compressed into the first instruction compression package, but is sent to the host separately for execution by the GPU. Then continue to traverse the rest of the rendering instructions, such as the D rendering instruction. Because the rendering instructions are executed asynchronously, the traversal continues until the next synchronous execution rendering instruction is encountered. . . Repeat the above steps.

Here are three specific scenarios:

The first scene is to use the emulator to play high-definition games (for example, the screen reaches 4K resolution or higher). For this scenario, after the user opens the high-definition game (or selects the high-definition mode in the game settings), the emulator guest can receive the message of the game start event. The guest can traverse the rendering instructions currently to be sent, and divide the current rendering instructions into synchronous execution rendering instructions and asynchronous execution rendering instructions according to the function symbol table. For asynchronously executed rendering instructions, it may not be sent temporarily, but compressed together with other asynchronously executed rendering instructions that have been traversed, and then sent to the host for rendering processing. The traversed synchronous execution rendering instructions can be sent separately.

The second scenario is to use the simulator to watch high-definition video (for example, the picture reaches a resolution of 4K or more). After the user opens the high-definition video (or selects the high-definition mode in the video playback settings), the emulator guest can receive the message of the video playback event. The guest can traverse the rendering instructions currently to be sent, and divide the current rendering instructions into synchronous execution rendering instructions and asynchronous execution rendering instructions according to the function symbol table. For asynchronously executed rendering instructions, it may not be sent temporarily, but compressed together with other asynchronously executed rendering instructions that have been traversed, and then sent to the host for rendering processing. The traversed synchronous execution rendering commands can be sent separately. Optionally, before starting to play, the bandwidth required by the video can be compared with the maximum bandwidth capability supported by the simulator. If the maximum bandwidth capability supported by the simulator is exceeded, the asynchronous execution rendering instructions obtained by the traversal need to be compressed.

The third scene is the office scene using the simulator. After the user opens the office software, the simulator guest can receive the message of the start of the office event. The guest can traverse the rendering instructions currently to be sent, and divide the current rendering instructions into synchronous execution rendering instructions and asynchronous execution rendering instructions according to the function symbol table. For asynchronously executed rendering instructions, it may not be sent temporarily, but compressed together with other asynchronously executed rendering instructions that have been traversed, and then sent to the host for rendering processing. The traversed synchronous execution rendering instructions can be sent separately. Optionally, before starting the office, you can compare the bandwidth required by the office software with the maximum bandwidth capability supported by the simulator. If the maximum bandwidth capability supported by the simulator is exceeded, the asynchronous execution rendering commands obtained from the traversal need to be compressed. .

This application also provides a mobile terminal. The mobile terminal includes one or more processors, a memory, and one or more programs; wherein, one or more programs are stored in the memory and configured to be configured by one or more The processor executes, and the one or more programs include instructions, and the instructions are used to execute the methods described in FIGS. 3-7.

This application also provides a storage medium for storing computer software instructions, which are used to execute the methods described in FIGS. 3-7.

The present application also provides a computer program product, which, when the computer program product is run, causes the computer to execute the method described in FIGS. 3-7.

Through the description of the above embodiments, those skilled in the art can understand that for the convenience and conciseness of the description, only the division of the above-mentioned functional modules is used as an example for illustration. In practical applications, the above-mentioned functions can be assigned to different functions as required. The function module is completed, that is, the internal structure of the device is divided into different function modules to complete all or part of the functions described above.

In the several embodiments provided in this application, it should be understood that the disclosed device and method can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods, for example, multiple units or components may be divided. It can be combined or integrated into another device, or some features can be omitted or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.

The units described as separate parts may or may not be physically separate. The parts displayed as units may be one physical unit or multiple physical units, that is, they may be located in one place, or they may be distributed to multiple different places. . Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.

In addition, the functional units in the various embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or software functional unit.

If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present application are essentially or the part that contributes to the prior art, or all or part of the technical solutions can be embodied in the form of a software product, and the software product is stored in a storage medium. It includes several instructions to make a device (may be a single-chip microcomputer, a chip, etc.) or a processor (processor) execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage media include: U disk, mobile hard disk, read only memory (read only memory, ROM), random access memory (random access memory, RAM), magnetic disk or optical disk and other media that can store program codes.

The above content is only the specific implementation of this application, but the protection scope of this application is not limited to this. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in this application. Covered in the scope of protection of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

A method for processing rendering instructions in a simulator, characterized in that the method includes:

Traverse the rendering instructions to be sent, where the rendering instructions to be sent include synchronously executing rendering instructions and asynchronously executing rendering instructions;

Compressing the asynchronously executed rendering instructions obtained by the traversal to obtain a first instruction compression package;

Sending the first instruction compressed package;

Decompress the first instruction compression package, and execute the asynchronous execution rendering instruction obtained by the decompression.
The method according to claim 1, wherein the traversing the rendering instructions to be sent comprises:

The rendering instructions to be sent are detected one by one according to the order to be sent.
The method according to claim 1, wherein the asynchronously executing rendering instruction obtained by executing the decompression comprises:

The asynchronous execution rendering instruction is sent to the GPU, and the GPU generates a rendering image.
The method according to claims 1-3, wherein the method further comprises:

The synchronous execution rendering instruction obtained by the traversal is sent.
The method according to claims 1-4, characterized in that, before the traversing the rendering instructions to be sent, the method further comprises:

Detect and mark the type of rendering instruction.
The method according to claim 5, wherein said detecting the type of rendering instruction comprises:

Rendering instructions with a return value of void are marked as asynchronously executed rendering instructions; for rendering instructions with a return value of non-void, they are marked as synchronously executed rendering instructions.
The method according to claim 5, wherein said detecting the type of rendering instruction comprises:

The instruction function symbol table is used to detect the type of the rendering instruction.
7. The method according to claims 1-7, characterized in that, before the traversing the rendering instructions to be sent, the method further comprises:

Detecting the first bandwidth required by the application currently running in the simulator;

It is determined whether the first bandwidth exceeds the maximum bandwidth capability supported by the simulator.
8. The method according to claims 1-8, wherein the traversing the rendering instructions to be sent comprises:

If the traversal obtains the synchronous execution rendering instruction, pause the traversal and send the synchronous execution rendering instruction;

If the traversal obtains the asynchronous execution rendering instruction, the traversal continues until the traversal is suspended when the synchronous execution instruction is encountered.
A mobile terminal includes one or more processors, a memory, and one or more programs; wherein the one or more programs are stored in the memory and configured to be used by the one or more Executed by multiple processors, the one or more programs include instructions for executing the method according to claims 1-9.
A storage medium for storing computer software instructions, and the computer software instructions are used to execute the method according to claims 1-9.
A computer program product, when running the computer program product, causes a computer to execute the method according to claims 1-9.