WO2018040428A1 - 屏幕界面显示方法和系统 - Google Patents

屏幕界面显示方法和系统 Download PDF

Info

Publication number
WO2018040428A1
WO2018040428A1 PCT/CN2016/113188 CN2016113188W WO2018040428A1 WO 2018040428 A1 WO2018040428 A1 WO 2018040428A1 CN 2016113188 W CN2016113188 W CN 2016113188W WO 2018040428 A1 WO2018040428 A1 WO 2018040428A1
Authority
WO
WIPO (PCT)
Prior art keywords
rendering
screen interface
screen
interface data
current terminal
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2016/113188
Other languages
English (en)
French (fr)
Inventor
扶宇琳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangzhou Shirui Electronics Co Ltd
Original Assignee
Guangzhou Shirui Electronics Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangzhou Shirui Electronics Co Ltd filed Critical Guangzhou Shirui Electronics Co Ltd
Publication of WO2018040428A1 publication Critical patent/WO2018040428A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance

Definitions

  • the present invention relates to the field of display control application technologies, and in particular, to a screen interface display method and system.
  • a host computer (a computer that can directly issue commands) can transmit its screen interface to a terminal device for synchronous display.
  • the terminal device receives the screen interface data of the upper computer, the screen interface data of the upper computer is rendered.
  • the resource occupancy rate of the terminal device processor is generally high during rendering. The performance of the terminal device is limited.
  • the processor idle resources of the terminal device may be insufficient. The quality of the application or service of the terminal device cannot be guaranteed.
  • a screen interface display method includes the following steps:
  • the screen interface data is rendered according to the first rendering manner, and the first rendered image is obtained, and the first rendered image is displayed on the screen of the current terminal;
  • the screen interface data is rendered according to the second rendering manner, and the second rendered image is obtained, and the second rendered image is displayed on the screen of the current terminal, wherein the resource usage of the first rendering mode is lower than the second rendering The resource occupancy rate of the method.
  • a screen interface display system comprising the following units:
  • An obtaining unit configured to acquire screen interface data
  • a determining unit configured to determine, in real time, whether a resource occupancy rate of the current terminal processor is higher than a first preset value
  • a first rendering unit configured to: when the determining unit determines that the resource occupancy rate of the current terminal processor is higher than the first preset value, rendering the screen interface data according to the first rendering manner, obtaining the first rendered image, and performing the first rendering The image is displayed on the screen of the current terminal;
  • a second rendering unit configured to: when the determining unit determines that the resource occupancy rate of the current terminal processor is higher than the first preset value, rendering the screen interface data according to the second rendering manner, obtaining the second rendered image, and performing the second rendering The image is displayed on the screen of the current terminal, where the resource usage of the first rendering mode is lower than the resource usage of the second rendering mode.
  • the rendering mode is selected according to whether the resource occupancy rate of the current terminal processor is higher than the first preset value, and the acquired screen interface data is rendered and displayed.
  • the second rendering mode has a high resource occupancy rate for the processor.
  • the second rendering mode can well render and display the screen interface image;
  • the first rendering mode is for the processor resource.
  • the occupancy rate is lower than the resource usage of the processor in the second rendering mode.
  • the remote application runs before rendering, the resource usage of the current terminal processor is high, and the first rendering mode can leave the processor with idle resources.
  • the invention can adjust the occupation of the processor resources, and ensure the running quality of the remote application while rendering.
  • FIG. 1 is a schematic flow chart of a screen interface display method of one embodiment
  • FIG. 2 is a schematic diagram of practical application of a screen interface display method of one embodiment
  • FIG. 3 is a schematic structural diagram of a screen interface display system of one embodiment
  • FIG. 4 is a schematic structural diagram of a screen interface display system of one embodiment
  • FIG. 5 is a schematic structural diagram of a screen interface display system of one embodiment.
  • FIG. 1 is a schematic flowchart diagram of a screen interface display method according to an embodiment of the present invention.
  • the screen interface display method in this embodiment includes the following steps:
  • Step S101 Acquire screen interface data
  • Step S102 determining, in real time, whether the resource occupancy rate of the current terminal processor is higher than a first preset value
  • Step S103 If yes, the screen interface data is rendered according to the first rendering manner, and the first rendered image is obtained, and the first rendered image is displayed on the screen of the current terminal;
  • Step S104 If no, the screen interface data is rendered according to the second rendering manner, and the second rendered image is obtained, and the second rendered image is displayed on the screen of the current terminal, where the resource usage rate of the first rendering mode is lower than Resource usage of the second rendering mode.
  • the rendering mode is selected according to whether the resource occupancy rate of the current terminal processor is higher than the first preset value, and the acquired screen interface data is rendered and displayed.
  • the second rendering mode has a high resource occupancy rate for the processor. When the current resource consumption of the terminal processor is low, the second rendering mode can well render and display the screen interface image; the first rendering mode is for the processor resource. The occupancy rate is lower than the resource usage of the processor in the second rendering mode.
  • the remote application runs before rendering, the resource usage of the current terminal processor is high, and the first rendering mode can leave the processor with idle resources. To ensure the quality of the remote application.
  • the invention can adjust the occupation of the processor resources, and ensure the running quality of the remote application while rendering.
  • the first preset value can be set according to the actual situation, such as 80%.
  • the screen interface data is rendered according to the first rendering manner to obtain a first rendered image
  • the step of displaying the first rendered image on the screen of the current terminal comprises the following steps:
  • the screen interface data is rendered according to the first rendering manner, and the first rendered image is obtained, and the first rendered image is displayed on the screen of the current terminal;
  • the screen interface data is rendered according to the second rendering manner to obtain a second rendered image, and the second rendered image is displayed on the screen of the current terminal.
  • the resource occupancy rate required for rendering the screen interface data in the second rendering manner may be calculated, and the Whether the sum of the resource usage ratios of the current terminal processor is higher than the second preset value. If yes, it indicates that the second rendering mode is used to render the screen interface data, which may result in insufficient idle resources of the current terminal processor.
  • a rendering mode renders and displays the screen interface data; if not, it indicates that the second rendering mode can still leave the current terminal processor with idle resources, and the second rendering mode can be used to render and display the screen interface data. .
  • the second preset value can be set according to the actual situation, such as 90%.
  • the screen interface data is screen interface change data or screen interface unchanged data.
  • the screen interface data may be screen interface change data or screen interface unchanged data.
  • the screen interface change data and the screen interface unchanged data may be separately rendered, and the screen interface change data and the screen interface unchanged data may be separately rendered, so that the screen interface may be changed according to the screen interface.
  • the actual situation of data and screen interface without changing data is flexible and the corresponding rendering mode is selected.
  • the first rendering mode is a SurfaceView rendering mode
  • the second rendering mode is an OpenGL rendering mode.
  • the step of rendering the screen interface data according to the first rendering manner includes the following steps:
  • the step of rendering the screen interface data according to the second rendering manner includes the following steps:
  • the first rendering mode is a SurfaceView rendering mode.
  • the view used for rendering is first converted into a SurfaceView, and then the screen interface data is transmitted to the SurfaceView for rendering, thereby implementing the SurfaceView.
  • the second rendering mode is OpenGL rendering mode.
  • When rendering first convert the view used for rendering to GLSurfaceView (graphics library surface view), and then pass the screen interface data to GLSurfaceView for rendering, which can realize OpenGL (open Graphics library) rendering.
  • the OpenGL rendering method has high resource utilization rate, low latency, and stable frame rate, which can render and display the screen interface image well.
  • the SurfaceView rendering mode has lower resource utilization rate than the OpenGL rendering mode, although SurfaceView The rendering mode is not stable, but its latency is completely negligible for remote applications, so the quality of the remote application can be guaranteed.
  • the step of determining in real time whether the resource occupancy rate of the current terminal processor is higher than the first preset value comprises the following steps:
  • the resource occupancy rate of the terminal processor when the remote application is running, the resource occupancy rate of the terminal processor is high, and the resource occupancy rate of the terminal processor when the remote application is running may be used as a reference value of the first preset value, that is, When the current terminal runs the remote application, the resource occupancy rate of the current terminal processor is higher than the first preset value. Moreover, this ensures the quality of the remote application's operation.
  • Remote applications can include applications such as remote voice, remote video, and remote desktop sharing.
  • the step of determining in real time whether the current terminal has a remote application running includes the following steps:
  • the system property it is determined in real time whether the current terminal has a remote application running, and the running state of the remote application is saved in the system attribute.
  • the running state of the remote application is saved in the system attribute, and when it is necessary to know whether the current terminal is running, the system attribute can be judged by the system attribute, and the remote application is not required to be associated. Relationships, which reduce the resource usage of the processor.
  • the remote application writes the operational status of the remote application to system properties in real time.
  • the running state of the remote application is written by the remote application, and the running state of the remote application and the non-running state are written into the system attribute in real time, that is, the system attribute is recorded in real time.
  • the running state of the remote application can improve the real-time performance of the remote application running state through system properties.
  • the step of obtaining screen interface data includes the following steps:
  • the screen interface data refers to the screen interface data of the upper computer
  • the screen interface display method can be applied to the intelligent terminal.
  • the terminal receives the transmission code stream of the upper computer, and the transmission code of the transmission screen The stream can be decoded to obtain the screen interface data of the upper computer. After the screen interface data of the upper computer is rendered, the screen interface of the upper computer can be displayed on the terminal screen to complete the screen transmission from the upper computer to the intelligent terminal.
  • the step of receiving the screen encoded code stream sent by the host computer comprises the following steps:
  • the transmission code stream transmitted by the host computer is received by means of wireless hotspot communication.
  • the smart terminal applying the screen interface display method receives the broadcast coded code stream sent by the upper computer through the wireless hotspot communication mode, and the receiving by the wireless hotspot communication mode may not limit the specificity of the upper computer and the terminal.
  • the location in turn, guarantees the complete transmission of data.
  • the screen interface display method can be applied to an intelligent terminal using an Android system platform, and the screen interface display method can be implemented in the wireless screen receiving software;
  • the system broadcast may be sent to notify other applications, and the running status of the remote application is written into the system attribute of the terminal;
  • the wireless transmission software is running in the upper computer.
  • the wireless transmission hardware is connected to the upper computer through USB communication.
  • the upper computer sends the data of the upper computer screen interface to the wireless transmission hardware, and the wireless transmission hardware will be on the upper position.
  • the screen interface data is sent to the terminal through wireless hotspot communication;
  • the wireless screen receiving software in the terminal receives the wireless screen data and decodes it to obtain Screen interface data of the host computer;
  • the wireless screen receiving software determines whether the current terminal has a high resource occupancy rate such as remote voice, remote video or remote desktop sharing through the system attribute;
  • the wireless screen receiving software changes the view used for rendering to SurfaceView; the decoded screen interface data is transmitted to the underlying SurfaceView.
  • Rendering displayed on the screen of the terminal device; during the rendering of the SurfaceView, the running state of the remote application can be determined in real time through the system property, and when the remote application exits, the rendering mode is switched to OpenGL rendering;
  • the wireless screen receiving software When the application of high resource usage such as remote voice, remote video or remote desktop sharing is not running, the wireless screen receiving software will convert the view used for rendering into GLSurfaceView; the decoded screen interface data is passed to GLSurfaceView for rendering. Displayed on the screen of the terminal device; during the rendering of the GLSurfaceView, the running state of the remote application can be determined in real time through the system property, and when the remote application starts running, the rendering mode is switched to SurfaceView rendering;
  • the invention can adjust the occupation of the processor resources, and ensure the running quality of the remote application and improve the user experience while performing wireless display and rendering.
  • the system in the terminal to which the present invention is applied is not limited to the Android system, but may be other operating systems, and is not limited to wireless screen transmission, and may also be a wired screen.
  • the upper computer may be a PC (personal computer). .
  • the present invention further provides a screen interface display system.
  • a screen interface display system According to the above screen interface display method, the present invention further provides a screen interface display system.
  • an embodiment of the screen interface display system of the present invention will be described in detail.
  • FIG. 3 it is a schematic structural diagram of a screen interface display system according to an embodiment of the present invention.
  • the screen interface display system in this embodiment includes the following units:
  • the acquiring unit 210 is configured to acquire screen interface data.
  • the determining unit 220 is configured to determine, in real time, whether the resource occupancy rate of the current terminal processor is higher than a first preset value
  • the first rendering unit 230 is configured to: when the determining unit 220 determines that the resource occupancy rate of the current terminal processor is higher than the first preset value, render the screen interface data according to the first rendering manner, to obtain the first rendered image, and the first A rendered image is displayed on the screen of the current terminal;
  • the second rendering unit 240 is configured to: when the determining unit 220 determines that the resource occupancy rate of the current terminal processor is lower than or equal to the first preset value, and render the screen interface data according to the second rendering manner to obtain the second rendered image, The second rendered image is displayed on the screen of the current terminal, wherein the resource usage rate of the first rendering mode is lower than the resource usage ratio of the second rendering mode.
  • the screen interface display system further includes a computing unit 250 and a second determining unit 260;
  • the calculating unit 250 is configured to calculate a resource occupancy rate required to render the screen interface data in the second rendering manner
  • the second determining unit 260 is configured to determine whether the sum of the resource occupancy rate of the current terminal processor and the resource occupancy rate required to render the screen interface data in the second rendering manner is higher than a second preset value;
  • the first rendering unit 230 determines, according to the second determining unit 260, that the sum of the resource occupancy rate of the current terminal processor and the resource occupancy rate required to render the screen interface data in the second rendering manner is higher than the second preset value, according to the first Rendering mode renders screen interface data, obtains a first rendered image, and displays the first rendered image on a screen of the current terminal;
  • the second rendering unit 240 determines, when the secondary determining unit 260 determines that the sum of the resource occupancy rate of the current terminal processor and the resource occupancy rate required to render the screen interface data in the second rendering manner is lower than or equal to the second preset value, The second rendering mode renders the screen interface data, obtains a second rendered image, and displays the second rendered image on the screen of the current terminal.
  • the screen interface data is screen interface change data or screen interface unchanged data.
  • the first rendering mode is a SurfaceView rendering mode
  • the second rendering mode is an OpenGL rendering mode.
  • the first rendering unit 230 converts the view used for rendering into a SurfaceView, and the number of screen interfaces It is passed to SurfaceView for rendering.
  • the second rendering unit 240 converts the view used for rendering into a GLSurfaceView, and passes the screen interface data to the GLSurfaceView for rendering.
  • the determining unit 220 is configured to determine in real time whether the current terminal has a remote application running.
  • the determining unit 220 determines in real time whether the current terminal has a remote application running according to the system attribute, wherein the running state of the remote application is saved in the system attribute.
  • the remote application writes the operational status of the remote application to system properties in real time.
  • the obtaining unit 210 includes a receiving unit 212 and a decoding unit 214;
  • the receiving unit 212 is configured to receive a screen coded code stream sent by the host computer, where the screen coded code stream includes screen interface data of the host computer;
  • the decoding unit 214 is configured to decode the screen encoded code stream to obtain screen interface data of the upper computer.
  • the receiving unit 212 receives the broadcast encoded code stream sent by the host computer by means of wireless hotspot communication.
  • the screen interface display system of the present invention has a one-to-one correspondence with the screen interface display method of the present invention.
  • the technical features and the beneficial effects described in the embodiments of the above screen interface display method are all applicable to the embodiment of the screen interface display system.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Information Transfer Between Computers (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

一种屏幕界面显示方法和系统，其获取屏幕界面数据(S101)，实时判断当前终端处理器的资源占用率是否高于第一预设值(S102)；若是，则根据第一渲染方式对所述屏幕界面数据进行渲染，获得第一渲染图像，将第一渲染图像显示在当前终端的屏幕上(S103)；若否，则根据第二渲染方式对所述屏幕界面数据进行渲染，获得第二渲染图像并显示。在当前终端处理器的资源占用率较低时采用第二渲染方式可以很好地渲染并显示屏幕界面图像；第一渲染方式对处理器的资源占用率比第二渲染方式对处理器的资源占用率低，在渲染之前有远程应用程序运行导致当前终端处理器的资源占用率较高时，采用第一渲染方式可以使处理器留有空闲资源，可以保证远程应用程序的运行质量。

Description

屏幕界面显示方法和系统 技术领域
本发明涉及显示控制应用程序技术领域，特别是涉及一种屏幕界面显示方法和系统。
背景技术
在显示技术领域中，上位机(可以直接发出命令的计算机)可以将其屏幕界面传送到终端设备上进行同步显示。终端设备接收到上位机的屏幕界面数据时，会对上位机的屏幕界面数据进行渲染，为了加快渲染速度，减小屏幕界面显示延时，一般渲染时对终端设备处理器的资源占用率很高；终端设备的性能有限，在接收到上位机的屏幕界面数据的同时，如果终端设备还运行着其它资源占用率较高的应用程序或服务，就会导致终端设备的处理器空闲资源不足，使得终端设备的应用程序或服务的运行质量无法得到保证。
发明内容
基于此，有必要针对现有的终端设备在进行屏幕界面渲染时处理器的资源占用率过高导致终端设备的应用程序或服务的质量无法得到保证的问题，提供一种屏幕界面显示方法和系统。
一种屏幕界面显示方法，包括以下步骤：
获取屏幕界面数据；
实时判断当前终端处理器的资源占用率是否高于第一预设值；
若是，则根据第一渲染方式对屏幕界面数据进行渲染，获得第一渲染图像，将第一渲染图像显示在当前终端的屏幕上；
若否，则根据第二渲染方式对屏幕界面数据进行渲染，获得第二渲染图像，将第二渲染图像显示在当前终端的屏幕上，其中，第一渲染方式的资源占用率低于第二渲染方式的资源占用率。
一种屏幕界面显示系统，包括以下单元：
获取单元，用于获取屏幕界面数据；
判断单元，用于实时判断当前终端处理器的资源占用率是否高于第一预设值；
第一渲染单元，用于在判断单元判定当前终端处理器的资源占用率高于第一预设值时，根据第一渲染方式对屏幕界面数据进行渲染，获得第一渲染图像，将第一渲染图像显示在当前终端的屏幕上；
第二渲染单元，用于在判断单元判定当前终端处理器的资源占用率高于第一预设值时，根据第二渲染方式对屏幕界面数据进行渲染，获得第二渲染图像，将第二渲染图像显示在当前终端的屏幕上，其中，第一渲染方式的资源占用率低于第二渲染方式的资源占用率。
根据上述本发明的屏幕界面显示方法和系统，其主要是根据当前终端处理器的资源占用率是否高于第一预设值来选择渲染方式，对获取的屏幕界面数据进行渲染并显示图像。第二渲染方式对处理器的资源占用率高，在当前终端处理器的资源占用率较低时采用第二渲染方式可以很好地渲染并显示屏幕界面图像；第一渲染方式对处理器的资源占用率比第二渲染方式对处理器的资源占用率低，在渲染之前有远程应用程序运行导致当前终端处理器的资源占用率较高时，采用第一渲染方式可以使处理器留有空闲资源，可以保证远程应用程序的运行质量。本发明通过改变渲染方式，可以调整对处理器资源占用的情况，在渲染的同时保证远程应用程序的运行质量。
附图说明
图1为其中一个实施例的屏幕界面显示方法的流程示意图；
图2为其中一个实施例的屏幕界面显示方法的实际应用示意图；
图3为其中一个实施例的屏幕界面显示系统的结构示意图；
图4为其中一个实施例的屏幕界面显示系统的结构示意图；
图5为其中一个实施例的屏幕界面显示系统的结构示意图。
具体实施方式
为使本发明的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本发明进行进一步的详细说明。应当理解，此处所描述的具体实施方式仅仅用以解释本发明，并不限定本发明的保护范围。
参见图1所示，为本发明一个实施例中屏幕界面显示方法的流程示意图。该实施例中的屏幕界面显示方法，包括以下步骤：
步骤S101：获取屏幕界面数据；
步骤S102：实时判断当前终端处理器的资源占用率是否高于第一预设值；
步骤S103：若是，则根据第一渲染方式对屏幕界面数据进行渲染，获得第一渲染图像，将第一渲染图像显示在当前终端的屏幕上；
步骤S104：若否，则根据第二渲染方式对屏幕界面数据进行渲染，获得第二渲染图像，将第二渲染图像显示在当前终端的屏幕上，其中，第一渲染方式的资源占用率低于第二渲染方式的资源占用率。
在本实施例中，主要是根据当前终端处理器的资源占用率是否高于第一预设值来选择渲染方式，对获取的屏幕界面数据进行渲染并显示图像。第二渲染方式对处理器的资源占用率高，在当前终端处理器的资源占用率较低时采用第二渲染方式可以很好地渲染并显示屏幕界面图像；第一渲染方式对处理器的资源占用率比第二渲染方式对处理器的资源占用率低，在渲染之前有远程应用程序运行导致当前终端处理器的资源占用率较高时，采用第一渲染方式可以使处理器留有空闲资源，可以保证远程应用程序的运行质量。本发明通过改变渲染方式，可以调整对处理器资源占用的情况，在渲染的同时保证远程应用程序的运行质量。
第一预设值可以根据实际情形进行设置，如80％。
在其中一个实施例中，根据第一渲染方式对屏幕界面数据进行渲染，获得第一渲染图像，将第一渲染图像显示在当前终端的屏幕上的步骤包括以下步骤：
计算以第二渲染方式渲染屏幕界面数据所需要的资源占用率，判断当前终端处理器的资源占用率与以第二渲染方式渲染屏幕界面数据所需要的资源 占用率之和是否高于第二预设值；
若是，则根据第一渲染方式对屏幕界面数据进行渲染，获得第一渲染图像，将第一渲染图像显示在当前终端的屏幕上；
若否，则根据第二渲染方式对屏幕界面数据进行渲染，获得第二渲染图像，将第二渲染图像显示在当前终端的屏幕上。
在本实施例中，在判定当前终端处理器的资源占用率高于第一预设值后，还可以对以第二渲染方式渲染屏幕界面数据所需要的资源占用率进行计算，并判断其与当前终端处理器的资源占用率之和是否高于第二预设值，若是，则说明此时采用第二渲染方式对屏幕界面数据进行渲染会导致当前终端处理器的空闲资源不足，可以采用第一渲染方式对屏幕界面数据进行渲染并显示；若否，则说明此时采用第二渲染方式仍可以使当前终端处理器留有空闲资源，可以采用第二渲染方式对屏幕界面数据进行渲染并显示。
第二预设值可以根据实际情形进行设置，如90％。
在其中一个实施例中，屏幕界面数据为屏幕界面变化数据或屏幕界面未变数据。
在本实施例中，屏幕界面数据可以为屏幕界面变化数据或屏幕界面未变数据。在对屏幕界面数据进行渲染时，可以对屏幕界面变化数据和屏幕界面未变数据分别进行渲染，对屏幕界面变化数据和屏幕界面未变数据分别进行渲染的方式可以不同，如此可以根据屏幕界面变化数据和屏幕界面未变数据的实际情形灵活选择相应的渲染方式。
在其中一个实施例中，第一渲染方式为SurfaceView渲染方式，第二渲染方式为OpenGL渲染方式
根据第一渲染方式对屏幕界面数据进行渲染的步骤包括以下步骤：
将用来渲染的视图转变为SurfaceView，将屏幕界面数据传给SurfaceView进行渲染；
根据第二渲染方式对屏幕界面数据进行渲染的步骤包括以下步骤：
将用来渲染的视图转变为GLSurfaceView，将屏幕界面数据传给GLSurfaceView进行渲染。
在本实施例中，第一渲染方式为SurfaceView渲染方式，进行渲染时，先将用来渲染的视图转变为SurfaceView(表面视图)，再将屏幕界面数据传给SurfaceView进行渲染，以此可以实现SurfaceView渲染；第二渲染方式为OpenGL渲染方式，进行渲染时，先将用来渲染的视图转变为GLSurfaceView(图形库表面视图)，再将屏幕界面数据传给GLSurfaceView进行渲染，以此可以实现OpenGL(开放图形库)渲染。OpenGL渲染方式对处理器的资源占用率高，延时小，帧率稳定，可以很好地渲染并显示屏幕界面图像，SurfaceView渲染方式对处理器的资源占用率比OpenGL渲染方式的低，虽然SurfaceView渲染方式的延时不稳定，但其延时对于远程应用程序来说完全可以忽略，因此可以保证远程应用程序的运行质量。
在其中一个实施例中，实时判断当前终端处理器的资源占用率是否高于第一预设值的步骤包括以下步骤：
实时判断当前终端是否有远程应用程序在运行。
在本实施例中，远程应用程序运行时对终端处理器的资源占用率较高，可以将远程应用程序运行时对终端处理器的资源占用率作为第一预设值的参照值，也就是，当前终端运行远程应用程序时，即等同于当前终端处理器的资源占用率高于第一预设值。而且，如此更可以确保远程应用程序的运行质量。远程应用程序可以包括远程语音、远程视频、远程桌面共享等应用程序。
在其中一个实施例中，实时判断当前终端是否有远程应用程序在运行的步骤包括以下步骤：
根据系统属性实时判断当前终端是否有远程应用程序在运行，其中，远程应用程序的运行状态保存在系统属性中。
在本实施例中，远程应用程序的运行状态保存在系统属性中，在需要了解当前终端是否有远程应用程序在运行时，通过系统属性就可以进行判断，而并不需要与远程应用程序建立关联关系，从而降低了对处理器的资源占用率。
在其中一个实施例中，远程应用程序将远程应用程序的运行状态实时写入系统属性。
在本实施例中，远程应用程序的运行状态是由远程应用程序写入系统属性的，远程应用程序的正在运行的状态和不在运行的状态都会实时写入系统属性，也就是系统属性会实时记录远程应用程序的运行状态，可以提高通过系统属性判断远程应用程序运行状态的实时性。
在其中一个实施例中，获取屏幕界面数据的步骤包括以下步骤：
接收上位机发送的传屏编码码流，其中，传屏编码码流包含上位机的屏幕界面数据；
对传屏编码码流进行解码，获得上位机的屏幕界面数据。
在本实施例中，屏幕界面数据是指上位机的屏幕界面数据，屏幕界面显示方法可以应用于智能终端，在具体应用时，终端接收上位机的传屏编码码流，对该传屏编码码流进行解码就可以获得上位机的屏幕界面数据，在对上位机的屏幕界面数据进行渲染后就可以在终端屏幕上显示上位机的屏幕界面，完成上位机到智能终端的屏幕传输。
在其中一个实施例中，接收上位机发送的传屏编码码流的步骤包括以下步骤：
通过无线热点通信的方式接收上位机发送的传屏编码码流。
在本实施例中，应用屏幕界面显示方法的智能终端是通过无线热点通信的方式来接收上位机发送的传屏编码码流，通过无线热点通信的方式进行接收可以不限制上位机与终端的具体位置，又可以保证数据的完整传输。
在一个优选的实施例中，屏幕界面显示方法可以应用于使用Android系统平台的智能终端，屏幕界面显示方法可以在无线传屏接收软件中实现；
在终端中，远程语音、远程视频或远程桌面共享等远程应用服务运行时，可以发送系统广播通知其他应用程序，并将远程应用程序的运行状态写入终端的系统属性；
如图2所示，上位机中运行无线传屏发送软件，无线传屏硬件通过USB通信与上位机进行连接，上位机将上位机屏幕界面数据发给无线传屏硬件，无线传屏硬件将上位机屏幕界面数据通过无线热点通信的方式发送至终端；
终端中的无线传屏接收软件接收无线传屏数据，并对其进行解码，获得 上位机的屏幕界面数据；
在对上位机的屏幕界面数据进行渲染之前，无线传屏接收软件会通过系统属性来判断当前终端是否有远程语音、远程视频或远程桌面共享等高资源占用率的应用在同时运行；
当远程语音、远程视频或远程桌面共享等高资源占用率的应用在同时运行时，无线传屏接收软件将用来渲染的视图变为SurfaceView；解码后得到的屏幕界面数据被传给底层SurfaceView进行渲染，显示在终端设备的屏幕上；在SurfaceView渲染的过程中，可以实时通过系统属性确定远程应用程序的运行状态，当远程应用程序退出时，会将渲染方式切换为OpenGL渲染；
当远程语音、远程视频或远程桌面共享等高资源占用率的应用没有在运行时，无线传屏接收软件将用来渲染的视图变为GLSurfaceView；解码后得到的屏幕界面数据被传给GLSurfaceView进行渲染，显示在终端设备的屏幕上；在GLSurfaceView渲染的过程中，可以实时通过系统属性确定远程应用程序的运行状态，当远程应用程序开始运行时，会将渲染方式切换为SurfaceView渲染；
本发明通过动态改变渲染方式，可以调整对处理器资源占用的情况，在无线传屏，进行渲染显示的同时，保证远程应用程序的运行质量，提高用户的体验。
另外，本发明所应用的终端中的系统不仅仅只限于Android系统，也可以是其他操作系统，而且，不限于无线传屏，也可以是有线传屏，上位机可以是PC机(个人电脑)。
根据上述屏幕界面显示方法，本发明还提供一种屏幕界面显示系统，以下就本发明的屏幕界面显示系统的实施例进行详细说明。
参见图3所示，为本发明一个实施例中屏幕界面显示系统的结构示意图。该实施例中的屏幕界面显示系统包括以下单元：
获取单元210，用于获取屏幕界面数据；
判断单元220，用于实时判断当前终端处理器的资源占用率是否高于第一预设值；
第一渲染单元230，用于在判断单元220判定当前终端处理器的资源占用率高于第一预设值时，根据第一渲染方式对屏幕界面数据进行渲染，获得第一渲染图像，将第一渲染图像显示在当前终端的屏幕上；
第二渲染单元240，用于在判断单元220判定当前终端处理器的资源占用率低于或等于第一预设值时，根据第二渲染方式对屏幕界面数据进行渲染，获得第二渲染图像，将第二渲染图像显示在当前终端的屏幕上，其中，第一渲染方式的资源占用率低于第二渲染方式的资源占用率。
在其中一个实施例中，如图4所示，屏幕界面显示系统还包括计算单元250和二次判断单元260；
计算单元250用于计算以第二渲染方式渲染屏幕界面数据所需要的资源占用率；
二次判断单元260用于判断当前终端处理器的资源占用率与以第二渲染方式渲染屏幕界面数据所需要的资源占用率之和是否高于第二预设值；
第一渲染单元230在二次判断单元260判定当前终端处理器的资源占用率与以第二渲染方式渲染屏幕界面数据所需要的资源占用率之和高于第二预设值时，根据第一渲染方式对屏幕界面数据进行渲染，获得第一渲染图像，将第一渲染图像显示在当前终端的屏幕上；
第二渲染单元240在二次判断单元260判定当前终端处理器的资源占用率与以第二渲染方式渲染屏幕界面数据所需要的资源占用率之和低于或等于第二预设值时，根据第二渲染方式对屏幕界面数据进行渲染，获得第二渲染图像，将第二渲染图像显示在当前终端的屏幕上。
在其中一个实施例中，屏幕界面数据为屏幕界面变化数据或屏幕界面未变数据。
在其中一个实施例中，第一渲染方式为SurfaceView渲染方式,第二渲染方式为OpenGL渲染方式；
第一渲染单元230将用来渲染的视图转变为SurfaceView，将屏幕界面数 据传给SurfaceView进行渲染。
第二渲染单元240将用来渲染的视图转变为GLSurfaceView，将屏幕界面数据传给GLSurfaceView进行渲染。
在其中一个实施例中，判断单元220用于实时判断当前终端是否有远程应用程序在运行。
在其中一个实施例中，判断单元220根据系统属性实时判断当前终端是否有远程应用程序在运行，其中，远程应用程序的运行状态保存在系统属性中。
在其中一个实施例中，远程应用程序将远程应用程序的运行状态实时写入系统属性。
在其中一个实施例中，如图5所示，获取单元210包括接收单元212和解码单元214；
接收单元212用于接收上位机发送的传屏编码码流，其中，传屏编码码流包含上位机的屏幕界面数据；
解码单元214用于对传屏编码码流进行解码，获得上位机的屏幕界面数据。
在其中一个实施例中，接收单元212通过无线热点通信的方式接收上位机发送的传屏编码码流。
本发明的屏幕界面显示系统与本发明的屏幕界面显示方法一一对应，在上述屏幕界面显示方法的实施例阐述的技术特征及其有益效果均适用于屏幕界面显示系统的实施例中。
以上所述实施例的各技术特征可以进行任意的组合，为使描述简洁，未对上述实施例中的各个技术特征所有可能的组合都进行描述，然而，只要这些技术特征的组合不存在矛盾，都应当认为是本说明书记载的范围。
以上所述实施例仅表达了本发明的几种实施方式，其描述较为具体和详细，但并不能因此而理解为对发明专利范围的限制。应当指出的是，对于本 领域的普通技术人员来说，在不脱离本发明构思的前提下，还可以做出若干变形和改进，这些都属于本发明的保护范围。因此，本发明专利的保护范围应以所附权利要求为准。

Claims (10)

  1. 一种屏幕界面显示方法，其特征在于，包括以下步骤：
    获取屏幕界面数据；
    实时判断当前终端处理器的资源占用率是否高于第一预设值；
    若是，则根据第一渲染方式对所述屏幕界面数据进行渲染，获得第一渲染图像，将所述第一渲染图像显示在所述当前终端的屏幕上；
    若否，则根据第二渲染方式对所述屏幕界面数据进行渲染，获得第二渲染图像，将所述第二渲染图像显示在所述当前终端的屏幕上，其中，所述第一渲染方式的资源占用率低于所述第二渲染方式的资源占用率。
  2. 根据权利要求1所述的屏幕界面显示方法，其特征在于，所述根据第一渲染方式对所述屏幕界面数据进行渲染，获得第一渲染图像，将所述第一渲染图像显示在所述当前终端的屏幕上的步骤包括以下步骤：
    计算以第二渲染方式渲染所述屏幕界面数据所需要的资源占用率，判断当前终端处理器的资源占用率与以第二渲染方式渲染所述屏幕界面数据所需要的资源占用率之和是否高于第二预设值；
    若是，则根据所述第一渲染方式对所述屏幕界面数据进行渲染，获得第一渲染图像，将所述第一渲染图像显示在所述当前终端的屏幕上；
    若否，则根据所述第二渲染方式对所述屏幕界面数据进行渲染，获得第二渲染图像，将所述第二渲染图像显示在所述当前终端的屏幕上。
  3. 根据权利要求2所述的屏幕界面显示方法，其特征在于，所述屏幕界面数据为屏幕界面变化数据或屏幕界面未变数据。
  4. 根据权利要求1所述的屏幕界面显示方法，其特征在于：
    第一渲染方式为SurfaceView渲染方式，第二渲染方式为OpenGL渲染方式；
    所述根据第一渲染方式对所述屏幕界面数据进行渲染的步骤包括以下步骤：
    将用来渲染的视图转变为SurfaceView，将所述屏幕界面数据传给SurfaceView进行渲染；
    所述根据第二渲染方式对所述屏幕界面数据进行渲染的步骤包括以下步骤：
    将用来渲染的视图转变为GLSurfaceView，将所述屏幕界面数据传给GLSurfaceView进行渲染。
  5. 根据权利要求1所述的屏幕界面显示方法，其特征在于，所述实时判断当前终端处理器的资源占用率是否高于第一预设值的步骤包括以下步骤：
    实时判断当前终端是否有远程应用程序在运行。
  6. 根据权利要求5所述的屏幕界面显示方法，其特征在于，所述实时判断当前终端是否有远程应用程序在运行的步骤包括以下步骤：
    根据系统属性实时判断当前终端是否有远程应用程序在运行，其中，所述远程应用程序的运行状态保存在所述系统属性中。
  7. 根据权利要求6所述的屏幕界面显示方法，其特征在于，所述远程应用程序将所述远程应用程序的运行状态实时写入所述系统属性。
  8. 根据权利要求1所述的屏幕界面显示方法，其特征在于，所述获取屏幕界面数据的步骤包括以下步骤：
    接收上位机发送的传屏编码码流，其中，所述传屏编码码流包含上位机的屏幕界面数据；
    对所述传屏编码码流进行解码，获得所述上位机的屏幕界面数据。
  9. 根据权利要求8所述的屏幕界面显示方法，其特征在于，所述接收上位机发送的传屏编码码流的步骤包括以下步骤：
    通过无线热点通信的方式接收上位机发送的传屏编码码流。
  10. 一种屏幕界面显示系统，其特征在于，包括以下单元：
    获取单元，用于获取屏幕界面数据；
    判断单元，用于实时判断当前终端处理器的资源占用率是否高于第一预设值；
    第一渲染单元，用于在所述判断单元判定当前终端处理器的资源占用率高于第一预设值时，根据第一渲染方式对所述屏幕界面数据进行渲染，获得第一渲染图像，将所述第一渲染图像显示在所述当前终端的屏幕上；
    第二渲染单元，用于在所述判断单元判定当前终端处理器的资源占用率低于或等于第一预设值时，根据第二渲染方式对所述屏幕界面数据进行渲染，获得第二渲染图像，将所述第二渲染图像显示在所述当前终端的屏幕上，其中，所述第一渲染方式的资源占用率低于所述第二渲染方式的资源占用率。
PCT/CN2016/113188 2016-08-29 2016-12-29 屏幕界面显示方法和系统 Ceased WO2018040428A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201610755063.7A CN106407003A (zh) 2016-08-29 2016-08-29 屏幕界面显示方法和系统
CN201610755063.7 2016-08-29

Publications (1)

Publication Number Publication Date
WO2018040428A1 true WO2018040428A1 (zh) 2018-03-08

Family

ID=58002594

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/113188 Ceased WO2018040428A1 (zh) 2016-08-29 2016-12-29 屏幕界面显示方法和系统

Country Status (2)

Country Link
CN (1) CN106407003A (zh)
WO (1) WO2018040428A1 (zh)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019184251A1 (en) * 2018-03-27 2019-10-03 Boe Technology Group Co., Ltd. Rendering method, computer product and display apparatus
CN110377258A (zh) * 2019-07-17 2019-10-25 Oppo广东移动通信有限公司 图像渲染方法、装置、电子设备及存储介质
CN111459428A (zh) * 2020-02-28 2020-07-28 通彩视听科技(上海)有限公司 显示界面同步方法、装置、计算机设备及存储介质
CN114281515A (zh) * 2020-09-18 2022-04-05 华为云计算技术有限公司 渲染资源分配方法及装置、计算机设备、存储介质

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107071556B (zh) * 2017-04-18 2019-12-10 腾讯科技（深圳）有限公司 一种界面渲染方法和装置
CN110413245A (zh) * 2019-07-17 2019-11-05 Oppo广东移动通信有限公司 图像合成方法、装置、电子设备及存储介质
CN115291791B (zh) * 2022-08-17 2024-08-06 维沃移动通信有限公司 文本识别方法、装置、电子设备及存储介质
CN119717565A (zh) * 2024-12-18 2025-03-28 珠海格力电器股份有限公司 界面显示方法、装置、智能家居设备、电子设备及介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130283318A1 (en) * 2012-04-23 2013-10-24 Comcast Cable Communications, LLC. Dynamic Mosaic for Creation of Video Rich User Interfaces
US8817021B1 (en) * 2011-11-11 2014-08-26 Google Inc. System for writing, interpreting, and translating three-dimensional (3D) scenes
CN104035768A (zh) * 2014-05-30 2014-09-10 四川长虹电器股份有限公司 Android平台兼容本地代码OpenGL程序的系统及方法
US20150178977A1 (en) * 2013-05-14 2015-06-25 Google Inc. Rendering Vector Maps in a Geographic Information System
CN105096373A (zh) * 2015-06-30 2015-11-25 华为技术有限公司 一种媒体内容渲染的方法、用户设备及系统

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8817021B1 (en) * 2011-11-11 2014-08-26 Google Inc. System for writing, interpreting, and translating three-dimensional (3D) scenes
US20130283318A1 (en) * 2012-04-23 2013-10-24 Comcast Cable Communications, LLC. Dynamic Mosaic for Creation of Video Rich User Interfaces
US20150178977A1 (en) * 2013-05-14 2015-06-25 Google Inc. Rendering Vector Maps in a Geographic Information System
CN104035768A (zh) * 2014-05-30 2014-09-10 四川长虹电器股份有限公司 Android平台兼容本地代码OpenGL程序的系统及方法
CN105096373A (zh) * 2015-06-30 2015-11-25 华为技术有限公司 一种媒体内容渲染的方法、用户设备及系统

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019184251A1 (en) * 2018-03-27 2019-10-03 Boe Technology Group Co., Ltd. Rendering method, computer product and display apparatus
US11436787B2 (en) 2018-03-27 2022-09-06 Beijing Boe Optoelectronics Technology Co., Ltd. Rendering method, computer product and display apparatus
CN110377258A (zh) * 2019-07-17 2019-10-25 Oppo广东移动通信有限公司 图像渲染方法、装置、电子设备及存储介质
CN110377258B (zh) * 2019-07-17 2023-05-02 Oppo广东移动通信有限公司 图像渲染方法、装置、电子设备及存储介质
CN111459428A (zh) * 2020-02-28 2020-07-28 通彩视听科技(上海)有限公司 显示界面同步方法、装置、计算机设备及存储介质
CN111459428B (zh) * 2020-02-28 2023-01-06 通彩视听科技(上海)有限公司 显示界面同步方法、装置、计算机设备及存储介质
CN114281515A (zh) * 2020-09-18 2022-04-05 华为云计算技术有限公司 渲染资源分配方法及装置、计算机设备、存储介质

Also Published As

Publication number Publication date
CN106407003A (zh) 2017-02-15

Similar Documents

Publication Publication Date Title
WO2018040428A1 (zh) 屏幕界面显示方法和系统
US8839112B2 (en) Cloning or extending a computer desktop on a wireless display surface
WO2021143182A1 (zh) 游戏的处理方法、装置、电子设备及计算机可读存储介质
CN103475953B (zh) 一种基于桌面云的媒体控制方法和设备
US9693080B2 (en) Distribution control system, distribution control method, and computer-readable storage medium
KR101668282B1 (ko) 화면 가상화 기반 어플리케이션 구동 시스템 및 방법
US9723337B2 (en) Distribution control system and distribution system
JP2017536007A (ja) クラウドストリーミングサービスシステム、最適ｇｐｕを利用したクラウドストリーミングサービス方法及びそのための装置
US20160044079A1 (en) Distribution control system, distribution control method, and computer-readable storage medium
US9578079B2 (en) Distribution control system, distribution system, distribution control method, and computer-readable storage medium
CN109587555B (zh) 视频处理方法、装置、电子设备及存储介质
WO2023071546A1 (zh) 重定向方法、装置、设备、存储介质及程序产品
CN114286128A (zh) 直播视频参数调整方法、系统、装置、设备及存储介质
WO2018040427A1 (zh) 屏幕界面共享方法和系统
CN111031389A (zh) 视频处理方法、电子装置和存储介质
KR101630638B1 (ko) 화면 가상화 기반 어플리케이션 실행 시스템 및 방법
CN116781973B (zh) 视频编解码方法、装置、存储介质和电子设备
CN113766266B (zh) 音视频处理方法、装置、设备以及存储介质
JP2014075737A (ja) 画像処理装置、画像処理方法及び画像処理システム
KR20160015128A (ko) 클라우드 스트리밍 서비스 시스템, 이미지 타입에 따른 클라우드 스트리밍 서비스 방법 및 이를 위한 장치
CN115412559B (zh) 端云资源协同方法、电子设备及可读存储介质
US10075325B2 (en) User terminal device and contents streaming method using the same
EP3264284A1 (en) Data processing method and device
WO2020107970A1 (zh) 视频解码方法、装置、电子设备及存储介质
JP6781445B1 (ja) 情報処理方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16914989

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 190719)

122 Ep: pct application non-entry in european phase

Ref document number: 16914989

Country of ref document: EP

Kind code of ref document: A1