WO2019020112A1 - Terminal display method, terminal and computer-readable storage medium - Google Patents

Terminal display method, terminal and computer-readable storage medium Download PDF

Info

Publication number
WO2019020112A1
WO2019020112A1 PCT/CN2018/097487 CN2018097487W WO2019020112A1 WO 2019020112 A1 WO2019020112 A1 WO 2019020112A1 CN 2018097487 W CN2018097487 W CN 2018097487W WO 2019020112 A1 WO2019020112 A1 WO 2019020112A1
Authority
WO
WIPO (PCT)
Prior art keywords
information
data
transcoding
terminal
area information
Prior art date
Application number
PCT/CN2018/097487
Other languages
French (fr)
Chinese (zh)
Inventor
李繁
Original Assignee
中兴通讯股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中兴通讯股份有限公司 filed Critical 中兴通讯股份有限公司
Publication of WO2019020112A1 publication Critical patent/WO2019020112A1/en

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440218Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440263Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering the spatial resolution, e.g. for displaying on a connected PDA

Definitions

  • the present disclosure relates to, but is not limited to, the field of terminal display technologies, and in particular, to a terminal display method, a terminal, and a computer readable storage medium.
  • terminal devices such as personal computers, PDAs, and smart phones has become more and more widespread, and has become an indispensable device for people.
  • These terminal devices generally have a display function.
  • the video information When the video information is received, it can be displayed in the terminal by decoding and transcoding, so that people can watch the video on the terminal.
  • the terminal will re-decode and transcode all the video information, and then display it in the terminal. In this way, the display efficiency of the terminal is relatively low, and the hardware requirements of the terminal are relatively high, thereby increasing the use cost of the terminal.
  • a main object of the present disclosure is to provide a display method of a terminal, a terminal, and a computer readable storage medium.
  • a display method of a terminal including: receiving real-time video frame data and first change area information in a desktop sent by a server, and decoding the real-time video frame data to obtain a Color coding YUV data; dividing the first YUV data into a preset number of video regions according to a preset rule, and calculating a second change region where the video region intersects with the first change region information by using a preset algorithm And transcoding the second YUV data corresponding to the second change region information into corresponding RGB data, and displaying according to the RGB data.
  • a terminal comprising: a memory, a processor, and a display program stored on the memory and executable by the processor, wherein the display program is The steps of the terminal display method as described above are implemented when the processor is executed.
  • a computer readable storage medium having stored thereon a display program, wherein the display program is executed by a processor to implement the terminal display method as described above A step of.
  • FIG. 1 is a schematic structural diagram of a hardware operating environment (ie, a terminal) according to an embodiment of the present disclosure
  • FIG. 2 is a flowchart of a terminal display method according to an embodiment of the present disclosure
  • FIG. 3 is a flowchart of steps of decoding the real-time video frame data to obtain first YUV data, in accordance with an embodiment of the present disclosure
  • FIG. 4 is a second method of dividing the first YUV data into a preset number of video regions according to a preset rule and calculating a intersection of the video region and the first change region information by using a preset algorithm, according to an embodiment of the present disclosure. a flow chart of the steps of changing the area information;
  • FIG. 5 is a flowchart of a step of transcoding second color coded YUV data corresponding to the second change region information into corresponding RGB data and displaying according to the RGB data, according to an embodiment of the present disclosure.
  • the terminal will re-decode and transcode all the video information, and then display it in the terminal.
  • the display efficiency of the terminal is relatively low, and the hardware requirements of the terminal are relatively high, thereby increasing the use cost of the terminal.
  • the amount of data processed is much larger. Therefore, improving the display efficiency of the terminal is a technical problem to be solved by those skilled in the art.
  • an embodiment of the present disclosure provides a terminal display method, including: receiving real-time video frame data and first change area information in a desktop sent by a server, and decoding the real-time video frame data, Obtaining first color coded YUV data; dividing the first YUV data into a preset number of video regions according to a preset rule, and calculating, by using a preset algorithm, a second intersection of the video region and the first change region information Changing the region information; transcoding the second YUV data corresponding to the second change region information into corresponding RGB data, and displaying the data according to the RGB data.
  • the terminal display method receives the real-time video frame data and the first change region information in the desktop sent by the server, and calculates the second change region information that the two intersect according to the preset algorithm. And transcoding only the second YUV data corresponding to the second change region information after decoding into corresponding RGB data.
  • the terminal only needs to transcode the second changed area information to complete the display. Therefore, the time from the reception of the desktop video data to the display by the terminal can be reduced, thereby improving the display efficiency of the terminal. At the same time, the requirements on the hardware resources of the terminal can be reduced, thereby reducing the use cost of the terminal.
  • FIG. 1 is a schematic structural diagram of a hardware operating environment (ie, a terminal) according to an embodiment of the present disclosure.
  • the terminal in the embodiment of the present disclosure may be a PC, or may be a mobile terminal device having a display function such as a smart phone, a tablet computer, a portable computer, or may be a cloud terminal.
  • the terminal may include a processor 1001 (eg, a CPU), a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002.
  • Communication bus 1002 is used to implement connection communication between these components.
  • the user interface 1003 can include a display, an input unit such as a keyboard, and the user interface 1003 can optionally include a standard wired interface, a wireless interface.
  • Network interface 1004 can optionally include a standard wired interface, a wireless interface (such as a WI-FI interface).
  • the memory 1005 may be a high speed RAM memory or a non-volatile memory such as a disk memory.
  • the memory 1005 can optionally be a storage device that is separate from the aforementioned processor 1001.
  • the terminal may further include a camera, an RF (Radio Frequency) circuit, a sensor, an audio circuit, a Wi-Fi module, and the like.
  • the sensor comprises, for example, a light sensor, a motion sensor or any other suitable sensor.
  • the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display according to the brightness of the ambient light, and the proximity sensor may turn off the display and/or when the mobile terminal moves to the ear. Backlighting.
  • a gravity acceleration sensor is configured to detect the magnitude of acceleration in each direction (generally three axes), the magnitude of gravity is detected at rest, and the application of the gesture of the mobile terminal is recognized (eg, horizontal and vertical screen switching, correlation) Game, magnetometer attitude calibration) and recognition of vibration (such as pedometer, tapping).
  • the mobile terminal can also be configured with other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, and the like, and details are not described herein again.
  • terminal structure shown in FIG. 1 does not constitute a limitation to the terminal, and may include more or less components than those illustrated, or a combination of certain components, or different component arrangements.
  • a memory 1005 as a computer storage medium may be installed with an operating system, a network communication module, a user interface module, and a display program of the terminal.
  • the network interface 1004 is mainly configured to connect to the background server for data communication with the background server;
  • the user interface 1003 is mainly configured to connect to the client (user end) to perform data communication with the client;
  • the processor 1001 may be configured to call a display program of the terminal stored in the memory 1005 and perform the following operations:
  • the second YUV data corresponding to the second change region information is transcoded into corresponding RGB data, and displayed according to the RGB data.
  • the processor 1001 may be configured to call a terminal display program stored in the memory 1005 and perform the following operations:
  • the real-time video frame data supports hardware decoding, decoding the real-time video frame data by using hardware decoding to obtain first YUV data;
  • the real-time video frame data does not support hardware decoding, the real-time video frame data is decoded in a software decoding manner to obtain first YUV data.
  • the processor 1001 may be configured to call a display program of the terminal stored in the memory 1005 and perform the following operations:
  • the processor 1001 may be configured as a display program of a terminal stored in the memory 1005 and perform the following operations:
  • the first YUV data is divided into a preset number of video regions according to the height or width of the terminal display resolution.
  • the processor 1001 may be configured to call a display program of the terminal stored in the memory 1005 and perform the following operations:
  • the second change area information supports hardware transcoding
  • the second YUV data corresponding to the second change area information is transcoded into corresponding RGB data by using hardware transcoding.
  • the processor 1001 may be configured to call a display program of the terminal stored in the memory 1005 and perform the following operations:
  • the second change area information does not support hardware transcoding, determining, according to the pre-stored hardware information, whether the second change area information supports assembly transcoding;
  • the second change area information supports the assembly transcoding
  • the second YUV data corresponding to the second change area information is transcoded into corresponding RGB data by means of assembly transcoding
  • the second YUV data corresponding to the second change area information is transcoded into corresponding RGB data by means of software transcoding.
  • the processor 1001 may be configured to call a display program of the terminal stored in the memory 1005 and perform the following operations:
  • the processor 1001 may be configured to call a display program of the terminal stored in the memory 1005 and perform the following operations:
  • the first change area information, the second change area information, and the video area each include parameters indicating upper, left, right, and lower coordinate information of the corresponding change area in the desktop.
  • FIG. 2 is a flowchart of a terminal display method according to an embodiment of the present disclosure.
  • the terminal display method includes:
  • Step S10 Receive real-time video frame data and first change area information in the desktop sent by the server, and decode the real-time video frame data to obtain first YUV data.
  • the terminal is activated to establish a network connection between the terminal and the server, and at the same time, the corresponding message communication data structure format is agreed. Further, after receiving the real-time video frame data and the first change area information in the desktop sent by the server, the terminal decodes the real-time video frame data according to the format information of the real-time video frame data and the pre-stored terminal hardware information, thereby A corresponding first color coding (Luminance, Chrominance; luminance, chrominance) YUV data is obtained.
  • the server may be a cloud server or a common physical server; the desktop may be a cloud desktop, or may be a corresponding desktop such as a PC or a tablet.
  • the server includes or is connected to the desktop, for example, the cloud server includes a cloud desktop, and the physical server is connected to the desktop of the computer.
  • FIG. 3 is a schematic flowchart of decoding the real-time video frame data to obtain first YUV data according to an embodiment of the present disclosure.
  • step S10 further includes:
  • Step S11 determining, according to format information of the real-time video frame data and pre-stored hardware information, whether the real-time video frame data supports hardware decoding;
  • Step S12 if the real-time video frame data supports hardware decoding, decoding the real-time video frame data by using hardware decoding to obtain first YUV data;
  • Step S13 If the real-time video frame data does not support hardware decoding, the real-time video frame data is decoded by using software decoding to obtain first YUV data.
  • the terminal receives the real-time video frame data and the first change area information of the current desktop sent by the server, where the first change area information is captured by the server in the desktop by using a preset program. Further, the terminal determines, according to format information of the real-time video frame data and pre-stored terminal hardware information, such as a central processing unit CPU and a graphics card GPU, whether the real-time video frame data supports hardware decoding, if the real-time video frame data Supporting hardware decoding, the hardware decoding method is used to decode the real-time video frame data to obtain first YUV data; if the real-time video frame data does not support hardware decoding, the real-time video frame is obtained by software decoding. The data is decoded to obtain the first YUV data.
  • format information of the real-time video frame data and pre-stored terminal hardware information such as a central processing unit CPU and a graphics card GPU
  • the hardware decoding is to decode the video by GPU operation
  • the software decoding is to enable the CPU to decode the video through software. Therefore, when determining whether the real-time video frame data supports hardware decoding, the terminal should consider both the format information of the video data and the pre-stored hardware information such as the GPU and the CPU. When both of them support hardware decoding, the real-time video frame data is decoded by means of hardware decoding; when one or both of the two do not support hardware decoding, the software decodes the Real-time video frame data is decoded.
  • the terminal when decoding the real-time video frame data, uses the hardware decoding and the software decoding manner to perform the real-time according to the format information of the video data and the pre-stored hardware information support conditions such as the CPU and the GPU.
  • the video frame data is decoded to improve the utilization of the terminal hardware resources.
  • the terminal display method further includes: step S20, dividing the first YUV data into a preset number of video regions according to a preset rule, and calculating, by using a preset algorithm, the video region and the Second change region information intersecting the first change region information;
  • the first YUV data is divided into a preset number of video regions according to the height or width of the resolution of the terminal display.
  • the first YUV data may be equally divided into n horizontal video regions according to the terminal display resolution height or divided into n vertical video regions according to the display resolution width.
  • the resolution of the terminal display is 1024 ⁇ 768, and when the resolution is highly divided by the terminal display, the first YUV data is equally divided into n horizontal video regions by 768/n; Then, the first YUV data is equally divided into n longitudinal video regions by 1024/n.
  • the second change region information that the video region intersects with the first change region information is calculated by using a preset algorithm.
  • the equation "Rect X and Rect Y" is used to obtain the intersection region Rect Z of Rect Y and Rect X.
  • Rect Y Since some areas in Rect Y may not change, there is no intersection with Rect X. Therefore, in the embodiment of the present disclosure, a preset algorithm is used to intersect Rect X with Rect Y to obtain Rect Z, and the region range of Rect Z at this time Smaller than Rect X. Therefore, the range of the real-time changing area of the desktop can be further reduced, thereby shortening the transcoding time and improving the display efficiency of the terminal.
  • the terminal display method further includes: Step S30, transcoding the second YUV data corresponding to the second change area information into corresponding RGB data, and performing display according to the RGB data.
  • the terminal determines whether the second change area information supports hardware decoding according to the support situation of the hardware information such as the pre-stored terminal CPU and the GPU. If the second change area information supports hardware transcoding, the hardware is adopted. Transcoding, the second YUV data corresponding to the second change area information is transcoded into corresponding RGB data; if the second change area information does not support hardware transcoding, further according to the pre-stored CPU and GPU And determining, by the hardware information, whether the second change area information supports assembly transcoding; if the second change area information supports assembly transcoding, the second YUV corresponding to the second change area information is used in an assembly transcoding manner. Data is transcoded into corresponding RGB data; if the second change region information does not support assembly transcoding, the second YUV data corresponding to the second change region information is transcoded into corresponding RGB by software transcoding. data.
  • YUV is a color coding method (belonging to PAL) adopted by European television systems.
  • YUV is mainly used to optimize the transmission of color video signals, making it backward compatible with old-fashioned black-and-white TVs.
  • RGB video signal transmission its biggest advantage is that it requires very little bandwidth (RGB requires three independent video signals to be transmitted simultaneously).
  • Y in YUV means brightness (Luminance or Luma), which is the gray level value, which is a baseband signal.
  • the "U” and “V” represent the chroma (Chrominance or Chroma), which is used to describe the color and saturation of the image and is used to specify the color of the pixel.
  • U and V are signals that are orthogonally modulated, rather than baseband signals.
  • the three components of YUV can restore R (red), G (green), and B (blue).
  • the conversion formula between YUV and RGB can be:
  • RGB data is converted into an image for refreshing and display on the display of the terminal.
  • RGB data can be converted into corresponding RGB images by using a program such as Matlab.
  • the first change area information, the second change area information, and the video area each include four parameters ⁇ top, left, right, bottom ⁇ , respectively
  • Corresponding change regions are upper, left, right, and lower coordinate information in the desktop.
  • the coordinate information can be used to accurately locate the first change area information, the second change area information, and an area corresponding to the video area in the desktop.
  • the embodiment of the present disclosure receives the real-time video frame data and the first change area information in the desktop sent by the server, and decodes the real-time video frame data to obtain the first YUV data; and presets the first YUV data according to the preset Dividing the rule into a preset number of video regions, and calculating, by using a preset algorithm, second change region information that intersects the video region with the first change region information; and second YUV data corresponding to the second change region information Transcoded into corresponding RGB data and displayed according to the RGB data.
  • the present disclosure receives the real-time video frame data and the first change area information in the desktop sent by the server, and calculates the second change area information that the two intersect according to the preset algorithm, and only decodes the first
  • the second YUV data corresponding to the two change area information is transcoded into the corresponding RGB data, so that the terminal only needs to transcode the second change area information to complete the display, and compares all the video area information in the prior art.
  • the method of code display can reduce the time from the receiving of the desktop video data to the display by the terminal, thereby improving the display efficiency of the terminal, and reducing the requirement on the hardware resources of the terminal, thereby reducing the use cost of the terminal.
  • step S20 is a second method of dividing the first YUV data into a preset number of video regions according to a preset rule and calculating a intersection of the video region and the first change region information by using a preset algorithm, according to an embodiment of the present disclosure. Flowchart of the steps to change the area information. Referring to FIG. 4, step S20 further includes:
  • Step S21 dividing the first YUV data into a preset number of video regions according to a preset rule
  • Step S22 creating a corresponding thread for each video area
  • Step S23 calculating, by using the thread and a preset algorithm, second change region information that intersects the video region with the first change region information.
  • the first YUV data is divided into a preset number of video regions according to the height or width of the resolution of the terminal display.
  • the first YUV data may be equally divided into n horizontal video regions according to the terminal display resolution height or divided into n vertical video regions according to the display resolution width.
  • the resolution of the terminal display is 1024 x 768.
  • the first YUV data is equally divided into n horizontal video regions by 768/n; when the resolution width is divided by the terminal, the first is performed with 1024/n.
  • the YUV data is equally divided into n longitudinal video regions.
  • a corresponding thread is created for each video area.
  • multi-thread processing is adopted, and the resources of the terminal CPU can be fully utilized, thereby improving the efficiency of the terminal system.
  • the second change area information that the video area intersects with the first change area information is calculated by using the thread and a preset algorithm.
  • Rect Z Since there may be no changes in certain areas of Rect Y, there is no intersection with Rect X. Therefore, in this embodiment, multi-threading and a preset algorithm are adopted, and Rect X and Rect Y are intersected to obtain Rect Z. At this time, the range of the Rect Z is smaller than that of the Rect X, so that the desktop can be further reduced. Real-time change of the range of the area, thereby shortening the transcoding time and improving the display efficiency of the terminal.
  • FIG. 5 is a flowchart of a step of transcoding a second YUV data corresponding to the second change region information into corresponding RGB data and displaying according to the RGB data, according to an embodiment of the present disclosure.
  • step S30 further includes:
  • Step S31 determining, according to the pre-stored hardware information, whether the second change area information supports hardware transcoding
  • Step S32 if the second change area information supports hardware transcoding, the second YUV data corresponding to the second change area information is transcoded into corresponding RGB data by means of hardware transcoding;
  • Step S33 if the second change area information does not support hardware transcoding, determine, according to the pre-stored hardware information, whether the second change area information supports assembly transcoding;
  • Step S34 if the second change area information supports assembly transcoding, the second YUV data corresponding to the second change area information is transcoded into corresponding RGB data by means of assembly transcoding;
  • Step S35 If the second change area information does not support the assembly transcoding, the second YUV data corresponding to the second change area information is transcoded into corresponding RGB data by means of software transcoding.
  • the terminal when the terminal receives the second change area information and performs video transcoding on the second YUV data corresponding to the second change area information, the terminal may obtain hardware information such as a pre-stored CPU and GPU.
  • the support situation automatically selects hardware transcoding, assembly transcoding and software transcoding, which can further improve the utilization of terminal hardware resources.
  • the terminal determines whether the second change area information supports hardware decoding according to the supported condition of the hardware information such as the pre-stored terminal CPU and the GPU. If the second change area information supports hardware transcoding, the hardware transcodes the hardware. The method further transcodes the second YUV data corresponding to the second change area information into corresponding RGB data; if the second change area information does not support hardware transcoding, further according to the pre-stored hardware information such as the CPU and the GPU.
  • the second change area information supports assembly transcoding; if the second change area information supports assembly transcoding, transcoding the second YUV data corresponding to the second change area information by means of assembly transcoding For the corresponding RGB data; if the second change region information does not support the assembly transcoding, the second YUV data corresponding to the second change region information is transcoded into the corresponding RGB data by means of software transcoding.
  • the hardware transcoding is performed by the GPU for transcoding
  • the assembly transcoding and software transcoding are transcoded by software and CPU.
  • hardware transcoding is faster than assembly transcoding and software transcoding, and does not take up too much CPU and memory.
  • Assembly transcoding is more efficient than software transcoding. Therefore, the order of judgment is to first determine whether to support hardware transcoding, and then determine whether to support assembly transcoding.
  • the present disclosure further provides a computer readable storage medium having a terminal display program stored thereon, the terminal display program being executed by the processor to implement the terminal display method according to any of the above embodiments. step.
  • portions of the technical solution of the present disclosure that contribute substantially or to the prior art may be embodied in the form of a software product stored in a storage medium (such as a ROM/RAM as described above). , a disk, an optical disk, including a number of instructions for causing a terminal device (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the methods described in various embodiments of the present disclosure.
  • a terminal device which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Controls And Circuits For Display Device (AREA)

Abstract

Provided are a terminal display method, comprising: receiving real-time video frame data and information about a first varying area in a desktop sent by a server, and decoding the real-time video frame data to obtain first colour code YUV data; dividing the first YUV data into a pre-set number of video areas according to a pre-set rule, and calculating, using a pre-set algorithm, information about a second varying area of the video area intersecting with the first varying area information; and transcoding second YUV data corresponding to the information about the second varying area into corresponding RGB data, and performing display according to the RGB data. Also disclosed are a terminal and a computer-readable storage medium.

Description

终端显示方法、终端及计算机可读存储介质Terminal display method, terminal and computer readable storage medium 技术领域Technical field
本公开涉及但不限于终端显示技术领域,尤其涉及一种终端显示方法、终端及计算机可读存储介质。The present disclosure relates to, but is not limited to, the field of terminal display technologies, and in particular, to a terminal display method, a terminal, and a computer readable storage medium.
背景技术Background technique
随着计算机技术的快速发展,个人电脑、掌上电脑、智能手机等终端设备的应用越来越广泛,成为人们身边不可缺少的设备。这些终端设备一般都具有显示功能。在接收到视频信息时,可通过解码和转码进而显示在终端中,因此,人们可以在终端上观看视频。With the rapid development of computer technology, the application of terminal devices such as personal computers, PDAs, and smart phones has become more and more widespread, and has become an indispensable device for people. These terminal devices generally have a display function. When the video information is received, it can be displayed in the terminal by decoding and transcoding, so that people can watch the video on the terminal.
但是,目前当全部视频信息中只有一部分发生变化时,终端也会重新对所有的视频信息进行解码和转码,进而显示在终端中。这种方式使得终端的显示效率比较低,而且对终端的硬件要求也比较高,因此增加了终端的使用成本。However, at present, when only a part of all video information changes, the terminal will re-decode and transcode all the video information, and then display it in the terminal. In this way, the display efficiency of the terminal is relatively low, and the hardware requirements of the terminal are relatively high, thereby increasing the use cost of the terminal.
发明内容Summary of the invention
本公开的主要目的在于提供一种终端的显示方法、终端及计算机可读存储介质。A main object of the present disclosure is to provide a display method of a terminal, a terminal, and a computer readable storage medium.
根据本公开的一个实施例,提供了一种终端的显示方法,包括:接收服务器发送的桌面中的实时视频帧数据及第一变化区域信息,并对所述实时视频帧数据进行解码,得到第一颜色编码YUV数据;将所述第一YUV数据按预设规则分成预设数量的视频区域,并利用预设算法计算出所述视频区域与所述第一变化区域信息相交的第二变化区域信息;以及将所述第二变化区域信息对应的第二YUV数据转码为对应的RGB数据,并根据所述RGB数据进行显示。According to an embodiment of the present disclosure, a display method of a terminal is provided, including: receiving real-time video frame data and first change area information in a desktop sent by a server, and decoding the real-time video frame data to obtain a Color coding YUV data; dividing the first YUV data into a preset number of video regions according to a preset rule, and calculating a second change region where the video region intersects with the first change region information by using a preset algorithm And transcoding the second YUV data corresponding to the second change region information into corresponding RGB data, and displaying according to the RGB data.
根据本公开的一个实施例,还提供一种终端,所述终端包括:存储器、处理器及存储在所述存储器上并可被所述处理器执行的显示程序,其中,所述显示程序被所述处理器执行时实现如上所述的终端显示方法的步骤。According to an embodiment of the present disclosure, there is also provided a terminal, the terminal comprising: a memory, a processor, and a display program stored on the memory and executable by the processor, wherein the display program is The steps of the terminal display method as described above are implemented when the processor is executed.
根据本公开的一个实施例,还提供一种计算机可读存储介质,所述计算机可读存储介质上存储有显示程序,其中,所述显示程序被处理器执行时实现如上所述的终端显示方法的步骤。According to an embodiment of the present disclosure, there is further provided a computer readable storage medium having stored thereon a display program, wherein the display program is executed by a processor to implement the terminal display method as described above A step of.
附图说明DRAWINGS
图1为根据本公开实施例的硬件运行环境(即,终端)的结构示意图;1 is a schematic structural diagram of a hardware operating environment (ie, a terminal) according to an embodiment of the present disclosure;
图2为根据本公开实施例的终端显示方法的流程图;2 is a flowchart of a terminal display method according to an embodiment of the present disclosure;
图3为根据本公开实施例的对所述实时视频帧数据进行解码以得到第一YUV数据的步骤的流程图;3 is a flowchart of steps of decoding the real-time video frame data to obtain first YUV data, in accordance with an embodiment of the present disclosure;
图4为根据本公开实施例的将所述第一YUV数据按预设规则分成预设数量的视频区域并利用预设算法计算出所述视频区域与所述第一变化区域信息相交的第二变化区域信息的步骤的流程图;以及4 is a second method of dividing the first YUV data into a preset number of video regions according to a preset rule and calculating a intersection of the video region and the first change region information by using a preset algorithm, according to an embodiment of the present disclosure. a flow chart of the steps of changing the area information;
图5为根据本公开实施例的将所述第二变化区域信息对应的第二颜色编码YUV数据转码为对应的RGB数据并根据所述RGB数据进行显示的的步骤的流程图。FIG. 5 is a flowchart of a step of transcoding second color coded YUV data corresponding to the second change region information into corresponding RGB data and displaying according to the RGB data, according to an embodiment of the present disclosure.
下面将参照附图并结合实施例对本公开的目的、功能特点及优点进行进一步说明。The objects, functional features and advantages of the present disclosure will be further described with reference to the accompanying drawings.
具体实施方式Detailed ways
应当理解,此处所描述的实施例仅仅用于解释本公开,并不用于限定本公开。It is understood that the embodiments described herein are merely illustrative of the disclosure and are not intended to limit the disclosure.
通常,当全部视频信息中只有一部分发生变化时,终端也会重新对所有的视频信息进行解码和转码,然后显示在终端中。这种方式使得终端的显示效率比较低,而且对终端的硬件要求也比较高,因此增加了终端的使用成本。特别对于云终端而言,处理的数据信息量更为庞大。因此,提高终端的显示效率是本领域人员亟待解决的技术问题。Generally, when only a part of all video information changes, the terminal will re-decode and transcode all the video information, and then display it in the terminal. In this way, the display efficiency of the terminal is relatively low, and the hardware requirements of the terminal are relatively high, thereby increasing the use cost of the terminal. Especially for cloud terminals, the amount of data processed is much larger. Therefore, improving the display efficiency of the terminal is a technical problem to be solved by those skilled in the art.
为了解决上述技术问题,本公开的一个实施例提供一种终端显示方法,包括:接收服务器发送的桌面中的实时视频帧数据及第一变化区域信息,并对所述实时视频帧数据进行解码,得到第一颜色编码YUV 数据;将所述第一YUV数据按预设规则分成预设数量的视频区域,并利用预设算法计算出所述视频区域与所述第一变化区域信息相交的第二变化区域信息;将所述第二变化区域信息对应的第二YUV数据转码为对应的RGB数据,并根据所述RGB数据进行显示。按照上述方式,根据本公开的实施例提供的终端显示方法,接收服务器中发送的桌面中的实时视频帧数据及第一变化区域信息,根据预设算法计算出两者相交的第二变化区域信息,并只将解码后所述第二变化区域信息对应的第二YUV数据转码为对应的RGB数据。终端只需要将第二变化区域信息进行转码即可完成显示。因此,可以减少终端从接收桌面视频数据到显示的时间,从而提高终端的显示效率。同时,还可以减少对终端硬件资源的要求,从而降低终端的使用成本。In order to solve the above technical problem, an embodiment of the present disclosure provides a terminal display method, including: receiving real-time video frame data and first change area information in a desktop sent by a server, and decoding the real-time video frame data, Obtaining first color coded YUV data; dividing the first YUV data into a preset number of video regions according to a preset rule, and calculating, by using a preset algorithm, a second intersection of the video region and the first change region information Changing the region information; transcoding the second YUV data corresponding to the second change region information into corresponding RGB data, and displaying the data according to the RGB data. According to the above manner, the terminal display method according to the embodiment of the present disclosure receives the real-time video frame data and the first change region information in the desktop sent by the server, and calculates the second change region information that the two intersect according to the preset algorithm. And transcoding only the second YUV data corresponding to the second change region information after decoding into corresponding RGB data. The terminal only needs to transcode the second changed area information to complete the display. Therefore, the time from the reception of the desktop video data to the display by the terminal can be reduced, thereby improving the display efficiency of the terminal. At the same time, the requirements on the hardware resources of the terminal can be reduced, thereby reducing the use cost of the terminal.
如图1所示,图1是根据本公开实施例的硬件运行环境(即,终端)的结构示意图。As shown in FIG. 1, FIG. 1 is a schematic structural diagram of a hardware operating environment (ie, a terminal) according to an embodiment of the present disclosure.
在本公开实施例中的终端可以是PC,也可以是智能手机、平板电脑、便携计算机等具有显示功能的可移动式终端设备,或者还可以是云终端。The terminal in the embodiment of the present disclosure may be a PC, or may be a mobile terminal device having a display function such as a smart phone, a tablet computer, a portable computer, or may be a cloud terminal.
如图1所示,该终端可以包括:处理器1001(例如,CPU)、网络接口1004、用户接口1003、存储器1005以及通信总线1002。通信总线1002用于实现这些组件之间的连接通信。用户接口1003可以包括显示屏(Display)、输入单元比如键盘(Keyboard),用户接口1003可选地可以包括标准的有线接口、无线接口。网络接口1004可选地可以包括标准的有线接口、无线接口(如WI-FI接口)。存储器1005可以是高速RAM存储器,也可以是稳定的存储器(non-volatile memory),例如磁盘存储器。存储器1005可选地可以是独立于前述处理器1001的存储装置。As shown in FIG. 1, the terminal may include a processor 1001 (eg, a CPU), a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. Communication bus 1002 is used to implement connection communication between these components. The user interface 1003 can include a display, an input unit such as a keyboard, and the user interface 1003 can optionally include a standard wired interface, a wireless interface. Network interface 1004 can optionally include a standard wired interface, a wireless interface (such as a WI-FI interface). The memory 1005 may be a high speed RAM memory or a non-volatile memory such as a disk memory. The memory 1005 can optionally be a storage device that is separate from the aforementioned processor 1001.
在一个示例性实施例中,终端还可以包括摄像头、RF(Radio Frequency,射频)电路、传感器、音频电路、Wi-Fi模块等等。其中,传感器比如包括光传感器、运动传感器或者其他任何适当传感器。具体地,光传感器可包括环境光传感器及接近传感器,其中,环境光传 感器可根据环境光线的明暗来调节显示屏的亮度,接近传感器可在移动终端移动到耳边时,关闭显示屏和/或背光。作为一种运动传感器,例如,重力加速度传感器被配制成检测各个方向上(一般为三轴)的加速度的大小,静止时检测出重力的大小,识别移动终端姿态的应用(比如横竖屏切换、相关游戏、磁力计姿态校准)以及识别振动(比如计步器、敲击)。当然,移动终端还可配置陀螺仪、气压计、湿度计、温度计、红外线传感器等其他传感器,在此不再赘述。In an exemplary embodiment, the terminal may further include a camera, an RF (Radio Frequency) circuit, a sensor, an audio circuit, a Wi-Fi module, and the like. Wherein the sensor comprises, for example, a light sensor, a motion sensor or any other suitable sensor. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display according to the brightness of the ambient light, and the proximity sensor may turn off the display and/or when the mobile terminal moves to the ear. Backlighting. As a motion sensor, for example, a gravity acceleration sensor is configured to detect the magnitude of acceleration in each direction (generally three axes), the magnitude of gravity is detected at rest, and the application of the gesture of the mobile terminal is recognized (eg, horizontal and vertical screen switching, correlation) Game, magnetometer attitude calibration) and recognition of vibration (such as pedometer, tapping). Of course, the mobile terminal can also be configured with other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, and the like, and details are not described herein again.
本领域技术人员可以理解,图1中示出的终端结构并不构成对终端的限定,可以包括比图示更多或更少的部件,或者组合某些部件,或者不同的部件布置。It will be understood by those skilled in the art that the terminal structure shown in FIG. 1 does not constitute a limitation to the terminal, and may include more or less components than those illustrated, or a combination of certain components, or different component arrangements.
如图1所示,作为一种计算机存储介质的存储器1005可以安装有操作系统、网络通信模块、用户接口模块以及终端的显示程序。As shown in FIG. 1, a memory 1005 as a computer storage medium may be installed with an operating system, a network communication module, a user interface module, and a display program of the terminal.
在图1所示的终端中,网络接口1004主要被配置成连接后台服务器,与后台服务器进行数据通信;用户接口1003主要被配置成连接客户端(用户端),与客户端进行数据通信;而处理器1001可以被配置成调用存储器1005中存储的终端的显示程序,并执行以下操作:In the terminal shown in FIG. 1, the network interface 1004 is mainly configured to connect to the background server for data communication with the background server; the user interface 1003 is mainly configured to connect to the client (user end) to perform data communication with the client; The processor 1001 may be configured to call a display program of the terminal stored in the memory 1005 and perform the following operations:
接收服务器发送的桌面中的实时视频帧数据及第一变化区域信息,并对所述实时视频帧数据进行解码,得到第一颜色编码YUV数据;Receiving real-time video frame data and first change area information in the desktop sent by the server, and decoding the real-time video frame data to obtain first color coded YUV data;
将所述第一YUV数据按预设规则分成预设数量的视频区域,并利用预设算法计算出所述视频区域与所述第一变化区域信息相交的第二变化区域信息;以及Dividing the first YUV data into a preset number of video regions according to a preset rule, and calculating, by using a preset algorithm, second change region information that intersects the video region with the first change region information;
将所述第二变化区域信息对应的第二YUV数据转码为对应的RGB数据,并根据所述RGB数据进行显示。The second YUV data corresponding to the second change region information is transcoded into corresponding RGB data, and displayed according to the RGB data.
进一步地,所述处理器1001可以被配置成调用存储器1005中存储的终端显示程序,并执行以下操作:Further, the processor 1001 may be configured to call a terminal display program stored in the memory 1005 and perform the following operations:
根据所述实时视频帧数据的格式信息及预存硬件信息判断所述实时视频帧数据是否支持硬件解码;Determining, according to the format information of the real-time video frame data and the pre-stored hardware information, whether the real-time video frame data supports hardware decoding;
若所述实时视频帧数据支持硬件解码,则采用硬件解码的方式对所述实时视频帧数据进行解码,得到第一YUV数据;If the real-time video frame data supports hardware decoding, decoding the real-time video frame data by using hardware decoding to obtain first YUV data;
若所述实时视频帧数据不支持硬件解码,则采用软件解码的方式 对所述实时视频帧数据进行解码,得到第一YUV数据。If the real-time video frame data does not support hardware decoding, the real-time video frame data is decoded in a software decoding manner to obtain first YUV data.
进一步地,所述处理器1001可以被配置成调用存储器1005中存储的终端的显示程序,并执行以下操作:Further, the processor 1001 may be configured to call a display program of the terminal stored in the memory 1005 and perform the following operations:
将所述第一YUV数据按预设规则分成预设数量的视频区域;Dividing the first YUV data into a preset number of video regions according to a preset rule;
对每个视频区域创建一对应的线程;Create a corresponding thread for each video area;
利用所述线程及预设算法计算出所述视频区域与所述第一变化区域信息相交的第二变化区域信息。And calculating, by the thread and a preset algorithm, second change area information that intersects the video area with the first change area information.
进一步地,所述处理器1001可以被配置成存储器1005中存储的终端的显示程序,并执行以下操作:Further, the processor 1001 may be configured as a display program of a terminal stored in the memory 1005 and perform the following operations:
将所述第一YUV数据按终端显示分辨率的高度或宽度等分成预设数量的视频区域。The first YUV data is divided into a preset number of video regions according to the height or width of the terminal display resolution.
进一步地,所述处理器1001可以被配置成调用存储器1005中存储的终端的显示程序,并执行以下操作:Further, the processor 1001 may be configured to call a display program of the terminal stored in the memory 1005 and perform the following operations:
根据所述预存硬件信息判断所述第二变化区域信息是否支持硬件转码;Determining, according to the pre-stored hardware information, whether the second change area information supports hardware transcoding;
若所述第二变化区域信息支持硬件转码,则采用硬件转码的方式将所述第二变化区域信息对应的第二YUV数据转码为对应的RGB数据。If the second change area information supports hardware transcoding, the second YUV data corresponding to the second change area information is transcoded into corresponding RGB data by using hardware transcoding.
进一步地,所述处理器1001可以被配置成调用存储器1005中存储的终端的显示程序,并执行以下操作:Further, the processor 1001 may be configured to call a display program of the terminal stored in the memory 1005 and perform the following operations:
若所述第二变化区域信息不支持硬件转码,则根据所述预存硬件信息判断所述第二变化区域信息是否支持汇编转码;If the second change area information does not support hardware transcoding, determining, according to the pre-stored hardware information, whether the second change area information supports assembly transcoding;
若所述第二变化区域信息支持汇编转码,则采用汇编转码的方式将所述第二变化区域信息对应的第二YUV数据转码为对应的RGB数据;If the second change area information supports the assembly transcoding, the second YUV data corresponding to the second change area information is transcoded into corresponding RGB data by means of assembly transcoding;
若所述第二变化区域信息不支持汇编转码,则采用软件转码的方式将所述第二变化区域信息对应的第二YUV数据转码为对应的RGB数据。If the second change area information does not support the assembly transcoding, the second YUV data corresponding to the second change area information is transcoded into corresponding RGB data by means of software transcoding.
进一步地,所述处理器1001可以被配置成调用存储器1005中存储的终端的显示程序,并执行以下操作:Further, the processor 1001 may be configured to call a display program of the terminal stored in the memory 1005 and perform the following operations:
接收服务器发送的桌面中的实时视频帧数据,及服务器通过预设程序抓取的桌面中的第一变化区域信息。Receiving real-time video frame data in the desktop sent by the server, and first change area information in the desktop captured by the server through a preset program.
进一步地,所述处理器1001可以被配置成调用存储器1005中存储的终端的显示程序,并执行以下操作:Further, the processor 1001 may be configured to call a display program of the terminal stored in the memory 1005 and perform the following operations:
所述第一变化区域信息、所述第二变化区域信息及所述视频区域均包含分别表示对应的变化区域在所述桌面中的上、左、右、下坐标信息的参数。The first change area information, the second change area information, and the video area each include parameters indicating upper, left, right, and lower coordinate information of the corresponding change area in the desktop.
请参阅图2,图2为根据本公开实施例的终端显示方法的流程图。Please refer to FIG. 2. FIG. 2 is a flowchart of a terminal display method according to an embodiment of the present disclosure.
在本实施例中,所述终端显示方法包括:In this embodiment, the terminal display method includes:
步骤S10,接收服务器发送的桌面中的实时视频帧数据及第一变化区域信息,并对所述实时视频帧数据进行解码,得到第一YUV数据。Step S10: Receive real-time video frame data and first change area information in the desktop sent by the server, and decode the real-time video frame data to obtain first YUV data.
在本实施例中,启动终端,使终端与服务器建立网络连接,同时约定相应消息通信数据结构格式。进一步地,终端接收到服务器发送的桌面中的实时视频帧数据及第一变化区域信息之后,会根据该实时视频帧数据的格式信息及预存的终端硬件信息对该实时视频帧数据进行解码,从而得到对应的第一颜色编码(Luminance,Chrominance;亮度,色度)YUV数据。在一个示例性实施例中,所述服务器可以是云服务器,也可以是普通的物理服务器;所述桌面可以是云桌面,也可以是PC、平板电脑等对应的桌面。此外,所述服务器包含所述桌面或与所述桌面相连接,例如云服务器包含云桌面,物理服务器与电脑桌面相连接。In this embodiment, the terminal is activated to establish a network connection between the terminal and the server, and at the same time, the corresponding message communication data structure format is agreed. Further, after receiving the real-time video frame data and the first change area information in the desktop sent by the server, the terminal decodes the real-time video frame data according to the format information of the real-time video frame data and the pre-stored terminal hardware information, thereby A corresponding first color coding (Luminance, Chrominance; luminance, chrominance) YUV data is obtained. In an exemplary embodiment, the server may be a cloud server or a common physical server; the desktop may be a cloud desktop, or may be a corresponding desktop such as a PC or a tablet. In addition, the server includes or is connected to the desktop, for example, the cloud server includes a cloud desktop, and the physical server is connected to the desktop of the computer.
图3为本公开实施例中对所述实时视频帧数据进行解码,得到第一YUV数据的流程示意图。请参阅图3,步骤S10进一步包括:FIG. 3 is a schematic flowchart of decoding the real-time video frame data to obtain first YUV data according to an embodiment of the present disclosure. Referring to FIG. 3, step S10 further includes:
步骤S11,根据所述实时视频帧数据的格式信息及预存硬件信息判断所述实时视频帧数据是否支持硬件解码;Step S11, determining, according to format information of the real-time video frame data and pre-stored hardware information, whether the real-time video frame data supports hardware decoding;
步骤S12,若所述实时视频帧数据支持硬件解码,则采用硬件解码的方式对所述实时视频帧数据进行解码,得到第一YUV数据;以及Step S12, if the real-time video frame data supports hardware decoding, decoding the real-time video frame data by using hardware decoding to obtain first YUV data;
步骤S13,若所述实时视频帧数据不支持硬件解码,则采用软件解码的方式对所述实时视频帧数据进行解码,得到第一YUV数据。Step S13: If the real-time video frame data does not support hardware decoding, the real-time video frame data is decoded by using software decoding to obtain first YUV data.
在本实施例中,终端接收到服务器发送的当前桌面的实时视频帧数据及第一变化区域信息,其中,所述第一变化区域信息是服务器通过预设程序在桌面中抓取到的。进一步地,终端会根据该实时视频帧数据的格式信息及预存的终端硬件信息,如中央处理器CPU和显卡GPU,来判断所述实时视频帧数据是否支持硬件解码,若所述实时视频帧数据支持硬件解码,则采用硬件解码的方式对所述实时视频帧数据进行解码,得到第一YUV数据;若所述实时视频帧数据不支持硬件解码,则采用软件解码的方式对所述实时视频帧数据进行解码,得到第一YUV数据。In this embodiment, the terminal receives the real-time video frame data and the first change area information of the current desktop sent by the server, where the first change area information is captured by the server in the desktop by using a preset program. Further, the terminal determines, according to format information of the real-time video frame data and pre-stored terminal hardware information, such as a central processing unit CPU and a graphics card GPU, whether the real-time video frame data supports hardware decoding, if the real-time video frame data Supporting hardware decoding, the hardware decoding method is used to decode the real-time video frame data to obtain first YUV data; if the real-time video frame data does not support hardware decoding, the real-time video frame is obtained by software decoding. The data is decoded to obtain the first YUV data.
这里,硬件解码是通过GPU运算对视频进行解码处理,而软件解码是通过软件使CPU对视频进行解码处理。因此,终端在判断所述实时视频帧数据是否支持硬件解码时,应同时考虑该视频数据的格式信息和预存的GPU和CPU等硬件信息的支持情况。当两者均支持硬件解码时,则采用硬件解码的方式对所述实时视频帧数据进行解码;当两者中有一个或两者均不支持硬件解码时,则采用软件解码的方式对所述实时视频帧数据进行解码。Here, the hardware decoding is to decode the video by GPU operation, and the software decoding is to enable the CPU to decode the video through software. Therefore, when determining whether the real-time video frame data supports hardware decoding, the terminal should consider both the format information of the video data and the pre-stored hardware information such as the GPU and the CPU. When both of them support hardware decoding, the real-time video frame data is decoded by means of hardware decoding; when one or both of the two do not support hardware decoding, the software decodes the Real-time video frame data is decoded.
在本公开实施例中,在对实时视频帧数据进行解码时,终端根据视频数据的格式信息及预存的终端CPU和GPU等硬件信息支持情况,分别采用硬件解码和软件解码的方式对所述实时视频帧数据进行解码,可提高对终端硬件资源的利用率。In the embodiment of the present disclosure, when decoding the real-time video frame data, the terminal uses the hardware decoding and the software decoding manner to perform the real-time according to the format information of the video data and the pre-stored hardware information support conditions such as the CPU and the GPU. The video frame data is decoded to improve the utilization of the terminal hardware resources.
在本实施例中,所述终端显示方法还包括:步骤S20,将所述第一YUV数据按预设规则分成预设数量的视频区域,并利用预设算法计算出所述视频区域与所述第一变化区域信息相交的第二变化区域信息;In this embodiment, the terminal display method further includes: step S20, dividing the first YUV data into a preset number of video regions according to a preset rule, and calculating, by using a preset algorithm, the video region and the Second change region information intersecting the first change region information;
在本实施例中,将所述第一YUV数据按终端显示器的分辨率的高度或宽度等分成预设数量的视频区域。具体地,可以将所述第一YUV数据按终端显示分辨率高度等分成n个横向视频区域或按显示分辨率宽度等分成n个纵向视频区域。例如,终端显示器的分辨率为1024×768,当按终端显示分辨率高度分时,则用768/n将所述第一YUV数据等分成n个横向视频区域;当按终端显示分辨率宽度分时,则用1024/n将所述第一YUV数据等分成n个纵向视频区域。In this embodiment, the first YUV data is divided into a preset number of video regions according to the height or width of the resolution of the terminal display. Specifically, the first YUV data may be equally divided into n horizontal video regions according to the terminal display resolution height or divided into n vertical video regions according to the display resolution width. For example, the resolution of the terminal display is 1024×768, and when the resolution is highly divided by the terminal display, the first YUV data is equally divided into n horizontal video regions by 768/n; Then, the first YUV data is equally divided into n longitudinal video regions by 1024/n.
进一步地,利用预设算法计算出所述视频区域与所述第一变化区域信息相交的第二变化区域信息。为了方便说明,我们将所述第一变化区域信息记为Rect X(X=1,2,…,N),所述视频区域记为Rect Y(Y=1,2,…,n),所述第二变化区域信息记为Rect Z(Z=1,2,…,M),其中,N,n,M分别代表对应区域信息的区域数。在计算Rect X与Rect Y的相交区域时,采用公式“Rect X and Rect Y”,得到Rect Y与Rect X的相交区域Rect Z。由于Rect Y中的某些区域可能没有变化,与Rect X无交集,因此,在本公开实施例中,采用了预设算法将Rect X与Rect Y相交得到Rect Z,此时Rect Z的区域范围相比于Rect X更小。因此,可以进一步地缩小桌面实时变化区域的范围,从而缩短转码时间,提高终端显示效率。Further, the second change region information that the video region intersects with the first change region information is calculated by using a preset algorithm. For convenience of explanation, we record the first change region information as Rect X (X=1, 2, . . . , N), and the video region is denoted as Rect Y (Y=1, 2, . . . , n). The second change region information is denoted as Rect Z (Z = 1, 2, ..., M), where N, n, M represent the number of regions corresponding to the region information, respectively. When calculating the intersection of Rect X and Rect Y, the equation "Rect X and Rect Y" is used to obtain the intersection region Rect Z of Rect Y and Rect X. Since some areas in Rect Y may not change, there is no intersection with Rect X. Therefore, in the embodiment of the present disclosure, a preset algorithm is used to intersect Rect X with Rect Y to obtain Rect Z, and the region range of Rect Z at this time Smaller than Rect X. Therefore, the range of the real-time changing area of the desktop can be further reduced, thereby shortening the transcoding time and improving the display efficiency of the terminal.
在本实施例中,所述终端显示方法还包括:步骤S30,将所述第二变化区域信息对应的第二YUV数据转码为对应的RGB数据,并根据所述RGB数据进行显示。In this embodiment, the terminal display method further includes: Step S30, transcoding the second YUV data corresponding to the second change area information into corresponding RGB data, and performing display according to the RGB data.
在本实施例中,终端会根据预存的终端CPU和GPU等硬件信息的支持情况判断所述第二变化区域信息是否支持硬件解码,若所述第二变化区域信息支持硬件转码,则采用硬件转码的方式将所述第二变化区域信息对应的第二YUV数据转码为对应的RGB数据;若所述第二变化区域信息不支持硬件转码,则进一步根据所述预存的CPU和GPU等硬件信息判断所述第二变化区域信息是否支持汇编转码;若所述第二变化区域信息支持汇编转码,则采用汇编转码的方式将所述第二变化区域信息对应的第二YUV数据转码为对应的RGB数据;若所述第二变化区域信息不支持汇编转码,则采用软件转码的方式将所述第二变化区域信息对应的第二YUV数据转码为对应的RGB数据。In this embodiment, the terminal determines whether the second change area information supports hardware decoding according to the support situation of the hardware information such as the pre-stored terminal CPU and the GPU. If the second change area information supports hardware transcoding, the hardware is adopted. Transcoding, the second YUV data corresponding to the second change area information is transcoded into corresponding RGB data; if the second change area information does not support hardware transcoding, further according to the pre-stored CPU and GPU And determining, by the hardware information, whether the second change area information supports assembly transcoding; if the second change area information supports assembly transcoding, the second YUV corresponding to the second change area information is used in an assembly transcoding manner. Data is transcoded into corresponding RGB data; if the second change region information does not support assembly transcoding, the second YUV data corresponding to the second change region information is transcoded into corresponding RGB by software transcoding. data.
其中,YUV是被欧洲电视系统所采用的一种颜色编码方法(属于PAL)。YUV主要用于优化彩色视频信号的传输,使其向后兼容老式黑白电视。与RGB视频信号传输相比,它最大的优点在于只需占用极少的带宽(RGB要求三个独立的视频信号同时传输)。Among them, YUV is a color coding method (belonging to PAL) adopted by European television systems. YUV is mainly used to optimize the transmission of color video signals, making it backward compatible with old-fashioned black-and-white TVs. Compared with RGB video signal transmission, its biggest advantage is that it requires very little bandwidth (RGB requires three independent video signals to be transmitted simultaneously).
YUV中"Y"表示明亮度(Luminance或Luma),也就是灰阶值,是个基带信号。而"U"和"V"表示的则是色度(Chrominance或Chroma), 作用是描述影像色彩及饱和度,用于指定像素的颜色。U和V是被正交调制的信号,而不是基带信号。"Y" in YUV means brightness (Luminance or Luma), which is the gray level value, which is a baseband signal. The "U" and "V" represent the chroma (Chrominance or Chroma), which is used to describe the color and saturation of the image and is used to specify the color of the pixel. U and V are signals that are orthogonally modulated, rather than baseband signals.
通过转码,YUV的三个分量可以还原出R(红),G(绿),B(蓝)。具体的,YUV与RGB之间的转换公式可以为:By transcoding, the three components of YUV can restore R (red), G (green), and B (blue). Specifically, the conversion formula between YUV and RGB can be:
R=Y+1.140*V,G=Y-0.394*U-0.581*V,B=Y+2.032*U。R = Y + 1.140 * V, G = Y - 0.394 * U - 0.581 * V, B = Y + 2.032 * U.
进一步地,将所述RGB(Red,Green,Blue;红,绿,蓝)数据转化为图像,以在终端的显示器上进行刷新和显示。这里,可以利用Matlab等程序将RGB数据转化为对应的RGB图像。Further, the RGB (Red, Green, Blue; Red, Green, Blue) data is converted into an image for refreshing and display on the display of the terminal. Here, RGB data can be converted into corresponding RGB images by using a program such as Matlab.
此外,需要说明的是,在本实施例中,所述第一变化区域信息、所述第二变化区域信息及所述视频区域均包含{top,left,right,bottom}四个参数,分别表示对应的变化区域在所述桌面中的上、左、右、下坐标信息。该坐标信息可用于准确定位所述第一变化区域信息、所述第二变化区域信息及所述视频区域在桌面中分别对应的区域。In addition, in this embodiment, the first change area information, the second change area information, and the video area each include four parameters {top, left, right, bottom}, respectively Corresponding change regions are upper, left, right, and lower coordinate information in the desktop. The coordinate information can be used to accurately locate the first change area information, the second change area information, and an area corresponding to the video area in the desktop.
本公开实施例通过接收服务器发送的桌面中的实时视频帧数据及第一变化区域信息,并对所述实时视频帧数据进行解码,得到第一YUV数据;将所述第一YUV数据按预设规则分成预设数量的视频区域,并利用预设算法计算出所述视频区域与所述第一变化区域信息相交的第二变化区域信息;将所述第二变化区域信息对应的第二YUV数据转码为对应的RGB数据,并根据所述RGB数据进行显示。通过上述方式,本公开接收服务器中发送的桌面中的实时视频帧数据及第一变化区域信息,并根据预设算法计算出两者相交的第二变化区域信息,并只将解码后所述第二变化区域信息对应的第二YUV数据转码为对应的RGB数据,从而终端只需要将第二变化区域信息进行转码即可完成显示,相比于现有技术中对所有视频区域信息进行转码显示的方式,可减少终端从接收桌面视频数据到显示的时间,从而提高终端的显示效率,同时减少对终端硬件资源的要求,从而降低终端的使用成本。The embodiment of the present disclosure receives the real-time video frame data and the first change area information in the desktop sent by the server, and decodes the real-time video frame data to obtain the first YUV data; and presets the first YUV data according to the preset Dividing the rule into a preset number of video regions, and calculating, by using a preset algorithm, second change region information that intersects the video region with the first change region information; and second YUV data corresponding to the second change region information Transcoded into corresponding RGB data and displayed according to the RGB data. In the above manner, the present disclosure receives the real-time video frame data and the first change area information in the desktop sent by the server, and calculates the second change area information that the two intersect according to the preset algorithm, and only decodes the first The second YUV data corresponding to the two change area information is transcoded into the corresponding RGB data, so that the terminal only needs to transcode the second change area information to complete the display, and compares all the video area information in the prior art. The method of code display can reduce the time from the receiving of the desktop video data to the display by the terminal, thereby improving the display efficiency of the terminal, and reducing the requirement on the hardware resources of the terminal, thereby reducing the use cost of the terminal.
图4为根据本公开实施例的将所述第一YUV数据按预设规则分成预设数量的视频区域并利用预设算法计算出所述视频区域与所述第 一变化区域信息相交的第二变化区域信息的步骤的流程图。请参阅图4,步骤S20进一步包括:4 is a second method of dividing the first YUV data into a preset number of video regions according to a preset rule and calculating a intersection of the video region and the first change region information by using a preset algorithm, according to an embodiment of the present disclosure. Flowchart of the steps to change the area information. Referring to FIG. 4, step S20 further includes:
步骤S21,将所述第一YUV数据按预设规则分成预设数量的视频区域;Step S21, dividing the first YUV data into a preset number of video regions according to a preset rule;
步骤S22,对每个视频区域创建一对应的线程;Step S22, creating a corresponding thread for each video area;
步骤S23,利用所述线程及预设算法计算出所述视频区域与所述第一变化区域信息相交的第二变化区域信息。Step S23, calculating, by using the thread and a preset algorithm, second change region information that intersects the video region with the first change region information.
在本实施例中,将所述第一YUV数据按终端显示器的分辨率的高度或宽度等分成预设数量的视频区域。具体地,可以将所述第一YUV数据按终端显示分辨率高度等分成n个横向视频区域或按显示分辨率宽度等分成n个纵向视频区域。例如,终端显示器的分辨率为1024×768。当按终端显示分辨率高度分时,则用768/n将所述第一YUV数据等分成n个横向视频区域;当按终端显示分辨率宽度分时,则用1024/n将所述第一YUV数据等分成n个纵向视频区域。In this embodiment, the first YUV data is divided into a preset number of video regions according to the height or width of the resolution of the terminal display. Specifically, the first YUV data may be equally divided into n horizontal video regions according to the terminal display resolution height or divided into n vertical video regions according to the display resolution width. For example, the resolution of the terminal display is 1024 x 768. When the resolution is highly divided by the terminal display, the first YUV data is equally divided into n horizontal video regions by 768/n; when the resolution width is divided by the terminal, the first is performed with 1024/n. The YUV data is equally divided into n longitudinal video regions.
进一步地,对每个视频区域创建一对应的线程。这里,采用了多线程处理,可充分利用终端CPU的资源,从而提高终端系统的效率。Further, a corresponding thread is created for each video area. Here, multi-thread processing is adopted, and the resources of the terminal CPU can be fully utilized, thereby improving the efficiency of the terminal system.
进一步地,利用所述线程及预设算法计算出所述视频区域与所述第一变化区域信息相交的第二变化区域信息。Further, the second change area information that the video area intersects with the first change area information is calculated by using the thread and a preset algorithm.
为了方便说明,我们将所述第一变化区域信息记为Rect X(X=1,2,…,N),所述视频区域记为Rect Y(Y=1,2,…,n),所述第二变化区域信息记为Rect Z(Z=1,2,…,M),其中,N,n,M分别代表对应区域信息的区域数。在计算Rect X与Rect Y的相交区域时,对Rect Y区域中的n个区域分别创建n个对应的线程,并利用该线程和公式“Rect X and Rect Y”,得到Rect Y与Rect X的相交区域Rect Z。由于在Rect Y的某些区域可能没有变化,与Rect X无交集。因此,在本实施例中,采用了多线程和预设算法,将Rect X与Rect Y相交得到Rect Z,此时Rect Z的区域范围相比于Rect X更小,因此,可以进一步地缩小桌面实时变化区域的范围,从而缩短转码时间,提高终端显示效率。For convenience of explanation, we record the first change region information as Rect X (X=1, 2, . . . , N), and the video region is denoted as Rect Y (Y=1, 2, . . . , n). The second change region information is denoted as Rect Z (Z = 1, 2, ..., M), where N, n, M represent the number of regions corresponding to the region information, respectively. When calculating the intersection of Rect X and Rect Y, create n corresponding threads for n regions in the Rect Y region, and use the thread and the formula "Rect X and Rect Y" to get Rect Y and Rect X. Intersecting area Rect Z. Since there may be no changes in certain areas of Rect Y, there is no intersection with Rect X. Therefore, in this embodiment, multi-threading and a preset algorithm are adopted, and Rect X and Rect Y are intersected to obtain Rect Z. At this time, the range of the Rect Z is smaller than that of the Rect X, so that the desktop can be further reduced. Real-time change of the range of the area, thereby shortening the transcoding time and improving the display efficiency of the terminal.
图5为根据本公开实施例的将所述第二变化区域信息对应的第二YUV数据转码为对应的RGB数据并根据所述RGB数据进行显示的步骤的流程图。请参阅图5,步骤S30进一步包括:FIG. 5 is a flowchart of a step of transcoding a second YUV data corresponding to the second change region information into corresponding RGB data and displaying according to the RGB data, according to an embodiment of the present disclosure. Referring to FIG. 5, step S30 further includes:
步骤S31,根据所述预存硬件信息判断所述第二变化区域信息是否支持硬件转码;Step S31, determining, according to the pre-stored hardware information, whether the second change area information supports hardware transcoding;
步骤S32,若所述第二变化区域信息支持硬件转码,则采用硬件转码的方式将所述第二变化区域信息对应的第二YUV数据转码为对应的RGB数据;Step S32, if the second change area information supports hardware transcoding, the second YUV data corresponding to the second change area information is transcoded into corresponding RGB data by means of hardware transcoding;
步骤S33,若所述第二变化区域信息不支持硬件转码,则根据所述预存硬件信息判断所述第二变化区域信息是否支持汇编转码;Step S33, if the second change area information does not support hardware transcoding, determine, according to the pre-stored hardware information, whether the second change area information supports assembly transcoding;
步骤S34,若所述第二变化区域信息支持汇编转码,则采用汇编转码的方式将所述第二变化区域信息对应的第二YUV数据转码为对应的RGB数据;Step S34, if the second change area information supports assembly transcoding, the second YUV data corresponding to the second change area information is transcoded into corresponding RGB data by means of assembly transcoding;
步骤S35,若所述第二变化区域信息不支持汇编转码,则采用软件转码的方式将所述第二变化区域信息对应的第二YUV数据转码为对应的RGB数据。Step S35: If the second change area information does not support the assembly transcoding, the second YUV data corresponding to the second change area information is transcoded into corresponding RGB data by means of software transcoding.
在本实施例中,当终端接收到所述第二变化区域信息,并对所述第二变化区域信息对应的第二YUV数据进行视频转码时,终端会根据预存的CPU和GPU等硬件信息的支持情况自动选择硬件转码、汇编转码和软件转码方式,可进一步提高对终端硬件资源的利用率。In this embodiment, when the terminal receives the second change area information and performs video transcoding on the second YUV data corresponding to the second change area information, the terminal may obtain hardware information such as a pre-stored CPU and GPU. The support situation automatically selects hardware transcoding, assembly transcoding and software transcoding, which can further improve the utilization of terminal hardware resources.
具体地,终端会根据预存的终端CPU和GPU等硬件信息的支持情况判断所述第二变化区域信息是否支持硬件解码,若所述第二变化区域信息支持硬件转码,则采用硬件转码的方式将所述第二变化区域信息对应的第二YUV数据转码为对应的RGB数据;若所述第二变化区域信息不支持硬件转码,则进一步根据所述预存的CPU和GPU等硬件信息判断所述第二变化区域信息是否支持汇编转码;若所述第二变化区域信息支持汇编转码,则采用汇编转码的方式将所述第二变化区域信息对应的第二YUV数据转码为对应的RGB数据;若所述第二变化区域信息不支持汇编转码,则采用软件转码的方式将所述第二变化区域信息对应的第二YUV数据转码为对应的RGB数据。Specifically, the terminal determines whether the second change area information supports hardware decoding according to the supported condition of the hardware information such as the pre-stored terminal CPU and the GPU. If the second change area information supports hardware transcoding, the hardware transcodes the hardware. The method further transcodes the second YUV data corresponding to the second change area information into corresponding RGB data; if the second change area information does not support hardware transcoding, further according to the pre-stored hardware information such as the CPU and the GPU. Determining whether the second change area information supports assembly transcoding; if the second change area information supports assembly transcoding, transcoding the second YUV data corresponding to the second change area information by means of assembly transcoding For the corresponding RGB data; if the second change region information does not support the assembly transcoding, the second YUV data corresponding to the second change region information is transcoded into the corresponding RGB data by means of software transcoding.
这里,硬件转码是通过GPU来进行转码操作,而汇编转码和软件转码均是通过软件和CPU进行转码操作。相对而言,硬件转码比汇编转码和软件转码的速度更快,同时不会占用太多的CPU和内存。而汇编转码相比于软件转码效率更高。因此,判断的先后顺序为先判断是否支持硬件转码,再判断是否支持汇编转码。Here, the hardware transcoding is performed by the GPU for transcoding, and the assembly transcoding and software transcoding are transcoded by software and CPU. In contrast, hardware transcoding is faster than assembly transcoding and software transcoding, and does not take up too much CPU and memory. Assembly transcoding is more efficient than software transcoding. Therefore, the order of judgment is to first determine whether to support hardware transcoding, and then determine whether to support assembly transcoding.
本公开还提出一种计算机可读存储介质,该计算机可读存储介质上存储有终端显示程序,所述终端显示程序被处理器执行时实现如以上任一项实施例所述的终端显示方法的步骤。The present disclosure further provides a computer readable storage medium having a terminal display program stored thereon, the terminal display program being executed by the processor to implement the terminal display method according to any of the above embodiments. step.
本公开计算机可读存储介质的具体实施例与上述终端显示方法各实施例基本相同,在此不作赘述。The specific embodiment of the computer readable storage medium of the present disclosure is substantially the same as the embodiment of the terminal display method, and is not described herein.
需要说明的是,在本文中,术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含,从而使得包括一系列要素的过程、方法、物品或者系统不仅包括那些要素,而且还包括没有明确列出的其他要素,或者是还包括为这种过程、方法、物品或者系统所固有的要素。在没有更多限制的情况下,由语句“包括一个……”限定的要素,并不排除在包括该要素的过程、方法、物品或者系统中还存在另外的相同要素。It is to be understood that the term "comprises", "comprising", or any other variants thereof, is intended to encompass a non-exclusive inclusion, such that a process, method, article, or It also includes other elements that are not explicitly listed, or elements that are inherent to such a process, method, item, or system. An element defined by the phrase "comprising a ..." does not exclude the presence of additional equivalent elements in a process, method, article, or system that includes the element, without further limitation.
上述本公开实施例序号仅仅为了描述,不代表实施例的优劣。The above-mentioned serial numbers of the embodiments of the present disclosure are merely for the description, and do not represent the advantages and disadvantages of the embodiments.
通过以上的实施方式的描述,本领域的技术人员可以清楚地了解到上述实施例方法可借助软件加必需的通用硬件平台的方式来实现,当然也可以通过硬件,但很多情况下前者是更佳的实施方式。基于这样的理解,本公开的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来,该计算机软件产品存储在如上所述的一个存储介质(如ROM/RAM、磁碟、光盘)中,包括若干指令用以使得一台终端设备(可以是手机,计算机,服务器,空调器,或者网络设备等)执行本公开各个实施例所述的方法。Through the description of the above embodiments, those skilled in the art can clearly understand that the foregoing embodiment method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be through hardware, but in many cases, the former is better. Implementation. Based on such understanding, portions of the technical solution of the present disclosure that contribute substantially or to the prior art may be embodied in the form of a software product stored in a storage medium (such as a ROM/RAM as described above). , a disk, an optical disk, including a number of instructions for causing a terminal device (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the methods described in various embodiments of the present disclosure.
以上仅为本公开的优选实施例,并非因此限制本公开的专利范围, 凡是利用本公开说明书及附图内容所作的等效结构或等效流程变换,或直接或间接运用在其他相关的技术领域,均同理包括在本公开的专利保护范围内。The above is only a preferred embodiment of the present disclosure, and is not intended to limit the scope of the disclosure of the present disclosure, and the equivalent structure or equivalent process transformations made by the present disclosure and the contents of the drawings may be directly or indirectly applied to other related technical fields. The same is included in the scope of patent protection of the present disclosure.

Claims (10)

  1. 一种终端显示方法,包括:A terminal display method includes:
    接收服务器发送的桌面中的实时视频帧数据及第一变化区域信息,并对所述实时视频帧数据进行解码,得到第一YUV数据;Receiving real-time video frame data and first change area information in the desktop sent by the server, and decoding the real-time video frame data to obtain first YUV data;
    将所述第一YUV数据按预设规则分成预设数量的视频区域,并利用预设算法计算出所述视频区域与所述第一变化区域信息相交的第二变化区域信息;以及Dividing the first YUV data into a preset number of video regions according to a preset rule, and calculating, by using a preset algorithm, second change region information that intersects the video region with the first change region information;
    将所述第二变化区域信息对应的第二YUV数据转码为对应的RGB数据,并根据所述RGB数据进行显示。The second YUV data corresponding to the second change region information is transcoded into corresponding RGB data, and displayed according to the RGB data.
  2. 如权利要求1所述的终端显示方法,其中,所述对所述实时视频帧数据进行解码,得到第一颜色编码YUV数据的步骤,包括:The terminal display method according to claim 1, wherein the step of decoding the real-time video frame data to obtain first color-coded YUV data comprises:
    根据所述实时视频帧数据的格式信息及预存硬件信息判断所述实时视频帧数据是否支持硬件解码;Determining, according to the format information of the real-time video frame data and the pre-stored hardware information, whether the real-time video frame data supports hardware decoding;
    若所述实时视频帧数据支持硬件解码,则采用硬件解码的方式对所述实时视频帧数据进行解码,得到第一YUV数据;以及If the real-time video frame data supports hardware decoding, decoding the real-time video frame data by using hardware decoding to obtain first YUV data;
    若所述实时视频帧数据不支持硬件解码,则采用软件解码的方式对所述实时视频帧数据进行解码,得到第一YUV数据。If the real-time video frame data does not support hardware decoding, the real-time video frame data is decoded by using software decoding to obtain first YUV data.
  3. 如权利要求1所述的终端显示方法,其中,所述将所述第一YUV数据按预设规则分成预设数量的视频区域,并利用预设算法计算出所述视频区域与第一变化区域信息相交的第二变化区域信息的步骤,包括:The terminal display method according to claim 1, wherein the first YUV data is divided into a preset number of video regions according to a preset rule, and the video region and the first change region are calculated by using a preset algorithm. The step of intersecting the second change area information of the information includes:
    将所述第一YUV数据按预设规则分成预设数量的视频区域;Dividing the first YUV data into a preset number of video regions according to a preset rule;
    对每个视频区域创建一对应的线程;以及Create a corresponding thread for each video area;
    利用所述线程及预设算法计算出所述视频区域与所述第一变化区域信息相交的第二变化区域信息。And calculating, by the thread and a preset algorithm, second change area information that intersects the video area with the first change area information.
  4. 如权利要求1所述的终端显示方法,其中,所述将所述第一 YUV数据按预设规则分成预设数量的视频区域的步骤,包括:The terminal display method according to claim 1, wherein the step of dividing the first YUV data into a preset number of video regions according to a preset rule comprises:
    将所述第一YUV数据按终端显示器的分辨率的高度或宽度等分成预设数量的视频区域。The first YUV data is divided into a preset number of video regions according to the height or width of the resolution of the terminal display.
  5. 如权利要求2所述的终端显示方法,其中,所述将所述第二变化区域信息对应的第二YUV数据转码为对应的RGB数据的步骤,包括:The terminal display method according to claim 2, wherein the step of transcoding the second YUV data corresponding to the second change region information into corresponding RGB data comprises:
    根据所述预存硬件信息判断所述第二变化区域信息是否支持硬件转码;以及Determining, according to the pre-stored hardware information, whether the second change area information supports hardware transcoding;
    若所述第二变化区域信息支持硬件转码,则采用硬件转码的方式将所述第二变化区域信息对应的第二YUV数据转码为对应的RGB数据。If the second change area information supports hardware transcoding, the second YUV data corresponding to the second change area information is transcoded into corresponding RGB data by using hardware transcoding.
  6. 如权利要求5所述的终端显示方法,其中,所述将所述第二变化区域信息对应的第二YUV数据转码为对应的RGB数据的步骤,还包括:The terminal display method of claim 5, wherein the step of transcoding the second YUV data corresponding to the second change region information into corresponding RGB data further comprises:
    若所述第二变化区域信息不支持硬件转码,则根据所述预存硬件信息判断所述第二变化区域信息是否支持汇编转码;If the second change area information does not support hardware transcoding, determining, according to the pre-stored hardware information, whether the second change area information supports assembly transcoding;
    若所述第二变化区域信息支持汇编转码,则采用汇编转码的方式将所述第二变化区域信息对应的第二YUV数据转码为对应的RGB数据;以及If the second change area information supports the assembly transcoding, the second YUV data corresponding to the second change area information is transcoded into corresponding RGB data by means of assembly transcoding;
    若所述第二变化区域信息不支持汇编转码,则采用软件转码的方式将所述第二变化区域信息对应的第二YUV数据转码为对应的RGB数据。If the second change area information does not support the assembly transcoding, the second YUV data corresponding to the second change area information is transcoded into corresponding RGB data by means of software transcoding.
  7. 如权利要求1所述的终端显示方法,其中,所述接收服务器发送的桌面中的实时视频帧数据及第一变化区域信息的步骤,包括:The terminal display method of claim 1, wherein the step of receiving the real-time video frame data and the first change area information in the desktop sent by the server comprises:
    接收服务器发送的桌面中的实时视频帧数据,及所述服务器通过预设程序抓取的桌面中的第一变化区域信息。Receiving real-time video frame data in the desktop sent by the server, and first change area information in the desktop captured by the server through a preset program.
  8. 如权利要求1所述的终端显示方法,其中,所述第一变化区域信息、所述第二变化区域信息及所述视频区域均包含分别表示对应的变化区域在所述桌面中的上、左、右、下坐标信息的参数。The terminal display method according to claim 1, wherein the first change region information, the second change region information, and the video region each include upper and left sides respectively indicating corresponding change regions in the desktop. The parameters of the right and bottom coordinate information.
  9. 一种终端,包括:存储器、处理器及存储在所述存储器上并可被所述处理器执行的显示程序,其中,所述显示程序被所述处理器执行时实现如权利要求1至8任一项所述的终端显示方法的步骤。A terminal comprising: a memory, a processor, and a display program stored on the memory and executable by the processor, wherein the display program is executed by the processor to implement any of claims 1 to 8 A step of the terminal display method described.
  10. 一种计算机可读存储介质,所述计算机可读存储介质上存储有显示程序,其中,所述显示程序被处理器执行时实现如权利要求1至8任一项所述的终端显示方法的步骤。A computer readable storage medium having stored thereon a display program, wherein the display program is executed by a processor to implement the steps of the terminal display method according to any one of claims 1 to 8. .
PCT/CN2018/097487 2017-07-28 2018-07-27 Terminal display method, terminal and computer-readable storage medium WO2019020112A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201710636716.4 2017-07-28
CN201710636716.4A CN109309867A (en) 2017-07-28 2017-07-28 Display methods, display terminal and the computer readable storage medium of terminal

Publications (1)

Publication Number Publication Date
WO2019020112A1 true WO2019020112A1 (en) 2019-01-31

Family

ID=65039409

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/097487 WO2019020112A1 (en) 2017-07-28 2018-07-27 Terminal display method, terminal and computer-readable storage medium

Country Status (2)

Country Link
CN (1) CN109309867A (en)
WO (1) WO2019020112A1 (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113050826A (en) * 2021-03-10 2021-06-29 深圳市绘王动漫科技有限公司 Digital screen connected with cloud
CN113076050B (en) * 2021-04-25 2022-05-20 深圳创维-Rgb电子有限公司 Man-machine interaction method and device based on memory copy, television and storage medium
CN113099238B (en) * 2021-06-09 2021-08-17 广州易方信息科技股份有限公司 Decoding method for judging video played by android mobile phone

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103247283A (en) * 2013-02-21 2013-08-14 北京君正集成电路股份有限公司 Screen refreshing method and device
CN103391442A (en) * 2013-07-24 2013-11-13 佳都新太科技股份有限公司 Rapid video image transmission compression algorithm based on regional division and difference comparison
WO2015100573A1 (en) * 2013-12-31 2015-07-09 华为终端有限公司 Display refreshing method and terminal
KR20160109805A (en) * 2015-03-13 2016-09-21 엔트릭스 주식회사 System for cloud streaming service, method of image cloud streaming service using split of changed image and apparatus for the same

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101977322A (en) * 2010-11-10 2011-02-16 上海交通大学 Screen coding system based on universal video coding standard
CN103281539B (en) * 2013-06-07 2016-10-05 华为技术有限公司 Method, device and the terminal that a kind of image coding and decoding processes
CN104253804B (en) * 2013-06-28 2017-12-12 广州华多网络科技有限公司 A kind of method and device for transmitting view data
CN104980788B (en) * 2015-02-11 2018-08-07 腾讯科技(深圳)有限公司 Video encoding/decoding method and device
JP6281518B2 (en) * 2015-03-30 2018-02-21 ソニー株式会社 Video transmission device, video reception device, video transmission method, video transmission method, and video transmission system
CN105245914B (en) * 2015-09-30 2019-02-22 上海有孚网络股份有限公司 Cloud desktop collaborative HD video transmission method and system
CN106713937A (en) * 2016-12-30 2017-05-24 广州虎牙信息科技有限公司 Video playing control method and device as well as terminal equipment

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103247283A (en) * 2013-02-21 2013-08-14 北京君正集成电路股份有限公司 Screen refreshing method and device
CN103391442A (en) * 2013-07-24 2013-11-13 佳都新太科技股份有限公司 Rapid video image transmission compression algorithm based on regional division and difference comparison
WO2015100573A1 (en) * 2013-12-31 2015-07-09 华为终端有限公司 Display refreshing method and terminal
KR20160109805A (en) * 2015-03-13 2016-09-21 엔트릭스 주식회사 System for cloud streaming service, method of image cloud streaming service using split of changed image and apparatus for the same

Also Published As

Publication number Publication date
CN109309867A (en) 2019-02-05

Similar Documents

Publication Publication Date Title
CN106941563B (en) control method and device for refresh rate of mobile terminal and mobile terminal
US9892716B2 (en) Image display program, image display method, and image display system
WO2018205878A1 (en) Method for transmitting video information, terminal, server and storage medium
KR102553273B1 (en) Method and apparatus for image enhancement of virtual reality images
WO2021114684A1 (en) Image processing method and apparatus, computing device, and storage medium
WO2019020112A1 (en) Terminal display method, terminal and computer-readable storage medium
CN107277399A (en) Television terminal and HDR image switch to SDR method and computer-readable recording medium
US20140258895A1 (en) Method and device for processing image data
KR102105168B1 (en) Display apparatus and control method of the same
CN113905268A (en) Black edge removing method for screen projection display of mobile terminal
US11887276B2 (en) Image processing method and display apparatus for adjusting format and resolution of input image in parallel and matrix manner
CN113645476B (en) Picture processing method and device, electronic equipment and storage medium
JP2018503281A (en) Image display method, user terminal, and video receiving device
WO2022052742A1 (en) Multi-terminal screen combination method, apparatus and device, and computer storage medium
US11303958B2 (en) Display device and image display method of the same
CN112328068A (en) Method, device and readable storage medium for reducing power consumption of display device
KR20200039638A (en) Display apparatus and control method of the same
CN114489549B (en) Screen projection image processing method and device, electronic equipment and storage medium
CN114647468B (en) Screen projection image display method and device, electronic equipment and storage medium
WO2021036551A1 (en) Power consumption adjustment method and device, and computer-readable storage medium
WO2021063135A1 (en) Saturation enhancement method and device, and computer readable storage medium
WO2023142370A1 (en) Screen mirroring display method and apparatus, and electronic device and readable storage medium
CN112788425A (en) Dynamic area display method, device, equipment and computer readable storage medium
CN110660364A (en) Method and device for adjusting regional backlight and storage medium
CN111327948B (en) Automatic matching method and device for movie image mode and computer readable storage medium

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18837968

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 24.06.2020)

122 Ep: pct application non-entry in european phase

Ref document number: 18837968

Country of ref document: EP

Kind code of ref document: A1