CN106488298B

CN106488298B - Method and device for drawing image in UI (user interface) and television

Info

Publication number: CN106488298B
Application number: CN201611000853.0A
Authority: CN
Inventors: 曹通; 仲维
Original assignee: Qingdao Hisense Electronics Co Ltd
Current assignee: Vidaa Netherlands International Holdings BV
Priority date: 2016-11-14
Filing date: 2016-11-14
Publication date: 2020-01-10
Anticipated expiration: 2036-11-14
Also published as: CN106488298A

Abstract

The embodiment of the invention provides a method and a device for drawing an image in a UI (user interface) and a television, relates to image display processing, and can reduce the performance of image rendering consumption and improve the UI fluency. The method comprises the following steps: determining a UI container node to be drawn in a rendering tree; and drawing an object of a hidden graphic container Canvas corresponding to the UI container node to be drawn on a User Interface (UI), wherein the content of the UI container node to be drawn is drawn in the hidden Canvas in advance. The embodiment of the invention is used for UI image drawing.

Description

Method and device for drawing image in UI (user interface) and television

Technical Field

The invention relates to the field of image display processing, in particular to a method and a device for drawing an image in a User Interface (UI), and a television.

Background

The HTML5 (English is called Hyper Text Markup Language version 5, Chinese: hypertext Markup Language version 5) standard brings revolutionary changes to the Web front end, and greatly improves the webpage display richness and interactivity. Due to the cross-platform property of the browser engine and the convenient development and debugging property of the JavaScript, the Web front-end technology is not only used for developing Web pages, but also becomes a preferred choice for front-end development in mobile terminals and intelligent equipment. With the HTML5 standard becoming mainstream, more and more manufacturers begin to adopt a Web UI development mode based on a browser in the field of smart tv UI (english: User Interface, chinese: User Interface).

A Canvas (Chinese: a graphic container or Canvas) is a new label in an HTML5 standard, and a drawing API (English: Application Programming Interface, Chinese: an Application Programming Interface) provided by the Canvas can directly call GPU (Graphics Processing Unit, Chinese: a graphic processor) resources, so that hardware acceleration of image drawing is realized, and the image Processing performance of a webpage is greatly improved. However, Canvas only provides basic drawing APIs, does not support layout management and control mechanisms such as HTML DOM (Document Object Model, chinese) and CSS (Cascading Style Sheets), and needs to be implemented by developers themselves. The Canvas itself has no concept of the UI, and is equivalent to a bitmap which can be operated by JavaScript, and HTML DOM characteristics such as CSS are not suitable for the Canvas, so the structure, interaction logic and the like of the Canvas UI need to be completely controlled by a UI developer, and the drawing of components, elements and animation also needs to be realized by the developer. This presents a certain difficulty to the drawing of the UI, since the drawing API provided by Canvas is very basic and primitive. The UI component is basically composed of pictures and text. For the UI, it is necessary for the user to perceive the currently operating interface, such as the scenes of the secondary Menu, the pop-up dialog box, etc., so the hierarchical design of the interface is inevitable, as shown in the scene of fig. 1, selecting Menu1 (Menu 1) from the Layer1 (Layer 1) interface opens the Layer2 interface on the basis of Layer1, and clicking Sub Menu1 in the Layer2 interface pops up a confirmation dialog box (confirm box), i.e., Layer 3. That is, in this scenario, Layer1, Layer2, Layer3 would be drawn in the UI interface at the same time. According to the scene, the structural relationship between the hierarchies and the components in the hierarchies can be known, and a rendering tree is formed. As shown in fig. 2, when the UI refreshes each frame, it is necessary to traverse a rendering tree under a root (root directory) of the UI, draw each component one by one according to the order of layers, and finally form an overall UI interface fed back to the user. In addition, as shown in fig. 3, each component is usually composed of basic elements such as a background, an icon, and a text, and when the UI is drawn, the basic elements inside the UI also need to be drawn one by one, so that the drawing of one component is completed. When fig. 2 and fig. 3 are combined, that is, the leaf nodes of the rendering tree structure shown in fig. 2 actually have respective child nodes, thereby forming a complete UI rendering tree.

When drawing in the UI, according to the conventional implementation manner, the whole UI rendering tree needs to be traversed, and drawing is performed node by node starting from child nodes. For a rendering engine, the pressure in performance is large, and when user interaction is processed, phenomena such as blocking easily occur, so that the smoothness of a UI is not good, and the user experience is influenced.

Disclosure of Invention

The embodiment of the invention provides a method and a device for drawing an image in a User Interface (UI) and a television, which can reduce the performance of image rendering consumption and improve the UI fluency.

In order to achieve the above purpose, the embodiment of the invention adopts the following technical scheme:

in a first aspect, a method for image rendering in a UI is provided, including:

determining a UI container node to be drawn in a rendering tree;

and drawing an object of a hidden graphic container Canvas corresponding to the UI container node to be drawn on a User Interface (UI), wherein the content of the UI container node to be drawn is drawn in the hidden Canvas in advance.

In a second aspect, there is provided an apparatus for image rendering in a UI,

the traversal unit is used for determining a UI container node to be drawn in the rendering tree;

and the processing unit is used for drawing the object of the hidden graphic container Canvas corresponding to the UI container node to be drawn, which is determined by the traversal unit, on the UI of the user interface, wherein the content of the UI container node to be drawn is drawn in the hidden Canvas in advance.

In a third aspect, a television is provided, which comprises any one of the above devices for image rendering in a UI. According to the method, the device and the television for drawing the image in the UI, the UI container node to be drawn is determined in the rendering tree; and then drawing the object of the hidden graphic container Canvas corresponding to the UI container node to be drawn on the user interface UI. Because the content of the UI container node to be rendered is pre-rendered in the hidden Canvas, when the UI performs traversal of the rendering tree and renders to the UI user interface container node to be rendered, the content of the UI user interface container node is already rendered in one hidden Canvas, so the hidden Canvas can be directly rendered on the UI. Therefore, a large amount of drawing processes for the UI container nodes to be drawn are avoided, the performance of image rendering consumption is reduced, and the UI fluency is improved.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

FIG. 1 is a diagram illustrating a UI scenario provided by the prior art;

FIG. 2 is a diagram illustrating a rendering tree of a UI provided by the prior art;

FIG. 3 is a diagram of another UI scenario provided by the prior art;

FIG. 4 is a flowchart illustrating a method for image rendering in a UI according to an embodiment of the present invention;

FIG. 5 is a flowchart illustrating a method for image rendering in a UI according to another embodiment of the invention;

FIG. 6 is a schematic diagram of a UI scenario provided by an embodiment of the invention;

FIG. 7 is a rendering tree diagram of a UI according to an embodiment of the invention;

fig. 8 is a schematic structural diagram of an apparatus for image drawing in a UI according to an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

It should be noted that, in the embodiments of the present invention, words such as "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" or "e.g.," an embodiment of the present invention is not necessarily to be construed as preferred or advantageous over other embodiments or designs. Rather, use of the word "exemplary" or "such as" is intended to present concepts related in a concrete fashion. "of", "corresponding" and "corresponding" may sometimes be used in combination, it being noted that the intended meaning is consistent when no distinction is made. The 'A and/or B' in the scheme refers to three conditions of a protection scheme 'A' or a protection scheme 'B' or protection schemes 'A and B'.

The invention relates to Web front-end development based on HTML Canvas, and describes a method for optimized image rendering in a UI. According to the scheme, in the intelligent television UI based on the HTML Canvas, a plurality of pictures are changed into a combined image in an image drawing mode by using the hidden Canvas, and then the combined image is drawn into the UI interface, so that the drawing efficiency of the UI on some complex scenes or elements is improved. Taking fig. 1 as an example, the level of interaction required for the user in the prior art is Layer3, and Layer1 and Layer2 are completely static when operating on components in Layer 3. For such a completely static hierarchy in the solution provided by the embodiments of the present application, we can consider whether it can be cached, and how the cached image can be drawn into the UI interface. Compared with a mode of sequentially drawing a plurality of pictures, the scheme described by the invention has the advantages that: complex UI scenes or elements are already inherently more performance consuming, and performing a large number of image rendering computations at the same time increases the CPU/GPU burden. In an embedded system device with relatively limited hardware performance resources, such as a smart television, the phenomena of unsmooth UI, slow response and the like are often caused, and the user experience is influenced. The optimized image drawing mode adopted by the invention can reduce the performance consumed by image rendering aiming at the complex UI scene and ensure the fluency of the UI to the maximum extent.

The protection scheme of the present invention is specifically illustrated with reference to the following examples:

referring to fig. 4, an embodiment of the present invention provides a method for image drawing in a UI, including the steps of:

101. and determining a UI container node to be drawn in the rendering tree.

102. And drawing an object of the hidden graphic container Canvas corresponding to the UI container node to be drawn on the user interface UI.

And the hidden Canvas is pre-drawn with the contents of the UI container nodes to be drawn. Before the method is executed, a hidden graphic container Canvas needs to be created for a UI user interface container node to be drawn in advance. The types of the UI containers comprise UI hierarchies and UI components; specifically, a Canvas tag may be created in the memory by using the document element method, and the size of the Canvas tag is equal to the size of the UI container node object to be drawn. This Canvas tag object is pointed to by this. In the API provided by the Canvas context, the interface for image drawing is context. The img parameter of this method, i.e. the object that needs to be rendered, may be passed in img tags, video tags or canvas tags. That is, the contents of one HTML5Canvas may be rendered in another HTML5 Canvas. With this feature, a hidden Canvas can be created in memory for a static UI container node.

According to the method for drawing the image in the user interface UI, firstly, a UI container node to be drawn is determined in a rendering tree; and then drawing the object of the hidden graphic container Canvas corresponding to the UI container node to be drawn on the user interface UI. Because the content of the UI container node to be rendered is pre-rendered in the hidden Canvas, when the UI performs traversal of the rendering tree and renders to the UI user interface container node to be rendered, the content of the UI user interface container node is already rendered in one hidden Canvas, so the hidden Canvas can be directly rendered on the UI. Therefore, a large amount of drawing processes for the UI container nodes to be drawn are avoided, the performance of image rendering consumption is reduced, and the UI fluency is improved.

A hidden Canvas is created in a memory for a certain static UI container node, a cache () method and an uncache () method are added for container types such as UI hierarchies and components, and an attribute cached is added. After the cache method is called, all elements in the UI container node hierarchy are drawn into a hidden Canvas, and the cached attribute points to the hidden Canvas. And the uncache method will leave the cached attribute empty. Thus, when the UI performs the whole rendering, if the cached attribute of the UI container node to be drawn is not empty, it indicates that the hierarchy or component has been drawn into the hidden Canvas, which is equivalent to performing the pre-rendering. At this time, the UI may draw the hidden Canvas (i.e., the object pointed by the cached attribute) directly without traversing its child nodes. Based on the above principle, before the step 101, the contents of the UI container node to be rendered need to be pre-rendered in a hidden Canvas, which refers to the flow shown in fig. 5, specifically as follows:

201. and judging whether the child node exists in the UI container node to be drawn.

Before step 201, a hidden graphic container Canvas needs to be created in advance for a UI user interface container node to be rendered. Specifically, a Canvas label is created in the memory by using the document. This Canvas tag object is pointed to by this.

202. And when determining that the UI container node to be drawn contains the child node, judging whether the cached attribute of the UI container node to be drawn is empty.

203. And when the cached attribute is determined not to be empty, drawing the hidden Canvas object pointed by the cached attribute in the UI container node onto the hidden Canvas.

If not, indicating that the UI container node is cached, calling context.

204. And when determining that the UI container node to be drawn does not contain the child node, determining whether the UI container node to be drawn contains the picture or the character.

205. And when the UI container node to be drawn is determined to be a picture, drawing the picture into the hidden Canvas.

206. When the predetermined UI container node is determined to be a text, the text is drawn into a hidden Canvas.

In step 204 and 206, if the UI container node has no child node, it indicates that the object in the UI container node is a basic picture element or text, and is not a container type, so it is impossible to have a cached attribute. Therefore, a context.

207. When the cached attribute is determined to be empty, the contents of the child nodes of the predetermined UI container node are drawn in another hidden Canvas.

And the uncache method is realized by setting the cached attribute of the container object to be null. At this point, the pointer to the hidden Canvas is cut off, and the hidden Canvas will be destroyed at a suitable time by the garbage collection mechanism of the browser JavaScript.

The steps 202 and 207 are executed circularly to traverse all child nodes of the UI container node, specifically, for the child nodes, create hidden Canvas corresponding to the object size of each child node, and respectively draw the child nodes into the corresponding hidden Canvas.

208. And traversing nodes in the rendering tree to which the UI container nodes to be drawn belong, and drawing all the UI container nodes in the rendering tree to the corresponding hidden Canvas.

After the steps 201 to 208 are completed, all UI container nodes in the rendering tree can be rendered and drawn into the UI, and when the UI is rendered as a whole, if the cached attribute of the UI container node to be drawn is not empty, it is indicated that the hierarchy or component is already drawn into the hidden Canvas, which is equivalent to pre-rendering. At this time, the UI may draw the hidden Canvas (i.e., the Canvas pointed by the cached attribute) directly without traversing its child nodes.

As per the scenario illustrated in fig. 3, the UI component is composed of multiple elements, and the conventional approach requires that each element inside it be rendered separately at each refresh. The rendering tree is shown in fig. 6, and List items (List items) include: background (Background), Icon (Icon), Title Text (Title Text), detailed Text (Detail Text), and Arrow (Arrow). However, from the perspective of the component, its internal elements are fixed and static, so the cache object of the component can be called and cached. Therefore, when the component is rendered, only one rendering process is needed to complete the drawing of the component. In actual application, a plurality of components are provided, and the hidden Canvas cache mode can greatly reduce the burden of a rendering engine and improve the fluency of a UI. In another example of the scenario illustrated in fig. 2, when the UI needs to render one frame, Layer1 and Layer2 in the UI are in a static state, and the operation of the user only works on Layer 3. In this scenario, we can call cache methods to Layer1 and Layer2, caching both levels to the hidden Canvas. Then the rendering tree that the UI needs to traverse becomes as shown in fig. 7, and the flow of the UI rendering interface is substantially the same as the flow shown in fig. 5, and is not described herein again. When the UI is used for user interaction or animation effect processing, in order to provide smooth experience for the user, the refreshing time of the UI per second is at least 24 frames. If the entire tree structure is rendered in a conventional manner, all nodes need to be rendered for each frame. As is evident from comparing fig. 7 with fig. 2, the number of nodes to be rendered per frame is greatly reduced. At this point, the rendering engine can render more frames per second, making the user experience more fluent. In addition, the HTML5Canvas is not very powerful for processing text, and rendering text is very resource-intensive compared to pictures. The characters are drawn into the hidden Canvas, namely the characters are converted into pictures, and then the hidden Canvas is drawn on the UI interface, so that the weak term of text rendering of the Canvas can be avoided, and the rendering speed is increased.

By the steps, the picture and the character elements are pre-rendered by hiding the Canvas by utilizing the characteristics of the API provided by the Canvas, so that the function similar to image caching is realized. In the process of rendering each frame of the UI, repeated rendering of a large number of static elements is avoided, and the consumption of the UI on performance is reduced. Particularly in a relatively complex scene, the drawing mode can effectively solve the Kanton phenomenon of the UI, and brings good interactive experience to the user.

Referring to fig. 8, an embodiment of the present invention provides an apparatus for image drawing in a UI, including:

a traversal unit 81, configured to determine a UI container node to be drawn in the rendering tree;

and the processing unit 82 is configured to draw an object of a hidden graphic container Canvas corresponding to the UI container node to be drawn, which is determined by the traversal unit, on the user interface UI, where the content of the UI container node to be drawn is pre-drawn in the hidden Canvas.

Optionally, the processing unit 82 is further configured to create the hidden Canvas for the UI container node to be drawn; and drawing the content of the UI container node to be drawn in the hidden Canvas.

Optionally, the processing unit 82 is specifically configured to determine whether a child node exists in the UI container node to be drawn; when the UI container node to be drawn is determined to contain the child node, judging whether the cache cached attribute of the UI container node to be drawn is empty; and when the cached attribute is determined not to be empty, drawing the hidden Canvas object pointed by the cached attribute in the UI container node to be drawn onto the hidden Canvas.

Optionally, the processing unit 82 is specifically configured to, when it is determined that the UI container node to be rendered does not include a child node, determine that a component of the UI container node to be rendered is a picture or a character; and when the UI container node to be drawn is determined to be a picture or a character, drawing the picture or the character into the hidden Canvas.

Optionally, the processing unit 82 is specifically configured to, when it is determined that the cached attribute is empty, draw the content of the child node of the UI container node to be drawn on another hidden Canvas.

According to the device for drawing the image in the user interface UI, firstly, the UI container node to be drawn is determined in the rendering tree; and then drawing the object of the hidden graphic container Canvas corresponding to the UI container node to be drawn on the user interface UI. Because the content of the UI container node to be rendered is pre-rendered in the hidden Canvas, when the UI performs traversal of the rendering tree and renders to the UI user interface container node to be rendered, the content of the UI user interface container node is already rendered in one hidden Canvas, so the hidden Canvas can be directly rendered on the UI. Therefore, a large amount of drawing processes for the UI container nodes to be drawn are avoided, the performance of image rendering consumption is reduced, and the UI fluency is improved.

An embodiment of the present invention further provides a television, including the apparatus for drawing an image in a UI provided in any of the above embodiments.

It should be noted that, in a specific implementation process, each step executed by the apparatus for drawing an image in a UI in the method flows shown in fig. 4 and 5 may be implemented by a processor in a hardware form executing a computer execution instruction in a software form stored in a memory, and is not described herein again to avoid repetition. The program corresponding to the action executed by the image rendering device in the UI may be stored in the memory of the image rendering device in the UI in a software form, so that the processor can call and execute the operations corresponding to the above modules.

The memory above may include volatile memory (volatile memory), such as random-access memory (RAM); a non-volatile memory (non-volatile memory) such as a read-only memory (ROM), a flash memory (flash memory), a hard disk (HDD) or a solid-state drive (SSD); combinations of the above categories of memory may also be included.

The processor in the above-provided apparatus may be a single processor or may be a collective term for a plurality of processing elements. For example, the processor may be a central processing unit (CPU; other general purpose processors, Digital Signal Processor (DSP), Application Specific Integrated Circuit (ASIC), field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, etc.

It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described apparatuses and modules may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is merely a logical division, and in actual implementation, there may be other divisions, for example, multiple modules or components may be combined or integrated into another system, or some features may be omitted, or not implemented. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

The modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be physically included alone, or two or more units may be integrated into one unit. The integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.

The above embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions in the embodiments of the present application.

Claims

1. A method of image rendering in a UI, comprising:

determining a UI container node to be drawn in a rendering tree; the UI container node to be drawn is static relative to other UI container nodes in the rendering tree;

drawing an object of a hidden graphic container Canvas corresponding to the UI container node to be drawn on a User Interface (UI), wherein the content of the UI container node to be drawn is drawn in the hidden Canvas in advance;

creating corresponding Canvas for the other UI container nodes, and drawing the contents of the other UI container nodes in the created Canvas so as to redraw the other UI container nodes on the UI;

before drawing the rendering tree, the method further comprises the following steps:

creating the hidden Canvas for the UI container node to be drawn;

drawing the content of the UI container node to be drawn in the hidden Canvas,

the method comprises the following steps:

judging whether child nodes exist in the UI container nodes to be drawn;

when the UI container node to be drawn is determined to contain the child node, judging whether the cache cached attribute of the UI container node to be drawn is empty;

and when the cached attribute is determined not to be empty, drawing the hidden Canvas object pointed by the cached attribute in the UI container node to be drawn onto the hidden Canvas.

2. The method for image rendering in a UI according to claim 1, wherein the determining whether the child node exists in the UI container node to be rendered further comprises;

and when the UI container node to be drawn does not contain the child node, drawing the picture or the character in the UI container node into the hidden Canvas.

3. The method according to claim 1, wherein after determining whether the cached attribute of the UI container node to be rendered is empty, the method further comprises:

and when the cached attribute is determined to be empty, drawing the content of the child node of the UI container node to be drawn on another hidden Canvas.

4. The method of image rendering in a UI according to any of the claims 1-3,

the types of the UI container comprise a UI hierarchy and a UI component.

5. An apparatus for image rendering in a UI,

the traversal unit is used for determining a UI container node to be drawn in the rendering tree; the UI container node to be drawn is static relative to other UI container nodes in the rendering tree;

the processing unit is used for drawing an object of a hidden graphic container Canvas corresponding to the UI container node to be drawn, which is determined by the traversal unit, on a user interface UI, wherein the content of the UI container node to be drawn is drawn in the hidden Canvas in advance;

the processing unit is further configured to create corresponding Canvas for the other UI container nodes, and draw the contents of the other UI container nodes in the created Canvas, so as to redraw the other UI container nodes on the UI.

6. The apparatus for image rendering in a UI of claim 5 wherein the processing unit is further configured to create the hidden Canvas for the UI container node to be rendered; and drawing the content of the UI container node to be drawn in the hidden Canvas.

7. The apparatus for image rendering in a UI according to claim 6,

the processing unit is specifically configured to determine whether a child node exists in the UI container node to be drawn; when the UI container node to be drawn is determined to contain the child node, judging whether the cache cached attribute of the UI container node to be drawn is empty; and when the cached attribute is determined not to be empty, drawing the hidden Canvas object pointed by the cached attribute in the UI container node to be drawn onto the hidden Canvas.

8. A television comprising the apparatus for image rendering in a UI according to any of the claims 5-7 above.