CN117851697A

CN117851697A - Display equipment and memory management method for loading webpage pictures

Info

Publication number: CN117851697A
Application number: CN202211214439.5A
Authority: CN
Inventors: 李星; 周金强
Original assignee: Vidaa Netherlands International Holdings BV
Current assignee: Vidaa Netherlands International Holdings BV
Priority date: 2022-09-30
Filing date: 2022-09-30
Publication date: 2024-04-09

Abstract

Some embodiments of the present application provide a display device and a memory management method for loading a web page picture, where the method may obtain web page data of a web page when a user opens the web page by using a browser with a constant visible area size, parse a download link of an original picture from the web page data, and parse an element size and an element position from a layout tree. And then downloading the compressed data of the original picture, and carrying out resolution reduction processing on the original picture obtained by decompression according to the element size so as to obtain the target picture. Finally, loading the target picture at the element position of the webpage to finish the display of the webpage. In the process of using the browser, the browser does not need to load the original pictures with oversized memory attributes, so that the system memory quantity of the browser is saved, the efficiency of a memory management mechanism is improved, the loading speed of the webpage is higher, and the user experience is improved.

Description

Display equipment and memory management method for loading webpage pictures

Technical Field

The application relates to the technical field of memory management of loading webpage pictures by a browser, in particular to display equipment and a memory management method of loading webpage pictures.

Background

In order to display the web page on the browsers of various platforms as normally as possible, different-sized browser windows can be dynamically adapted. The actual size of the pictures in such dynamically adapted web pages is usually large, and the purpose is to display a clear picture when encountering a large browser window or when the user enlarges the scene of the web page. After loading a web page picture, the browser typically needs to scale the picture to fit the HTML element size used to display the picture.

However, for some browsers with full-screen display and constant size of the visible area, the HTML element size of the display picture is also constant when the web page is loaded. Therefore, the browser can meet the normal display requirement of the browser only by loading the pictures with the sizes of the HTML elements which can be matched with the display pictures, and the original pictures are not required to be loaded to cope with the scene with repeatedly changed display windows. In practical application, the browser does not perform special optimization processing on the scene with the fixed size of the visible area, and still loads an oversized original picture, so that unnecessary memory consumption is increased, and the efficiency of a memory management mechanism is low.

Disclosure of Invention

The application provides display equipment and a memory management method for loading webpage pictures, which are used for solving the problem that when a user uses a browser with a unchanged visible area size, the browser still loads oversized original pictures, unnecessary memory consumption of the browser is increased, and the efficiency of a memory management mechanism is lower.

In a first aspect, the present application provides a display device comprising a display and a controller, wherein the controller is configured to:

acquiring a control instruction input by a user and used for opening a webpage;

responding to the control instruction, acquiring webpage data of the webpage through a browser, wherein the browser comprises a first cache area and a second cache area;

analyzing a download link and a layout tree of the original picture from the webpage data; the layout tree includes element sizes and element positions in the web page;

downloading the compressed data of the original picture according to the download link and storing the compressed data in the first buffer area;

decompressing the compressed data, and performing resolution reduction processing on the original picture obtained by decompression according to the element size to obtain a target picture;

storing the target picture to the second buffer;

And loading the target picture in the webpage according to the element position, and controlling the display to display the webpage.

In some embodiments of the present application, the web page data includes hypertext markup language HTML elements, cascading style sheet CSS data, and Javascript script language, and the controller is further configured in the step of parsing the download link and the layout tree of the original picture from the web page data:

analyzing the webpage data through a browser;

constructing a document object model tree of the webpage according to the HTML elements, the CSS data and the Javascript script language;

the document object model tree is built as a layout tree based on the HTML elements.

In some embodiments of the present application, the controller is further configured to:

layering the layout tree to construct a layer tree;

downloading layout resources of the webpage through the browser; the layout resources comprise the text resources and link resources;

and loading the text resource and the link resource in the webpage in turn according to the hierarchical relationship in the layer tree so as to draw the webpage.

In some embodiments of the present application, in the step of performing the resolution reduction process on the original picture obtained by decompression according to the element size, the controller is further configured to:

Inquiring the background image attribute in the HTML element;

if the background image size exists in the background image attribute, taking the background image size as an element size; wherein the element dimensions include an element width and an element height;

calculating an element height-width ratio according to the element width and the element height;

and performing resolution reduction processing on the original picture according to the element height-width ratio.

if the background image attribute is inquired that the background image size does not exist, taking the initial size of the HTML element as the background image size; the width of the background image size is the element width, and the height of the background image size is the element height;

If the background image attribute is not queried in the HTML element, querying an img image element;

if the img image element is queried, reading an img image size of the img image element, and taking the img image size as an element size;

extracting an original picture size of the original picture; the original picture size comprises an original picture height and an original picture width;

if the original picture height-to-width ratio is greater than or equal to the element height-to-width ratio, comparing the original picture height with the element height;

if the original picture height is larger than the element height, calculating a first ratio of the element height to the original picture height, and setting the first ratio as a first reduction ratio;

and performing resolution reduction processing on the original picture according to the first reduction ratio.

if the original picture height-width ratio is smaller than the element height-width ratio, comparing the original picture width with the element width;

if the original picture width is larger than the element width, calculating a second ratio of the element width to the original picture width, and setting the second ratio as a second reduction ratio;

and performing resolution reduction processing on the original picture according to the second reduction ratio.

In some embodiments of the present application, before the step of performing the resolution reduction process on the original picture obtained by decompression according to the element size, the controller is further configured to:

acquiring the system memory quantity of a browser;

when the system memory amount is smaller than a memory amount threshold, performing a shrinking operation on the original picture height according to 1/N times of the first shrinking ratio, and performing a shrinking operation on the original picture width according to 1/N times of the second shrinking ratio, wherein N is a preset value.

In a second aspect, the present application provides a memory management method for loading a web page picture, which is applied to a display device, where the display device includes a display and a controller, and the method includes:

Acquiring a control instruction input by a user and used for opening a webpage;

storing the target picture to the second buffer;

As can be seen from the above solution, according to the display device and the memory management method for loading a web page picture provided in some embodiments of the present application, when a user opens a web page by using a browser with a constant visible area size, the user may obtain web page data of the web page, parse a download link of an original picture from the web page data, and parse an element size and an element position from a layout tree. And then downloading the compressed data of the original picture according to the download link, and carrying out resolution reduction processing on the original picture obtained by decompression according to the element size so as to obtain the target picture. Finally, loading the target picture at the element position of the webpage to finish the display of the webpage. According to the method and the device, in the browser with the unchanged visible area size, the original picture can be subjected to resolution reduction according to the element size of the webpage. In the process of using the browser, the browser does not need to load the original pictures with oversized memory attributes, so that the system memory quantity of the browser is saved, the efficiency of a memory management mechanism is improved, the loading speed of the webpage is higher, and the user experience is improved.

Drawings

In order to more clearly illustrate the technical solutions of the present application, the drawings that are needed in the embodiments will be briefly described below, and it will be obvious to those skilled in the art that other drawings can be obtained from these drawings without inventive effort.

Fig. 1 is a usage scenario of a display device according to an embodiment of the present application;

fig. 2 is a hardware configuration diagram of a display device in an embodiment of the present application;

FIG. 3 is a schematic diagram of a maximizing pattern for a user clicking a web page in a browser;

FIG. 4 is a schematic diagram of a maximizing mode for a web page in a browser;

FIG. 5 is a hierarchical relationship diagram of a layer tree of a web page in a browser;

fig. 6 is a flow chart illustrating the saving of an original picture to a first buffer in the embodiment of the present application;

fig. 7 is a schematic flow chart of storing a target picture in a second buffer in an embodiment of the present application;

FIG. 8 is a flow chart of another embodiment of determining element sizes in an embodiment of the present application;

FIG. 9 is a schematic diagram of determining similarity of web page templates according to an embodiment of the present application;

FIG. 10 is a schematic diagram illustrating analysis of a picture layer of a web page according to an embodiment of the present application;

FIG. 11 is a schematic flow chart of calculating the aspect ratio of the element in the embodiment of the application;

Fig. 12 is a flowchart of the original picture after performing the resolution reduction process in the embodiment of the present application.

Detailed Description

For purposes of clarity and implementation of the present application, the following description will make clear and complete descriptions of exemplary implementations of the present application with reference to the accompanying drawings in which exemplary implementations of the present application are illustrated, it being apparent that the exemplary implementations described are only some, but not all, of the examples of the present application.

It should be noted that the brief description of the terms in the present application is only for convenience in understanding the embodiments described below, and is not intended to limit the embodiments of the present application. Unless otherwise indicated, these terms should be construed in their ordinary and customary meaning.

The terms "first," second, "" third and the like in the description and in the claims and in the above-described figures are used for distinguishing between similar or similar objects or entities and not necessarily for limiting a particular order or sequence, unless otherwise indicated. It is to be understood that the terms so used are interchangeable under appropriate circumstances.

The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a product or apparatus that comprises a list of elements is not necessarily limited to all elements explicitly listed, but may include other elements not expressly listed or inherent to such product or apparatus.

The display device provided in the embodiment of the application may have various implementation forms, for example, may be a television, a laser projection device, a display (monitor), an electronic whiteboard (electronic bulletin board), an electronic desktop (electronic table), and the like.

Fig. 1 is a schematic diagram of an operation scenario between a display device and a control apparatus according to an embodiment. As shown in fig. 1, a user may operate the display apparatus 200 through the control apparatus 300 or the control device 100.

In some embodiments, the control apparatus 100 may be a remote controller, and the communication between the remote controller and the display device includes infrared protocol communication or bluetooth protocol communication, and other short-range communication modes, and the display device 200 is controlled by a wireless or wired mode. The user may control the display device 200 by inputting user instructions through keys on a remote control, voice input, control panel input, etc.

In some embodiments, the control device 300 (e.g., mobile phone, tablet, computer, notebook, etc.) may also be used to control the display device 200. For example, the display device 200 is controlled using an application running on the control device 300.

In some embodiments, the display apparatus 200 may receive not an instruction using the control apparatus 300 or the control device 100 described above, but a control of a user through a touch or gesture or the like.

In some embodiments, the display device 200 may also perform control in a manner other than the control apparatus 100 and the control device 300, for example, the voice instruction control of the user may be directly received through a module configured inside the display device 200 device to obtain voice instructions, or the voice instruction control of the user may be received through a voice control device configured outside the display device 200 device.

In some embodiments, the display device 200 is also in data communication with a server 400. The display device 200 may be permitted to make communication connections via a Local Area Network (LAN), a Wireless Local Area Network (WLAN), and other networks. The server 400 may provide various contents and interactions to the display device 200. The server 400 may be a cluster, or may be multiple clusters, and may include one or more types of servers.

As shown in fig. 2, the display apparatus 200 may include at least one of a modem 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface.

In some embodiments, the controller 250 may include a processor, a video processor, an audio processor, a graphic processor, a RAM, a ROM, and first to nth interfaces for input/output.

Display 260 may include the following components, namely: a display screen assembly for presenting a picture; a driving assembly driving the image display; a component for receiving an image signal outputted from the controller 250, performing display of video content, image content, and a menu manipulation interface, a component for manipulating a UI interface by a user, and the like.

The display 260 may be a liquid crystal display, an OLED display, a projection device, or a projection screen.

The communicator 220 is a component for communicating with external devices or servers according to various communication protocol types. For example: the communicator may include at least one of a Wifi module, a bluetooth module, a wired ethernet module, or other network communication protocol chip or a near field communication protocol chip, and an infrared receiver. The display device 200 may establish transmission and reception of control signals and data signals with the external control device 100 or the server 400 through the communicator 220.

A user interface, which may be used to receive control signals from the control device 100 (e.g., an infrared remote control, etc.).

The detector 230 is used to collect signals of the external environment or interaction with the outside. For example, detector 230 includes a light receiver, a sensor for capturing the intensity of ambient light; alternatively, the detector 230 includes an image collector such as a camera, which may be used to collect external environmental scenes, user attributes, or user interaction gestures, or alternatively, the detector 230 includes a sound collector such as a microphone, or the like, which is used to receive external sounds.

The external device interface 240 may include, but is not limited to, the following: high Definition Multimedia Interface (HDMI), analog or data high definition component input interface (component), composite video input interface (CVBS), USB input interface (USB), RGB port, etc. The input/output interface may be a composite input/output interface formed by a plurality of interfaces.

The modem 210 receives broadcast television signals through a wired or wireless reception manner, and demodulates audio and video signals, such as EPG data signals, from a plurality of wireless or wired broadcast television signals. In some embodiments, the controller 250 and the modem 210 may be located in separate devices, i.e., the modem 210 may also be located in an external device to the main device in which the controller 250 is located, such as an external set-top box or the like.

The controller 250 controls the operation of the display device and responds to the user's operations through various software control programs stored on the memory. The controller 250 controls the overall operation of the display apparatus 200. For example: in response to receiving a user command to select a UI object to be displayed on the display 260, the controller 250 may perform an operation related to the object selected by the user command.

In some embodiments, the controller 250 includes at least one of a central processing unit (Central Processing Unit, CPU), a video processor, an audio processor, a graphics processor (Graphics Processing Unit, GPU), RAM Random Access Memory, RAM), ROM (Read-Only Memory, ROM), a first interface to an nth interface for input/output, a communication Bus (Bus), and the like.

The browser is an application program for retrieving, displaying and transmitting Web information resources, and is one of channels for obtaining information sources by users. When a user uses a browser to retrieve a material to be queried, the browser opens corresponding content in the browser in the form of a Web page, such as a piece of text, a piece of picture, a piece of video, or any content presented on the Web. The user can also open the corresponding webpage content in the browser by clicking the hyperlink in the application program, or copy the hyperlink and paste the hyperlink to the corresponding position of the search engine in the browser, so as to open the content corresponding to the hyperlink.

In some embodiments, web pages may be dynamically adapted to different sized browser windows for normal display on the browser of multiple platforms as possible. Referring to fig. 3 and 4, a web page is taken as an exemplary illustration of a user opening a web page through a browser, where the web page includes some media pictures or text content. The actual size of the pictures within the web page is typically large due to the need to dynamically adapt the different size browser windows. In this way, when the web page encounters a larger window of the browser, or when the user clicks the maximization mode of the web page by controlling the cursor through a mouse, the control device 100, or other devices as shown in fig. 3, the pictures in the web page can be clearly displayed. The enlarged web page is shown in fig. 4, so that the resolution of the picture in the web page is not reduced due to the enlargement of the window of the browser. After loading a web page picture, the browser typically needs to scale the picture to fit the HTML element size used to display the picture.

In order to improve the efficiency of the memory management mechanism, some embodiments of the present application provide a display device 200, which is used to solve the problem that when a user uses a browser with a unchanged visible area size, the browser still loads an oversized original picture, and unnecessary memory consumption of the browser is increased, so that the efficiency of the memory management mechanism is low. The display device 200 should include at least a controller 250 and a display 260. In some embodiments of the present application, the browser used is a browser with a constant size of the visible area. It should be understood that the solutions of some embodiments of the present application are equally applicable to browsers in which the viewable area may be changed.

The display is used for displaying a webpage opened by a user through the browser. The user may open a web page with the desired query material by entering keywords of the desired query material in the search engine of the browser. The user may input keywords of the desired query material, such as a keyboard, using an external device connected to the display device 200. If the display device 200 is provided with a touch screen input function, a virtual keyboard may be used to input keywords or the like in a touch handwriting manner or through a virtual keyboard of 9 keys or 26 keys.

The user may also input a piece of voice with the inquiry content, for example, "i want to inquire about the movie and television material of a movie" through an audio input interface of the display device, such as a microphone, an external audio device input device, such as a bluetooth headset, etc. "at this time, the controller 250 may recognize the content of the voice input by the user and convert the voice content into corresponding text content. The browser will search the movie data of the a movie in the network based on the text content, and after obtaining the search result, the controller 250 will control the display 260 to display a web page matching the movie data of the a movie.

In some embodiments of the present application, a user may also upload a picture with a content to be queried at a search engine of a browser, and the browser may identify text within the picture according to its own text identification function, search a web page containing a corresponding content based on the identified text, and may also identify a scene or person in the picture, and search a web page containing the corresponding content. For example, a user uploads a screenshot of a movie work. The browser can identify the movie characters in the scene, the actors playing the movie characters, the movie scene and the clothes worn by the movie characters according to the screenshot pictures, and search a plurality of webpages with corresponding information based on the information. The controller 250 may control the display 260 to display the plurality of web pages according to the search result. In the process of displaying the plurality of web pages, the display 260 may be selected to display in an array arrangement or in a tiled manner.

The controller 250 is configured to obtain a control instruction input by a user for opening a web page, or generate a corresponding control instruction according to information such as text, voice, or picture input by the user. The controller 250 obtains web page data of a web page through a browser in response to a control instruction, wherein the browser includes a first buffer and a second buffer, and the web page data includes hypertext markup language HTML elements, cascading style sheet CSS data, and Javascript language.

HTML, collectively referred to as hypertext markup language (Hyper Text Markup Language), is a standard markup language for creating web pages, which is described by markup tags that are used to describe words, graphics, animations, sounds, forms, links, etc., a browser reads an HTML document, interprets the contents of the tags within the document, and displays the contents as web pages.

CSS, collectively referred to as cascading style sheets (Cascading Style Sheets), is a computer language used to represent the style of HTML files and may be used to define style structures such as fonts, colors, positions, and the like. The CSS style can be directly stored in an HTML webpage or a separate style file, so that the control of the style in the webpage is realized.

JavaScript, a scripting language applied to Web page programming, can be inserted into HTML pages and interpreted by a browser. The interaction logic of the Web application is realized through JavaScript. The JavaScript can encapsulate the JavaScript extension interface through the browser to achieve communication with the browser operating system.

In some embodiments of the present application, the controller 250 is further configured to parse web page data and construct a document object model tree (Document Object Model Tree) according to hypertext markup language HTML elements, cascading style sheet CSS data, and Javascript scripting language in the web page data, wherein the document object model tree contains hierarchical relationships between the HTML elements.

After constructing the document object model tree, the controller 250 may further construct a layout tree including the element size and element position of the web page from the document object model tree and based on the HTML elements through the browser. The element size is the display area range of the target picture in the webpage, the size of the display area is the element size, and the element position is the appointed position of the picture in the webpage. In this embodiment, after the layout tree is constructed, the controller 250 may further parse the layout tree to obtain the element size and the element position of the web page, and send the element size and the element position to the browser.

In some embodiments of the present application, the controller 250 may also perform hierarchical processing on the layout tree to construct a layer tree, and draw the web page through the browser according to the hierarchical relationship in the layer tree. Before drawing the web page, the controller 250 also needs to download layout resources of the web page through the browser, where the layout resources may include text resources and link resources including web page presentation content, and may also include picture resources. The link resource is used for jumping to another web page or an application program of a third party, etc., for example, the user wants to download the game program a in the display device 200 through the web page, and the browser can jump to the corresponding download interface according to the download link by controlling the cursor to click on the download link in the web page through the control device 100, so as to start downloading the game program a.

As shown in fig. 5, the controller 250 may sequentially be a text layer, a link layer, and a picture layer according to a hierarchical relationship of the picture layer tree. In the process of the controller 250 drawing the web page, the corresponding layout resources are arranged in the corresponding layers, for example, text resources are arranged in the text layer, link resources are arranged in the link layer, and picture resources are arranged in the picture layer. The above-mentioned layer tree distribution relationship is merely illustrative, and the controller 250 may set the sequence of each layer according to the actual needs when drawing the web page through the browser.

In the process of drawing the web page by the controller 250, it should be ensured that the contents in each level are not blocked, if the phenomenon that the contents between the levels are blocked occurs, the controller 250 can perform layering processing on the layout tree again, or perform up-shifting operation or down-shifting operation on the level with the content blocking on the basis of the original level, so as to ensure that the user can accurately obtain effective information from the web page.

In some embodiments of the present application, the controller 250 is further configured to parse a download link of the original picture from the web page data, where the download link may be obtained from the network, obtain a URL link of the original picture, and download the compressed data of the original picture according to the URL link. The controller 250 may also send a download request to a server or a cloud server, where the download request may include a URL of an original picture, the server or the cloud server matches the URL of the original picture in a picture URL database, and after obtaining a matching result, the download link of the original picture is fed back to the controller 250 in a signal form, so that the controller 250 downloads compressed data of the original picture.

It should be noted that, in some embodiments of the present application, the compressed data format of the original picture may be suffix of ".7z", ". Rar", ". Zip", ". Apz", ". Ar", ". Bz", ". Car", ". Dar", ". Cpgz", ". F", ". Ha", ". Hbc", ". Hbc", ". Hbe", ". Hpk", ". Hyp", etc. The compression format of the original picture is not particularly limited. Correspondingly, the decompression mode is also applicable to all formats.

Referring to fig. 6, in some embodiments of the present application, after downloading the compressed data of the original picture, the controller 250 stores the compressed data of the original picture in the first buffer, decompresses the compressed data of the original picture to obtain the original picture when the web page needs to be displayed, and then performs the resolution reduction processing on the original picture, where the image after the resolution reduction processing may be stored in the second buffer for the browser to render the web page.

In some embodiments of the present application, a first buffer in the browser is used to store the downloaded original image compression data. Because most web pages are intended to dynamically fit different sized browser windows, the actual size of the original pictures in the web pages tends to be large. For a browser with a constant size of the visible area, the size of the picture is only adapted to the fixed display size of the browser. It should be noted that, the first buffer area and the second buffer area of the browser are both running in the background of the browser, and after the compressed data of the original picture is decompressed by the controller 250, the obtained compressed data of the original picture is temporarily stored in the first buffer area and is not displayed in the web page.

As shown in fig. 7, after the original picture is acquired, the controller 250 further needs to perform a resolution reduction process on the decompressed original picture according to the element size to obtain a target picture, and store the target picture in the second buffer. The method can reduce the memory quantity of the original picture while reducing the resolution ratio, so as to reduce the memory quantity of the picture occupying the browser system memory, accelerate the loading speed of the webpage and improve the user experience.

In some embodiments, the controller 250 may determine the element size and element position from the HTML element.

Because the web page is made up of a variety of HTML elements, the HTML elements that can be used to load the picture include background-image attributes (background-image attributes) and img image elements. There are a variety of HTML elements that can set the background-image attribute. The format is as follows (exemplified by < div > element):

< divstyle= "background-image: URL (original picture URL address); backspace-size: width height; ">

The background-image attribute is used for setting background pictures of the HTML elements and comprises picture URL information. The attributes that may appear with the background-image are also background-size, which is used to specify the size of the final display of the picture.

Therefore, in performing the resolution reduction process on the original picture according to the element size, the controller 250 also needs to query whether the background image attribute exists in the HTML element. If there is a background image attribute in the HTML element, the controller 250 further queries the background image size in the background image attribute. If the background image size exists in the background image attribute, the background image size is taken as the element size. Wherein the element dimensions include an element width and an element height. The controller 250 also needs to calculate the element aspect ratio according to the element height and the element width, and the element aspect ratio is used for determining whether the original picture downloaded by the controller 250 can be directly filled into the element position of the web page. If the original picture aspect ratio is the same as the element aspect ratio, the controller 250 may extract an original picture size including an original picture height and an original picture width. And performing resolution reduction processing on the original picture based on the original picture size until the original picture height is equal to the element height or the original picture width is equal to the element width, obtaining a target picture, and storing the target picture in the second buffer area. At this time, the other size of the original picture needs to perform the resolution reduction processing according to the original picture aspect ratio, for example, when the original picture width is equal to the element width, the original picture height needs to perform the resolution reduction processing according to the original picture own aspect ratio, so as to ensure that the original picture width and the original picture height perform the equal-proportion resolution reduction processing, and obtain the target picture with the same aspect ratio as the original picture but lower resolution.

The following background image attributes are taken as an exemplary illustration:

< divstyle= "background-image: URL; backspace-size with 500 height 500; from this, the element sizes in this example are the element height 500 and the element width 500. The calculated element height-width ratio is 500/500=1. After calculating the element aspect ratio, the controller 250 downloads the original picture downloaded by the link according to the URL. If the aspect ratio of the original picture is 1, the shape of the original picture and the element position directly filled into the webpage are described. The controller 250 may also obtain the original picture height and the original picture width of the original picture based on the above conditions, and perform the resolution reduction process on the original picture such that the original picture height reaches 500 and the original picture width reaches 500. After performing resolution processing on the original picture, a target picture is obtained, and the controller 250 stores the target picture in the second buffer.

Another method of determining element size is shown in fig. 8. In some embodiments shown in fig. 8, if the controller 250 does not have a background image size in the query background image attribute, the reduced resolution process cannot be performed on the original picture according to the background image size. At this time, the controller 250 may also query the background image size of the HTML element, and take the background image size of the HTML element as the element size of the resolution-reducing process, the width of the background image size being the element width, and the height of the background image size being the element height.

The initial size of the HTML element may be determined according to the layout of the web page, and the producer of the web page may set the initial size of the HTML element of each target picture according to the content to be displayed by the web page. When a web page producer produces a web page, if there are too many target pictures to be placed in the web page, the produced template is usually applied, and the target pictures, characters and other contents to be displayed are placed at the corresponding positions of the template. However, because of too many target pictures in the web page, the process of obtaining the initial size of the HTML element is complicated, and the loading speed of the web page is reduced.

In this regard, the controller 250 may perform template recognition on the target web page through the browser and acquire a layout style of the template. The controller 250 performs comparison in the web page template library according to the layout style through the browser, obtains a comparison sample template, and then obtains all the picture information in the comparison sample template. The picture information may include a download source of the picture, a layout size of the picture, an estimated memory footprint of the picture, or an optional format of the picture, etc. The controller 250 determines the initial sizes of all HTML elements at one time according to all the picture information, so as to solve the problem that the process of obtaining the initial sizes of the HTML elements is complicated because a large number of pictures exist in the web page, and improve the obtaining efficiency.

In the process that the controller 250 can perform template recognition on the web page through the browser, the controller 250 may also set a similarity threshold of the comparison result, so as to improve accuracy of obtaining the comparison sample template. Fig. 9 shows a similarity comparison result of the layout style of the web page template library and the template. As shown in fig. 9, after receiving a control instruction input by a user through the control device 100, the controller 250 obtains a layout style of a template of a web page and compares the layout style with a template library of the web page. In the embodiment shown in fig. 9, the similarity threshold is set to 80%, and the similarity is 90%, 70%, 50% and 60% in the template 1, the template 2, the template 3 and the template 4, respectively, and only the template 1 which meets the similarity threshold is set. The controller 250 determines the template 1 as a comparison sample template, obtains the sizes of all the picture display areas in the template 1, and correspondingly sets the initial sizes of all the HTML elements according to the sizes of all the picture display areas.

Since most web pages contain multiple target pictures, in some embodiments of the present application, the controller 250 may further analyze the picture layers in the hierarchical tree without obtaining the layout style of the web page template, so as to obtain the number of target pictures, the positions of the target pictures, and the element sizes of each target picture contained in the picture layers. For each target picture, one element position is corresponding. As shown in fig. 10, there are a picture 1, a picture 2, a picture 3, a picture 4, and a picture 5 in the web page. After the controller 250 obtains the control instruction, it analyzes the picture layer, and analyzes the picture layer of the web page, and identifies all the pictures in the picture layer. In this embodiment, the analysis results are from picture 1 to picture 5, and the controller 250 obtains all the element sizes of picture 1 to picture 5.

In some embodiments, the controller 250 may further establish a coordinate system capable of determining a position in the web page after acquiring a control instruction for opening the web page input by the user, and acquire position coordinates of the target picture according to the element position of the target picture. When the controller 250 loads the target picture at the element position of the web page through the browser, the base point coordinates of the target picture can also be obtained, and in the loading process, the upper edge and the lower edge of the target picture are ensured to be parallel to the X axis of the coordinate system, the left edge and the right edge of the target picture are ensured to be parallel to the Y axis of the coordinate system, and the base point coordinates of the target picture are positioned at the coordinate points indicated by the element position. The base point coordinates are usually coordinates of a point where the upper left corner of the target picture is located, and in some embodiments of the present application, coordinates of other corner points in the target picture may also be used as base point coordinates. After the base point coordinates of the target picture are overlapped or correlated with the coordinate point positions indicated by the element positions, determining the final display state of the target picture according to the element width or the element height and the target picture width or the target picture height.

In some embodiments of the present application, multiple target pictures may be displayed in the same element position in the web page, for example, multiple target pictures are displayed in a scrolling manner in a display area with a fixed display size, and each target picture is displayed completely. To achieve the above technical effect, the controller 250 also needs to obtain original pictures corresponding to all the target pictures rolled and displayed in the display area, and perform resolution reduction processing on the original pictures according to the element size of the display area, so as to facilitate the display and switching of the target pictures, and release the memory of the browser. In this embodiment, the position coordinates for the plurality of target pictures are the same.

In this embodiment, if the user wants to view a clearer version of the target picture in the web page, the user can click on the target picture in the web page by controlling the cursor, at this time, the controller 250 will establish a transmission connection channel with the second buffer area or the newly-built third buffer area through the browser, and transmit the original picture corresponding to the target picture clicked by the user to the controller 250 through the transmission connection channel, and after receiving the original picture, the controller 250 controls the display 260 to display the original picture.

If the information contained in the web page is excessive, but the size of the visible area of the browser is unchanged. The display device 200 cannot fully present all the content in the web page. The user needs to control the cursor in the browser through the control device 100, so that the cursor can move the position of the webpage in the visible area, and the purpose of browsing the website content is achieved. In the process that the user moves the webpage through the cursor, the position of the target picture in the webpage also moves according to the movement of the cursor in the browser. Therefore, in some embodiments of the present application, the controller 250 may also acquire the position of the current window of the display device 200 and determine the web page screen located at the position of the window in the web page during the process of loading the target picture in the web page according to the element position. And then, identifying the webpage picture, and acquiring the number of target pictures in the webpage picture and the element position of each target picture.

The controller 250 may also monitor the vector distance moved by the cursor in the user's browser before the element position of each target picture is acquired. The subsequent controller 250 adjusts the display position of the target picture in the current window according to the vector distance, and the display position is used as the element position of the target picture in the current display picture. Therefore, in the process of browsing the webpage, a user only needs to load the target pictures in the current window, and does not need to load the target pictures outside the current window, so that the memory occupied by the browser when the user browses the webpage is reduced, and the smoothness of webpage browsing is improved.

It should be noted that, the vector distance in the above embodiment may be determined by the key of the user. The user may calculate the vector distance by controlling the "up", "down", "left" and "right" buttons on the apparatus 100, for example, the user clicks the "down" button twice and the "right" button once during browsing the web page, and the controller 250 may directly calculate the vector distance according to the above-mentioned button conditions, or may first calculate the moving distance in the corresponding direction, and finally calculate the vector distance. The vector distance may also be determined by some external device that establishes a communication connection with the display device 200, and the user may slide a component in the external device to control cursor movement, such as a mouse, a joystick with a joystick, or the like. The controller 250 may monitor the sliding condition of the external device, and calculate the corresponding distance of the external device in the real device 200 according to the sliding sensing degree, thereby calculating the vector distance. In addition, the user can control the cursor to be positioned at the scroll icon in the scroll bar of the webpage, and further pull the icon to achieve the aim of browsing. The controller 250 may now calculate the vector distance by calculating the displacement of the scroll icon in the scroll bar.

In some embodiments of the present application, if controller 250 does not query that there are background image attributes in the HTML element, then query the img image element in the HTML element. The format of the image elements is: < imgsrc= "picture URL" width= "attribute value" height= "attribute value" >. The URL address of the picture can be obtained through the src attribute, the width of the img image element can be obtained through the width attribute, and the height of the img image element can be obtained through the height attribute. The size of the final display of the picture is determined by the width attribute and the height attribute of the img image element. The width attribute and the height attribute are img image width and img image height of img image size.

As shown in fig. 11, in determining the original size by the above embodiment, the controller 250 sequentially prioritizes the background image attribute-background image size, the background image attribute-the initial size of the HTML element, and the img image element. In some embodiments of the present application, the query steps are as follows:

s101: preferentially inquires whether the HTML element has a background image attribute, and if so, step S102 is executed.

S102: inquiring whether the background image size exists in the background image attribute, and taking the image background size as the element size if the background image size exists.

S103: if there is no background image size, the initial size of the HTML element is taken as the element size.

S104: if the controller 250 does not query the background image attribute, the img image element in the HTML element is queried, thereby identifying the img image size of the img image element as the element size.

S105: if the img image element is not queried, the background image attribute and the img image element are queried again.

After the controller 250 determines the element size, it is also necessary to determine a ratio at which to perform the resolution reduction process on the original picture. Because performing the resolution reduction process requires determining a ratio at which the resolution reduction is performed on the original picture, in some embodiments of the present application, the controller 250 also requires extracting the original picture size of the original picture.

The original picture size includes an original picture height and an original picture width, and the controller 250 may calculate an original picture aspect ratio of the original picture according to the original picture height and the original picture width. If the aspect ratio of the original picture is equal to the aspect ratio of the element, which indicates that the original picture is matched with the area that can be displayed by the element size, the controller 250 may directly perform the zoom-out operation on the original picture, so that the original picture is equal to the element height and the original picture is equal to the element width, to obtain the target picture, then fill the target picture into the element position in the web page, and control the display 260 to display the web page, so that the user can browse the picture.

And comparing the original picture height with the element height if the original picture height to width ratio is greater than the element height to width ratio. For example, the original picture has an aspect ratio of 300:100 and an element aspect ratio of 200:100, which means that if the original picture is higher than the element height, the original picture cannot be displayed in a part exceeding the display area in the height direction of the original picture if the original picture is directly loaded according to the element size, and the user cannot see the complete picture content. The controller 250 needs to calculate a first ratio of the element height to the original picture height, in this embodiment, the first ratio is 2/3. As shown in fig. 12, after calculating the first ratio, the controller 250 sets the first ratio to a first reduction ratio, performs a resolution reduction process on the original picture to obtain a target picture, and stores the target picture in the second buffer.

In the above embodiment, when the controller 250 performs the resolution reduction process on the original picture, it is necessary to reduce the original picture to be the same as the element height, and display the target picture obtained after the resolution reduction of the original picture in the designated area according to the element size.

In the course that the original picture is subjected to the resolution reduction, the scaling is performed. In the process of reducing the original picture, if the original picture height is compressed to be aligned with the element height, the original picture width is also compressed in equal proportion. If the original picture height is compressed to align with the element height, but the original picture width is smaller than the element width, a blank area appears in the display area of the target picture.

The blank area does not display any content, and in order to reduce the blank area, if the element height is greater than the element width, the process of calculating the area of the blank area is as follows: and calculating a difference value of the element width minus the target picture width, and calculating a product of the difference value and the element height. The product result is the area of the blank area. In some embodiments of the present application, in order to reduce the area of the blank area, the original picture width may also be used as a basis for performing the resolution reduction. At this time, the controller 250 compresses the original picture width to be aligned with the element width, and since the original picture is scaled in equal proportion and the element height is larger than the element width, the area of the blank region where the resolution reduction is performed according to the element width is smaller than the area of the blank region where the resolution reduction is performed according to the element height. The height of the original picture is larger than that of the target picture display area, and the upper and lower parts of the original picture exceed the picture display area and cannot be displayed. Therefore, when the original picture width is aligned with the element width, a blank area does not appear in the display area of the target picture after the resolution reduction process is performed.

And if the original picture height-width ratio is smaller than the element height-width ratio, comparing the original picture width with the element width. For example, the original picture has an aspect ratio of 100:300 and an element aspect ratio of 100:200, which indicates that the original picture width is larger than the element width. The controller 250 needs to calculate a second ratio of the element width to the original picture width, in this embodiment, the second ratio is 2:3. The controller 250 sets the second reduction ratio to the second ratio, and performs a resolution reduction process on the original picture to obtain the target picture.

It should be noted that the aspect ratio may be a ratio of width to height or a ratio of height to width, which is not limited in this application. When comparing the aspect ratio, only the aspect ratio of the element and the aspect ratio of the original picture are required to be ensured to be the ratio of the width to the height, or the aspect ratio of the element and the aspect ratio of the original picture are required to be the ratio of the height to the width.

If the memory of the browser is already in a low state, the browser is set to a low resolution mode in order to be able to normally display the web page. In some embodiments of the present application, the controller 250 may further set a memory amount threshold of the browser, and obtain the system memory amount of the browser before performing the resolution reduction process on the original picture. If the system memory size is greater than the memory size threshold, then the reduced resolution processing may be performed normally on the original picture. And if the system memory size is smaller than the memory size threshold, the controller 250 performs a scaling operation on the original picture height by 1/N times the first scaling-down ratio and performs a scaling operation on the original picture width by 1/N times the second scaling-down ratio. Thus, the original picture can execute the resolution reduction processing according to 1/N times of the original first reduction ratio, the memory quantity occupied by the loading of the obtained target picture is 1/N times of the original target picture, and the resolution is correspondingly lower. In order to display other contents in the webpage preferentially and clearly under the condition of insufficient system memory of the browser, the target picture in the webpage is displayed secondarily. Thus, most of the system memory of the browser is allocated to other contents in the displayed web page, and a small portion of the system memory is used for displaying the target picture in the web page.

It should be noted that, N is a preset value, and the preset value may be set manually or may be set according to the system memory allowance of the browser. For example, when the system memory margin for displaying the target pictures is 300m, and all target pictures in the current web page are required to be displayed by 1200m, the required memory amount of 1200m after 1/N times execution is at least less than 300m, so N can be set to be greater than or equal to 4.

Based on the display device 200, some embodiments of the present application further provide a method for managing the memory of the loaded web page picture, which is applied to any one of the display devices 200, where the display device 200 includes at least a display 260 and a controller 250, and the method includes:

s100: acquiring a control instruction input by a user and used for opening a webpage;

s200: responding to the control instruction, acquiring webpage data of the webpage through a browser, wherein the browser comprises a first cache area and a second cache area;

s300: analyzing a download link and a layout tree of the original picture from the webpage data; the layout tree includes element sizes and element positions in the web page;

s400: downloading the compressed data of the original picture according to the download link and storing the compressed data in the first buffer area;

S500: decompressing the compressed data, and performing resolution reduction processing on the original picture obtained by decompression according to the element size to obtain a target picture;

s600: storing the target picture to the second buffer;

s700: and loading the target picture in the webpage according to the element position, and controlling the display to display the webpage.

It will be apparent to those skilled in the art that the techniques of embodiments of the present invention may be implemented in software plus a necessary general purpose hardware platform. Based on such understanding, the technical solutions in the embodiments of the present invention may be embodied in the form of a software product, which may be stored in a computer-readable storage medium, in essence or contributing to the prior art.

The same or similar parts between the various embodiments in this specification are referred to each other. In particular, for the display control method embodiment, since it is substantially similar to the display device 200 and terminal device 300 embodiments, the description is relatively simple, and reference is made to the description in the display device 200 and terminal device 300 embodiments for the matters.

Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.

The foregoing description, for purposes of explanation, has been presented in conjunction with specific embodiments. However, the illustrative discussions above are not intended to be exhaustive or to limit the embodiments to the precise forms disclosed above. Many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the present disclosure and to enable others skilled in the art to best utilize the embodiments.

Claims

1. A display device, characterized by comprising:

a display;

a controller configured to:

acquiring a control instruction input by a user and used for opening a webpage;

Storing the target picture to the second buffer;

2. The display device of claim 1, wherein the web page data includes hypertext markup language HTML elements, cascading style sheet CSS data, and Javascript scripting language, and wherein the controller is further configured to, in the step of parsing the download links and layout trees of the original pictures from the web page data:

analyzing the webpage data through a browser;

3. The display device of claim 2, wherein the controller is further configured to:

layering the layout tree to construct a layer tree;

4. The display device according to claim 2, wherein in the step of performing the resolution-reducing process on the original picture obtained by decompression according to the element size, the controller is further configured to:

inquiring the background image attribute in the HTML element;

5. The display device according to claim 4, wherein in the step of performing the resolution reduction process on the original picture obtained by decompression according to the element size, the controller is further configured to:

6. The display device according to claim 4, wherein in the step of performing the resolution reduction process on the original picture obtained by decompression according to the element size, the controller is further configured to:

7. The display device according to any one of claims 4 to 6, wherein in the step of performing the resolution reduction processing on the original picture obtained by decompression according to the element size, the controller is further configured to:

8. The display device according to claim 7, wherein in the step of performing the resolution reduction processing on the original picture obtained by decompression according to the element size, the controller is further configured to:

9. The display device according to claim 8, wherein before the step of performing the resolution reduction process on the original picture obtained by decompression according to the element size, the controller is further configured to:

acquiring the system memory quantity of a browser;

10. A memory management method for loading web page pictures, which is applied to a display device, wherein the display device comprises a display and a controller, and is characterized in that the method comprises the following steps:

acquiring a control instruction input by a user and used for opening a webpage;

storing the target picture to the second buffer;