CN109948087B - Webpage resource obtaining method and device and terminal - Google Patents

Webpage resource obtaining method and device and terminal Download PDF

Info

Publication number
CN109948087B
CN109948087B CN201711270485.6A CN201711270485A CN109948087B CN 109948087 B CN109948087 B CN 109948087B CN 201711270485 A CN201711270485 A CN 201711270485A CN 109948087 B CN109948087 B CN 109948087B
Authority
CN
China
Prior art keywords
webpage
address
preloaded
historical
webpage address
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201711270485.6A
Other languages
Chinese (zh)
Other versions
CN109948087A (en
Inventor
陈岩
刘耀勇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangdong Oppo Mobile Telecommunications Corp Ltd
Original Assignee
Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangdong Oppo Mobile Telecommunications Corp Ltd filed Critical Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority to CN201711270485.6A priority Critical patent/CN109948087B/en
Priority to PCT/CN2018/116224 priority patent/WO2019109797A1/en
Publication of CN109948087A publication Critical patent/CN109948087A/en
Application granted granted Critical
Publication of CN109948087B publication Critical patent/CN109948087B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/95—Retrieval from the web
    • G06F16/953—Querying, e.g. by the use of web search engines
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/95—Retrieval from the web
    • G06F16/955—Retrieval from the web using information identifiers, e.g. uniform resource locators [URL]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/95—Retrieval from the web
    • G06F16/957—Browsing optimisation, e.g. caching or content distillation

Abstract

The application discloses a method, a device and a terminal for acquiring webpage resources. The method comprises the following steps: acquiring at least one historical webpage address; selecting a common webpage address from historical webpage addresses, wherein the common webpage address refers to the historical webpage address with the access frequency being greater than the preset frequency in a historical period; downloading webpage resources corresponding to the common webpage addresses; crawling at least one webpage resource corresponding to a target webpage address according to the webpage resource corresponding to the common webpage address; and storing webpage resources corresponding to the preloaded webpage addresses, wherein the preloaded webpage addresses comprise common webpage addresses and target webpage addresses. In the embodiment of the application, the webpage resources corresponding to the preloaded webpage addresses are downloaded and stored in advance, so that when a webpage access request is received subsequently, the requested webpage resources can be acquired from the prestored webpage resources, the webpage resources do not need to be requested to be downloaded from a network, the time consumed for acquiring the webpage resources is reduced, and the webpage display efficiency is improved.

Description

Webpage resource obtaining method and device and terminal
Technical Field
The embodiment of the application relates to the technical field of internet, in particular to a method, a device and a terminal for acquiring webpage resources.
Background
With the development of network technology, web browsing has become an essential part of people's life.
In the related art, when a user browses a web page on a terminal, for example, the user browses news in an application program of a mobile phone, and the terminal displays the web page when the user selects one web page. Before displaying the web page, the terminal needs to acquire the web page resource of the web page first to display the web page. Therefore, after a user selects a webpage, the terminal downloads the webpage Resource of the webpage according to a Uniform Resource Locator (URL) of the webpage, and the terminal needs to consume a certain amount of time to download the webpage Resource according to the URL. The URL is also called a web page address.
In the manner of acquiring the web page resource provided by the related art, after the user selects the web page, the terminal downloads the web page resource according to the web page address of the web page, so that the terminal needs to download the web page resource for a while after the user selects the web page, thereby causing a long waiting time before displaying the web page and a low efficiency of displaying the web page.
Disclosure of Invention
The embodiment of the application provides a method and a device for acquiring webpage resources and a terminal, and can be used for solving the problems that the waiting time before the webpage is displayed is long and the efficiency of displaying the webpage is low because the terminal downloads the webpage resources after a user selects the webpage. The technical scheme is as follows:
in a first aspect, a method for acquiring a web page resource is provided, where the method includes:
acquiring at least one historical webpage address, wherein the historical webpage address refers to a webpage address visited in a historical period;
selecting a common webpage address from the historical webpage addresses, wherein the common webpage address refers to the historical webpage address with the access frequency being greater than the preset frequency in the historical period;
downloading the webpage resources corresponding to the common webpage addresses;
according to the webpage resources corresponding to the common webpage addresses, crawling the webpage resources corresponding to at least one target webpage address;
and storing webpage resources corresponding to a preloaded webpage address, wherein the preloaded webpage address comprises the common webpage address and the target webpage address.
In a second aspect, an apparatus for acquiring a web resource is provided, the apparatus includes:
the system comprises an address acquisition module, a storage module and a processing module, wherein the address acquisition module is used for acquiring at least one historical webpage address, and the historical webpage address refers to a webpage address visited in a historical period;
the address selection module is used for selecting a commonly-used webpage address from the historical webpage addresses, wherein the commonly-used webpage address refers to the historical webpage address with the access frequency greater than the preset frequency in the historical period;
the resource acquisition module is used for downloading the webpage resources corresponding to the common webpage addresses;
the resource acquisition module is also used for crawling at least one webpage resource corresponding to the target webpage address according to the webpage resource corresponding to the common webpage address;
and the resource storage module is used for storing webpage resources corresponding to the pre-loading webpage addresses, and the pre-loading webpage addresses comprise the common webpage addresses and the target webpage addresses.
In a third aspect, a terminal is provided, where the terminal includes a processor and a memory, and the memory stores at least one instruction, where the instruction is loaded and executed by the processor to implement the method for acquiring a web page resource according to the first aspect.
In a fourth aspect, a computer-readable storage medium is provided, where at least one instruction is stored in the computer-readable storage medium, and the instruction is loaded and executed by a processor to implement the method for acquiring a web resource according to the first aspect.
In a fifth aspect, a computer program product is provided, which, when executed, is adapted to implement the method for acquiring web page resources as described in the first aspect above.
The technical scheme provided by the embodiment of the application can bring the following beneficial effects:
the webpage resources corresponding to the common webpage addresses and the webpage resources corresponding to the target webpage addresses and crawled from the webpage resources corresponding to the common webpage addresses are downloaded and stored in advance, so that when a webpage access request is received subsequently, the requested webpage resources can be acquired from the prestored webpage resources, the webpage resources do not need to be requested to be downloaded from a network, the time consumed for acquiring the webpage resources is reduced, and the webpage display efficiency is improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
FIG. 1 is a flow diagram illustrating a method for obtaining web page resources in accordance with an exemplary embodiment;
FIG. 2 is a diagram illustrating a web page crawl in accordance with an exemplary embodiment;
FIG. 3 is a flowchart illustrating a method for obtaining web page resources in accordance with another exemplary embodiment;
FIG. 4 is a block diagram illustrating an apparatus for obtaining web page resources in accordance with an exemplary embodiment;
fig. 5 is a schematic diagram illustrating a structure of a terminal according to an exemplary embodiment.
Detailed Description
To make the objects, technical solutions and advantages of the present application more clear, embodiments of the present application will be described in further detail below with reference to the accompanying drawings.
In the method provided by the embodiment of the application, the execution main body of each step may be a terminal. Alternatively, the execution subject of each step may be an application program running in the terminal. The application refers to an application capable of displaying a web page, such as a browser, a news application, instant messaging software, social application software, video playing software, and the like. The terminal can be an electronic device such as a mobile phone, a tablet computer, an electronic book reader, a multimedia playing device, a laptop portable computer, and the like.
FIG. 1 is a flowchart illustrating a method for acquiring web page resources according to an exemplary embodiment. The method may include the steps of:
step 101, at least one historical webpage address is obtained.
The terminal acquires at least one historical webpage address, wherein the historical webpage address refers to a webpage address accessed by the terminal in a historical period.
Optionally, this step includes several substeps as follows:
1. obtaining historical access data, wherein the historical access data comprises at least one access record;
each access record comprises a webpage address corresponding to one webpage access operation; optionally, each access record further includes at least one of a timestamp, an application program, and a network type corresponding to a web page access operation.
The timestamp indicates the specific time at which the access recorded for that access record occurred. For example, in the record of web page browsing by the user, a historical web page address has 2 access records in the history period, and the time indicated by the time stamp of the 2 access records is 2017/6/1613: 24:25 and 2017/6/178: 29:30 respectively. The user has accessed the historical web page addresses at 2017/6/1613: 24:25 and 2017/6/178: 29:30, respectively.
The application indicates an application that performs the access recorded in the access record.
The Network type indicates a Network type used by The terminal when performing The access recorded in The access record, for example, a Wireless Local Area Network (WLAN), The 4th Generation Mobile Communication (4G), a Local Area Network (LAN), and The like.
Illustratively, the historical access data acquired by the terminal is shown in the following table-1:
network type Application program Web page address Time stamp
4G XX browser www.baid.com 2017/6/16 13:24:25
WLAN XX News www.XXnews.com 2017/6/17 8:29:30
TABLE-1
As shown in Table-1, the user accessed the web page with web page address www.baid.com through the 4G network at 2017/6/1613: 24:25 using the XX browser.
2. And acquiring an access record in a history period.
And the terminal acquires the access record in the history period from the history access data. The history period may be set according to actual experience, for example, if the history period is set to be 7 days, the terminal acquires an access record within 7 days from the history access data.
3. And determining at least one historical webpage address according to the access records in the historical period.
And the terminal determines the webpage addresses in the access records in the acquired history period as history webpage addresses, and each history webpage address can have multiple access records in the history period.
And 102, selecting a common webpage address from the historical webpage addresses.
After the historical webpage addresses are obtained, the terminal selects the common webpage addresses according to the access frequency of the historical webpage addresses in the historical period. The access frequency of a historical webpage address is used for indicating the frequency of accessing the corresponding webpage by the terminal by using the historical webpage address. And the terminal selects the historical webpage address with the access frequency greater than the preset frequency in the historical period as the common webpage address. The preset frequency can be set according to actual experience. For example, if the preset frequency is set to 10 times per day on average, the selected commonly used web page address is a historical web page address with an access frequency greater than 10 times per day on average. Through the method, the terminal selects the webpage addresses frequently visited by the user from the historical webpage addresses. Optionally, the access frequency of the historical webpage address may be any one of an average number of accesses per day, an average number of accesses per hour, a total number of accesses in the historical period, an average time interval of each access, and a minimum number of accesses per day, which is not limited by the embodiment of the present invention.
And 103, downloading the webpage resources corresponding to the common webpage addresses.
And after the terminal selects the common webpage address, downloading the corresponding webpage resource according to the common webpage address. The webpage address comprises the resource type of the webpage resource, the host domain name for storing the resource and the resource file name, and the terminal can determine the position of the corresponding webpage resource in the network according to the webpage address so as to download the webpage resource. Alternatively, the web page address may be a URL. The downloaded web page resources include video, audio, pictures, and web page addresses of other web pages, among others. And the terminal can display the webpage corresponding to the webpage resource according to the webpage resource.
And 104, crawling the webpage resources corresponding to at least one target webpage address according to the webpage resources corresponding to the common webpage addresses.
And the terminal downloads the webpage resource corresponding to the webpage address according to the webpage address included in the downloaded webpage resource, downloads the webpage resource corresponding to the webpage address included in the newly downloaded webpage resource according to the webpage address included in the newly downloaded webpage resource, and repeats the steps until the downloaded webpage resource meets the preset condition, for example, the data volume of the downloaded webpage resource reaches the upper limit of the storage capacity. The process of downloading web page resources is referred to as web page crawling. Through the webpage crawling, after downloading the webpage resources corresponding to the common webpage addresses, the terminal crawls the webpage resources corresponding to at least one target webpage address. Illustratively, the web resource corresponding to the common web address a includes a web address B, the web resource corresponding to the web address B includes a web address C, and the web resource corresponding to the web address C includes a web address D, so that the web address B, the web address C, and the web address D are target web addresses crawled according to the common web address a.
Optionally, in the process of crawling the web resource corresponding to at least one target web address, the terminal first extracts the web address included in the web resource corresponding to the common web address, and uses the web address as the target web address extracted in the 1 st round. And the terminal downloads the webpage resource corresponding to the target webpage address extracted in the 1 st round, extracts the webpage address contained in the webpage resource, and takes the webpage address as the target webpage address extracted in the 2 nd round. And the terminal downloads the webpage resources corresponding to the target webpage address extracted in the 2 nd round. And then the terminal extracts the target webpage address in the 3 rd round according to the webpage resource corresponding to the target webpage address extracted in the 2 nd round, and so on, and stops downloading until the data volume of the stored webpage resource is larger than the preset data volume. At this time, the web resource corresponding to the target web address extracted in each round is the web resource corresponding to at least one target web address crawled by the terminal. Illustratively, as shown in fig. 2, a web page address a is a commonly used web page address, the terminal first downloads a web page resource corresponding to the web page address a, and the web page resource corresponding to the web page address a includes a web page address B and a web page address C. The webpage address B and the webpage address C are target webpage addresses extracted in the 1 st round. And the terminal downloads a webpage resource corresponding to the webpage address B and a webpage resource corresponding to the webpage address C, wherein the webpage resource corresponding to the webpage address B comprises a webpage address D and a webpage address E, and the webpage resource corresponding to the webpage address C comprises a webpage address F and a webpage address G. The webpage address D, the webpage address E, the webpage address F and the webpage address G are target webpage addresses extracted in the 2 nd round. And the terminal downloads the webpage resources corresponding to the webpage address D, the webpage address E, the webpage address F and the webpage address G respectively. And analogizing until the data volume of the stored webpage resources is larger than the preset data volume, and stopping webpage crawling by the terminal. Finally, the webpage resources corresponding to the target webpage address crawled by the terminal are respectively corresponding to the webpage address B, the webpage address C, the webpage address D, the webpage address E, the webpage address F and the webpage address G.
It should be noted that the terminal may stop the web page crawling after downloading the web page resource corresponding to the target web page address in one round, or may stop the web page crawling in the process of downloading the web page resource corresponding to the target web page address in one round.
In addition, the above-mentioned condition of stopping downloading, "when the data amount of the stored web page resource is greater than the preset data amount," is only exemplary, in other possible embodiments, the condition of stopping downloading may also be that the number of the downloaded target web page addresses is greater than the preset number, the number of rounds of crawling is greater than the preset number of rounds, the terminal finishes downloading the web page resource corresponding to the target web page address in one round, and the remaining storage capacity of the terminal is less than the preset capacity, and the like, which is not limited in this embodiment of the present invention.
And 105, storing the webpage resources corresponding to the preloaded webpage addresses.
The preloaded webpage address comprises a common webpage address and a target webpage address. And the terminal stores the webpage resources corresponding to the downloaded common webpage addresses and the webpage resources corresponding to the target webpage addresses downloaded in the crawling process, and stores the webpage resources into a nonvolatile memory of the terminal as the webpage resources corresponding to the preloaded webpage addresses. For example, the web page resource corresponding to the target web page address is stored in the hard disk. And the terminal stores the webpage resources corresponding to the preloaded webpage addresses and records the preloaded webpage addresses. In addition, the terminal also records the corresponding relation between the preloaded webpage address and the stored webpage resource.
Optionally, the step 105 further includes the following steps (not shown in fig. 1):
and step 106, acquiring a webpage access request carrying the request webpage address.
When a user requests to display a webpage, the terminal acquires a webpage access request. The web page access request carries a request web page address, namely a web page address of a web page opened by a user. Illustratively, the user browses the web page by using a browser, and inputs a web page address in the address input field, and the requested web page address carried in the web page access request is the web page address input by the user.
Step 107, detecting whether the preloaded web page address includes the request web page address.
And the terminal detects whether the request webpage address is contained or not according to the recorded preloaded webpage address.
And step 108, if the preloaded webpage address contains the request webpage address, displaying the webpage corresponding to the request webpage address according to the stored webpage resource corresponding to the request webpage address.
And if the terminal detects that the preloaded webpage address contains the request webpage address, the terminal is indicated to download and store the webpage resource corresponding to the request webpage address. The terminal directly displays the webpage of the request webpage address according to the webpage resource corresponding to the request webpage address stored in the nonvolatile memory without downloading the webpage resource corresponding to the request webpage address again.
Step 109, if the preloaded web page address does not include the requested web page address, downloading the web page resource corresponding to the requested web page address, and displaying the web page corresponding to the requested web page address.
In summary, according to the method provided by the embodiment of the application, the webpage resources corresponding to the common webpage addresses and the webpage resources corresponding to the target webpage addresses crawled from the webpage resources corresponding to the common webpage addresses are downloaded and stored in advance, so that when a webpage access request is subsequently received, the requested webpage resources can be acquired from the prestored webpage resources, the webpage resources do not need to be requested to be downloaded from a network, the time consumed for acquiring the webpage resources is reduced, and the efficiency for displaying the webpage is improved.
In addition, the webpage resources corresponding to the target webpage address and crawled from the webpage resources corresponding to the common webpage addresses can enrich the content of the prestored webpage resources and accord with the operation habit of the user when the user actually browses the webpage, so that the webpage resources of the webpage which is most likely to be browsed by the user are downloaded as far as possible, and the utilization rate and the effectiveness of the prestored webpage resources are ensured.
FIG. 3 is a flowchart illustrating a method for acquiring web page resources according to another exemplary embodiment. The method may include the steps of:
step 301, at least one historical webpage address is obtained.
Step 301 is the same as step 101 in the embodiment provided in fig. 1, and please refer to the embodiment in fig. 1 for detailed description, which is not repeated herein.
Step 302, for each historical webpage address, acquiring a first access timestamp and a last access timestamp of the historical webpage address in a historical period.
And the first access time stamp of the historical webpage address acquired by the terminal in the historical period represents the time when the user accesses the historical webpage address for the first time in the historical period. And the last access timestamp of the historical webpage address acquired by the terminal in the historical period represents the last time when the user accesses the historical webpage address in the historical period.
Step 303, calculating the time interval between the first access time stamp and the last access time stamp.
And the terminal calculates the interval duration of the first access to the historical webpage address and the last access to the historical webpage address of the user in the historical period according to the time represented by the first access timestamp and the time represented by the last access timestamp. For example, the time represented by the first access time stamp and the time represented by the last access time stamp are 2017/6/1613: 24:25 and 2017/6/1713: 24:25, respectively, and the calculated interval duration is 1 day.
And step 304, calculating the access frequency of the historical webpage address according to the total access times and interval duration of the historical webpage address in the historical period.
After the terminal calculates the interval duration, the access frequency can be calculated by dividing the total access times of the historical webpage addresses in the historical period by the interval duration. Illustratively, the total number of accesses and the interval duration of a historical webpage address in the historical period are respectively 10 and 2 days, and the access frequency of the historical webpage address is 5 times per day.
In step 305, if the access frequency of the historical webpage address is greater than the preset frequency, the historical webpage address is selected as a common webpage address.
And the terminal selects the historical webpage address with the access frequency greater than the preset frequency in the historical period as the common webpage address. Illustratively, the terminal obtains two historical web page addresses: historical web page address A and historical web page address B. The access frequency of the historical webpage address A is 5 times per day, the access frequency of the historical webpage address B is 3 times per day, the preset frequency is 4 times per day, and then the terminal selects the historical webpage address A as a common webpage address.
Optionally, before step 302, the terminal may further detect whether the total number of access times of the historical webpage address in the historical period is greater than a preset number of times; if the total access times of the historical webpage addresses in the historical period are greater than the preset times, executing step 302; if the total number of accesses of the historical webpage address in the historical period is not greater than the preset number, the step 302 is not executed, and the historical webpage address is not selected as the commonly used webpage address. Since the access frequency is obtained by dividing the total number of accesses by the interval duration, if the interval duration of a historical webpage address is too short, the calculated access frequency is higher even if the total number of accesses is small. However, the historical webpage addresses are accessed less frequently in the historical period, and are not webpage addresses frequently accessed by the user. If the terminal selects the historical webpage address as the common webpage address, the fact that the selected common webpage address has errors is meant. By the mode of firstly detecting whether the total access times are greater than the preset times, the error can be avoided, and the accuracy of the selected common webpage address is ensured.
And step 306, downloading the webpage resources corresponding to the common webpage addresses.
Step 307, crawling a web resource corresponding to at least one target web address according to a web resource corresponding to a common web address.
And 308, storing the webpage resources corresponding to the preloaded webpage addresses.
The above steps 306 to 308 are the same as the steps 103 to 105 in the embodiment provided in fig. 1, and please refer to the embodiment in fig. 1 for detailed description. This embodiment will not be described in detail.
Optionally, for each preloaded webpage address, the terminal obtains a historical access speed corresponding to the preloaded webpage address. And the historical access speed corresponding to the preloaded webpage address is used for indicating the downloading speed of the webpage resource corresponding to the preloaded webpage address. If the historical access speed corresponding to the preloaded webpage address is lower than the preset speed, the speed of downloading the webpage resource corresponding to the preloaded webpage address is low, and the waiting time before the webpage is displayed is too long, the terminal stores the webpage resource corresponding to the preloaded webpage address; if the historical access speed corresponding to the preloaded webpage address is higher than the preset speed, the speed of downloading the webpage resource corresponding to the preloaded webpage address is higher, the waiting time before the webpage is displayed is not too long, and the webpage resource corresponding to the preloaded webpage address is not stored by the terminal. By the method, the webpage resources corresponding to the webpage addresses with slower downloading speed can be selectively stored. And for the webpage resources corresponding to the webpage addresses which are downloaded quickly, the terminal does not store the webpage resources, so that the storage space is saved. Optionally, the terminal only uses the above manner for the commonly used web page address. The terminal stores the access records of the commonly used webpage addresses, and the access records comprise historical access speeds corresponding to the commonly used webpage addresses, so that the terminal can directly obtain the historical access speeds corresponding to the commonly used webpage addresses.
Optionally, after storing the web resource corresponding to the preloaded web address, the terminal starts a restart timer. And when the restart timer reaches the preset time length. The terminal deletes the stored web resource corresponding to the preloaded web address, and starts execution again from step 301. The preset time length can be set according to actual experience or user requirements. Through the method, the terminal can periodically determine the preloaded webpage address according to the latest historical webpage address of the user, and download and store the webpage resource corresponding to the preloaded webpage address, so that the timeliness of the webpage resource corresponding to the preloaded webpage address is guaranteed.
In summary, according to the method provided by the embodiment of the application, the webpage resources corresponding to the common webpage addresses and the webpage resources corresponding to the target webpage addresses crawled from the webpage resources corresponding to the common webpage addresses are downloaded and stored in advance, so that when a webpage access request is subsequently received, the requested webpage resources can be acquired from the prestored webpage resources, the webpage resources do not need to be requested to be downloaded from a network, the time consumed for acquiring the webpage resources is reduced, and the efficiency for displaying the webpage is improved.
In addition, the access frequency is determined according to the interval duration between the first access timestamp and the last access timestamp, so that the frequently-used webpage address is determined, the selected frequently-used webpage address is ensured to be the address frequently accessed by the user as far as possible, and the utilization rate and the effectiveness of the prestored webpage resource are further ensured.
The above embodiments describe the process of downloading and storing the web page resource corresponding to the preloaded web page address in advance. Optionally, for each preloaded web page address, the terminal periodically updates its corresponding web page resource. The method for acquiring the webpage resources provided by the embodiment of the application can further comprise the following steps:
1. and for each preloading webpage address, acquiring a resource updating time length corresponding to the preloading webpage address.
The content of a web page may change over time, and as the content of a web page changes, so too may the web page resources. In order to ensure that the latest contents of the web page can be displayed, the terminal needs to update the stored web page resources. And for each preloaded webpage address, the terminal acquires the resource updating duration corresponding to the preloaded webpage address after storing the corresponding webpage resource. And after the storage, the terminal updates the webpage resources corresponding to the preloaded webpage addresses according to the resource updating duration.
Optionally, the terminal obtains the resource update duration corresponding to the preloaded webpage address by the following method:
step a, when the stored duration of the webpage resource corresponding to the preloaded webpage address reaches the ith resource updating duration corresponding to the preloaded webpage address, the webpage resource corresponding to the preloaded webpage address is downloaded again, wherein the initial value of i is 1, and the 1 st resource updating duration is a preset value.
The stored duration of the webpage resource refers to a time interval from the last time the terminal downloads the webpage resource to the current time. And after downloading the webpage resources corresponding to the preloaded webpage address for the ith time by the terminal, the terminal downloads the webpage resources corresponding to the preloaded webpage address again after the resource updating duration for the ith time. Illustratively, the 1 st time resource updating duration is preset to be 10 minutes, and after the 1 st time downloading of the webpage resource corresponding to the preloaded webpage address, the terminal downloads the webpage resource corresponding to the preloaded webpage address again after 10 minutes.
And b, detecting whether the webpage resources corresponding to the reloaded preloaded webpage addresses change relative to the webpage resources corresponding to the stored preloaded webpage addresses.
And c, if the change does not occur, determining the (i + 1) th resource updating time length corresponding to the preloaded webpage address, wherein the (i + 1) th resource updating time length is greater than the ith resource updating time length.
And after determining the resource updating time length of the (i + 1) th time, the terminal sets i to be i +1 and starts to execute the step a again. And adding the resource updating time lengths from the 1 st time to the (i-1) th time until the webpage resource corresponding to the reloaded preloaded webpage address changes relative to the webpage resource corresponding to the stored preloaded webpage address to obtain the resource updating time length corresponding to the preloaded webpage address.
Optionally, the resource update duration of the ith time is TiThen the terminal determines the resource updating time length T of the (i + 1) th timei+1＝a×TiAnd a is a constant greater than 1, for example, a ═ 2.
Illustratively, the terminal downloads the webpage resource corresponding to the preloaded webpage address for the 1 st time, and the preset 1 st time resource updating time is 10 minutes. And after 10 minutes, the terminal downloads the webpage resources corresponding to the preloaded webpage addresses again. And if the webpage resources corresponding to the re-downloaded preloaded webpage addresses do not change relative to the webpage resources corresponding to the stored preloaded webpage addresses, the terminal determines that the resource updating time of the 2 nd time is 20 minutes. And after 20 minutes, the terminal downloads the webpage resources corresponding to the preloaded webpage addresses again. And if the webpage resources corresponding to the re-downloaded preloaded webpage addresses do not change relative to the webpage resources corresponding to the stored preloaded webpage addresses, the terminal determines that the 3 rd time resource updating time is 40 minutes. And after 40 minutes, the terminal downloads the webpage resources corresponding to the preloaded webpage addresses again. And the webpage resource corresponding to the reloaded preloading webpage address changes relative to the webpage resource corresponding to the stored preloading webpage address, and the value of i is 3 at the moment. And the terminal adds the 1 st resource updating time length and the 2 nd resource updating time length to obtain the resource updating time length corresponding to the preloaded webpage address of 30 minutes.
And d, if the change occurs, determining the (i + 1) th resource updating time length corresponding to the preloaded webpage address, wherein the (i + 1) th resource updating time length is less than the ith resource updating time length.
And after determining the resource updating time length of the (i + 1) th time, the terminal sets i to be i +1 and starts to execute the step a again. And determining the resource updating time length of the ith time as the resource updating time length corresponding to the preloaded webpage address until the webpage resource corresponding to the reloaded preloaded webpage address does not change relative to the webpage resource corresponding to the stored preloaded webpage address.
Optionally, the resource update duration of the ith time is TiThen the terminal determines the resource updating time length T of the (i + 1) th timei+1＝b×TiAnd b is a constant which is lower than 1 and greater than 0, for example, a is 0.5.
Illustratively, the terminal downloads the webpage resource corresponding to the preloaded webpage address for the 1 st time, and the preset 1 st time resource updating time is 10 minutes. And after 10 minutes, the terminal downloads the webpage resources corresponding to the preloaded webpage addresses again. And if the webpage resources corresponding to the re-downloaded preloaded webpage addresses change relative to the webpage resources corresponding to the stored preloaded webpage addresses, the terminal determines that the resource updating time of the 2 nd time is 5 minutes. And after 5 minutes, the terminal downloads the webpage resources corresponding to the preloaded webpage addresses again. And if the webpage resources corresponding to the re-downloaded preloaded webpage addresses change relative to the webpage resources corresponding to the stored preloaded webpage addresses, the terminal determines that the 3 rd time resource updating time is 2.5 minutes. And after 2.5 minutes, the terminal downloads the webpage resources corresponding to the preloaded webpage addresses again. And the webpage resource corresponding to the reloaded preloading webpage address is unchanged relative to the webpage resource corresponding to the stored preloading webpage address, and the value of i is 3 at the moment. The terminal takes the 3 rd time resource updating duration as the resource updating duration corresponding to the preloaded webpage address: 2.5 minutes.
Through the method, the terminal can independently determine the resource updating time length for each preloaded webpage address, the accuracy of the resource updating time length is ensured, the webpage resources corresponding to the preloaded webpage addresses can be ensured to be updated in time, and the processing overhead of updating the webpage resources can be saved.
2. And when the stored duration of the webpage resources corresponding to the preloaded webpage addresses reaches the resource updating duration corresponding to the preloaded webpage addresses, the webpage resources corresponding to the preloaded webpage addresses are downloaded again.
3. And updating the stored webpage resources corresponding to the preloaded webpage addresses by adopting the webpage resources corresponding to the reloaded preloaded webpage addresses.
And after the resource updating duration corresponding to the pre-loading webpage address is determined, the terminal updates the webpage resource corresponding to the stored pre-loading webpage address according to the resource updating duration. And for the changed webpage resources, the terminal deletes the webpage resources downloaded last time and stores the newly downloaded webpage resources. For the unchanged webpage resources, the terminal can delete the webpage resources downloaded last time, store the newly downloaded webpage resources, and also can still store the webpage resources downloaded last time.
By the method, the corresponding webpage resources can be updated according to the resource updating duration of each preloaded webpage address, the stored webpage resources corresponding to the preloaded webpage addresses are the latest webpage resources as far as possible, and the timeliness of the webpage resources is ensured.
The following are embodiments of the apparatus of the present application that may be used to perform embodiments of the method of the present application. For details which are not disclosed in the embodiments of the apparatus of the present application, reference is made to the embodiments of the method of the present application.
Fig. 4 is a block diagram illustrating an apparatus for acquiring web page resources according to an example embodiment. The device has the functions of realizing the method examples, and the functions can be realized by hardware or by hardware executing corresponding software. The apparatus may include: an address acquisition module 401, an address selection module 402, a resource acquisition module 403 and a resource storage module 404.
The address obtaining module 401 is configured to obtain at least one historical webpage address, where the historical webpage address refers to a webpage address visited in a historical period.
An address selecting module 402, configured to select a commonly used web page address from the historical web page addresses, where the commonly used web page address is a historical web page address whose access frequency is greater than a preset frequency in the historical period.
A resource obtaining module 403, configured to download the web resource corresponding to the commonly used web address.
The resource obtaining module 403 is further configured to crawl, according to the web resource corresponding to the common web address, a web resource corresponding to at least one target web address.
The resource storage module 404 is configured to store a web resource corresponding to a preloaded web address, where the preloaded web address includes the common web address and the target web address.
In summary, the device provided in the embodiment of the present application downloads and stores the web resource corresponding to the common web address and the web resource corresponding to the target web address crawled from the web resource corresponding to the common web address in advance, so that when a web access request is subsequently received, the requested web resource can be obtained from the pre-stored web resource, and the web resource does not need to be downloaded from the network, thereby reducing the time consumed for obtaining the web resource and improving the efficiency for displaying the web page.
In an optional embodiment provided based on the embodiment of fig. 4, the resource obtaining module 403 is configured to: extracting a target webpage address contained in the webpage resource corresponding to the common webpage address; downloading the webpage resources corresponding to the target webpage address extracted in the 1 st round; extracting target webpage addresses contained in webpage resources corresponding to the target webpage addresses extracted in the 1 st round; and downloading the webpage resources corresponding to the target webpage address extracted in the 2 nd round, and repeating the steps until the data volume of the stored webpage resources is larger than the preset data volume, and stopping downloading.
In an alternative embodiment provided based on the embodiment of fig. 4, the address selecting module 402 is configured to: for each historical webpage address, acquiring a first access timestamp and a last access timestamp of the historical webpage address in the historical period; calculating the interval duration between the first access timestamp and the last access timestamp; calculating the access frequency of the historical webpage address according to the total access times of the historical webpage address in the historical period and the interval duration; and when the access frequency of the historical webpage address is greater than the preset frequency, selecting the historical webpage address as the common webpage address.
In some embodiments of the present application, the address selecting module 402 is further configured to: detecting whether the total access times of the historical webpage addresses in the historical period are greater than a preset time or not; and when the total access times of the historical webpage addresses in the historical period are greater than the preset times, starting to execute from the step of acquiring the first access timestamp and the last access timestamp of the historical webpage addresses in the historical period.
In an optional embodiment provided based on the embodiment of fig. 4, the apparatus further includes: and a duration obtaining module.
And the duration acquisition module is used for acquiring the resource updating duration corresponding to each preloaded webpage address.
The resource obtaining module 403 is further configured to re-download the web resource corresponding to the preloaded web address when the stored duration of the web resource corresponding to the preloaded web address reaches the resource updating duration corresponding to the preloaded web address.
The resource storage module 404 is further configured to update the stored webpage resource corresponding to the preloaded webpage address by using the webpage resource corresponding to the reloaded preloaded webpage address.
In some embodiments of the present application, the duration obtaining module is configured to: when the stored time length of the webpage resource corresponding to the preloaded webpage address reaches the ith resource updating time length corresponding to the preloaded webpage address, the webpage resource corresponding to the preloaded webpage address is downloaded again, wherein the initial value of i is 1, and the 1 st resource updating time length is a preset value;
detecting whether the webpage resources corresponding to the reloaded preloaded webpage addresses change relative to the stored webpage resources corresponding to the preloaded webpage addresses;
if not, determining the i +1 th resource updating time length corresponding to the preloaded webpage address, wherein the i +1 th resource updating time length is greater than the i-th resource updating time length; when the stored duration of the webpage resources corresponding to the preloaded webpage address reaches the ith resource updating duration corresponding to the preloaded webpage address, the step of downloading the webpage resources corresponding to the preloaded webpage address again starts to be executed, and when the webpage resources corresponding to the preloaded webpage address which are downloaded again change relative to the stored webpage resources corresponding to the preloaded webpage address, the 1 st resource updating duration to the (i-1) th resource updating duration are added to obtain the resource updating duration corresponding to the preloaded webpage address;
if the change occurs, determining the resource updating time length of the (i + 1) th time corresponding to the preloaded webpage address, wherein the resource updating time length of the (i + 1) th time is less than the resource updating time length of the ith time; and when the stored duration of the webpage resources corresponding to the preloaded webpage address reaches the ith resource updating duration corresponding to the preloaded webpage address, the step of re-downloading the webpage resources corresponding to the preloaded webpage address is started to be executed again until the re-downloaded webpage resources corresponding to the preloaded webpage address do not change relative to the stored webpage resources corresponding to the preloaded webpage address, and the ith resource updating duration is determined as the resource updating duration corresponding to the preloaded webpage address.
In another optional embodiment provided based on the embodiment of fig. 4, the resource storage module 404 is further configured to: for each preloaded webpage address, obtaining the historical access speed corresponding to the preloaded webpage address, wherein the historical access speed corresponding to the preloaded webpage address is used for indicating the downloading speed of the webpage resource corresponding to the preloaded webpage address; if the historical access speed corresponding to the preloaded webpage address is lower than the preset speed, storing the webpage resource corresponding to the preloaded webpage address; and if the historical access speed corresponding to the preloaded webpage address is greater than the preset speed, not storing the webpage resource corresponding to the preloaded webpage address.
In another optional embodiment provided based on the embodiment of fig. 4, the apparatus further comprises: and a webpage display module.
A web page display module for: acquiring a webpage access request carrying a request webpage address; detecting whether the preloading webpage address contains the request webpage address or not; and if the preloading webpage address contains the request webpage address, displaying a webpage corresponding to the request webpage address according to the stored webpage resource corresponding to the request webpage address.
It should be noted that, when the apparatus provided in the foregoing embodiment implements the functions thereof, only the division of the above functional modules is illustrated, and in practical applications, the above functions may be distributed by different functional modules according to actual needs, that is, the content structure of the device is divided into different functional modules, so as to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided by the above embodiments belong to the same concept, and specific implementation processes thereof are described in the method embodiments for details, which are not described herein again.
Fig. 5 shows a block diagram of a terminal 500 according to an exemplary embodiment of the present invention. The terminal 500 may be a portable mobile terminal such as: a smart phone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III, motion video Experts compression standard Audio Layer 3), an MP4 player (Moving Picture Experts Group Audio Layer IV, motion video Experts compression standard Audio Layer 4), a notebook computer, or a desktop computer. Terminal 500 may also be referred to by other names such as user equipment, portable terminal, laptop terminal, desktop terminal, and the like.
In general, the terminal 500 includes: a processor 501 and a memory 502.
The processor 501 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and so on. The processor 501 may be implemented in at least one hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), and a PLA (Programmable Logic Array). The processor 501 may also include a main processor and a coprocessor, where the main processor is a processor for Processing data in an awake state, and is also called a Central Processing Unit (CPU); a coprocessor is a low power processor for processing data in a standby state. In some embodiments, the processor 501 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content required to be displayed on the display screen. In some embodiments, processor 501 may also include an AI (Artificial Intelligence) processor for processing computational operations related to machine learning.
Memory 502 may include one or more computer-readable storage media, which may be non-transitory. Memory 502 may also include high-speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In some embodiments, a non-transitory computer readable storage medium in the memory 502 is used to store at least one instruction for execution by the processor 501 to implement the method for obtaining web page resources provided by the method embodiments of the present application.
In some embodiments, the terminal 500 may further optionally include: a peripheral interface 503 and at least one peripheral. The processor 501, memory 502 and peripheral interface 503 may be connected by a bus or signal lines. Each peripheral may be connected to the peripheral interface 503 by a bus, signal line, or circuit board. Specifically, the peripheral device includes: at least one of radio frequency circuitry 504, touch screen display 505, camera 506, audio circuitry 507, positioning components 508, and power supply 509.
The peripheral interface 503 may be used to connect at least one peripheral related to I/O (Input/Output) to the processor 501 and the memory 502. In some embodiments, the processor 501, memory 502, and peripheral interface 503 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 501, the memory 502, and the peripheral interface 503 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.
The Radio Frequency circuit 504 is used for receiving and transmitting RF (Radio Frequency) signals, also called electromagnetic signals. The radio frequency circuitry 504 communicates with communication networks and other communication devices via electromagnetic signals. The rf circuit 504 converts an electrical signal into an electromagnetic signal to transmit, or converts a received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 504 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a subscriber identity module card, and so forth. The radio frequency circuitry 504 may communicate with other terminals via at least one wireless communication protocol. The wireless communication protocols include, but are not limited to: the world wide web, metropolitan area networks, intranets, generations of mobile communication networks (2G, 3G, 4G, and 5G), Wireless local area networks, and/or WiFi (Wireless Fidelity) networks. In some embodiments, the rf circuit 504 may further include NFC (Near Field Communication) related circuits, which are not limited in this application.
The display screen 505 is used to display a UI (User Interface). The UI may include graphics, text, icons, video, and any combination thereof. When the display screen 505 is a touch display screen, the display screen 505 also has the ability to capture touch signals on or over the surface of the display screen 505. The touch signal may be input to the processor 501 as a control signal for processing. At this point, the display screen 505 may also be used to provide virtual buttons and/or a virtual keyboard, also referred to as soft buttons and/or a soft keyboard. In some embodiments, the display screen 505 may be one, providing the front panel of the terminal 500; in other embodiments, the display screens 505 may be at least two, respectively disposed on different surfaces of the terminal 500 or in a folded design; in still other embodiments, the display 505 may be a flexible display disposed on a curved surface or on a folded surface of the terminal 500. Even more, the display screen 505 can be arranged in a non-rectangular irregular figure, i.e. a shaped screen. The Display screen 505 may be made of LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), and other materials.
The camera assembly 506 is used to capture images or video. Optionally, camera assembly 506 includes a front camera and a rear camera. Generally, a front camera is disposed at a front panel of the terminal, and a rear camera is disposed at a rear surface of the terminal. In some embodiments, the number of the rear cameras is at least two, and each rear camera is any one of a main camera, a depth-of-field camera, a wide-angle camera and a telephoto camera, so that the main camera and the depth-of-field camera are fused to realize a background blurring function, and the main camera and the wide-angle camera are fused to realize panoramic shooting and VR (Virtual Reality) shooting functions or other fusion shooting functions. In some embodiments, camera assembly 506 may also include a flash. The flash lamp can be a monochrome temperature flash lamp or a bicolor temperature flash lamp. The double-color-temperature flash lamp is a combination of a warm-light flash lamp and a cold-light flash lamp, and can be used for light compensation at different color temperatures.
Audio circuitry 507 may include a microphone and a speaker. The microphone is used for collecting sound waves of a user and the environment, converting the sound waves into electric signals, and inputting the electric signals to the processor 501 for processing, or inputting the electric signals to the radio frequency circuit 504 to realize voice communication. For the purpose of stereo sound collection or noise reduction, a plurality of microphones may be provided at different portions of the terminal 500. The microphone may also be an array microphone or an omni-directional pick-up microphone. The speaker is used to convert electrical signals from the processor 501 or the radio frequency circuit 504 into sound waves. The loudspeaker can be a traditional film loudspeaker or a piezoelectric ceramic loudspeaker. When the speaker is a piezoelectric ceramic speaker, the speaker can be used for purposes such as converting an electric signal into a sound wave audible to a human being, or converting an electric signal into a sound wave inaudible to a human being to measure a distance. In some embodiments, audio circuitry 507 may also include a headphone jack.
The positioning component 508 is used for positioning the current geographic Location of the terminal 500 for navigation or LBS (Location Based Service). The Positioning component 508 may be a Positioning component based on the Global Positioning System (GPS) in the united states, the beidou System in china, or the galileo System in russia.
Power supply 509 is used to power the various components in terminal 500. The power source 509 may be alternating current, direct current, disposable or rechargeable. When power supply 509 includes a rechargeable battery, the rechargeable battery may be a wired rechargeable battery or a wireless rechargeable battery. The wired rechargeable battery is a battery charged through a wired line, and the wireless rechargeable battery is a battery charged through a wireless coil. The rechargeable battery may also be used to support fast charge technology.
In some embodiments, terminal 500 also includes one or more sensors 510. The one or more sensors 510 include, but are not limited to: acceleration sensor 511, gyro sensor 512, pressure sensor 513, fingerprint sensor 514, optical sensor 515, and proximity sensor 516.
The acceleration sensor 511 may detect the magnitude of acceleration on three coordinate axes of the coordinate system established with the terminal 500. For example, the acceleration sensor 511 may be used to detect components of the gravitational acceleration in three coordinate axes. The processor 501 may control the touch screen 505 to display the user interface in a landscape view or a portrait view according to the gravitational acceleration signal collected by the acceleration sensor 511. The acceleration sensor 511 may also be used for acquisition of motion data of a game or a user.
The gyro sensor 512 may detect a body direction and a rotation angle of the terminal 500, and the gyro sensor 512 may cooperate with the acceleration sensor 511 to acquire a 3D motion of the user on the terminal 500. The processor 501 may implement the following functions according to the data collected by the gyro sensor 512: motion sensing (such as changing the UI according to a user's tilting operation), image stabilization at the time of photographing, game control, and inertial navigation.
The pressure sensor 513 may be disposed on a side bezel of the terminal 500 and/or an underlying layer of the touch display screen 505. When the pressure sensor 513 is disposed on the side frame of the terminal 500, a user's holding signal of the terminal 500 may be detected, and the processor 501 performs left-right hand recognition or shortcut operation according to the holding signal collected by the pressure sensor 513. When the pressure sensor 513 is disposed at the lower layer of the touch display screen 505, the processor 501 controls the operability control on the UI interface according to the pressure operation of the user on the touch display screen 505. The operability control comprises at least one of a button control, a scroll bar control, an icon control and a menu control.
The fingerprint sensor 514 is used for collecting a fingerprint of the user, and the processor 501 identifies the identity of the user according to the fingerprint collected by the fingerprint sensor 514, or the fingerprint sensor 514 identifies the identity of the user according to the collected fingerprint. Upon recognizing that the user's identity is a trusted identity, the processor 501 authorizes the user to perform relevant sensitive operations including unlocking the screen, viewing encrypted information, downloading software, paying, and changing settings, etc. The fingerprint sensor 514 may be provided on the front, back, or side of the terminal 500. When a physical button or a vendor Logo is provided on the terminal 500, the fingerprint sensor 514 may be integrated with the physical button or the vendor Logo.
The optical sensor 515 is used to collect the ambient light intensity. In one embodiment, the processor 501 may control the display brightness of the touch display screen 505 based on the ambient light intensity collected by the optical sensor 515. Specifically, when the ambient light intensity is high, the display brightness of the touch display screen 505 is increased; when the ambient light intensity is low, the display brightness of the touch display screen 505 is turned down. In another embodiment, processor 501 may also dynamically adjust the shooting parameters of camera head assembly 506 based on the ambient light intensity collected by optical sensor 515.
A proximity sensor 516, also referred to as a distance sensor, is typically disposed on the front panel of the terminal 500. The proximity sensor 516 is used to collect the distance between the user and the front surface of the terminal 500. In one embodiment, when the proximity sensor 516 detects that the distance between the user and the front surface of the terminal 500 gradually decreases, the processor 501 controls the touch display screen 505 to switch from the bright screen state to the dark screen state; when the proximity sensor 516 detects that the distance between the user and the front surface of the terminal 500 becomes gradually larger, the processor 501 controls the touch display screen 505 to switch from the screen-rest state to the screen-on state.
Those skilled in the art will appreciate that the configuration shown in fig. 5 is not intended to be limiting of terminal 500 and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components may be used.
In an exemplary embodiment, a computer-readable storage medium is further provided, where at least one instruction is stored in the storage medium, and the instruction is executed by a processor of a terminal to implement the method for acquiring a web page resource provided in the foregoing embodiment. Alternatively, the computer-readable storage medium may be a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.
In an exemplary embodiment, a computer program product is also provided, which is used to implement the method for acquiring web page resources provided in the above embodiments when the computer program product is executed.
It should be understood that reference to "a plurality" herein means two or more. "and/or" describes the association relationship of the associated objects, meaning that there may be three relationships, e.g., a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship.
The above-mentioned serial numbers of the embodiments of the present application are merely for description and do not represent the merits of the embodiments.
The above description is only exemplary of the present application and should not be taken as limiting the present application, and any modifications, equivalents, improvements and the like that are made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims (8)

1. A method for acquiring webpage resources is characterized by comprising the following steps:
acquiring at least one historical webpage address, wherein the historical webpage address refers to a webpage address visited in a historical period;
detecting whether the total access times of each historical webpage address in a historical period are greater than a preset time or not;
if the total access times of the historical webpage addresses in the historical period are larger than the preset times, acquiring a first access timestamp and a last access timestamp of the historical webpage addresses in the historical period;
calculating the interval duration between the first access timestamp and the last access timestamp;
calculating the access frequency of the historical webpage address according to the total access times of the historical webpage address in the historical period and the interval duration;
if the access frequency of the historical webpage address is greater than the preset frequency, selecting the historical webpage address as a common webpage address;
downloading the webpage resources corresponding to the common webpage addresses;
extracting a target webpage address contained in the webpage resource corresponding to the common webpage address;
downloading the webpage resources corresponding to the target webpage address extracted in the 1 st round;
extracting target webpage addresses contained in webpage resources corresponding to the target webpage addresses extracted in the 1 st round;
downloading the webpage resources corresponding to the target webpage address extracted in the 2 nd round, and repeating the steps until the data volume of the stored webpage resources is larger than the preset data volume, and stopping downloading;
storing webpage resources corresponding to a preloaded webpage address, wherein the preloaded webpage address comprises the common webpage address and the target webpage address, the webpage resources corresponding to the preloaded webpage address stop downloading when a downloading stop condition is met, and the downloading stop condition comprises at least one of the conditions that the data volume of the webpage resources is larger than a preset data volume, the number of crawling rounds is larger than a preset number of rounds, the number of the target webpage addresses is larger than a preset number, and the residual storage capacity of a terminal is smaller than a preset capacity; and the resource updating duration corresponding to the preloaded webpage address is a dynamic duration, and the resource updating duration is determined based on the change condition of the webpage resources of the preloaded webpage address downloaded twice, wherein the resource updating duration is shortened when the webpage resources are changed, and the resource updating duration is increased when the webpage resources are not changed.
2. The method of claim 1, after storing the web page resource corresponding to the preloaded web page address, further comprising:
for each preloaded webpage address, acquiring a resource updating time length corresponding to the preloaded webpage address;
when the stored duration of the webpage resources corresponding to the preloaded webpage address reaches the resource updating duration corresponding to the preloaded webpage address, the webpage resources corresponding to the preloaded webpage address are downloaded again;
and updating the stored webpage resources corresponding to the preloaded webpage addresses by adopting the webpage resources corresponding to the reloaded preloaded webpage addresses.
3. The method of claim 2, wherein the obtaining of the resource update duration corresponding to the preloaded web page address comprises:
when the stored time length of the webpage resource corresponding to the preloaded webpage address reaches the ith resource updating time length corresponding to the preloaded webpage address, the webpage resource corresponding to the preloaded webpage address is downloaded again, wherein the initial value of i is 1, and the 1 st resource updating time length is a preset value;
detecting whether the webpage resources corresponding to the reloaded preloaded webpage addresses change relative to the stored webpage resources corresponding to the preloaded webpage addresses;
if not, determining the i +1 th resource updating time length corresponding to the preloaded webpage address, wherein the i +1 th resource updating time length is greater than the i-th resource updating time length; when the stored duration of the webpage resources corresponding to the preloaded webpage address reaches the ith resource updating duration corresponding to the preloaded webpage address, the step of downloading the webpage resources corresponding to the preloaded webpage address again starts to be executed, and when the webpage resources corresponding to the preloaded webpage address which are downloaded again change relative to the stored webpage resources corresponding to the preloaded webpage address, the 1 st resource updating duration to the (i-1) th resource updating duration are added to obtain the resource updating duration corresponding to the preloaded webpage address;
if the change occurs, determining the resource updating time length of the (i + 1) th time corresponding to the preloaded webpage address, wherein the resource updating time length of the (i + 1) th time is less than the resource updating time length of the ith time; and when the stored duration of the webpage resources corresponding to the preloaded webpage address reaches the ith resource updating duration corresponding to the preloaded webpage address, the step of re-downloading the webpage resources corresponding to the preloaded webpage address is started to be executed again until the re-downloaded webpage resources corresponding to the preloaded webpage address do not change relative to the stored webpage resources corresponding to the preloaded webpage address, and the ith resource updating duration is determined as the resource updating duration corresponding to the preloaded webpage address.
4. The method according to any one of claims 1 to 3, further comprising:
for each preloaded webpage address, obtaining the historical access speed corresponding to the preloaded webpage address, wherein the historical access speed corresponding to the preloaded webpage address is used for indicating the downloading speed of the webpage resource corresponding to the preloaded webpage address;
if the historical access speed corresponding to the preloaded webpage address is lower than the preset speed, storing the webpage resource corresponding to the preloaded webpage address;
and if the historical access speed corresponding to the preloaded webpage address is greater than the preset speed, not storing the webpage resource corresponding to the preloaded webpage address.
5. The method according to any one of claims 1 to 3, wherein after storing the web page resource corresponding to the preloaded web page address, further comprising:
acquiring a webpage access request carrying a request webpage address;
detecting whether the preloading webpage address contains the request webpage address or not;
and if the preloading webpage address contains the request webpage address, displaying a webpage corresponding to the request webpage address according to the stored webpage resource corresponding to the request webpage address.
6. An apparatus for acquiring web page resources, the apparatus comprising:
the system comprises an address acquisition module, a storage module and a processing module, wherein the address acquisition module is used for acquiring at least one historical webpage address, and the historical webpage address refers to a webpage address visited in a historical period;
the address selection module is used for detecting whether the total access times of each historical webpage address in the historical period are greater than the preset times;
if the total access times of the historical webpage addresses in the historical period are larger than the preset times, acquiring a first access timestamp and a last access timestamp of the historical webpage addresses in the historical period;
calculating the interval duration between the first access timestamp and the last access timestamp;
calculating the access frequency of the historical webpage address according to the total access times of the historical webpage address in the historical period and the interval duration;
if the access frequency of the historical webpage address is greater than the preset frequency, selecting the historical webpage address as a common webpage address to select a common webpage address from the historical webpage addresses, wherein the common webpage address refers to the historical webpage address with the access frequency greater than the preset frequency in the historical period;
the resource acquisition module is used for downloading the webpage resources corresponding to the common webpage addresses;
the resource acquisition module is also used for extracting a target webpage address contained in the webpage resource corresponding to the common webpage address; downloading the webpage resources corresponding to the target webpage address extracted in the 1 st round; extracting target webpage addresses contained in webpage resources corresponding to the target webpage addresses extracted in the 1 st round; downloading the webpage resources corresponding to the target webpage address extracted in the 2 nd round, and repeating the steps until the data volume of the stored webpage resources is larger than the preset data volume, and stopping downloading;
the resource storage module is used for storing webpage resources corresponding to a preloaded webpage address, wherein the preloaded webpage address comprises the common webpage address and the target webpage address, the webpage resources corresponding to the preloaded webpage address stop downloading when a downloading stop condition is met, and the downloading stop condition comprises at least one of the conditions that the data volume of the webpage resources is larger than a preset data volume, the number of crawling rounds is larger than a preset round number, the number of the target webpage addresses is larger than a preset number, and the residual storage capacity of the terminal is smaller than a preset capacity; and the resource updating duration corresponding to the preloaded webpage address is a dynamic duration, and the resource updating duration is determined based on the change condition of the webpage resources of the preloaded webpage address downloaded twice, wherein the resource updating duration is shortened when the webpage resources are changed, and the resource updating duration is increased when the webpage resources are not changed.
7. A terminal, characterized in that the terminal comprises a processor and a memory, the memory stores at least one instruction, the instruction is loaded and executed by the processor to realize the method for acquiring webpage resources according to any one of claims 1 to 5.
8. A computer-readable storage medium having at least one instruction stored thereon, wherein the instruction is loaded and executed by a processor to implement the method for acquiring web page resources according to any one of claims 1 to 5.
CN201711270485.6A 2017-12-05 2017-12-05 Webpage resource obtaining method and device and terminal Active CN109948087B (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201711270485.6A CN109948087B (en) 2017-12-05 2017-12-05 Webpage resource obtaining method and device and terminal
PCT/CN2018/116224 WO2019109797A1 (en) 2017-12-05 2018-11-19 Method, device, terminal and storage medium for obtaining webpage resource

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201711270485.6A CN109948087B (en) 2017-12-05 2017-12-05 Webpage resource obtaining method and device and terminal

Publications (2)

Publication Number Publication Date
CN109948087A CN109948087A (en) 2019-06-28
CN109948087B true CN109948087B (en) 2021-11-16

Family

ID=66750775

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201711270485.6A Active CN109948087B (en) 2017-12-05 2017-12-05 Webpage resource obtaining method and device and terminal

Country Status (2)

Country Link
CN (1) CN109948087B (en)
WO (1) WO2019109797A1 (en)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11362869B2 (en) * 2018-07-31 2022-06-14 Telefonaktiebolaget Lm Ericsson (Publ) Method, transmitter, structure, transceiver and access point for provision of multi-carrier on-off keying signal
CN112148395A (en) * 2019-06-26 2020-12-29 杭州萤石软件有限公司 Page display method, device, equipment and storage medium
CN110458360B (en) * 2019-08-13 2023-07-18 腾讯科技（深圳）有限公司 Method, device, equipment and storage medium for predicting hot resources
CN111030936B (en) * 2019-11-18 2023-04-07 腾讯云计算（北京）有限责任公司 Current-limiting control method and device for network access and computer-readable storage medium
CN110968815B (en) * 2019-12-04 2023-06-20 北京达佳互联信息技术有限公司 Page refreshing method, device, terminal and storage medium
CN115329179B (en) * 2022-10-14 2023-04-28 卡奥斯工业智能研究院(青岛)有限公司 Data acquisition resource amount control method, device, equipment and storage medium

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101329674A (en) * 2007-06-18 2008-12-24 北京搜狗科技发展有限公司 System and method for providing personalized searching
CN101882162A (en) * 2010-06-29 2010-11-10 北京搜狗科技发展有限公司 Method and system for transmitting network information
CN101996193A (en) * 2009-08-21 2011-03-30 北京搜狗科技发展有限公司 Processing method and system for expressing network resource link and internet terminal
CN102567521A (en) * 2011-12-29 2012-07-11 维构（上海）文化传媒有限公司 Webpage data capturing and filtering method
CN102752288A (en) * 2012-06-06 2012-10-24 华为技术有限公司 Method and device for identifying network access action

Family Cites Families (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101369284B (en) * 2008-09-28 2010-12-22 北京搜狗科技发展有限公司 Method and apparatus for loading web pages
CN102402518A (en) * 2010-09-09 2012-04-04 中国移动通信有限公司 Method and device for accessing webpage
CN102880618A (en) * 2011-07-15 2013-01-16 国际商业机器公司 Method and system for searching webpage document
CN102591963A (en) * 2011-12-30 2012-07-18 奇智软件（北京）有限公司 Method and device for controlling webpage content loading
CN102737116B (en) * 2012-05-29 2016-04-13 深圳市同洲电子股份有限公司 A kind of web page resources store method and device
CN104036160B (en) * 2013-03-07 2019-03-15 腾讯科技（深圳）有限公司 A kind of Web browser method, device and browser
CN103281303A (en) * 2013-04-28 2013-09-04 乐视网信息技术（北京）股份有限公司 Method and equipment for obtaining data
CN103544320A (en) * 2013-11-05 2014-01-29 从兴技术有限公司 Webpage generation method and device
CN105095226B (en) * 2014-04-25 2019-08-02 广州市动景计算机科技有限公司 Web page resources loading method and device
CN104753922A (en) * 2015-03-10 2015-07-01 百度在线网络技术（北京）有限公司 Method for pre-loading, server side, client and system
CN104794165B (en) * 2015-03-26 2018-08-10 百度在线网络技术（北京）有限公司 A kind of webpage representation method, apparatus and system
CN105955766B (en) * 2016-04-22 2019-08-30 北京小米移动软件有限公司 Using preloading method and device
CN105956152A (en) * 2016-05-13 2016-09-21 北京金山安全软件有限公司 Webpage loading method and device and electronic equipment
CN106294609B (en) * 2016-07-29 2020-06-19 北京小米移动软件有限公司 Page loading method and device

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101329674A (en) * 2007-06-18 2008-12-24 北京搜狗科技发展有限公司 System and method for providing personalized searching
CN101996193A (en) * 2009-08-21 2011-03-30 北京搜狗科技发展有限公司 Processing method and system for expressing network resource link and internet terminal
CN101882162A (en) * 2010-06-29 2010-11-10 北京搜狗科技发展有限公司 Method and system for transmitting network information
CN102567521A (en) * 2011-12-29 2012-07-11 维构（上海）文化传媒有限公司 Webpage data capturing and filtering method
CN102752288A (en) * 2012-06-06 2012-10-24 华为技术有限公司 Method and device for identifying network access action

Also Published As

Publication number Publication date
CN109948087A (en) 2019-06-28
WO2019109797A1 (en) 2019-06-13

Similar Documents

Publication Publication Date Title
CN109948087B (en) Webpage resource obtaining method and device and terminal
CN107885533B (en) Method and device for managing component codes
CN109977333B (en) Webpage display method and device, computer equipment and storage medium
CN110674022B (en) Behavior data acquisition method and device and storage medium
CN110308956B (en) Application interface display method and device and mobile terminal
WO2019179237A1 (en) Satellite-view electronic map acquisition method and device, apparatus, and storage medium
CN110362762B (en) Content data display method and device, electronic equipment and storage medium
CN110362366B (en) Application interface display method and device
CN111880712B (en) Page display method and device, electronic equipment and storage medium
CN110147503B (en) Information issuing method and device, computer equipment and storage medium
CN110968815B (en) Page refreshing method, device, terminal and storage medium
CN110245255B (en) Song display method, device, equipment and storage medium
CN108038232B (en) Webpage editing method, device and system and storage medium
CN110191236B (en) Song playing queue management method and device, terminal equipment and storage medium
CN112181442B (en) Unloading page display method, device, terminal, server and storage medium
CN111158780B (en) Method, device, electronic equipment and medium for storing application data
CN112597417B (en) Page updating method and device, electronic equipment and storage medium
CN111159551B (en) User-generated content display method and device and computer equipment
CN110390065B (en) Webpage acquisition method, device and system
CN113836426A (en) Information pushing method and device and electronic equipment
CN109189525B (en) Method, device and equipment for loading sub-page and computer readable storage medium
CN110889060A (en) Webpage display method and device, computer equipment and storage medium
CN113378085B (en) Page loading method, device, terminal and storage medium
CN113285853B (en) Log reporting method, device, terminal and storage medium
CN115379274B (en) Picture-based interaction method and device, electronic equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
CB02 Change of applicant information

Address after: Changan town in Guangdong province Dongguan 523860 usha Beach Road No. 18

Applicant after: GUANGDONG OPPO MOBILE TELECOMMUNICATIONS Corp.,Ltd.

Address before: Changan town in Guangdong province Dongguan 523860 usha Beach Road No. 18

Applicant before: GUANGDONG OPPO MOBILE TELECOMMUNICATIONS Corp.,Ltd.

CB02 Change of applicant information
GR01 Patent grant
GR01 Patent grant