CN114491373A

CN114491373A - Resource writing method and device, electronic equipment and computer readable medium

Info

Publication number: CN114491373A
Application number: CN202210127943.5A
Authority: CN
Inventors: 郝帅卫; 陈旭
Original assignee: Beijing Jingdong Century Trading Co Ltd; Beijing Wodong Tianjun Information Technology Co Ltd
Current assignee: Beijing Jingdong Century Trading Co Ltd; Beijing Wodong Tianjun Information Technology Co Ltd
Priority date: 2022-02-11
Filing date: 2022-02-11
Publication date: 2022-05-13

Abstract

The embodiment of the disclosure discloses a resource writing method, a resource writing device, an electronic device and a computer readable medium. One embodiment of the method comprises: in response to determining that the target rendering page engine is started, acquiring page resources and network request resources associated with the target domain name; performing static resource filtering on the page resource to obtain a filtered resource; performing resource classification on the filtered resources and the network request resources to obtain first resources and second resources, wherein the first resources belong to the resources under the network station corresponding to the target domain name, and the second resources do not belong to the resources under the network station; performing anomaly detection on the second resource to obtain an abnormal resource and a normal resource; writing the processed normal resource and the processed first resource into a first resource queue to be uploaded, and writing the abnormal resource into a second resource queue to be uploaded. The implementation method can rapidly and efficiently write the processed resources into the corresponding queues respectively.

Description

Resource writing method, device, electronic equipment and computer readable medium

Technical Field

The embodiment of the disclosure relates to the technical field of computers, in particular to a resource writing method, a resource writing device, electronic equipment and a computer readable medium.

Background

Network resource collection refers to collecting various target resources (e.g., link address resources, network request resources, etc.) that are under the target network site. Here, the abnormal target resources are screened from the collected target resources, and may be subsequently used to secure the network security of a World Wide Web (World Wide Web) system. For the collection of target resources, the following methods are generally adopted: and actively triggering and collecting all target resources related to the target network station by related developers.

However, when collecting the respective target resources in the above manner, there are often technical problems as follows:

the manual collection has the problems of large workload and low efficiency of collecting target resources. In addition, the security of the web system is greatly affected by the low efficiency of target resource collection.

Disclosure of Invention

This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

Some embodiments of the present disclosure propose a resource writing method, apparatus, electronic device and computer readable medium to solve the technical problems mentioned in the background section above.

In a first aspect, some embodiments of the present disclosure provide a resource writing method, including: in response to the fact that a target rendering page engine is started, acquiring page resources and network request resources associated with a target domain name according to engine resources loaded by the target rendering page engine in the starting process; static resource filtering is carried out on the page resources to obtain filtered resources; performing resource classification on the filtered resources and the network request resources to obtain first resources and second resources, wherein the first resources belong to resources under a network station corresponding to the target domain name, and the second resources do not belong to the network station; carrying out abnormity detection on the second resource to obtain an abnormal resource and a normal resource; writing the processed normal resource and the processed first resource into a first resource queue to be uploaded, and writing the abnormal resource into a second resource queue to be uploaded.

Optionally, the method further includes: and uploading the resources in the first resource queue and the resources in the second resource queue to a target server.

Optionally, the engine resource includes: engine resources of the interface-less schema engine; the obtaining of the page resource and the network request resource associated with the target domain name according to the engine resource loaded by the target rendering page engine in the starting process includes: acquiring a label page associated with the target domain name; acquiring page resources associated with the label page by using the interface-free mode engine; and acquiring the page resource associated with the target domain name and the network request resource according to the page resource associated with the label page.

Optionally, before the writing the processed normal resource and the processed first resource into the first resource queue to be uploaded, and writing the abnormal resource into the second resource queue to be uploaded, the method further includes: and performing resource deduplication processing on the normal resource and the first resource to obtain a processed normal resource and a processed first resource.

Optionally, the obtaining, according to the page resource associated with the tab page, the page resource associated with the target domain name and the network request resource includes: in response to determining that a link resource exists in the page resources associated with the tab page, determining at least one link resource in the page resources associated with the tab page; intercepting a network request corresponding to the tag page to obtain a network request resource corresponding to the tag page; for each link resource in the at least one link resource, executing a page resource acquisition step: generating a label page corresponding to the link resource as a target label page; acquiring page resources associated with the target tag page; intercepting a network request corresponding to the target tag page to obtain a network request resource corresponding to the target tag page; determining whether the page resources associated with the target tag page have link resources; in response to determining that no link resource exists in the page resource associated with the target tab page, determining the page resource associated with the tab page and the page resource associated with the target tab page as sub-page resources, and combining the network request resource corresponding to the tab page and the network request resource corresponding to the target tab page to obtain a combined resource as a sub-network request resource, wherein the sub-page resource is a resource in the page resource associated with the target domain name, and the sub-network request resource is a resource in the network request resource associated with the target domain name.

Optionally, the method further includes: and in response to determining that the link resource exists in the page resource associated with the target tab page, determining at least one link resource in the page resource associated with the target tab page, and continuing to execute the page resource acquisition step.

Optionally, the method further includes: and in response to determining that no link resource exists in the page resources associated with the tab page, determining the page resources associated with the tab page as the page resources associated with the target domain name, and determining the network request resources corresponding to the tab page as the network request resources associated with the target domain name.

In a second aspect, some embodiments of the present disclosure provide a resource writing apparatus, including: the acquisition unit is configured to respond to the starting of a target rendering page engine and acquire page resources and network request resources associated with a target domain name according to engine resources loaded by the target rendering page engine in the starting process; the resource filtering unit is configured to perform static resource filtering on the page resources to obtain filtered resources; a resource classifying unit configured to perform resource classification on the filtered resource and the network request resource to obtain a first resource and a second resource, where the first resource belongs to a resource under a network site corresponding to the target domain name, and the second resource does not belong to the resource under the network site; an anomaly detection unit configured to perform anomaly detection on the second resource to obtain an abnormal resource and a normal resource; and the resource writing unit is configured to write the processed normal resource and the processed first resource into a first resource queue to be uploaded, and write the abnormal resource into a second resource queue to be uploaded.

Optionally, the apparatus further comprises: and uploading the resources in the first resource queue and the resources in the second resource queue to a target server.

Optionally, the engine resource includes: engine resources of the interface-less schema engine; and the acquisition unit is configured to: acquiring a label page associated with the target domain name; acquiring page resources associated with the label page by using the interface-free mode engine; and acquiring the page resource associated with the target domain name and the network request resource according to the page resource associated with the label page.

Optionally, the apparatus further comprises: and performing resource deduplication processing on the normal resource and the first resource to obtain a processed normal resource and a processed first resource.

Optionally, the obtaining unit is configured to: in response to determining that a link resource exists in the page resources associated with the tab page, determining at least one link resource in the page resources associated with the tab page; intercepting a network request corresponding to the tag page to obtain a network request resource corresponding to the tag page; for each link resource in the at least one link resource, executing a page resource acquisition step: generating a label page corresponding to the link resource as a target label page; acquiring page resources associated with the target tag page; intercepting a network request corresponding to the target tag page to obtain a network request resource corresponding to the target tag page; determining whether the page resources associated with the target tag page have link resources; in response to determining that no link resource exists in the page resource associated with the target tab page, determining the page resource associated with the tab page and the page resource associated with the target tab page as sub-page resources, and combining the network request resource corresponding to the tab page and the network request resource corresponding to the target tab page to obtain a combined resource as a sub-network request resource, wherein the sub-page resource is a resource in the page resource associated with the target domain name, and the sub-network request resource is a resource in the network request resource associated with the target domain name.

Optionally, the obtaining unit is configured to: and in response to determining that the link resource exists in the page resource associated with the target tab page, determining at least one link resource in the page resource associated with the target tab page, and continuing to execute the page resource acquisition step.

Optionally, the obtaining unit is configured to: and in response to determining that no link resource exists in the page resources associated with the tab page, determining the page resources associated with the tab page as the page resources associated with the target domain name, and determining the network request resources corresponding to the tab page as the network request resources associated with the target domain name.

In a third aspect, some embodiments of the present disclosure provide an electronic device, comprising: one or more processors; a storage device having one or more programs stored thereon, which when executed by one or more processors, cause the one or more processors to implement the method as described in any of the implementations of the first aspect.

In a fourth aspect, some embodiments of the disclosure provide a computer readable medium having a computer program stored thereon, where the program when executed by a processor implements a method as described in any of the implementations of the first aspect.

The above embodiments of the present disclosure have the following beneficial effects: according to the resource writing method of some embodiments of the disclosure, the processed resources can be quickly and efficiently written into the corresponding queues respectively. Specifically, the reason for the inefficient writing of resources is: the manual collection has the problems of large workload and low efficiency of collecting target resources. Thus, resource writing is not efficient enough. Based on this, the resource writing method according to some embodiments of the present disclosure may, in response to determining that the target rendering page engine is started, efficiently obtain the page resource and the network request resource associated with the target domain name according to the engine resource loaded by the target rendering page engine in the starting process. Here, the page resource and the network request resource include a resource to be uploaded. And then, performing static resource filtering on the page resources to obtain filtered resources. Here, the static resource in the page resource has a small effect on securing the network security of the web system. Therefore, static resources with small functions are removed from the page resources, and the network security of the web system can be guaranteed more efficiently. And further, performing resource classification on the filtered resources and the network request resources to obtain first resources and second resources, wherein the first resources belong to the resources under the network station corresponding to the target domain name, and the second resources do not belong to the resources under the network station. Here, the abnormal resources can be efficiently distinguished by the resource classification. And then, the second resource is subjected to abnormity detection, so that the abnormal resource and the normal resource can be efficiently obtained. And finally, writing the processed normal resources and the processed first resources into a first resource queue to be uploaded, and writing the abnormal resources into a second resource queue to be uploaded. Through the first resource queue and the second resource queue, the network security of the web system can be guaranteed subsequently and efficiently.

Drawings

The above and other features, advantages and aspects of various embodiments of the present disclosure will become more apparent by referring to the following detailed description when taken in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numbers refer to the same or similar elements. It should be understood that the drawings are schematic and that elements and elements are not necessarily drawn to scale.

FIG. 1 is a schematic diagram of one application scenario of a resource writing method according to some embodiments of the present disclosure;

FIG. 2 is a flow diagram of some embodiments of a resource writing method according to the present disclosure;

FIG. 3 is a flow diagram of further embodiments of a resource writing method according to the present disclosure;

FIG. 4 is a schematic illustration of a tab page in some embodiments of a resource writing method according to the present disclosure;

FIG. 5 is a schematic block diagram of some embodiments of a resource writing apparatus according to the present disclosure;

FIG. 6 is a schematic structural diagram of an electronic device suitable for use in implementing some embodiments of the present disclosure.

Detailed Description

Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it is to be understood that the disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the disclosure are for illustration purposes only and are not intended to limit the scope of the disclosure.

It should be noted that, for convenience of description, only the portions related to the related invention are shown in the drawings. The embodiments and features of the embodiments in the present disclosure may be combined with each other without conflict.

It should be noted that the terms "first", "second", and the like in the present disclosure are only used for distinguishing different devices, modules or units, and are not used for limiting the order or interdependence relationship of the functions performed by the devices, modules or units.

It is noted that references to "a", "an", and "the" modifications in this disclosure are intended to be illustrative rather than limiting, and that those skilled in the art will recognize that "one or more" may be used unless the context clearly dictates otherwise.

The names of messages or information exchanged between devices in the embodiments of the present disclosure are for illustrative purposes only, and are not intended to limit the scope of the messages or information.

The present disclosure will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.

Fig. 1 is a schematic diagram of one application scenario of a resource writing method according to some embodiments of the present disclosure.

In the application scenario of fig. 1, in response to determining that the target rendering page engine 102 is started, the client 101 may first obtain the page resource 105 and the network request resource 106 associated with the target domain name 104 according to the engine resource 103 loaded by the target rendering page engine 102 during the starting process. Then, the client 101 may perform static resource filtering on the page resource 105 to obtain a filtered resource 107. Next, the client 101 may classify the filtered resources 107 and the network request resources 106 to obtain a first resource 108 and a second resource 109. The first resource 108 is a resource belonging to a website corresponding to the target domain name 104, and the second resource 109 is a resource not belonging to the website. Further, the client 101 may perform an anomaly detection on the second resource 109 to obtain an anomalous resource 110 and a normal resource 111. Finally, the client 101 may write the processed normal resource 113 and the processed first resource 112 into the first resource queue 114 to be uploaded, and write the above abnormal resource 110 into the second resource queue 115 to be uploaded.

The client may be hardware or software. When the client is hardware, the client can be implemented as a distributed device cluster consisting of a plurality of devices, or can be implemented as a single server/single device. When the client is software, it may be implemented as multiple pieces of software or software modules, for example, to provide distributed services, or as a single piece of software or software module. And is not particularly limited herein.

It should be understood that the number of clients in fig. 1 is merely illustrative. There may be any number of clients, as desired for the implementation.

With continued reference to fig. 2, a flow 200 of some embodiments of a resource writing method according to the present disclosure is shown. The resource writing method comprises the following steps:

step 201, in response to determining that the target rendering page engine is started, acquiring a page resource and a network request resource associated with a target domain name according to an engine resource loaded by the target rendering page engine in a starting process.

In some embodiments, in response to determining that the target rendering page engine is started, an execution subject (e.g., the client 101 shown in fig. 1) of the resource writing method may obtain a page resource and a network request resource associated with the target domain name according to an engine resource loaded by the target rendering page engine during the starting process. The target rendering page engine may be a Go-Driver engine. The Go-Driver engine may be a server rendering page engine developed using a Go language and a target browser engine. Here, the Go-Driver engine may render an incoming URL (Uniform Resource Locator) to simulate a scene of an open page, so that a human does not need to open a browser to obtain a corresponding Resource. The target domain name may be a domain name corresponding to the target web site. The page resource may be various types of resources of the page corresponding to the target domain name. The page resources may include, but are not limited to, at least one of: the target domain name corresponds to the static resource of the page, and the target domain name corresponds to the link resource of the page. The network request resource may be an Application Programming Interface (API) resource for data request. The target domain name exists corresponding to a target Internet Protocol (IP). And acquiring the corresponding page through the target network interconnection protocol. The engine resource is a resource to be configured by the Go-Driver engine during the starting process. The engine resources may include, but are not limited to, at least one of: the resource collector corresponds to the resource and the resource deduplication filter corresponds to the resource. Here, the configuration of the engine resource may further include: central Processing Unit (CPU) resource configuration, and the like.

Step 202, performing static resource filtering on the page resource to obtain a filtered resource.

In some embodiments, the execution subject may perform static resource filtering on the page resource to obtain a filtered resource. The static resources may include, but are not limited to, at least one of: picture resources and video resources.

As an example, the execution body may remove a static resource from the page resource to obtain a filtered resource.

Here, the purpose of removing static resources is to: the static resources have small determination effect on the safety of the web system, so that the efficiency of ensuring the safety of the web system can be greatly improved by removing the static resources.

Step 203, performing resource classification on the filtered resources and the network request resources to obtain first resources and second resources.

In some embodiments, the execution subject may classify the filtered resource and the network request resource to obtain a first resource and a second resource. The first resource belongs to a resource under a network site corresponding to the target domain name, and the second resource does not belong to the network site.

As an example, the execution main body may perform resource classification on the filtered resource and the network request resource by using a preset resource partition rule, so as to obtain a first resource and a second resource.

And 204, performing anomaly detection on the second resource to obtain an abnormal resource and a normal resource.

In some embodiments, the execution subject may perform exception detection on the second resource to obtain an exception resource and a normal resource.

As an example, the execution subject may first obtain a website corresponding to the target domain name. Then, the execution subject may determine a resource list to which the website belongs. And finally, comparing the second resource with the belonged resource list to obtain an abnormal resource and a normal resource. And the sub-resource which is not in the resource list is an abnormal resource.

Step 205, writing the processed normal resource and the processed first resource into a first resource queue to be uploaded, and writing the abnormal resource into a second resource queue to be uploaded.

In some embodiments, the execution main body may write the processed normal resource and the processed first resource into a first resource queue to be uploaded, and write the abnormal resource into a second resource queue to be uploaded. The processed normal resource may be a resource processed from the normal resource. The processed first resource may be a resource processed with respect to the first resource.

As an example, the execution subject may perform resource supplementation on the normal resource and the first resource to obtain a processed normal resource and a processed first resource.

In some optional implementation manners of some embodiments, the execution main body may upload the resources in the first resource queue and the resources in the second resource queue to a target server. The target server can manage all the clients in a unified mode. The various clients described above may be deployed in a clustered fashion.

In some optional implementation manners of some embodiments, before writing the processed normal resource and the processed first resource into a first resource queue to be uploaded, and writing the abnormal resource into a second resource queue to be uploaded, the steps further include:

and the execution main body performs resource duplication elimination processing on the normal resource and the first resource to obtain a processed normal resource and a processed first resource.

With further reference to FIG. 3, a flow 300 of further embodiments of a resource writing method according to the present disclosure is shown. The resource writing method comprises the following steps:

step 301, obtaining a label page associated with the target domain name.

In some embodiments, an executing principal (e.g., client 101 shown in fig. 1) may retrieve a tabbed page associated with the target domain name. Wherein the engine resources include: engine resources of the interface-less schema engine. The above-described mode engine without interface may be a Headless engine.

As an example, the executing entity may obtain the tab page associated with the target domain name by:

firstly, performing domain name resolution on a target domain name to acquire an ip address corresponding to the target domain name.

And secondly, acquiring a label page corresponding to the target domain name through a pre-packaged label page management engine according to the ip address.

As an example, as shown in FIG. 4, FIG. 4 shows a schematic view of a tab page.

Step 302, obtaining the page resource associated with the tab page by using the interface-free mode engine.

In some embodiments, the execution agent may obtain the page resource associated with the tab page by using the interface-free mode engine.

As an example, the execution subject may obtain the page resource associated with the tab page by using an automatic page click script pre-packaged in the interface-free mode engine. The automatic page clicking script can be a JS script. When the non-interface mode engine is started, the domain name, the Cookie (data stored on the local terminal of the user), the Host information and the JS automatic click script are packaged into the non-interface mode engine together.

Step 303, obtaining the page resource associated with the target domain name and the network request resource according to the page resource associated with the tab page.

In some embodiments, the execution subject may obtain the page resource associated with the target domain name and the network request resource according to the page resource associated with the tab page.

As an example, the execution subject may first intercept a network request corresponding to a tab page. Then, the execution subject may determine the network request corresponding to the tabbed page as the network request resource corresponding to the target domain name, and determine the page resource associated with the tabbed page as the page resource associated with the target domain name.

In some optional implementation manners of some embodiments, the obtaining, according to the page resource associated with the tab page, the page resource associated with the target domain name and the network request resource may include the following steps:

in a first step, in response to determining that a link resource exists in the page resource associated with the tab page, the execution subject may determine at least one link resource in the page resource associated with the tab page.

As an example, the execution subject may determine at least one link resource in the page resources associated with the tab page by means of a page query.

And secondly, the execution main body intercepts the network request corresponding to the tag page to obtain the network request resource corresponding to the tag page.

Thirdly, for each link resource in the at least one link resource, executing a page resource acquisition step:

the first substep, generate the label page corresponding to above-mentioned link resource, as the goal label page.

And a second substep of acquiring page resources associated with the target tab page.

As an example, the execution agent may utilize the automated page click script to obtain page resources associated with the target tab page.

In the third substep, the execution main body may intercept the network request corresponding to the target tab page to obtain a network request resource corresponding to the target tab page.

In a fourth substep, the executing agent may determine whether a link resource exists in the page resource associated with the target tab page.

As an example, the execution subject may determine whether a link resource exists in a page resource associated with the target tab page by means of a link query.

In the fifth substep, in response to determining that there is no link resource in the page resource associated with the target tab page, the execution main body may determine the page resource associated with the tab page and the page resource associated with the target tab page as sub-page resources, and combine the network request resource corresponding to the tab page and the network request resource corresponding to the target tab page to obtain a combined resource, which is used as a sub-network request resource. And the sub-page resource is a resource in the page resource associated with the target domain name. The sub-network request resource is a resource in the network request resource associated with the target domain name.

Optionally, the foregoing steps further include:

in response to determining that the link resource exists in the page resource associated with the target tab page, the execution subject determines at least one link resource in the page resource associated with the target tab page, and continues to execute the page resource acquisition step.

Optionally, the foregoing steps further include:

in response to determining that no link resource exists in the page resource associated with the tab page, the execution subject may determine the page resource associated with the tab page as the page resource associated with the target domain name, and determine the network request resource corresponding to the tab page as the network request resource associated with the target domain name.

And 304, performing static resource filtering on the page resources to obtain filtered resources.

Step 305, performing resource classification on the filtered resources and the network request resources to obtain first resources and second resources.

And step 306, performing anomaly detection on the second resource to obtain an abnormal resource and a normal resource.

Step 307, writing the processed normal resource and the processed first resource into a first resource queue to be uploaded, and writing the abnormal resource into a second resource queue to be uploaded.

In some embodiments, the specific implementation of steps 304-307 and the technical effects thereof can refer to steps 202-205 in the embodiment corresponding to fig. 2, which are not described herein again.

As can be seen from fig. 3, compared with the description of some embodiments corresponding to fig. 2, the process 300 of the resource writing method in some embodiments corresponding to fig. 3 highlights the specific steps of acquiring the page resource and the network request resource associated with the target domain name according to the interface-less mode engine. Therefore, the solutions described in the embodiments can efficiently and quickly realize automatic acquisition of page resources and network request resources without manual clicking.

With further reference to fig. 5, as an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a resource writing apparatus, which correspond to those shown in fig. 2, and which may be applied in various electronic devices in particular.

As shown in fig. 5, a resource writing apparatus 500 includes: an acquisition unit 501, a resource filtering unit 502, a resource classification unit 503, an abnormality detection unit 504, and a resource writing unit 505. The obtaining unit 501 is configured to, in response to determining that a target rendering page engine is started, obtain a page resource and a network request resource associated with a target domain name according to an engine resource loaded by the target rendering page engine in a starting process; a resource filtering unit 502, configured to perform static resource filtering on the page resource to obtain a filtered resource; a resource classifying unit 503, configured to perform resource classification on the filtered resource and the network request resource to obtain a first resource and a second resource, where the first resource is a resource under a network site corresponding to the target domain name, and the second resource is a resource not under the network site; an anomaly detection unit 504 configured to perform anomaly detection on the second resource to obtain an anomalous resource and a normal resource; and a resource writing unit 505 configured to write the processed normal resource and the processed first resource into a first resource queue to be uploaded, and write the abnormal resource into a second resource queue to be uploaded.

In some optional implementations of some embodiments, the apparatus 500 further includes: an upload unit (not shown). Wherein the upload unit may be configured to: and uploading the resources in the first resource queue and the resources in the second resource queue to a target server.

In some optional implementations of some embodiments, the engine resource includes: engine resources of the interface-less mode engine, and the obtaining unit 501 is further configured to: acquiring a label page associated with the target domain name; acquiring page resources associated with the label page by using the interface-free mode engine; and acquiring the page resource associated with the target domain name and the network request resource according to the page resource associated with the label page.

In some optional implementations of some embodiments, the apparatus 500 further includes: a deduplication unit (not shown). Wherein the deduplication unit may be configured to: and performing resource deduplication processing on the normal resource and the first resource to obtain a processed normal resource and a processed first resource.

In some optional implementations of some embodiments, the obtaining unit 501 is further configured to: in response to determining that a link resource exists in the page resources associated with the tab page, determining at least one link resource in the page resources associated with the tab page; intercepting a network request corresponding to the tag page to obtain a network request resource corresponding to the tag page; for each link resource in the at least one link resource, executing a page resource acquisition step: generating a label page corresponding to the link resource as a target label page; acquiring page resources associated with the target tag page; intercepting a network request corresponding to the target tag page to obtain a network request resource corresponding to the target tag page; determining whether the page resources associated with the target tag page have link resources; in response to determining that no link resource exists in the page resource associated with the target tab page, determining the page resource associated with the tab page and the page resource associated with the target tab page as sub-page resources, and combining the network request resource corresponding to the tab page and the network request resource corresponding to the target tab page to obtain a combined resource as a sub-network request resource, wherein the sub-page resource is a resource in the page resource associated with the target domain name, and the sub-network request resource is a resource in the network request resource associated with the target domain name.

In some optional implementations of some embodiments, the obtaining unit 501 is further configured to: and in response to determining that the link resource exists in the page resource associated with the target tab page, determining at least one link resource in the page resource associated with the target tab page, and continuing to execute the page resource acquisition step.

In some optional implementations of some embodiments, the obtaining unit 501 is further configured to: and in response to determining that no link resource exists in the page resources associated with the tab page, determining the page resources associated with the tab page as the page resources associated with the target domain name, and determining the network request resources corresponding to the tab page as the network request resources associated with the target domain name.

It will be understood that the elements described in the apparatus 500 correspond to various steps in the method described with reference to fig. 2. Thus, the operations, features and resulting advantages described above with respect to the method are also applicable to the apparatus 500 and the units included therein, and are not described herein again.

Referring now to FIG. 6, a block diagram of an electronic device (e.g., client 101 of FIG. 1)600 suitable for use in implementing some embodiments of the present disclosure is shown. The electronic device shown in fig. 6 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.

As shown in fig. 6, electronic device 600 may include a processing means (e.g., central processing unit, graphics processor, etc.) 601 that may perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)602 or a program loaded from a storage means 608 into a Random Access Memory (RAM) 603. In the RAM603, various programs and data necessary for the operation of the electronic apparatus 600 are also stored. The processing device 601, the ROM 602, and the RAM603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.

Generally, the following devices may be connected to the I/O interface 605: input devices 606 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 607 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage 608 including, for example, tape, hard disk, etc.; and a communication device 609. The communication means 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. While fig. 6 illustrates an electronic device 600 having various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided. Each block shown in fig. 6 may represent one device or may represent multiple devices as desired.

In particular, according to some embodiments of the present disclosure, the processes described above with reference to the flow diagrams may be implemented as computer software programs. For example, some embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In some such embodiments, the computer program may be downloaded and installed from a network through the communication device 609, or installed from the storage device 608, or installed from the ROM 602. The computer program, when executed by the processing device 601, performs the above-described functions defined in the methods of some embodiments of the present disclosure.

It should be noted that the computer readable medium described above in some embodiments of the present disclosure may be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In some embodiments of the disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In some embodiments of the present disclosure, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.

In some embodiments, the clients, servers may communicate using any currently known or future developed network Protocol, such as HTTP (HyperText Transfer Protocol), and may interconnect any form or medium of digital data communication (e.g., a communications network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the Internet (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed network.

The computer readable medium may be embodied in the electronic device; or may exist separately without being assembled into the electronic device. The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: in response to the fact that a target rendering page engine is started, acquiring page resources and network request resources associated with a target domain name according to engine resources loaded by the target rendering page engine in the starting process; static resource filtering is carried out on the page resources to obtain filtered resources; performing resource classification on the filtered resources and the network request resources to obtain first resources and second resources, wherein the first resources belong to resources under a network station corresponding to the target domain name, and the second resources do not belong to the network station; carrying out abnormity detection on the second resource to obtain an abnormal resource and a normal resource; writing the processed normal resource and the processed first resource into a first resource queue to be uploaded, and writing the abnormal resource into a second resource queue to be uploaded.

Computer program code for carrying out operations for embodiments of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The units described in some embodiments of the present disclosure may be implemented by software, and may also be implemented by hardware. The described units may also be provided in a processor, and may be described as: a processor includes an acquisition unit, a resource filtering unit, a resource classifying unit, an abnormality detecting unit, and a resource writing unit. For example, the obtaining unit may be further described as "a unit that obtains, in response to determining that the target rendering page engine starts, a page resource associated with the target domain name and a network request resource according to an engine resource loaded by the target rendering page engine in the starting process".

The functions described herein above may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), systems on a chip (SOCs), Complex Programmable Logic Devices (CPLDs), and the like.

The foregoing description is only exemplary of the preferred embodiments of the disclosure and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention in the embodiments of the present disclosure is not limited to the specific combinations of the above-mentioned features, and other embodiments in which the above-mentioned features or their equivalents are combined arbitrarily without departing from the spirit of the invention are also encompassed. For example, the above features and (but not limited to) technical features with similar functions disclosed in the embodiments of the present disclosure are mutually replaced to form the technical solution.

Claims

1. A resource writing method, comprising:

in response to the fact that a target rendering page engine is started, acquiring page resources and network request resources associated with a target domain name according to engine resources loaded by the target rendering page engine in the starting process;

performing static resource filtering on the page resource to obtain a filtered resource;

performing resource classification on the filtered resources and the network request resources to obtain first resources and second resources, wherein the first resources belong to the resources under the network station corresponding to the target domain name, and the second resources do not belong to the resources under the network station;

performing anomaly detection on the second resource to obtain an abnormal resource and a normal resource;

writing the processed normal resource and the processed first resource into a first resource queue to be uploaded, and writing the abnormal resource into a second resource queue to be uploaded.

2. The method of claim 1, wherein the method further comprises:

and uploading the resources in the first resource queue and the resources in the second resource queue to a target server.

3. The method of claim 1, wherein the engine resources comprise: engine resources of the interface-less schema engine; and

the acquiring a page resource and a network request resource associated with a target domain name according to an engine resource loaded by the target rendering page engine in a starting process includes:

acquiring a label page associated with the target domain name;

acquiring page resources associated with the label page by using the interface-free mode engine;

and acquiring the page resource associated with the target domain name and the network request resource according to the page resource associated with the label page.

4. The method of claim 1, wherein prior to the writing the processed normal resources and the processed first resources to a first resource queue to be uploaded, and writing the abnormal resources to a second resource queue to be uploaded, the method further comprises:

and performing resource duplicate removal processing on the normal resource and the first resource to obtain a processed normal resource and a processed first resource.

5. The method of claim 3, wherein the obtaining the page resource associated with the target domain name and the network request resource according to the page resource associated with the tabbed page comprises:

in response to determining that a link resource exists in the page resources associated with the tab page, determining at least one link resource in the page resources associated with the tab page;

intercepting a network request corresponding to the tag page to obtain a network request resource corresponding to the tag page;

for each link resource in the at least one link resource, executing a page resource acquisition step:

generating a label page corresponding to the link resource as a target label page;

acquiring page resources associated with the target tag page;

intercepting a network request corresponding to the target tag page to obtain a network request resource corresponding to the target tag page;

determining whether a link resource exists in a page resource associated with the target tab page;

and in response to determining that no link resource exists in the page resource associated with the target tab page, determining the page resource associated with the tab page and the page resource associated with the target tab page as sub-page resources, and combining the network request resource corresponding to the tab page and the network request resource corresponding to the target tab page to obtain a combined resource as a sub-network request resource, wherein the sub-page resource is a resource in the page resource associated with the target domain name, and the sub-network request resource is a resource in the network request resource associated with the target domain name.

6. The method of claim 5, wherein the method further comprises:

and in response to determining that the link resource exists in the page resource associated with the target tab page, determining at least one link resource in the page resource associated with the target tab page, and continuing to execute the page resource acquisition step.

7. The method of claim 5, wherein the method further comprises:

and in response to determining that no link resource exists in the page resources associated with the tab page, determining the page resources associated with the tab page as the page resources associated with the target domain name, and determining the network request resources corresponding to the tab page as the network request resources associated with the target domain name.

8. A resource writing apparatus, comprising:

the obtaining unit is configured to respond to the starting of the target rendering page engine, and obtain page resources and network request resources related to a target domain name according to engine resources loaded by the target rendering page engine in the starting process;

the resource filtering unit is configured to perform static resource filtering on the page resource to obtain a filtered resource;

a resource classification unit configured to perform resource classification on the filtered resource and the network request resource to obtain a first resource and a second resource, where the first resource belongs to a resource under a network site corresponding to the target domain name, and the second resource does not belong to the network site;

an anomaly detection unit configured to perform anomaly detection on the second resource to obtain an abnormal resource and a normal resource;

and the resource writing unit is configured to write the processed normal resources and the processed first resources into a first resource queue to be uploaded, and write the abnormal resources into a second resource queue to be uploaded.

9. An electronic device, comprising:

one or more processors;

a storage device having one or more programs stored thereon,

when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-7.

10. A computer-readable medium, on which a computer program is stored, wherein the program, when executed by a processor, implements the method of any one of claims 1-7.