CN114491373A - Resource writing method, apparatus, electronic device and computer readable medium - Google Patents

Resource writing method, apparatus, electronic device and computer readable medium Download PDF

Info

Publication number
CN114491373A
CN114491373A CN202210127943.5A CN202210127943A CN114491373A CN 114491373 A CN114491373 A CN 114491373A CN 202210127943 A CN202210127943 A CN 202210127943A CN 114491373 A CN114491373 A CN 114491373A
Authority
CN
China
Prior art keywords
resource
page
resources
target
network request
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210127943.5A
Other languages
Chinese (zh)
Inventor
郝帅卫
陈旭
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Jingdong Century Trading Co Ltd
Beijing Wodong Tianjun Information Technology Co Ltd
Original Assignee
Beijing Jingdong Century Trading Co Ltd
Beijing Wodong Tianjun Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Jingdong Century Trading Co Ltd, Beijing Wodong Tianjun Information Technology Co Ltd filed Critical Beijing Jingdong Century Trading Co Ltd
Priority to CN202210127943.5A priority Critical patent/CN114491373A/en
Publication of CN114491373A publication Critical patent/CN114491373A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/95Retrieval from the web
    • G06F16/958Organisation or management of web site content, e.g. publishing, maintaining pages or automatic linking
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/95Retrieval from the web
    • G06F16/957Browsing optimisation, e.g. caching or content distillation
    • G06F16/9577Optimising the visualization of content, e.g. distillation of HTML documents

Landscapes

  • Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

The embodiment of the disclosure discloses a resource writing method, a resource writing device, an electronic device and a computer readable medium. One embodiment of the method comprises: in response to determining that the target rendering page engine is started, acquiring page resources and network request resources associated with the target domain name; performing static resource filtering on the page resource to obtain a filtered resource; performing resource classification on the filtered resources and the network request resources to obtain first resources and second resources, wherein the first resources belong to the resources under the network station corresponding to the target domain name, and the second resources do not belong to the resources under the network station; performing anomaly detection on the second resource to obtain an abnormal resource and a normal resource; writing the processed normal resource and the processed first resource into a first resource queue to be uploaded, and writing the abnormal resource into a second resource queue to be uploaded. The implementation method can rapidly and efficiently write the processed resources into the corresponding queues respectively.

Description

资源写入方法、装置、电子设备和计算机可读介质Resource writing method, apparatus, electronic device and computer readable medium

技术领域technical field

本公开的实施例涉及计算机技术领域,具体涉及资源写入方法、装置、电子设备和计算机可读介质。Embodiments of the present disclosure relate to the field of computer technology, and in particular, to a resource writing method, apparatus, electronic device, and computer-readable medium.

背景技术Background technique

网络资源收集是指将处于目标网络站点下的各个目标资源(例如,链接地址资源,网络请求资源等等)收集起来。在这里,从收集到的各个目标资源中筛选出异常的目标资源,可以后续用于保障web(World Wide Web,万维网)系统的网络安全。对于目标资源的收集,通常采用的方式为:由相关开发人员主动去触发收集目标网络站点相关的各个目标资源。Network resource collection refers to collecting various target resources (eg, link address resources, network request resources, etc.) under the target network site. Here, the abnormal target resources are screened out from the collected target resources, which can be subsequently used to ensure the network security of the web (World Wide Web, World Wide Web) system. For the collection of target resources, a method is usually adopted: relevant developers actively trigger the collection of various target resources related to the target network site.

然而,当采用上述方式来收集各个目标资源时,经常会存在如下技术问题:However, when using the above method to collect various target resources, there are often the following technical problems:

人工收集存在工作量较大,收集目标资源效率太低的问题。除此之外,目标资源收集效率较低会较大程度影响web系统的安全。Manual collection has the problem that the workload is large and the efficiency of collecting target resources is too low. In addition, the low efficiency of target resource collection will greatly affect the security of the web system.

发明内容SUMMARY OF THE INVENTION

本公开的内容部分用于以简要的形式介绍构思,这些构思将在后面的具体实施方式部分被详细描述。本公开的内容部分并不旨在标识要求保护的技术方案的关键特征或必要特征,也不旨在用于限制所要求的保护的技术方案的范围。This summary of the disclosure serves to introduce concepts in a simplified form that are described in detail in the detailed description that follows. The content section of this disclosure is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to be used to limit the scope of the claimed technical solution.

本公开的一些实施例提出了资源写入方法、装置、电子设备和计算机可读介质,来解决以上背景技术部分提到的技术问题。Some embodiments of the present disclosure propose a resource writing method, apparatus, electronic device, and computer-readable medium to solve the technical problems mentioned in the background section above.

第一方面,本公开的一些实施例提供了一种资源写入方法,包括:响应于确定目标渲染页面引擎启动,根据上述目标渲染页面引擎在启动过程中加载的引擎资源,获取目标域名相关联的页面资源和网络请求资源;对上述页面资源进行静态资源过滤,得到过滤后资源;对上述过滤后资源和上述网络请求资源进行资源分类,得到第一资源和第二资源,其中,上述第一资源为属于上述目标域名对应网络站点下的资源,上述第二资源为不属于上述网络站点下的资源;对上述第二资源进行异常检测,以得到异常资源和正常资源;将处理后的正常资源和处理后的第一资源写入待上传的第一资源队列,以及将上述异常资源写入待上传的第二资源队列。In a first aspect, some embodiments of the present disclosure provide a resource writing method, comprising: in response to determining that the target rendering page engine is started, obtaining a target domain name associated with the engine according to the engine resources loaded by the target rendering page engine during the startup process performing static resource filtering on the above-mentioned page resources to obtain filtered resources; performing resource classification on the above-mentioned filtered resources and the above-mentioned network request resources to obtain the first resource and the second resource, wherein the above-mentioned first resource The resource is a resource belonging to the network site corresponding to the target domain name, and the second resource is a resource that does not belong to the network site; abnormality detection is performed on the second resource to obtain abnormal resources and normal resources; and the processed first resource is written into the first resource queue to be uploaded, and the above-mentioned abnormal resource is written into the second resource queue to be uploaded.

可选地,上述方法还包括:将上述第一资源队列中的资源和上述第二资源队列中的资源上传至目标服务端。Optionally, the above method further includes: uploading the resources in the first resource queue and the resources in the second resource queue to the target server.

可选地,上述引擎资源包括:无界面模式引擎的引擎资源;以及上述根据上述目标渲染页面引擎在启动过程中加载的引擎资源,获取目标域名相关联的页面资源和网络请求资源,包括:获取上述目标域名相关联的标签页面;利用上述无界面模式引擎,获取上述标签页面相关联的页面资源;根据上述标签页面相关联的页面资源,获取上述目标域名相关联的页面资源和上述网络请求资源。Optionally, the above-mentioned engine resources include: the engine resources of the interface-less mode engine; and the above-mentioned engine resources loaded according to the above-mentioned target rendering page engine during the startup process, and obtaining the page resources and network request resources associated with the target domain name, including: obtaining The tab page associated with the target domain name; using the interfaceless mode engine to obtain the page resource associated with the tab page; according to the page resource associated with the tab page, obtain the page resource associated with the target domain name and the network request resource. .

可选地,在上述将处理后的正常资源和处理后的第一资源写入待上传的第一资源队列,以及将上述异常资源写入待上传的第二资源队列之前,上述方法还包括:对上述正常资源和上述第一资源进行资源去重处理,得到处理后的正常资源和处理后的第一资源。Optionally, before writing the processed normal resources and the processed first resources into the first resource queue to be uploaded, and before writing the above-mentioned abnormal resources into the second resource queue to be uploaded, the method further includes: Resource deduplication processing is performed on the normal resource and the first resource to obtain the normal resource after processing and the first resource after processing.

可选地,上述根据上述标签页面相关联的页面资源,获取上述目标域名相关联的页面资源和上述网络请求资源,包括:响应于确定上述标签页面相关联的页面资源中存在链接资源,确定上述标签页面相关联的页面资源中的至少一个链接资源;拦截上述标签页面对应的网络请求,以得到上述标签页面对应的网络请求资源;对于上述至少一个链接资源中的每个链接资源,执行页面资源获取步骤:生成与上述链接资源相对应的标签页面,作为目标标签页面;获取与上述目标标签页面相关联的页面资源;拦截上述目标标签页面对应的网络请求,以得到上述目标标签页面对应的网络请求资源;确定上述目标标签页面相关联的页面资源是否存在链接资源;响应于确定上述目标标签页面相关联的页面资源中不存在链接资源,将上述标签页面相关联的页面资源和上述目标标签页面相关联的页面资源确定为子页面资源,以及将上述标签页面对应的网络请求资源和上述目标标签页面对应的网络请求资源进行组合,得到组合资源,作为子网络请求资源,其中,上述子页面资源为上述目标域名相关联的页面资源中的资源,上述子网络请求资源为上述目标域名相关联的网络请求资源中的资源。Optionally, obtaining the page resource associated with the target domain name and the network request resource according to the page resource associated with the label page includes: in response to determining that there is a link resource in the page resource associated with the label page, determining the above-mentioned page resource. at least one link resource in the page resources associated with the tab page; intercept the network request corresponding to the tab page to obtain the network request resource corresponding to the tab page; for each link resource in the at least one link resource, execute the page resource Obtaining steps: generating a tab page corresponding to the above-mentioned link resource as a target tab page; obtaining page resources associated with the above-mentioned target tab page; intercepting the network request corresponding to the above-mentioned target tab page, so as to obtain the network corresponding to the above-mentioned target tab page requesting resources; determining whether there is a link resource in the page resource associated with the above-mentioned target tab page; in response to determining that there is no link resource in the page resource associated with the above-mentioned target tab page, link the page resource associated with the above-mentioned target tab page with the above-mentioned target tab page The associated page resource is determined as a sub-page resource, and the network request resource corresponding to the above-mentioned tab page and the network request resource corresponding to the above-mentioned target tab page are combined to obtain a combined resource as a sub-network request resource, wherein the above-mentioned sub-page resource is a resource in the page resource associated with the target domain name, and the sub-network request resource is a resource in the network request resource associated with the target domain name.

可选地,上述方法还包括:响应于确定上述目标标签页面相关联的页面资源中存在链接资源,确定上述目标标签页面相关联的页面资源中的至少一个链接资源,以及继续执行上述页面资源获取步骤。Optionally, the above method further includes: in response to determining that there is a link resource in the page resources associated with the target tab page, determining at least one link resource in the page resources associated with the target tab page, and continuing to perform the page resource acquisition. step.

可选地,上述方法还包括:响应于确定上述标签页面相关联的页面资源中不存在链接资源,将上述标签页面相关联的页面资源确定为上述目标域名相关联的页面资源,以及将上述标签页面对应的网络请求资源确定为上述目标域名相关联的网络请求资源。Optionally, the above method further includes: in response to determining that there is no link resource in the page resource associated with the above-mentioned tab page, determining the page resource associated with the above-mentioned tab page as the page resource associated with the above-mentioned target domain name, and adding the above-mentioned label page resource. The network request resource corresponding to the page is determined to be the network request resource associated with the target domain name.

第二方面,本公开的一些实施例提供了一种资源写入装置,包括:获取单元,被配置成响应于确定目标渲染页面引擎启动,根据上述目标渲染页面引擎在启动过程中加载的引擎资源,获取目标域名相关联的页面资源和网络请求资源;资源过滤单元,被配置成对上述页面资源进行静态资源过滤,得到过滤后资源;资源分类单元,被配置成对上述过滤后资源和上述网络请求资源进行资源分类,得到第一资源和第二资源,其中,上述第一资源为属于上述目标域名对应网络站点下的资源,上述第二资源为不属于上述网络站点下的资源;异常检测单元,被配置成对上述第二资源进行异常检测,以得到异常资源和正常资源;资源写入单元,被配置成将处理后的正常资源和处理后的第一资源写入待上传的第一资源队列,以及将上述异常资源写入待上传的第二资源队列。In a second aspect, some embodiments of the present disclosure provide a resource writing apparatus, including: an acquisition unit configured to, in response to determining that a target rendering page engine is started, render engine resources loaded by the page engine during the startup process according to the above target rendering , obtain the page resources and network request resources associated with the target domain name; the resource filtering unit is configured to perform static resource filtering on the above page resources to obtain filtered resources; the resource classification unit is configured to filter the above-mentioned resources and the above-mentioned network resources. Requesting resources to perform resource classification to obtain a first resource and a second resource, wherein the first resource is a resource belonging to a network site corresponding to the target domain name, and the second resource is a resource that does not belong to the network site; anomaly detection unit , is configured to perform anomaly detection on the above-mentioned second resources to obtain abnormal resources and normal resources; the resource writing unit is configured to write the processed normal resources and the processed first resources into the first resource to be uploaded queue, and write the above-mentioned abnormal resource into the second resource queue to be uploaded.

可选地,上述装置还包括:将上述第一资源队列中的资源和上述第二资源队列中的资源上传至目标服务端。Optionally, the above-mentioned apparatus further includes: uploading the resources in the above-mentioned first resource queue and the resources in the above-mentioned second resource queue to the target server.

可选地,上述引擎资源包括:无界面模式引擎的引擎资源;以及获取单元被配置成:获取上述目标域名相关联的标签页面;利用上述无界面模式引擎,获取上述标签页面相关联的页面资源;根据上述标签页面相关联的页面资源,获取上述目标域名相关联的页面资源和上述网络请求资源。Optionally, the above-mentioned engine resources include: the engine resources of the interfaceless mode engine; and the acquiring unit is configured to: acquire the tab page associated with the above-mentioned target domain name; use the above-mentioned interfaceless mode engine to obtain the page resource associated with the above-mentioned tabbed page ; Obtain the page resources associated with the target domain name and the network request resources according to the page resources associated with the label page.

可选地,上述装置还包括:对上述正常资源和上述第一资源进行资源去重处理,得到处理后的正常资源和处理后的第一资源。Optionally, the above-mentioned apparatus further includes: performing resource deduplication processing on the above-mentioned normal resources and the above-mentioned first resources, so as to obtain the processed normal resources and the processed first resources.

可选地,获取单元被配置成:响应于确定上述标签页面相关联的页面资源中存在链接资源,确定上述标签页面相关联的页面资源中的至少一个链接资源;拦截上述标签页面对应的网络请求,以得到上述标签页面对应的网络请求资源;对于上述至少一个链接资源中的每个链接资源,执行页面资源获取步骤:生成与上述链接资源相对应的标签页面,作为目标标签页面;获取与上述目标标签页面相关联的页面资源;拦截上述目标标签页面对应的网络请求,以得到上述目标标签页面对应的网络请求资源;确定上述目标标签页面相关联的页面资源是否存在链接资源;响应于确定上述目标标签页面相关联的页面资源中不存在链接资源,将上述标签页面相关联的页面资源和上述目标标签页面相关联的页面资源确定为子页面资源,以及将上述标签页面对应的网络请求资源和上述目标标签页面对应的网络请求资源进行组合,得到组合资源,作为子网络请求资源,其中,上述子页面资源为上述目标域名相关联的页面资源中的资源,上述子网络请求资源为上述目标域名相关联的网络请求资源中的资源。Optionally, the obtaining unit is configured to: in response to determining that there is a link resource in the page resources associated with the above-mentioned tab page, determine at least one link resource in the page resources associated with the above-mentioned tab page; intercept the network request corresponding to the above-mentioned tab page. , in order to obtain the network request resource corresponding to the above-mentioned tag page; for each link resource in the above-mentioned at least one link resource, perform the page resource acquisition step: generate a tag page corresponding to the above-mentioned link resource as the target tag page; The page resource associated with the target tab page; intercepting the network request corresponding to the target tab page to obtain the network request resource corresponding to the target tab page; determining whether the page resource associated with the target tab page has a link resource; in response to determining the above There is no link resource in the page resource associated with the target tab page, the page resource associated with the above tab page and the page resource associated with the target tab page are determined as sub-page resources, and the network request resource corresponding to the tab page and the page resource are determined. The network request resources corresponding to the above-mentioned target tag pages are combined to obtain the combined resources as sub-network request resources, wherein the above-mentioned sub-page resources are resources in the page resources associated with the above-mentioned target domain name, and the above-mentioned sub-network request resources are the above-mentioned target domain name. The resource in the associated network request resource.

可选地,获取单元被配置成:响应于确定上述目标标签页面相关联的页面资源中存在链接资源,确定上述目标标签页面相关联的页面资源中的至少一个链接资源,以及继续执行上述页面资源获取步骤。Optionally, the obtaining unit is configured to: in response to determining that there is a link resource in the page resource associated with the target tab page, determine at least one link resource in the page resource associated with the target tab page, and continue to execute the page resource. Get steps.

可选地,获取单元被配置成:响应于确定上述标签页面相关联的页面资源中不存在链接资源,将上述标签页面相关联的页面资源确定为上述目标域名相关联的页面资源,以及将上述标签页面对应的网络请求资源确定为上述目标域名相关联的网络请求资源。Optionally, the obtaining unit is configured to: in response to determining that there is no link resource in the page resource associated with the above-mentioned tab page, determine the page resource associated with the above-mentioned tab page as the page resource associated with the above-mentioned target domain name, and The network request resource corresponding to the label page is determined as the network request resource associated with the target domain name.

第三方面,本公开的一些实施例提供了一种电子设备,包括:一个或多个处理器;存储装置,其上存储有一个或多个程序,当一个或多个程序被一个或多个处理器执行,使得一个或多个处理器实现如第一方面中任一实现方式描述的方法。In a third aspect, some embodiments of the present disclosure provide an electronic device, comprising: one or more processors; a storage device on which one or more programs are stored, when one or more programs are stored by one or more The processor executes such that the one or more processors implement a method as described in any implementation of the first aspect.

第四方面,本公开的一些实施例提供了一种计算机可读介质,其上存储有计算机程序,其中,程序被处理器执行时实现如第一方面中任一实现方式描述的方法。In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method described in any implementation manner of the first aspect.

本公开的上述各个实施例中具有如下有益效果:本公开的一些实施例的资源写入方法可以快捷、高效的将处理后的资源分别写入对应的队列。具体来说,造成资源写入不够高效的原因在于:人工收集存在工作量较大,收集目标资源效率太低的问题。由此,导致资源写入不够高效。基于此,本公开的一些实施例的资源写入方法可以响应于确定目标渲染页面引擎启动,根据上述目标渲染页面引擎在启动过程中加载的引擎资源,可以高效地获取目标域名相关联的页面资源和网络请求资源。在这里,上述页面资源和网络请求资源包括待上传的资源。然后,对上述页面资源进行静态资源过滤,得到过滤后资源。在这里,页面资源中的静态资源对于保障web系统的网络安全的作用较小。所以,在页面资源中去除作用较小的静态资源,可以更为高效的保障web系统的网络安全。进而,对上述过滤后资源和上述网络请求资源进行资源分类,得到第一资源和第二资源,其中,上述第一资源为属于上述目标域名对应网络站点下的资源,上述第二资源为不属于上述网络站点下的资源。在这里,通过资源分类可以高效地区分出异常资源。接着,对上述第二资源进行异常检测,可以高效的得到异常资源和正常资源。最后,将处理后的正常资源和处理后的第一资源写入待上传的第一资源队列,以及将上述异常资源写入待上传的第二资源队列。通过第一资源队列和第二资源队列,可以后续高效地保障web系统的网络安全。The foregoing embodiments of the present disclosure have the following beneficial effects: the resource writing methods of some embodiments of the present disclosure can quickly and efficiently write processed resources into corresponding queues respectively. Specifically, the reason for the inefficient writing of resources is that manual collection has a large workload and the efficiency of collecting target resources is too low. As a result, resource writing is not efficient enough. Based on this, the resource writing methods of some embodiments of the present disclosure can respond to determining that the target rendering page engine is started, and can efficiently obtain page resources associated with the target domain name according to the engine resources loaded by the target rendering page engine during the startup process. and network request resources. Here, the above-mentioned page resources and network request resources include resources to be uploaded. Then, static resource filtering is performed on the above page resources to obtain filtered resources. Here, the static resources in the page resources have little effect on ensuring the network security of the web system. Therefore, removing the static resources with less effect from the page resources can more efficiently ensure the network security of the web system. Further, the above-mentioned filtered resources and the above-mentioned network request resources are classified as resources to obtain a first resource and a second resource, wherein the above-mentioned first resource is a resource belonging to the network site corresponding to the above-mentioned target domain name, and the above-mentioned second resource is not belonging to Resources under the aforementioned web site. Here, abnormal resources can be efficiently distinguished by resource classification. Next, abnormality detection is performed on the above-mentioned second resource, so that abnormal resources and normal resources can be obtained efficiently. Finally, the processed normal resources and the processed first resources are written into the first resource queue to be uploaded, and the above abnormal resources are written into the second resource queue to be uploaded. Through the first resource queue and the second resource queue, the network security of the web system can be effectively guaranteed subsequently.

附图说明Description of drawings

结合附图并参考以下具体实施方式,本公开各实施例的上述和其他特征、优点及方面将变得更加明显。贯穿附图中,相同或相似的附图标记表示相同或相似的元素。应当理解附图是示意性的,元件和元素不一定按照比例绘制。The above and other features, advantages and aspects of various embodiments of the present disclosure will become more apparent when taken in conjunction with the accompanying drawings and with reference to the following detailed description. Throughout the drawings, the same or similar reference numbers refer to the same or similar elements. It should be understood that the drawings are schematic and that elements and elements are not necessarily drawn to scale.

图1是根据本公开的一些实施例的资源写入方法的一个应用场景的示意图;1 is a schematic diagram of an application scenario of a resource writing method according to some embodiments of the present disclosure;

图2是根据本公开的资源写入方法的一些实施例的流程图;2 is a flowchart of some embodiments of resource writing methods according to the present disclosure;

图3是根据本公开的资源写入方法的另一些实施例的流程图;3 is a flowchart of other embodiments of resource writing methods according to the present disclosure;

图4是根据本公开的资源写入方法的一些实施例中的标签页面的示意图;4 is a schematic diagram of a tab page in some embodiments of the resource writing method according to the present disclosure;

图5是根据本公开的资源写入装置的一些实施例的结构示意图;5 is a schematic structural diagram of some embodiments of a resource writing apparatus according to the present disclosure;

图6是适于用来实现本公开的一些实施例的电子设备的结构示意图。6 is a schematic structural diagram of an electronic device suitable for implementing some embodiments of the present disclosure.

具体实施方式Detailed ways

下面将参照附图更详细地描述本公开的实施例。虽然附图中显示了本公开的某些实施例,然而应当理解的是,本公开可以通过各种形式来实现,而且不应该被解释为限于这里阐述的实施例。相反,提供这些实施例是为了更加透彻和完整地理解本公开。应当理解的是,本公开的附图及实施例仅用于示例性作用,并非用于限制本公开的保护范围。Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided for a thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes, and are not intended to limit the protection scope of the present disclosure.

另外还需要说明的是,为了便于描述,附图中仅示出了与有关发明相关的部分。在不冲突的情况下,本公开中的实施例及实施例中的特征可以相互组合。In addition, it should be noted that, for the convenience of description, only the parts related to the related invention are shown in the drawings. The embodiments of this disclosure and features of the embodiments may be combined with each other without conflict.

需要注意,本公开中提及的“第一”、“第二”等概念仅用于对不同的装置、模块或单元进行区分,并非用于限定这些装置、模块或单元所执行的功能的顺序或者相互依存关系。It should be noted that concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or interdependence.

需要注意,本公开中提及的“一个”、“多个”的修饰是示意性而非限制性的,本领域技术人员应当理解,除非在上下文另有明确指出,否则应该理解为“一个或多个”。It should be noted that the modifications of "a" and "a plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless the context clearly indicates otherwise, they should be understood as "one or a plurality of". multiple".

本公开实施方式中的多个装置之间所交互的消息或者信息的名称仅用于说明性的目的,而并不是用于对这些消息或信息的范围进行限制。The names of messages or information exchanged between multiple devices in the embodiments of the present disclosure are only for illustrative purposes, and are not intended to limit the scope of these messages or information.

下面将参考附图并结合实施例来详细说明本公开。The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

图1是根据本公开一些实施例的资源写入方法的一个应用场景的示意图。FIG. 1 is a schematic diagram of an application scenario of a resource writing method according to some embodiments of the present disclosure.

在图1的应用场景中,响应于确定目标渲染页面引擎102启动,客户端101可以首先根据上述目标渲染页面引擎102在启动过程中加载的引擎资源103,获取目标域名104相关联的页面资源105和网络请求资源106。然后,客户端101可以对上述页面资源105进行静态资源过滤,得到过滤后资源107。接着,客户端101可以对上述过滤后资源107和上述网络请求资源106进行资源分类,得到第一资源108和第二资源109。其中,上述第一资源108为属于上述目标域名104对应网络站点下的资源,上述第二资源109为不属于上述网络站点下的资源。进而,客户端101可以对上述第二资源109进行异常检测,以得到异常资源110和正常资源111。最后,客户端101可以将处理后的正常资源113和处理后的第一资源112写入待上传的第一资源队列114,以及将上述异常资源110写入待上传的第二资源队列115。In the application scenario of FIG. 1 , in response to determining that the target rendering page engine 102 is started, the client 101 may first obtain the page resource 105 associated with the target domain name 104 according to the engine resource 103 loaded by the target rendering page engine 102 during the startup process. and network request resource 106. Then, the client 101 may perform static resource filtering on the above-mentioned page resource 105 to obtain the filtered resource 107 . Next, the client 101 may perform resource classification on the above-mentioned filtered resource 107 and the above-mentioned network request resource 106 to obtain the first resource 108 and the second resource 109 . The above-mentioned first resource 108 is a resource belonging to the network site corresponding to the above-mentioned target domain name 104, and the above-mentioned second resource 109 is a resource that does not belong to the above-mentioned network site. Further, the client 101 may perform anomaly detection on the second resource 109 to obtain an anomaly resource 110 and a normal resource 111 . Finally, the client 101 can write the processed normal resource 113 and the processed first resource 112 into the first resource queue 114 to be uploaded, and write the above-mentioned abnormal resource 110 into the second resource queue 115 to be uploaded.

需要说明的是,客户端可以是硬件,也可以是软件。当客户端为硬件时,可以实现成多个设备组成的分布式设备集群,也可以实现成单个服务器/单个设备。当客户端为软件时,可以实现成例如用来提供分布式服务的多个软件或软件模块,也可以实现成单个软件或软件模块。在此不做具体限定。It should be noted that the client can be hardware or software. When the client is hardware, it can be implemented as a distributed device cluster composed of multiple devices, or as a single server/single device. When the client is software, it may be implemented as multiple software or software modules for providing distributed services, or may be implemented as a single software or software module. There is no specific limitation here.

应该理解,图1中的客户端的数目仅仅是示意性的。根据实现需要,可以具有任意数目的客户端。It should be understood that the number of clients in FIG. 1 is merely illustrative. There can be any number of clients depending on the implementation needs.

继续参考图2,示出了根据本公开的资源写入方法的一些实施例的流程200。该资源写入方法,包括以下步骤:With continued reference to FIG. 2, a flow 200 of some embodiments of resource writing methods according to the present disclosure is shown. The resource writing method includes the following steps:

步骤201,响应于确定目标渲染页面引擎启动,根据上述目标渲染页面引擎在启动过程中加载的引擎资源,获取目标域名相关联的页面资源和网络请求资源。Step 201 , in response to determining that the target rendering page engine is started, acquire page resources and network request resources associated with the target domain name according to the engine resources loaded by the target rendering page engine during the startup process.

在一些实施例中,响应于确定目标渲染页面引擎启动,上述资源写入方法的执行主体(例如图1所示的客户端101)可以根据上述目标渲染页面引擎在启动过程中加载的引擎资源,获取目标域名相关联的页面资源和网络请求资源。其中,上述目标渲染页面引擎可以是Go-Driver引擎。上述Go-Driver引擎可以是使用go语言和目标浏览器引擎研发的服务端渲染页面引擎。在这里,上述Go-Driver引擎可以对传入的URL(Uniform ResourceLocator,统一资源定位器)进行渲染,以模拟打开页面的场景,这样可以不需要人为去打开浏览器来获取对应的资源。上述目标域名可以是目标web站点对应的域名。上述页面资源可以是目标域名对应页面的各种类型的资源。上述页面资源可以包括但不限于以下至少一项:目标域名对应页面的静态资源,目标域名对应页面的链接资源。上述网络请求资源可以是数据请求的接口(Application Programming Interface,API)资源。上述目标域名存在对应着目标网际互连协议(IP,Internet Protocol)。通过目标网络互联协议可以获取对应的页面。上述引擎资源是Go-Driver引擎在启动过程中要配置的资源。上述引擎资源可以包括但不限于以下至少一项:资源收集器对应资源、资源去重过滤器对应资源。这里引擎资源的配置还可以包括:中央处理器(Central Processing Unit,CPU)资源的配置等等。In some embodiments, in response to determining that the target rendering page engine is started, the execution body of the above resource writing method (for example, the client 101 shown in FIG. 1 ) may render engine resources loaded by the page engine during the startup process according to the above target rendering method, Get the page resources and network request resources associated with the target domain name. Wherein, the above-mentioned target rendering page engine may be a Go-Driver engine. The above-mentioned Go-Driver engine may be a server-side rendering page engine developed by using the go language and the target browser engine. Here, the above-mentioned Go-Driver engine can render the incoming URL (Uniform ResourceLocator, Uniform Resource Locator) to simulate the scene of opening the page, so that there is no need to manually open the browser to obtain the corresponding resources. The above target domain name may be a domain name corresponding to the target website. The above-mentioned page resources may be various types of resources of the page corresponding to the target domain name. The above-mentioned page resources may include, but are not limited to, at least one of the following: static resources of the page corresponding to the target domain name, and link resources of the page corresponding to the target domain name. The above-mentioned network request resource may be a data request interface (Application Programming Interface, API) resource. The existence of the above target domain name corresponds to the target Internet Protocol (IP, Internet Protocol). The corresponding page can be obtained through the target Internet Internet Protocol. The above engine resources are the resources to be configured by the Go-Driver engine during startup. The above engine resources may include, but are not limited to, at least one of the following: resources corresponding to resource collectors, resources corresponding to resource deduplication filters. The configuration of engine resources here may also include: configuration of central processing unit (Central Processing Unit, CPU) resources, and so on.

步骤202,对上述页面资源进行静态资源过滤,得到过滤后资源。Step 202: Perform static resource filtering on the above page resources to obtain filtered resources.

在一些实施例中,上述执行主体可以对上述页面资源进行静态资源过滤,得到过滤后资源。上述静态资源可以包括但不限于以下至少一项:图片资源,视频资源。In some embodiments, the above-mentioned execution body may perform static resource filtering on the above-mentioned page resources to obtain filtered resources. The above-mentioned static resources may include, but are not limited to, at least one of the following: picture resources, video resources.

作为示例,上述执行主体可以将页面资源中的静态资源进行去除,得到过滤后资源。As an example, the above-mentioned execution body may remove the static resources in the page resources to obtain the filtered resources.

在这里,去除静态资源的目的在于:静态资源对web系统安全的确定作用较小,所以去除静态资源可以大大提高保障web系统安全的效率。Here, the purpose of removing static resources is that static resources have little effect on the determination of web system security, so removing static resources can greatly improve the efficiency of ensuring web system security.

步骤203,对上述过滤后资源和上述网络请求资源进行资源分类,得到第一资源和第二资源。Step 203: Perform resource classification on the above-mentioned filtered resources and the above-mentioned network request resources to obtain a first resource and a second resource.

在一些实施例中,上述执行主体可以对上述过滤后资源和上述网络请求资源进行资源分类,得到第一资源和第二资源。其中,上述第一资源为属于上述目标域名对应网络站点下的资源,上述第二资源为不属于上述网络站点下的资源。In some embodiments, the above-mentioned execution body may perform resource classification on the above-mentioned filtered resources and the above-mentioned network request resources to obtain the first resource and the second resource. The first resource is a resource that belongs to the network site corresponding to the target domain name, and the second resource is a resource that does not belong to the network site.

作为示例,上述执行主体可以利用预先设置的资源划分规则,来对上述过滤后资源和上述网络请求资源进行资源分类,得到第一资源和第二资源。As an example, the above-mentioned executive body may use a preset resource division rule to classify the above-mentioned filtered resources and the above-mentioned network request resources, and obtain the first resource and the second resource.

步骤204,对上述第二资源进行异常检测,以得到异常资源和正常资源。Step 204: Perform abnormality detection on the above-mentioned second resource to obtain abnormal resources and normal resources.

在一些实施例中,上述执行主体可以对上述第二资源进行异常检测,以得到异常资源和正常资源。In some embodiments, the above-mentioned execution body may perform abnormality detection on the above-mentioned second resource, so as to obtain abnormal resources and normal resources.

作为示例,上述执行主体可以首先获取目标域名对应web站点。然后,上述执行主体可以确定上述web站点对应所属资源列表。最后,通过对比上述第二资源与上述所属资源列表,以得到异常资源和正常资源。其中,上述第二资源中的、不存在于上述所属资源列表的子资源为异常资源。As an example, the above-mentioned execution body may first obtain the website corresponding to the target domain name. Then, the above-mentioned executive body may determine the resource list corresponding to the above-mentioned web site. Finally, the abnormal resource and the normal resource are obtained by comparing the above-mentioned second resource and the above-mentioned resource list. Wherein, the sub-resources in the above-mentioned second resource that do not exist in the above-mentioned resource list are abnormal resources.

步骤205,将处理后的正常资源和处理后的第一资源写入待上传的第一资源队列,以及将上述异常资源写入待上传的第二资源队列。Step 205: Write the processed normal resource and the processed first resource into the first resource queue to be uploaded, and write the above abnormal resource into the second resource queue to be uploaded.

在一些实施例中,上述执行主体可以将处理后的正常资源和处理后的第一资源写入待上传的第一资源队列,以及将上述异常资源写入待上传的第二资源队列。其中,上述处理后的正常资源可以是对正常资源处理后的资源。上述处理后的第一资源可以是对第一资源处理后的资源。In some embodiments, the above-mentioned execution body may write the processed normal resources and the processed first resources into the first resource queue to be uploaded, and write the above-mentioned abnormal resources into the second resource queue to be uploaded. The above-mentioned processed normal resources may be resources processed on normal resources. The above-mentioned processed first resource may be a resource processed on the first resource.

作为示例,上述执行主体可以对正常资源和第一资源进行资源补充,以得到处理后的正常资源和处理后的第一资源。As an example, the above-mentioned execution body may perform resource supplementation on the normal resource and the first resource, so as to obtain the processed normal resource and the processed first resource.

在一些实施例的一些可选的实现方式中,上述执行主体可以将上述第一资源队列中的资源和上述第二资源队列中的资源上传至目标服务端。其中,上述目标服务端可以统一管理各个客户端。上述各个客户端可以是集群化部署的。In some optional implementation manners of some embodiments, the execution body may upload the resources in the first resource queue and the resources in the second resource queue to the target server. The above-mentioned target server can manage each client in a unified manner. Each of the above clients may be deployed in a cluster.

在一些实施例的一些可选的实现方式中,在上述将处理后的正常资源和处理后的第一资源写入待上传的第一资源队列,以及将上述异常资源写入待上传的第二资源队列之前,上述步骤还包括:In some optional implementations of some embodiments, the above-mentioned normal resources after processing and the processed first resources are written into the first resource queue to be uploaded, and the above-mentioned abnormal resources are written into the second to-be-uploaded resource queue. Before the resource queue, the above steps also include:

上述执行主体对上述正常资源和上述第一资源进行资源去重处理,得到处理后的正常资源和处理后的第一资源。The execution body performs resource deduplication processing on the normal resource and the first resource, and obtains the processed normal resource and the processed first resource.

本公开的上述各个实施例中具有如下有益效果:本公开的一些实施例的资源写入方法可以快捷、高效的将处理后的资源分别写入对应的队列。具体来说,造成资源写入不够高效的原因在于:人工收集存在工作量较大,收集目标资源效率太低的问题。由此,导致资源写入不够高效。基于此,本公开的一些实施例的资源写入方法可以响应于确定目标渲染页面引擎启动,根据上述目标渲染页面引擎在启动过程中加载的引擎资源,可以高效地获取目标域名相关联的页面资源和网络请求资源。在这里,上述页面资源和网络请求资源包括待上传的资源。然后,对上述页面资源进行静态资源过滤,得到过滤后资源。在这里,页面资源中的静态资源对于保障web系统的网络安全的作用较小。所以,在页面资源中去除作用较小的静态资源,可以更为高效的保障web系统的网络安全。进而,对上述过滤后资源和上述网络请求资源进行资源分类,得到第一资源和第二资源,其中,上述第一资源为属于上述目标域名对应网络站点下的资源,上述第二资源为不属于上述网络站点下的资源。在这里,通过资源分类可以高效地区分出异常资源。接着,对上述第二资源进行异常检测,可以高效的得到异常资源和正常资源。最后,将处理后的正常资源和处理后的第一资源写入待上传的第一资源队列,以及将上述异常资源写入待上传的第二资源队列。通过第一资源队列和第二资源队列,可以后续高效地保障web系统的网络安全。The foregoing embodiments of the present disclosure have the following beneficial effects: the resource writing methods of some embodiments of the present disclosure can quickly and efficiently write processed resources into corresponding queues respectively. Specifically, the reason for the inefficient writing of resources is that manual collection has a large workload and the efficiency of collecting target resources is too low. As a result, resource writing is not efficient enough. Based on this, the resource writing methods of some embodiments of the present disclosure can respond to determining that the target rendering page engine is started, and can efficiently obtain page resources associated with the target domain name according to the engine resources loaded by the target rendering page engine during the startup process. and network request resources. Here, the above-mentioned page resources and network request resources include resources to be uploaded. Then, static resource filtering is performed on the above page resources to obtain filtered resources. Here, the static resources in the page resources have little effect on ensuring the network security of the web system. Therefore, removing the static resources with less effect from the page resources can more efficiently ensure the network security of the web system. Further, the above-mentioned filtered resources and the above-mentioned network request resources are classified as resources to obtain a first resource and a second resource, wherein the above-mentioned first resource is a resource belonging to the network site corresponding to the above-mentioned target domain name, and the above-mentioned second resource is not belonging to Resources under the aforementioned web site. Here, abnormal resources can be efficiently distinguished by resource classification. Next, abnormality detection is performed on the above-mentioned second resource, so that abnormal resources and normal resources can be obtained efficiently. Finally, the processed normal resources and the processed first resources are written into the first resource queue to be uploaded, and the above abnormal resources are written into the second resource queue to be uploaded. Through the first resource queue and the second resource queue, the network security of the web system can be effectively guaranteed subsequently.

进一步参考图3,示出了根据本公开的资源写入方法的另一些实施例的流程300。该资源写入方法,包括以下步骤:With further reference to FIG. 3, a flow 300 of other embodiments of resource writing methods according to the present disclosure is shown. The resource writing method includes the following steps:

步骤301,获取上述目标域名相关联的标签页面。Step 301: Acquire a tab page associated with the target domain name.

在一些实施例中,执行主体(例如图1所示的客户端101)可以获取上述目标域名相关联的标签页面。其中,引擎资源包括:无界面模式引擎的引擎资源。上述无界面模式引擎可以是Headless引擎。In some embodiments, the execution body (eg, the client 101 shown in FIG. 1 ) can obtain the tag page associated with the target domain name. The engine resources include: engine resources of the interfaceless mode engine. The above interfaceless mode engine may be a Headless engine.

作为示例,上述执行主体可以通过以下步骤来获取上述目标域名相关联的标签页面:As an example, the above-mentioned execution body may obtain the tab page associated with the above-mentioned target domain name through the following steps:

第一步,对目标域名进行域名解析,以获取目标域名对应的ip地址。The first step is to perform domain name resolution on the target domain name to obtain the IP address corresponding to the target domain name.

第二步,根据ip地址,通过预先封装的标签页面管理引擎,来获取目标域名对应的标签页面。The second step is to obtain the label page corresponding to the target domain name through the pre-packaged label page management engine according to the IP address.

作为示例,如图4所示,图4示出了标签页面的示意图。As an example, as shown in FIG. 4, FIG. 4 shows a schematic diagram of a tab page.

步骤302,利用上述无界面模式引擎,获取上述标签页面相关联的页面资源。Step 302, using the above-mentioned no-interface mode engine to acquire the page resource associated with the above-mentioned tab page.

在一些实施例中,上述执行主体可以利用上述无界面模式引擎,获取上述标签页面相关联的页面资源。In some embodiments, the above-mentioned execution body may use the above-mentioned interfaceless mode engine to obtain the page resource associated with the above-mentioned tab page.

作为示例,上述执行主体可以利用上述无界面模式引擎中预先封装的自动化页面点击脚本来获取上述标签页面相关联的页面资源。其中,上述自动化页面点击脚本可以是JS脚本。其中,在无界面模式引擎启动时,会将域名、Cookie(储存在用户本地终端上的数据)、Host(主机)信息、JS自动化点击脚本一起封装到无界面模式引擎。As an example, the above-mentioned execution body may obtain the page resource associated with the above-mentioned tab page by using the automated page click script pre-packaged in the above-mentioned interfaceless mode engine. The above-mentioned automated page click script may be a JS script. Among them, when the no-interface mode engine is started, the domain name, Cookie (data stored on the user's local terminal), Host (host) information, and JS automatic click script are encapsulated into the no-interface mode engine together.

步骤303,根据上述标签页面相关联的页面资源,获取上述目标域名相关联的页面资源和上述网络请求资源。Step 303: Acquire the page resource associated with the target domain name and the network request resource according to the page resource associated with the tag page.

在一些实施例中,上述执行主体可以根据上述标签页面相关联的页面资源,获取上述目标域名相关联的页面资源和上述网络请求资源。In some embodiments, the execution body may acquire the page resource associated with the target domain name and the network request resource according to the page resource associated with the tag page.

作为示例,上述执行主体可以首先拦截标签页面对应的网络请求。然后,上述执行主体可以将标签页面对应的网络请求确定为目标域名对应的网络请求资源,以及将标签页面相关联的页面资源确定为目标域名相关联的页面资源。As an example, the above-mentioned execution body may first intercept the network request corresponding to the tab page. Then, the execution body may determine the network request corresponding to the tab page as the network request resource corresponding to the target domain name, and determine the page resource associated with the tab page as the page resource associated with the target domain name.

在一些实施例的一些可选的实现方式中,上述根据上述标签页面相关联的页面资源,获取上述目标域名相关联的页面资源和上述网络请求资源,可以包括以下步骤:In some optional implementations of some embodiments, obtaining the page resource associated with the target domain name and the network request resource according to the page resource associated with the label page may include the following steps:

第一步,响应于确定上述标签页面相关联的页面资源中存在链接资源,上述执行主体可以确定上述标签页面相关联的页面资源中的至少一个链接资源。In the first step, in response to determining that there is a link resource in the page resource associated with the tab page, the execution body may determine at least one link resource in the page resource associated with the tab page.

作为示例,上述执行主体可以通过页面查询的方式来确定上述标签页面相关联的页面资源中的至少一个链接资源。As an example, the above-mentioned execution body may determine at least one link resource in the page resources associated with the above-mentioned tab page by means of page query.

第二步,上述执行主体拦截上述标签页面对应的网络请求,以得到上述标签页面对应的网络请求资源。In the second step, the above-mentioned execution body intercepts the network request corresponding to the above-mentioned tab page, so as to obtain the network request resource corresponding to the above-mentioned tab page.

第三步,对于上述至少一个链接资源中的每个链接资源,执行页面资源获取步骤:The third step, for each link resource in the above at least one link resource, execute the page resource acquisition step:

第一子步骤,生成与上述链接资源相对应的标签页面,作为目标标签页面。In the first sub-step, a tab page corresponding to the above linked resource is generated as a target tab page.

第二子步骤,获取与上述目标标签页面相关联的页面资源。The second sub-step is to acquire page resources associated with the above target tab page.

作为示例,上述执行主体可以利用上述自动化页面点击脚本,来获取与上述目标标签页面相关联的页面资源。As an example, the above-mentioned execution body may use the above-mentioned automated page click script to acquire the page resource associated with the above-mentioned target tab page.

第三子步骤,上述执行主体可以拦截上述目标标签页面对应的网络请求,以得到上述目标标签页面对应的网络请求资源。In the third sub-step, the execution body may intercept the network request corresponding to the target tab page, so as to obtain the network request resource corresponding to the target tab page.

第四子步骤,上述执行主体可以确定上述目标标签页面相关联的页面资源是否存在链接资源。In the fourth sub-step, the execution body may determine whether there is a link resource in the page resource associated with the target tab page.

作为示例,上述执行主体可以通过链接查询的方式来确定上述目标标签页面相关联的页面资源是否存在链接资源。As an example, the above-mentioned execution body may determine whether there is a link resource in the page resource associated with the above-mentioned target tab page by means of a link query.

第五子步骤,响应于确定上述目标标签页面相关联的页面资源中不存在链接资源,上述执行主体可以将上述标签页面相关联的页面资源和上述目标标签页面相关联的页面资源确定为子页面资源,以及将上述标签页面对应的网络请求资源和上述目标标签页面对应的网络请求资源进行组合,得到组合资源,作为子网络请求资源。其中,上述子页面资源为上述目标域名相关联的页面资源中的资源。上述子网络请求资源为上述目标域名相关联的网络请求资源中的资源。In the fifth sub-step, in response to determining that there is no link resource in the page resource associated with the target tab page, the execution body may determine the page resource associated with the target tab page and the page resource associated with the target tab page as a sub-page resource, and combining the network request resource corresponding to the above tab page and the network request resource corresponding to the target tab page to obtain the combined resource as the sub-network request resource. The above-mentioned sub-page resources are resources in the page resources associated with the above-mentioned target domain name. The above-mentioned sub-network request resource is a resource in the network request resource associated with the above-mentioned target domain name.

可选地,上述步骤还包括:Optionally, the above steps also include:

响应于确定上述目标标签页面相关联的页面资源中存在链接资源,上述执行主体确定上述目标标签页面相关联的页面资源中的至少一个链接资源,以及继续执行上述页面资源获取步骤。In response to determining that there is a link resource in the page resource associated with the target tab page, the execution body determines at least one link resource in the page resource associated with the target tab page, and continues to perform the page resource obtaining step.

可选地,上述步骤还包括:Optionally, the above steps also include:

响应于确定上述标签页面相关联的页面资源中不存在链接资源,上述执行主体可以将上述标签页面相关联的页面资源确定为上述目标域名相关联的页面资源,以及将上述标签页面对应的网络请求资源确定为上述目标域名相关联的网络请求资源。In response to determining that there is no link resource in the page resource associated with the above-mentioned tab page, the above-mentioned execution body may determine the page resource associated with the above-mentioned tab page as the page resource associated with the above-mentioned target domain name, and send the network request corresponding to the above-mentioned tab page. The resource is determined to be the network request resource associated with the above target domain name.

步骤304,对上述页面资源进行静态资源过滤,得到过滤后资源。Step 304: Perform static resource filtering on the above page resources to obtain filtered resources.

步骤305,对上述过滤后资源和上述网络请求资源进行资源分类,得到第一资源和第二资源。Step 305: Perform resource classification on the above-mentioned filtered resources and the above-mentioned network request resources to obtain a first resource and a second resource.

步骤306,对上述第二资源进行异常检测,以得到异常资源和正常资源。Step 306: Perform abnormality detection on the above-mentioned second resource to obtain abnormal resources and normal resources.

步骤307,将处理后的正常资源和处理后的第一资源写入待上传的第一资源队列,以及将上述异常资源写入待上传的第二资源队列。Step 307: Write the processed normal resource and the processed first resource into the first resource queue to be uploaded, and write the above-mentioned abnormal resource into the second resource queue to be uploaded.

在一些实施例中,步骤304-307的具体实现及其所带来的技术效果,可以参考图2对应的实施例中的步骤202-205,在此不再赘述。In some embodiments, for the specific implementation of steps 304-307 and the technical effects brought about by them, reference may be made to steps 202-205 in the embodiment corresponding to FIG. 2, and details are not repeated here.

从图3中可以看出,与图2对应的一些实施例的描述相比,图3对应的一些实施例中的资源写入方法的流程300更加突出了根据无界面模式引擎获取目标域名相关联的页面资源和网络请求资源的具体步骤。由此,这些实施例描述的方案可以在不需要人工点击的情况下,高效、快捷的实现自动化获取页面资源和网络请求资源。As can be seen from FIG. 3 , compared with the description of some embodiments corresponding to FIG. 2 , the process 300 of the resource writing method in some embodiments corresponding to FIG. 3 more highlights the acquisition of the target domain name association according to the interfaceless mode engine The specific steps of the page resources and network request resources. Therefore, the solutions described in these embodiments can efficiently and quickly realize automatic acquisition of page resources and network request resources without requiring manual clicks.

进一步参考图5,作为对上述各图所示方法的实现,本公开提供了一种资源写入装置的一些实施例,这些装置实施例与图2所示的那些方法实施例相对应,该装置具体可以应用于各种电子设备中。Further referring to FIG. 5 , as an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a resource writing apparatus, these apparatus embodiments correspond to those method embodiments shown in FIG. 2 , the apparatus Specifically, it can be applied to various electronic devices.

如图5所示,一种资源写入装置500包括:获取单元501、资源过滤单元502、资源分类单元503、异常检测单元504和资源写入单元505。其中,获取单元501,被配置成响应于确定目标渲染页面引擎启动,根据上述目标渲染页面引擎在启动过程中加载的引擎资源,获取目标域名相关联的页面资源和网络请求资源;资源过滤单元502,被配置成对上述页面资源进行静态资源过滤,得到过滤后资源;资源分类单元503,被配置成对上述过滤后资源和上述网络请求资源进行资源分类,得到第一资源和第二资源,其中,上述第一资源为属于上述目标域名对应网络站点下的资源,上述第二资源为不属于上述网络站点下的资源;异常检测单元504,被配置成对上述第二资源进行异常检测,以得到异常资源和正常资源;资源写入单元505,被配置成将处理后的正常资源和处理后的第一资源写入待上传的第一资源队列,以及将上述异常资源写入待上传的第二资源队列。As shown in FIG. 5 , a resource writing device 500 includes: an acquisition unit 501 , a resource filtering unit 502 , a resource classification unit 503 , an abnormality detection unit 504 and a resource writing unit 505 . The obtaining unit 501 is configured to, in response to determining that the target rendering page engine is started, obtain the page resources and network request resources associated with the target domain name according to the engine resources loaded by the target rendering page engine during the startup process; the resource filtering unit 502 , is configured to perform static resource filtering on the above-mentioned page resources to obtain the filtered resources; the resource classification unit 503 is configured to perform resource classification on the above-mentioned filtered resources and the above-mentioned network request resources to obtain the first resource and the second resource, wherein , the above-mentioned first resource is a resource that belongs to the network site corresponding to the above-mentioned target domain name, and the above-mentioned second resource is a resource that does not belong to the above-mentioned network site; the abnormality detection unit 504 is configured to perform abnormality detection on the above-mentioned second resource to obtain Abnormal resources and normal resources; the resource writing unit 505 is configured to write the processed normal resources and the processed first resources into the first resource queue to be uploaded, and write the above abnormal resources into the second to-be-uploaded resource queue resource queue.

在一些实施例的一些可选的实现方式中,上述装置500还包括:上传单元(图中未显示)。其中,上述上传单元可以被配置成:将上述第一资源队列中的资源和上述第二资源队列中的资源上传至目标服务端。In some optional implementations of some embodiments, the above-mentioned apparatus 500 further includes: an uploading unit (not shown in the figure). The uploading unit may be configured to: upload the resources in the first resource queue and the resources in the second resource queue to the target server.

在一些实施例的一些可选的实现方式中,上述引擎资源包括:无界面模式引擎的引擎资源,以及获取单元501以进一步被配置成:获取上述目标域名相关联的标签页面;利用上述无界面模式引擎,获取上述标签页面相关联的页面资源;根据上述标签页面相关联的页面资源,获取上述目标域名相关联的页面资源和上述网络请求资源。In some optional implementations of some embodiments, the above-mentioned engine resources include: engine resources of the interfaceless mode engine, and the acquiring unit 501 is further configured to: acquire the tab page associated with the above-mentioned target domain name; use the above-mentioned interfaceless mode The schema engine obtains the page resources associated with the above-mentioned tab pages; according to the page resources associated with the above-mentioned tab pages, obtains the page resources associated with the above-mentioned target domain name and the above-mentioned network request resources.

在一些实施例的一些可选的实现方式中,上述装置500还包括:去重单元(图中未显示)。其中,上述去重单元可以被配置成:对上述正常资源和上述第一资源进行资源去重处理,得到处理后的正常资源和处理后的第一资源。In some optional implementations of some embodiments, the above-mentioned apparatus 500 further includes: a deduplication unit (not shown in the figure). The deduplication unit may be configured to: perform resource deduplication processing on the normal resources and the first resources to obtain the processed normal resources and the processed first resources.

在一些实施例的一些可选的实现方式中,获取单元501以进一步被配置成:响应于确定上述标签页面相关联的页面资源中存在链接资源,确定上述标签页面相关联的页面资源中的至少一个链接资源;拦截上述标签页面对应的网络请求,以得到上述标签页面对应的网络请求资源;对于上述至少一个链接资源中的每个链接资源,执行页面资源获取步骤:生成与上述链接资源相对应的标签页面,作为目标标签页面;获取与上述目标标签页面相关联的页面资源;拦截上述目标标签页面对应的网络请求,以得到上述目标标签页面对应的网络请求资源;确定上述目标标签页面相关联的页面资源是否存在链接资源;响应于确定上述目标标签页面相关联的页面资源中不存在链接资源,将上述标签页面相关联的页面资源和上述目标标签页面相关联的页面资源确定为子页面资源,以及将上述标签页面对应的网络请求资源和上述目标标签页面对应的网络请求资源进行组合,得到组合资源,作为子网络请求资源,其中,上述子页面资源为上述目标域名相关联的页面资源中的资源,上述子网络请求资源为上述目标域名相关联的网络请求资源中的资源。In some optional implementations of some embodiments, the obtaining unit 501 is further configured to: in response to determining that a link resource exists in the page resource associated with the above-mentioned tab page, determine at least one of the page resources associated with the above-mentioned tab page a link resource; intercept the network request corresponding to the above-mentioned tab page to obtain the network request resource corresponding to the above-mentioned tab page; for each link resource in the above-mentioned at least one link resource, perform a page resource acquisition step: generate a page resource corresponding to the above-mentioned link resource The tab page is used as the target tab page; obtain the page resources associated with the above-mentioned target tab page; intercept the network request corresponding to the above-mentioned target tab page to obtain the network request resource corresponding to the above-mentioned target tab page; determine that the above-mentioned target tab page is associated with Whether there is a link resource in the page resource of the above-mentioned target tab page; in response to determining that there is no link resource in the page resource associated with the above-mentioned target tab page, determine the page resource associated with the above-mentioned tab page and the page resource associated with the above-mentioned target tab page as a sub-page resource , and combine the network request resource corresponding to the above-mentioned tab page and the network request resource corresponding to the above-mentioned target tab page to obtain the combined resource as a sub-network request resource, wherein the above-mentioned sub-page resource is one of the page resources associated with the above-mentioned target domain name The above-mentioned sub-network request resource is a resource in the network request resource associated with the above-mentioned target domain name.

在一些实施例的一些可选的实现方式中,获取单元501以进一步被配置成:响应于确定上述目标标签页面相关联的页面资源中存在链接资源,确定上述目标标签页面相关联的页面资源中的至少一个链接资源,以及继续执行上述页面资源获取步骤。In some optional implementations of some embodiments, the obtaining unit 501 is further configured to: in response to determining that a link resource exists in the page resource associated with the target tab page, determine whether the page resource associated with the target tab page is in the page resource. at least one link resource, and continue to perform the above steps of obtaining page resources.

在一些实施例的一些可选的实现方式中,获取单元501以进一步被配置成:响应于确定上述标签页面相关联的页面资源中不存在链接资源,将上述标签页面相关联的页面资源确定为上述目标域名相关联的页面资源,以及将上述标签页面对应的网络请求资源确定为上述目标域名相关联的网络请求资源。In some optional implementations of some embodiments, the obtaining unit 501 is further configured to: in response to determining that there is no link resource in the page resource associated with the above tab page, determine the page resource associated with the above tab page as The page resource associated with the target domain name, and the network request resource corresponding to the label page is determined as the network request resource associated with the target domain name.

可以理解的是,该装置500中记载的诸单元与参考图2描述的方法中的各个步骤相对应。由此,上文针对方法描述的操作、特征以及产生的有益效果同样适用于装置500及其中包含的单元,在此不再赘述。It can be understood that the units recorded in the apparatus 500 correspond to the respective steps in the method described with reference to FIG. 2 . Therefore, the operations, features and beneficial effects described above with respect to the method are also applicable to the apparatus 500 and the units included therein, and details are not described herein again.

下面参考图6,其示出了适于用来实现本公开的一些实施例的电子设备(例如图1中的客户端101)600的结构示意图。图6示出的电子设备仅仅是一个示例,不应对本公开的实施例的功能和使用范围带来任何限制。Referring next to FIG. 6 , it shows a schematic structural diagram of an electronic device (eg, client 101 in FIG. 1 ) 600 suitable for implementing some embodiments of the present disclosure. The electronic device shown in FIG. 6 is just an example, and should not impose any limitation on the function and scope of use of the embodiments of the present disclosure.

如图6所示,电子设备600可以包括处理装置(例如中央处理器、图形处理器等)601,其可以根据存储在只读存储器(ROM)602中的程序或者从存储装置608加载到随机访问存储器(RAM)603中的程序而执行各种适当的动作和处理。在RAM 603中,还存储有电子设备600操作所需的各种程序和数据。处理装置601、ROM 602以及RAM603通过总线604彼此相连。输入/输出(I/O)接口605也连接至总线604。As shown in FIG. 6 , an electronic device 600 may include a processing device (eg, a central processing unit, a graphics processor, etc.) 601 that may be loaded into random access according to a program stored in a read only memory (ROM) 602 or from a storage device 608 Various appropriate actions and processes are executed by the programs in the memory (RAM) 603 . In the RAM 603, various programs and data necessary for the operation of the electronic device 600 are also stored. The processing device 601 , the ROM 602 , and the RAM 603 are connected to each other through a bus 604 . An input/output (I/O) interface 605 is also connected to bus 604 .

通常,以下装置可以连接至I/O接口605:包括例如触摸屏、触摸板、键盘、鼠标、摄像头、麦克风、加速度计、陀螺仪等的输入装置606;包括例如液晶显示器(LCD)、扬声器、振动器等的输出装置607;包括例如磁带、硬盘等的存储装置608;以及通信装置609。通信装置609可以允许电子设备600与其他设备进行无线或有线通信以交换数据。虽然图6示出了具有各种装置的电子设备600,但是应理解的是,并不要求实施或具备所有示出的装置。可以替代地实施或具备更多或更少的装置。图6中示出的每个方框可以代表一个装置,也可以根据需要代表多个装置。Typically, the following devices can be connected to the I/O interface 605: input devices 606 including, for example, a touch screen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; including, for example, a liquid crystal display (LCD), speakers, vibration An output device 607 of a computer, etc.; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609. Communication means 609 may allow electronic device 600 to communicate wirelessly or by wire with other devices to exchange data. While FIG. 6 shows electronic device 600 having various means, it should be understood that not all of the illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided. Each block shown in FIG. 6 may represent one device, or may represent multiple devices as required.

特别地,根据本公开的一些实施例,上文参考流程图描述的过程可以被实现为计算机软件程序。例如,本公开的一些实施例包括一种计算机程序产品,其包括承载在计算机可读介质上的计算机程序,该计算机程序包含用于执行流程图所示的方法的程序代码。在这样的一些实施例中,该计算机程序可以通过通信装置609从网络上被下载和安装,或者从存储装置608被安装,或者从ROM 602被安装。在该计算机程序被处理装置601执行时,执行本公开的一些实施例的方法中限定的上述功能。In particular, according to some embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, some embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the method illustrated in the flowchart. In some such embodiments, the computer program may be downloaded and installed from the network via the communication device 609 , or from the storage device 608 , or from the ROM 602 . When the computer program is executed by the processing device 601, the above-described functions defined in the methods of some embodiments of the present disclosure are performed.

需要说明的是,本公开的一些实施例上述的计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质或者是上述两者的任意组合。计算机可读存储介质例如可以是——但不限于——电、磁、光、电磁、红外线、或半导体的系统、装置或器件,或者任意以上的组合。计算机可读存储介质的更具体的例子可以包括但不限于:具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机访问存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑磁盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本公开的一些实施例中,计算机可读存储介质可以是任何包含或存储程序的有形介质,该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。而在本公开的一些实施例中,计算机可读信号介质可以包括在基带中或者作为载波一部分传播的数据信号,其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式,包括但不限于电磁信号、光信号或上述的任意合适的组合。计算机可读信号介质还可以是计算机可读存储介质以外的任何计算机可读介质,该计算机可读信号介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。计算机可读介质上包含的程序代码可以用任何适当的介质传输,包括但不限于:电线、光缆、RF(射频)等等,或者上述的任意合适的组合。It should be noted that, in some embodiments of the present disclosure, the computer-readable medium described above may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the foregoing two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or a combination of any of the above. More specific examples of computer readable storage media may include, but are not limited to, electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable Programmable read only memory (EPROM or flash memory), fiber optics, portable compact disk read only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing. In some embodiments of the present disclosure, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. Rather, in some embodiments of the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code therein. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device . Program code embodied on a computer readable medium may be transmitted using any suitable medium including, but not limited to, electrical wire, optical fiber cable, RF (radio frequency), etc., or any suitable combination of the foregoing.

在一些实施方式中,客户端、服务端可以利用诸如HTTP(HyperText TransferProtocol,超文本传输协议)之类的任何当前已知或未来研发的网络协议进行通信,并且可以与任意形式或介质的数字数据通信(例如,通信网络)互连。通信网络的示例包括局域网(“LAN”),广域网(“WAN”),网际网(例如,互联网)以及端对端网络(例如,ad hoc端对端网络),以及任何当前已知或未来研发的网络。In some embodiments, the client and server can use any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol) to communicate, and can communicate with digital data in any form or medium Communication (eg, a communication network) interconnects. Examples of communication networks include local area networks ("LAN"), wide area networks ("WAN"), the Internet (eg, the Internet), and peer-to-peer networks (eg, ad hoc peer-to-peer networks), as well as any currently known or future development network of.

上述计算机可读介质可以是上述电子设备中所包含的;也可以是单独存在,而未装配入该电子设备中。上述计算机可读介质承载有一个或者多个程序,当上述一个或者多个程序被该电子设备执行时,使得该电子设备:响应于确定目标渲染页面引擎启动,根据上述目标渲染页面引擎在启动过程中加载的引擎资源,获取目标域名相关联的页面资源和网络请求资源;对上述页面资源进行静态资源过滤,得到过滤后资源;对上述过滤后资源和上述网络请求资源进行资源分类,得到第一资源和第二资源,其中,上述第一资源为属于上述目标域名对应网络站点下的资源,上述第二资源为不属于上述网络站点下的资源;对上述第二资源进行异常检测,以得到异常资源和正常资源;将处理后的正常资源和处理后的第一资源写入待上传的第一资源队列,以及将上述异常资源写入待上传的第二资源队列。The above-mentioned computer-readable medium may be included in the above-mentioned electronic device; or may exist alone without being assembled into the electronic device. The above-mentioned computer-readable medium carries one or more programs, and when the above-mentioned one or more programs are executed by the electronic device, the electronic device: in response to determining that the target rendering page engine is started, according to the above-mentioned target rendering page engine in the startup process The engine resources loaded in the target domain name are obtained, and the page resources and network request resources associated with the target domain name are obtained; the static resource filtering is performed on the above page resources to obtain the filtered resources; resources and a second resource, wherein the first resource is a resource belonging to a network site corresponding to the target domain name, and the second resource is a resource that does not belong to the network site; anomaly detection is performed on the second resource to obtain anomalies resources and normal resources; write the processed normal resources and the processed first resources into the first resource queue to be uploaded, and write the above abnormal resources into the second resource queue to be uploaded.

可以以一种或多种程序设计语言或其组合来编写用于执行本公开的一些实施例的操作的计算机程序代码,上述程序设计语言包括面向对象的程序设计语言—诸如Java、Smalltalk、C++,还包括常规的过程式程序设计语言—诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务端上执行。在涉及远程计算机的情形中,远程计算机可以通过任意种类的网络——包括局域网(LAN)或广域网(WAN)——连接到用户计算机,或者,可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。Computer program code for carrying out operations of some embodiments of the present disclosure may be written in one or more programming languages, including object-oriented programming languages—such as Java, Smalltalk, C++, or a combination thereof, Also included are conventional procedural programming languages - such as the "C" language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (eg, using an Internet service provider to via Internet connection).

附图中的流程图和框图,图示了按照本公开各种实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上,流程图或框图中的每个方框可以代表一个模块、程序段、或代码的一部分,该模块、程序段、或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意,在有些作为替换的实现中,方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如,两个接连地表示的方框实际上可以基本并行地执行,它们有时也可以按相反的顺序执行,这依所涉及的功能而定。也要注意的是,框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合,可以用执行规定的功能或操作的专用的基于硬件的系统来实现,或者可以用专用硬件与计算机指令的组合来实现。The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code that contains one or more logical functions for implementing the specified functions executable instructions. It should also be noted that, in some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It is also noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented in dedicated hardware-based systems that perform the specified functions or operations , or can be implemented in a combination of dedicated hardware and computer instructions.

描述于本公开的一些实施例中的单元可以通过软件的方式实现,也可以通过硬件的方式来实现。所描述的单元也可以设置在处理器中,例如,可以描述为:一种处理器包括获取单元、资源过滤单元、资源分类单元、异常检测单元和资源写入单元。其中,这些单元的名称在某种情况下并不构成对该单元本身的限定,例如,获取单元还可以被描述为“响应于确定目标渲染页面引擎启动,根据上述目标渲染页面引擎在启动过程中加载的引擎资源,获取目标域名相关联的页面资源和网络请求资源的单元”。The units described in some embodiments of the present disclosure may be implemented by means of software, and may also be implemented by means of hardware. The described unit can also be set in the processor, for example, it can be described as: a processor includes an acquisition unit, a resource filtering unit, a resource classification unit, an anomaly detection unit and a resource writing unit. Wherein, the names of these units do not constitute a limitation on the unit itself under certain circumstances. For example, the acquisition unit may also be described as "in response to determining that the target rendering page engine is started, according to the above target rendering page engine during the startup process Loaded engine resources, obtain the page resources associated with the target domain name and the unit of network request resources".

本文中以上描述的功能可以至少部分地由一个或多个硬件逻辑部件来执行。例如,非限制性地,可以使用的示范类型的硬件逻辑部件包括:现场可编程门阵列(FPGA)、专用集成电路(ASIC)、专用标准产品(ASSP)、片上系统(SOC)、复杂可编程逻辑设备(CPLD)等等。The functions described herein above may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), Systems on Chips (SOCs), Complex Programmable Logical Devices (CPLDs) and more.

以上描述仅为本公开的一些较佳实施例以及对所运用技术原理的说明。本领域技术人员应当理解,本公开的实施例中所涉及的发明范围,并不限于上述技术特征的特定组合而成的技术方案,同时也应涵盖在不脱离上述发明构思的情况下,由上述技术特征或其等同特征进行任意组合而形成的其它技术方案。例如上述特征与本公开的实施例中公开的(但不限于)具有类似功能的技术特征进行互相替换而形成的技术方案。The above descriptions are merely some preferred embodiments of the present disclosure and illustrations of the applied technical principles. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solution formed by the specific combination of the above-mentioned technical features, and should also cover, without departing from the above-mentioned inventive concept, the above-mentioned Other technical solutions formed by any combination of technical features or their equivalent features. For example, a technical solution is formed by replacing the above-mentioned features with the technical features disclosed in the embodiments of the present disclosure (but not limited to) with similar functions.

Claims (10)

1.一种资源写入方法,包括:1. A resource writing method, comprising: 响应于确定目标渲染页面引擎启动,根据所述目标渲染页面引擎在启动过程中加载的引擎资源,获取目标域名相关联的页面资源和网络请求资源;In response to determining that the target rendering page engine is started, obtain the page resources and network request resources associated with the target domain name according to the engine resources loaded by the target rendering page engine during the startup process; 对所述页面资源进行静态资源过滤,得到过滤后资源;Perform static resource filtering on the page resources to obtain filtered resources; 对所述过滤后资源和所述网络请求资源进行资源分类,得到第一资源和第二资源,其中,所述第一资源为属于所述目标域名对应网络站点下的资源,所述第二资源为不属于所述网络站点下的资源;Perform resource classification on the filtered resource and the network request resource to obtain a first resource and a second resource, wherein the first resource is a resource belonging to a network site corresponding to the target domain name, and the second resource For resources that do not belong to the said network site; 对所述第二资源进行异常检测,以得到异常资源和正常资源;performing anomaly detection on the second resource to obtain anomalous resources and normal resources; 将处理后的正常资源和处理后的第一资源写入待上传的第一资源队列,以及将所述异常资源写入待上传的第二资源队列。The processed normal resources and the processed first resources are written into the first resource queue to be uploaded, and the abnormal resources are written into the second resource queue to be uploaded. 2.根据权利要求1所述的方法,其中,所述方法还包括:2. The method of claim 1, wherein the method further comprises: 将所述第一资源队列中的资源和所述第二资源队列中的资源上传至目标服务端。Upload the resources in the first resource queue and the resources in the second resource queue to the target server. 3.根据权利要求1所述的方法,其中,所述引擎资源包括:无界面模式引擎的引擎资源;以及3. The method of claim 1, wherein the engine resources comprise: engine resources of a no-interface mode engine; and 所述根据所述目标渲染页面引擎在启动过程中加载的引擎资源,获取目标域名相关联的页面资源和网络请求资源,包括:The rendering of the engine resources loaded by the page engine during the startup process according to the target, and obtaining the page resources and network request resources associated with the target domain name, include: 获取所述目标域名相关联的标签页面;obtain the tag page associated with the target domain name; 利用所述无界面模式引擎,获取所述标签页面相关联的页面资源;Utilize the interfaceless mode engine to obtain the page resource associated with the tab page; 根据所述标签页面相关联的页面资源,获取所述目标域名相关联的页面资源和所述网络请求资源。According to the page resource associated with the tab page, the page resource associated with the target domain name and the network request resource are acquired. 4.根据权利要求1所述的方法,其中,在所述将处理后的正常资源和处理后的第一资源写入待上传的第一资源队列,以及将所述异常资源写入待上传的第二资源队列之前,所述方法还包括:4. The method according to claim 1, wherein, in the process of writing the processed normal resources and the processed first resources into the first resource queue to be uploaded, and writing the abnormal resources into the to-be-uploaded first resource queue Before the second resource queue, the method further includes: 对所述正常资源和所述第一资源进行资源去重处理,得到处理后的正常资源和处理后的第一资源。Perform resource deduplication processing on the normal resource and the first resource to obtain the processed normal resource and the processed first resource. 5.根据权利要求3所述的方法,其中,所述根据所述标签页面相关联的页面资源,获取所述目标域名相关联的页面资源和所述网络请求资源,包括:5. The method according to claim 3, wherein the obtaining the page resource associated with the target domain name and the network request resource according to the page resource associated with the label page comprises: 响应于确定所述标签页面相关联的页面资源中存在链接资源,确定所述标签页面相关联的页面资源中的至少一个链接资源;In response to determining that a linked resource exists in the page resource associated with the tab page, determining at least one linked resource in the page resource associated with the tab page; 拦截所述标签页面对应的网络请求,以得到所述标签页面对应的网络请求资源;Intercepting the network request corresponding to the tab page to obtain the network request resource corresponding to the tab page; 对于所述至少一个链接资源中的每个链接资源,执行页面资源获取步骤:For each linked resource in the at least one linked resource, a page resource acquisition step is performed: 生成与所述链接资源相对应的标签页面,作为目标标签页面;generating a tab page corresponding to the linked resource as a target tab page; 获取与所述目标标签页面相关联的页面资源;obtaining page resources associated with the target tab page; 拦截所述目标标签页面对应的网络请求,以得到所述目标标签页面对应的网络请求资源;Intercepting the network request corresponding to the target tab page to obtain the network request resource corresponding to the target tab page; 确定所述目标标签页面相关联的页面资源是否存在链接资源;determining whether there is a link resource in the page resource associated with the target tab page; 响应于确定所述目标标签页面相关联的页面资源中不存在链接资源,将所述标签页面相关联的页面资源和所述目标标签页面相关联的页面资源确定为子页面资源,以及将所述标签页面对应的网络请求资源和所述目标标签页面对应的网络请求资源进行组合,得到组合资源,作为子网络请求资源,其中,所述子页面资源为所述目标域名相关联的页面资源中的资源,所述子网络请求资源为所述目标域名相关联的网络请求资源中的资源。In response to determining that a linked resource does not exist in the page resource associated with the target tab page, determining the page resource associated with the tab page and the page resource associated with the target tab page as sub-page resources, and determining the page resource associated with the target tab page The network request resource corresponding to the tab page and the network request resource corresponding to the target tab page are combined to obtain a combined resource as a sub-network request resource, wherein the sub-page resource is one of the page resources associated with the target domain name resource, the sub-network request resource is a resource in the network request resource associated with the target domain name. 6.根据权利要求5所述的方法,其中,所述方法还包括:6. The method of claim 5, wherein the method further comprises: 响应于确定所述目标标签页面相关联的页面资源中存在链接资源,确定所述目标标签页面相关联的页面资源中的至少一个链接资源,以及继续执行所述页面资源获取步骤。In response to determining that a link resource exists in the page resources associated with the target tab page, determining at least one link resource in the page resources associated with the target tab page, and continuing to perform the page resource obtaining step. 7.根据权利要求5所述的方法,其中,所述方法还包括:7. The method of claim 5, wherein the method further comprises: 响应于确定所述标签页面相关联的页面资源中不存在链接资源,将所述标签页面相关联的页面资源确定为所述目标域名相关联的页面资源,以及将所述标签页面对应的网络请求资源确定为所述目标域名相关联的网络请求资源。In response to determining that there is no link resource in the page resource associated with the label page, determining the page resource associated with the label page as the page resource associated with the target domain name, and sending a network request corresponding to the label page The resource is determined to be the network request resource associated with the target domain name. 8.一种资源写入装置,包括:8. A resource writing device, comprising: 获取单元,被配置成响应于确定目标渲染页面引擎启动,根据所述目标渲染页面引擎在启动过程中加载的引擎资源,获取目标域名相关联的页面资源和网络请求资源;an obtaining unit, configured to, in response to determining that the target rendering page engine is started, obtain the page resources and network request resources associated with the target domain name according to the engine resources loaded by the target rendering page engine during the startup process; 资源过滤单元,被配置成对所述页面资源进行静态资源过滤,得到过滤后资源;a resource filtering unit, configured to perform static resource filtering on the page resources to obtain filtered resources; 资源分类单元,被配置成对所述过滤后资源和所述网络请求资源进行资源分类,得到第一资源和第二资源,其中,所述第一资源为属于所述目标域名对应网络站点下的资源,所述第二资源为不属于所述网络站点下的资源;The resource classification unit is configured to perform resource classification on the filtered resources and the network request resources to obtain a first resource and a second resource, wherein the first resource belongs to the network site corresponding to the target domain name. resource, the second resource is a resource that does not belong to the network site; 异常检测单元,被配置成对所述第二资源进行异常检测,以得到异常资源和正常资源;an abnormality detection unit, configured to perform abnormality detection on the second resource to obtain abnormal resources and normal resources; 资源写入单元,被配置成将处理后的正常资源和处理后的第一资源写入待上传的第一资源队列,以及将所述异常资源写入待上传的第二资源队列。The resource writing unit is configured to write the processed normal resources and the processed first resources into the first resource queue to be uploaded, and to write the abnormal resources into the second resource queue to be uploaded. 9.一种电子设备,包括:9. An electronic device comprising: 一个或多个处理器;one or more processors; 存储装置,其上存储有一个或多个程序,a storage device on which one or more programs are stored, 当所述一个或多个程序被所述一个或多个处理器执行,使得所述一个或多个处理器实现如权利要求1-7中任一所述的方法。The one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method of any of claims 1-7. 10.一种计算机可读介质,其上存储有计算机程序,其中,所述程序被处理器执行时实现如权利要求1-7中任一所述的方法。10. A computer-readable medium having stored thereon a computer program, wherein the program, when executed by a processor, implements the method of any one of claims 1-7.
CN202210127943.5A 2022-02-11 2022-02-11 Resource writing method, apparatus, electronic device and computer readable medium Pending CN114491373A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210127943.5A CN114491373A (en) 2022-02-11 2022-02-11 Resource writing method, apparatus, electronic device and computer readable medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210127943.5A CN114491373A (en) 2022-02-11 2022-02-11 Resource writing method, apparatus, electronic device and computer readable medium

Publications (1)

Publication Number Publication Date
CN114491373A true CN114491373A (en) 2022-05-13

Family

ID=81481360

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210127943.5A Pending CN114491373A (en) 2022-02-11 2022-02-11 Resource writing method, apparatus, electronic device and computer readable medium

Country Status (1)

Country Link
CN (1) CN114491373A (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101158974A (en) * 2007-11-21 2008-04-09 深圳市茁壮网络技术有限公司 Method and device for quoting resource
CN103001817A (en) * 2011-09-16 2013-03-27 厦门市美亚柏科信息股份有限公司 Method and device for real-time detection of webpage cross-domain requests
CN112115266A (en) * 2020-09-25 2020-12-22 奇安信科技集团股份有限公司 Malicious website classification method and device, computer equipment and readable storage medium

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101158974A (en) * 2007-11-21 2008-04-09 深圳市茁壮网络技术有限公司 Method and device for quoting resource
CN103001817A (en) * 2011-09-16 2013-03-27 厦门市美亚柏科信息股份有限公司 Method and device for real-time detection of webpage cross-domain requests
CN112115266A (en) * 2020-09-25 2020-12-22 奇安信科技集团股份有限公司 Malicious website classification method and device, computer equipment and readable storage medium

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
公衍磊: "跨站脚本漏洞与攻击的客户端检测方法研究", 中国优秀硕士学位论文全文数据库信息科技辑, 15 September 2011 (2011-09-15), pages 30 - 50 *

Similar Documents

Publication Publication Date Title
US12225025B2 (en) Enhanced cloud infrastructure security through runtime visibility into deployed software
WO2022105591A1 (en) Cache server performance test method and apparatus, device, and medium
US11695803B2 (en) Extension framework for an information technology and security operations application
CN111131320B (en) Asset identification method, device, system and medium
WO2022231903A1 (en) On-premises action execution agent for cloud-based information technology and security operations applications
US20110138457A1 (en) Securing Communications Between Different Network Zones
CN110928934A (en) Data processing method and device for business analysis
WO2022105590A1 (en) Domain name certificate detection method and apparatus, electronic device and computer-readable medium
CN105577799A (en) Method and device for fault detection of database cluster
US20120266186A1 (en) Providing inter-platform application launch in context
CN110765334A (en) Data capture method, system, medium and electronic device
US20200028743A1 (en) Dynamic product installation based on user feedback
CN114490718A (en) Data output method, data output device, electronic equipment and computer readable medium
CN116150513A (en) Data processing method, device, electronic equipment and computer readable storage medium
CN111049949A (en) Domain name identification method, device, electronic device and medium
CN112668194B (en) Page-based automatic driving scene library information display method, device and device
CN113190771B (en) Resource processing method, device, electronic device and computer readable medium
CN111373377A (en) Error handling
CN112230891A (en) Interface document integration method and device, server and computer storage medium
CN114491373A (en) Resource writing method, apparatus, electronic device and computer readable medium
CN111611585A (en) Monitoring method, device, electronic device and medium for terminal equipment
CN112817874B (en) User interface testing method, device, equipment and medium
CN110730251B (en) Method, device, medium and electronic equipment for analyzing domain name
CN112182002A (en) Data disaster tolerance method and device, electronic equipment and computer readable medium
CN112532734A (en) Message sensitive information detection method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination