CN109977677A - Vulnerability information collection method, device, equipment and readable storage medium storing program for executing - Google Patents

Vulnerability information collection method, device, equipment and readable storage medium storing program for executing Download PDF

Info

Publication number
CN109977677A
CN109977677A CN201711468775.1A CN201711468775A CN109977677A CN 109977677 A CN109977677 A CN 109977677A CN 201711468775 A CN201711468775 A CN 201711468775A CN 109977677 A CN109977677 A CN 109977677A
Authority
CN
China
Prior art keywords
vulnerability
loophole
database website
vulnerability database
collection
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201711468775.1A
Other languages
Chinese (zh)
Inventor
荣夫弟
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ping An Technology Shenzhen Co Ltd
Original Assignee
Ping An Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ping An Technology Shenzhen Co Ltd filed Critical Ping An Technology Shenzhen Co Ltd
Priority to CN201711468775.1A priority Critical patent/CN109977677A/en
Priority to PCT/CN2018/076396 priority patent/WO2019127869A1/en
Publication of CN109977677A publication Critical patent/CN109977677A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/50Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems
    • G06F21/57Certifying or maintaining trusted computer platforms, e.g. secure boots or power-downs, version controls, system software checks, secure updates or assessing vulnerabilities
    • G06F21/577Assessing vulnerabilities and evaluating computer system security
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/50Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems
    • G06F21/57Certifying or maintaining trusted computer platforms, e.g. secure boots or power-downs, version controls, system software checks, secure updates or assessing vulnerabilities
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00Network architectures or network communication protocols for network security
    • H04L63/14Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
    • H04L63/1433Vulnerability analysis
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2221/00Indexing scheme relating to security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F2221/03Indexing scheme relating to G06F21/50, monitoring users, programs or devices to maintain the integrity of platforms
    • G06F2221/034Test or assess a computer or a system

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Security & Cryptography (AREA)
  • Computer Hardware Design (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Computer And Data Communications (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

The present invention discloses a kind of vulnerability information collection method, device, equipment and readable storage medium storing program for executing, which comprises according to the pace of change of vulnerability database website orientation loophole, determines the current collection node of vulnerability database website;According to the redaction rule of vulnerability database website, the HTML extracting rule and URL extracting rule of vulnerability database website are determined;When reaching prefixed time interval, according to HTML extracting rule and URL extracting rule, controls current agent IP and access vulnerability database website, collection is extracted to the vulnerability information of each current collection node.This programme determines current collection node according to the loophole renewal speed of vulnerability database website, it can be ensured that collects newest vulnerability information in time, improves collection efficiency;Furthermore vulnerability database website is accessed using Agent IP, extract collection according to the HTML extracting rule of vulnerability database website and URL extracting rule, extraction it is with strong points, and when current agent IP is prohibited, replaceable other Agent IPs access, it is ensured that realize the collection of vulnerability information.

Description

Vulnerability information collection method, device, equipment and readable storage medium storing program for executing
Technical field
The invention mainly relates to financial air control systems technology fields, specifically, be related to a kind of vulnerability information collection method, Device, equipment and readable storage medium storing program for executing.
Background technique
Currently, popularizing with network, various loopholes cover the sky and the earth above network, if various loopholes cannot be found in time Repairing, by malicious exploitation, will cause heavy losses.Existing loophole is all to scan by single machine hole scanner or manually come It was found that lacking discovery in time and processing for some new loopholes.In addition the information of existing loophole platform anti-collection strategy mostly Compare strictly, simple single machine collection system is easy to be forbidden by loophole platform, and cannot achieve the collection of vulnerability information.
Summary of the invention
The main object of the present invention is to provide a kind of vulnerability information collection method, device, equipment and readable storage medium storing program for executing, purport In the vulnerability information for solving that update cannot be collected in time in the prior art, collects operation and be easy forbidden problem.
To achieve the above object, the present invention provides a kind of vulnerability information collection method, the vulnerability information collection method packet Include following steps:
According to the pace of change of vulnerability database website orientation loophole, the current collection node of vulnerability database website is determined;
According to the redaction rule of vulnerability database website, the HTML extracting rule and URL extracting rule of vulnerability database website are determined;
When reaching prefixed time interval, according to HTML extracting rule and URL extracting rule, current agent IP access is controlled Vulnerability database website extracts collection to the vulnerability information of each current collection node.
Preferably, the speed according to vulnerability database website orientation loophole determines the current collection node of vulnerability database website The step of include:
Loophole quantity collected by according to prefixed time interval and in prefixed time interval, determines vulnerability database website orientation The pace of change of loophole;
When the speed of vulnerability database website orientation loophole becomes larger, increase the current collection node of vulnerability database website;
When the speed of vulnerability database website orientation loophole becomes smaller, the current collection node of vulnerability database website is reduced.
Preferably, described when reaching prefixed time interval, according to HTML extracting rule and URL extracting rule, control is worked as Include: before the step of preceding Agent IP access vulnerability database website
Judge whether current agent IP is invalid, if current agent IP is invalid, other generations are randomly choosed from Agent IP library IP is managed as current agent IP.
Preferably, the step for judging whether current agent IP is invalid includes:
It controls current agent IP and accesses vulnerability database website, and receive the feedback information that access vulnerability database website is returned;
When the identifier for carrying access failure in feedback information, then determine that current agent IP is invalid.
Preferably, include: after the step of vulnerability information to each current collection node extracts collection
Vulnerability information is resolved into loophole reference information, and the loophole reference information is inserted into each of predetermined system Data source, to be monitored to each data source.
Preferably, described the step of being monitored to each data source, includes:
By in each data source data and loophole reference information comparison, with judge in data source with the presence or absence of with it is any one The item consistent data of loophole reference information;
When there are data consistent with any one loophole reference information in data source, to using making for the data source Prompt information is sent with side.
In addition, to achieve the above object, the present invention also proposes that a kind of vulnerability information collection device, the vulnerability information are collected Device includes:
First determining module determines working as vulnerability database website for the pace of change according to vulnerability database website orientation loophole Preceding collector node;
Second determining module determines that the HTML of vulnerability database website extracts rule for the redaction rule according to vulnerability database website Then with URL extracting rule;
Collection module, for according to HTML extracting rule and URL extracting rule, controlling when reaching prefixed time interval Current agent IP accesses vulnerability database website, extracts collection to the vulnerability information of each current collection node.
Preferably, first determining module includes:
Determination unit is determined for loophole quantity collected by according to prefixed time interval and in prefixed time interval The pace of change of vulnerability database website orientation loophole;
Adding unit, for increasing the current receipts of vulnerability database website when the speed of vulnerability database website orientation loophole becomes larger Collect node;
Unit is reduced, for reducing the current receipts of vulnerability database website when the speed of vulnerability database website orientation loophole becomes smaller Collect node.
In addition, to achieve the above object, the present invention also proposes that a kind of vulnerability information collecting device, the vulnerability information are collected Equipment includes: memory, processor, communication bus and the vulnerability information collection procedure being stored on the memory;
The communication bus is for realizing the connection communication between processor and memory;
The processor is for executing the vulnerability information collection procedure, to perform the steps of
According to the pace of change of vulnerability database website orientation loophole, the current collection node of vulnerability database website is determined;
According to the redaction rule of vulnerability database website, the HTML extracting rule and URL extracting rule of vulnerability database website are determined;
When reaching prefixed time interval, according to HTML extracting rule and URL extracting rule, current agent IP access is controlled Vulnerability database website extracts collection to the vulnerability information of each current collection node.
In addition, to achieve the above object, the present invention also provides a kind of readable storage medium storing program for executing, the readable storage medium storing program for executing storage Have one perhaps more than one program the one or more programs can be held by one or more than one processor Row is to be used for:
According to the pace of change of vulnerability database website orientation loophole, the current collection node of vulnerability database website is determined;
According to the redaction rule of vulnerability database website, the HTML extracting rule and URL extracting rule of vulnerability database website are determined;
When reaching prefixed time interval, according to HTML extracting rule and URL extracting rule, current agent IP access is controlled Vulnerability database website extracts collection to the vulnerability information of each current collection node.
The vulnerability information collection method of the present embodiment determines loophole according to the pace of change of vulnerability database website orientation loophole The current collection node of library website, while also according to the redaction rule of vulnerability database website, determine that the HTML of vulnerability database website is extracted Rule and URL extracting rule;To realize that, according to HTML extracting rule and URL extracting rule, control is worked as in prefixed time interval Preceding Agent IP accesses vulnerability database website, extracts collection to the vulnerability information of each current collection node.The vulnerability database of this programme The pace of change of website orientation loophole can embody the renewal speed of vulnerability database website vulnerability, be determined according to the renewal speed that this is embodied Current collection node, it can be ensured that collect newest vulnerability information in time, improve collection efficiency;Furthermore it is carried out using current agent IP Access extracts collection according to the HTML extracting rule of vulnerability database website and URL extracting rule, extraction it is with strong points, and When current agent IP is prohibited, replaceable other Agent IPs access, it is ensured that realize the collection of vulnerability information.
Detailed description of the invention
Fig. 1 is the flow diagram of vulnerability information collection method first embodiment of the invention;
Fig. 2 is the flow diagram of vulnerability information collection method second embodiment of the invention;
Fig. 3 is the flow diagram of vulnerability information collection method 3rd embodiment of the invention;
Fig. 4 is the functional block diagram of vulnerability information collection device first embodiment of the invention;
Fig. 5 is the device structure schematic diagram for the hardware running environment that present invention method is related to.
The embodiments will be further described with reference to the accompanying drawings for the realization, the function and the advantages of the object of the present invention.
Specific embodiment
It should be appreciated that the specific embodiments described herein are merely illustrative of the present invention, it is not intended to limit the present invention.
The present invention provides a kind of vulnerability information collection method.
Fig. 1 is please referred to, Fig. 1 is the flow diagram of vulnerability information collection method first embodiment of the present invention.In this implementation In example, the vulnerability information collection method includes:
Step S10 determines the current collection section of vulnerability database website according to the pace of change of vulnerability database website orientation loophole Point;
The vulnerability information of the present embodiment is collected by arranging collector node on vulnerability database website, to the loophole of collector node Information is collected, and vulnerability database website issues loophole on its site, and the loophole quantity that the different time is issued may not Equally, the loophole quantity loophole quantity reductions that are more, and being issued in future time issued in certain a period of time.When quantity is more When, the required loophole amount collected of each collector node is big;When data volume is few, the loophole amount of collection required for each collector node It is small.If to which the quantity of collector node is set as constant, when loophole quantity increases, in fact it could happen that collect the feelings not come Condition;And when loophole quantity is reduced, in fact it could happen that the case where node is not efficiently used.To which the present embodiment is according to vulnerability database The pace of change of website orientation loophole determines the current collection node for being currently used in and collecting vulnerability database website vulnerability.It is specifically walked Suddenly include:
Step S11, according to prefixed time interval and in prefixed time interval collected by loophole quantity, determine vulnerability database The pace of change of website orientation loophole;
Understandably, because vulnerability database website will not real-time release update, when being collected to loophole, if using real-time The strategy of collection may be wasted compared with multi-system resource.To set interval collection strategy, prefixed time interval is set, arrival is worked as When prefixed time interval, then loophole collection is carried out.This prefixed time interval can be preset according to demand, such as according to vulnerability database net The loophole renewal time interval stood sets this prefixed time interval, i.e. collectable to ensure to issue the first time updated in loophole To the loophole of update.When reaching prefixed time interval, i.e., the loophole of vulnerability database website is once collected, statistics is this time received The loophole quantity of collection.It is subsequent when arriving again at prefixed time interval, the loophole of vulnerability database website is collected again, and unites Count this loophole quantity collected again.Collection quantity twice is compared, determines the size relation of the two, the size of the two Relationship can embody the pace of change of publication loophole;The loophole quantity wherein collected becomes larger, then illustrates that the speed for issuing loophole becomes Greatly;When the roof quantity of collection becomes smaller, then illustrate that the speed for issuing loophole becomes smaller.
Step S12 increases the current collection section of vulnerability database website when the speed of vulnerability database website orientation loophole becomes larger Point;
When the speed of vulnerability database website orientation loophole becomes larger, that is, the loophole quantity issued increases, then vulnerability database is worked as Preceding collector node is increased, and increases more collector nodes as current collection node, to ensure to carry out increased loophole Comprehensive collection prevents from omitting.Judge the position that the increased loophole of vulnerability database issuing web site occurs, arranges and collect on this position Node is collected increased loophole using the collector node of arrangement as increased current collection node.Furthermore after increasing Current collection node in a distributed manner node form exist, the same time to multiple nodes carry out loophole collection, improve collect Efficiency.
Step S13 reduces the current collection section of vulnerability database website when the speed of vulnerability database website orientation loophole becomes smaller Point.
Correspondingly, when the loophole quantity issued is reduced, then working as to vulnerability database when the pace of change of publication loophole becomes smaller Preceding collector node is reduced, using the collector node after reduction as current collection node, to prevent from occupying compared with multi-system resource. Judge the position that the loophole of vulnerability database issuing web site reduction occurs, the collector node arranged on this position to be reduced, it will Collector node after reducing collector node is collected the loophole of reduction as current collection node.After corresponding reduction Also the form of node exists current collection node in a distributed manner, carries out loophole collection to multiple nodes in the same time, improves and receive Collect efficiency.
Step S20 determines that the HTML extracting rule of vulnerability database website and URL mention according to the redaction rule of vulnerability database website Take rule;
Understandably, vulnerability database website is write by program code, including multiple HTML (HyperText Markup Language, HyperText Markup Language) webpage, different html web page contents has differences in the mode of writing, such as certain Content uses XML format using HTTP transformat, other contents.Furthermore multiple web page address URL be may include in webpage (Uniform Resource Locator, uniform resource locator), can be linked to other webpages by web page address URL. When carrying out loophole collection to vulnerability database website, substantive process is the mistake extracted to the webpage and web page address of vulnerability database Journey.Because the rule of written in code is different, the rule that will lead to extraction is different, thus according to the redaction rule of vulnerability database website, really Its fixed HTML extracting rule and URL extracting rule.The content of HTML and URL is read, by writing program code to determine its volume Rule is write, and then corresponding extracting rule is determined according to redaction rule.
Step S30, when reaching prefixed time interval, according to HTML extracting rule and URL extracting rule, former generation is worked as in control It manages IP and accesses vulnerability database website, collection is extracted to the vulnerability information of each current collection node.
When reaching preset time interval, according to HTML extracting rule and URL extracting rule, controls current agent IP and visit It asks vulnerability database website, collection is extracted to the vulnerability information of each current collection node.Agent IP is gone by agency network user The network information is obtained, client is first connected with proxy server creation, then according to agency agreement used in proxy server, It requests to create connection to destination server or obtains the specified resource of destination server.The present embodiment is by working as former generation It manages IP and accesses vulnerability database website, obtain the vulnerability information for being arranged in each collector node on vulnerability database website, i.e., to vulnerability database website Collection is extracted in the vulnerability information of each current collection node publication.
The vulnerability information collection method of the present embodiment determines loophole according to the pace of change of vulnerability database website orientation loophole The current collection node of library website, while also according to the redaction rule of vulnerability database website, determine that the HTML of vulnerability database website is extracted Rule and URL extracting rule;To realize that, according to HTML extracting rule and URL extracting rule, control is worked as in prefixed time interval Preceding Agent IP accesses vulnerability database website, extracts collection to the vulnerability information of each current collection node.The vulnerability database of this programme The pace of change of website orientation loophole can embody the renewal speed of vulnerability database website vulnerability, be determined according to the renewal speed that this is embodied Current collection node, it can be ensured that collect newest vulnerability information in time, improve collection efficiency;Furthermore it is carried out using current agent IP Access extracts collection according to the HTML extracting rule of vulnerability database website and URL extracting rule, extraction it is with strong points, and When current agent IP is prohibited, replaceable other Agent IPs access, it is ensured that realize the collection of vulnerability information.
Further, referring to figure 2., on the basis of vulnerability information collection method first embodiment of the present invention, this is proposed Invention vulnerability information collection method second embodiment, it is in a second embodiment, described when reaching prefixed time interval, according to Include: before the step of HTML extracting rule and URL extracting rule, the control access vulnerability database website current agent IP
Step S40 judges whether current agent IP is invalid, if current agent IP is invalid, selects at random from Agent IP library Other Agent IPs are selected as current agent IP.
It further, may be by vulnerability database when the lasting access vulnerability database website using the same agent IP address , there is the case where being disabled by vulnerability database website in anti-collection strategy limitation, so that the present embodiment is in control current agent IP access leakage Before cave depot website, whether the time is prohibited the mechanism judged in vain to current agent IP at regular intervals for setting, with Ensure to control the validity of the current agent IP of access vulnerability database website.This interval time can set according to demand, and such as setting exists Reach first five minute of prefixed time interval.When judging that current agent IP is not prohibited, is still effective, then still use The access of current agent IP progress vulnerability database website.And when sentencing that current agent IP is prohibited invalid out, then from Agent IP library Other Agent IPs are randomly choosed as current agent IP.It include multiple Agent IPs in Agent IP library, if currently used Agent IP It is prohibited, i.e., randomly chooses other Agent IPs from Agent IP library and access as new current agent IP, and consider Still original Agent IP is chosen when random selection, to then select again more when new current agent IP is still invalid Agent IP is changed, that is, uses the mechanism of dynamic proxy IP.Using the dynamic proxy IP mode of multiple selection replacement Agent IP, generation is prevented Reason IP is prohibited, it is ensured that the validity of access vulnerability database website.Wherein judging current agent IP, whether invalid step includes:
Step S41, control current agent IP access vulnerability database website, and receive the feedback that access vulnerability database website is returned Information;
When the ineffectivity for carrying out current agent IP judges, control current agent IP accesses vulnerability database website, and judgement is It is no to access successfully, access success flag symbol is generated when accessing successfully, and the identifier of access failure is then generated when accessing identification, This success flag is accorded with or failure identifier is added to feedback information and returns, receives what this access vulnerability database website was returned Feedback information, to judge whether current agent IP is invalid according to this feedback information.
Step S42 then determines that current agent IP is invalid when the identifier for carrying access failure in feedback information.
When carrying the successful identifier of access in received feedback information, then illustrate the accessible leakage of current agent IP Cave depot website, current agent IP is effective, does not need to be replaced.And when the mark for carrying access failure in received feedback information When knowing symbol, then illustrates that current agent IP is prohibited, vulnerability database website cannot be accessed, current agent IP is invalid, does not need from agency The library IP randomly chooses other Agent IPs and is replaced.
Further, referring to figure 3., on the basis of vulnerability information collection method second embodiment of the present invention, this is proposed Invention vulnerability information collection method 3rd embodiment, in the third embodiment, the vulnerability information to each current collection node Include: after the step of extracting collection
Vulnerability information is resolved to loophole reference information, and the loophole reference information is inserted into default system by step S50 Each data source of system, to be monitored to each data source.
According to HTML extracting rule and URL extracting rule, after extracting collection to vulnerability information, need to this loophole Information is parsed, and loophole reference information is obtained.It is corresponding to use for the vulnerability information extracted by HTML extracting rule HTML mode is parsed;For the vulnerability information extracted by URL extracting rule, then parsed using URL mode.Cause The address URL can exist or exist in the form of details, the leakage that will first be extracted according to URL extracting rule in the form of a list Hole information and url list rule match illustrate that the vulnerability information extracted is list type, for list type if successful match Vulnerability information do not need to be parsed, can by its directly as loophole reference information carry out using.If matching is unsuccessful, Illustrate that the vulnerability information extracted is details formula, the vulnerability information of details formula is then needed to use after parsing.Parsing Obtain the loopholes reference informations such as loophole title, loophole details, loophole severity level, loophole issuing time and pageview.This is leaked Hole reference information is inserted into each data source of predetermined system, is monitored to data source, check wherein whether carry it is leaky Any one in information.Predetermined system is the local system or third party system for needing to carry out vulnerability monitoring, and data source is For constructing the source of each data of predetermined system webpage, can be deposited in the form of text file, database or message queue For a plurality of types of data sources, priority can be set according to significance level, i.e., loophole reference information is preferentially being inserted into weight It is monitored in the data source for wanting degree high.Specifically, the step of being monitored to each data source include:
Step S51, by the data and the comparison of loophole reference information in each data source, to judge to whether there is in data source With the consistent data of any one loophole reference information;
Because loophole reference information is there are multinomial, and data source is there is also multiple types, and a plurality of types of data sources include each Plant the data for constructing predetermined system webpage.The data in each data source are referred to each loophole one by one when to monitor Information compares one by one, to judge to whether there is and the consistent data of any one in loophole reference information in data source.
Step S52, when there are data consistent with any one loophole reference information in data source, to using the number Prompt information is sent according to the user in source.
When judging to there are data consistent with any one loophole reference information in a certain data source, then illustrate this number Exist according to source and have leaky data, carries out webpage building using the data of this data source, webpage can be made to start a leak, lead to net The user that page uses loses.To be taken because the data source exists to using the user of the data source to send prompt information Data with leaky reference information, if this user can be had risk, be sent to it prompt information using this data source, Prompt information may include that there are the loopholes such as the data name of loophole, loophole title, loophole details, loophole severity level with reference to letter Breath avoids losing to prompt it according to this loophole reference information patching bugs as early as possible.
In addition, referring to figure 4., the present invention provides a kind of vulnerability information collection device, collects and fill in vulnerability information of the present invention It sets in first embodiment, the vulnerability information collection device includes:
First determining module 10 determines vulnerability database website for the pace of change according to vulnerability database website orientation loophole Current collection node;
The vulnerability information of the present embodiment is collected by arranging collector node on vulnerability database website, to the loophole of collector node Information is collected, and vulnerability database website issues loophole on its site, and the loophole quantity that the different time is issued may not Equally, the loophole quantity loophole quantity reductions that are more, and being issued in future time issued in certain a period of time.When quantity is more When, the required loophole amount collected of each collector node is big;When data volume is few, the loophole amount of collection required for each collector node It is small.If to which the quantity of collector node is set as constant, when loophole quantity increases, in fact it could happen that collect the feelings not come Condition;And when loophole quantity is reduced, in fact it could happen that the case where node is not efficiently used.To which the present embodiment is according to vulnerability database The pace of change of website orientation loophole, the first determining module 10 determine the current collection for being currently used in and collecting vulnerability database website vulnerability Node.This first determining module 10 includes:
Determination unit 11, for loophole quantity collected by according to prefixed time interval and in prefixed time interval, really Determine the pace of change of vulnerability database website orientation loophole;
Understandably, because vulnerability database website will not real-time release update, when being collected to loophole, if using real-time The strategy of collection may be wasted compared with multi-system resource.To set interval collection strategy, prefixed time interval is set, arrival is worked as When prefixed time interval, then loophole collection is carried out.This prefixed time interval can be preset according to demand, such as according to vulnerability database net The loophole renewal time interval stood sets this prefixed time interval, i.e. collectable to ensure to issue the first time updated in loophole To the loophole of update.When reaching prefixed time interval, i.e., the loophole of vulnerability database website is once collected, statistics is this time received The loophole quantity of collection.It is subsequent when arriving again at prefixed time interval, the loophole of vulnerability database website is collected again, and unites Count this loophole quantity collected again.Collection quantity twice is compared, determination unit 11 determines the size relation of the two, The size relation of the two can embody the pace of change of publication loophole;The loophole quantity wherein collected becomes larger, then illustrates publication leakage The speed in hole becomes larger;When the roof quantity of collection becomes smaller, then illustrate that the speed for issuing loophole becomes smaller.
Adding unit 12, for increasing the current of vulnerability database website when the speed of vulnerability database website orientation loophole becomes larger Collector node;
When the speed of vulnerability database website orientation loophole becomes larger, that is, the loophole quantity issued increases, adding unit 12 is then right The current collection node of vulnerability database is increased, and increases more collector nodes as current collection node, to ensure to increase Loophole carry out comprehensive collection, prevent from omitting.The position that the increased loophole of vulnerability database issuing web site occurs is judged, in this position Upper arrangement collector node is collected increased loophole using the collector node of arrangement as increased current collection node.This The form of current collection node after outer increase node in a distributed manner exists, and carries out loophole receipts to multiple nodes in the same time Collection improves collection efficiency.
Unit 13 is reduced, for reducing the current of vulnerability database website when the speed of vulnerability database website orientation loophole becomes smaller Collector node.
Correspondingly, when the loophole quantity issued is reduced, it is then right to reduce unit 13 when the pace of change of publication loophole becomes smaller The current collection node of vulnerability database is reduced, using the collector node after reduction as current collection node, with prevent from occupying compared with Multi-system resource.Judge the position that the loophole of vulnerability database issuing web site reduction occurs, the collector node that will be arranged on this position It is reduced, using the collector node after reduction collector node as current collection node, the loophole of reduction is collected.Accordingly Reduction after current collection node also in a distributed manner node form exist, the same time to multiple nodes carry out loophole receipts Collection improves collection efficiency.
Second determining module 20 determines that the HTML of vulnerability database website is extracted for the redaction rule according to vulnerability database website Rule and URL extracting rule;
Understandably, vulnerability database website is write by program code, including multiple HTML (HyperText Markup Language, HyperText Markup Language) webpage, different html web page contents has differences in the mode of writing, such as certain Content uses XML format using HTTP transformat, other contents.Furthermore multiple web page address URL be may include in webpage (Uniform Resource Locator, uniform resource locator), can be linked to other webpages by web page address URL. When carrying out loophole collection to vulnerability database website, substantive process is the mistake extracted to the webpage and web page address of vulnerability database Journey.Because the rule of written in code is different, the rule that will lead to extraction is different, thus according to the redaction rule of vulnerability database website, the Two determining modules 20 determine its HTML extracting rule and URL extracting rule.It is read in HTML and URL by writing program code Hold, to determine its redaction rule, and then corresponding extracting rule is determined according to redaction rule.
Collection module 30, for according to HTML extracting rule and URL extracting rule, controlling when reaching prefixed time interval Current agent IP processed accesses vulnerability database website, extracts collection to the vulnerability information of each current collection node.
When reaching preset time interval, according to HTML extracting rule and URL extracting rule, the control of collection module 30 is worked as Preceding Agent IP accesses vulnerability database website, extracts collection to the vulnerability information of each current collection node.Agent IP passes through agency The network user goes to obtain the network information, and client is first connected with proxy server creation, used then according to proxy server Agency agreement, request to destination server create connection or obtain destination server specified resource.The present embodiment is Pair vulnerability database website is accessed by current agent IP, obtains and is arranged in the vulnerability information of each collector node on vulnerability database website, i.e., Vulnerability database website extracts collection in the vulnerability information that each current collection node is issued.
The vulnerability information collection device of the present embodiment, the first determining module 10 is according to the variation of vulnerability database website orientation loophole Speed, determines the current collection node of vulnerability database website, while the second determining module 20 is according to the redaction rule of vulnerability database website, Determine the HTML extracting rule and URL extracting rule of vulnerability database website;To realize in prefixed time interval, collection module 30 According to HTML extracting rule and URL extracting rule, controls current agent IP and access vulnerability database website, the leakage to each current collection node Hole information extracts collection.The pace of change of the vulnerability database website orientation loophole of this programme can embody vulnerability database website vulnerability Renewal speed, the renewal speed embodied according to this determine current collection node, it can be ensured that collect newest vulnerability information in time, mention High collection efficiency;Furthermore it is accessed using current agent IP, is extracted and advised according to the HTML extracting rule and URL of vulnerability database website Then extract collection, extraction it is with strong points, and when current agent IP is prohibited, replaceable other Agent IPs are visited It asks, it is ensured that realize the collection of vulnerability information.
Referring to Fig. 5, Fig. 5 is the device structure schematic diagram for the hardware running environment that present invention method is related to.
Vulnerability information collecting device of the embodiment of the present invention can be PC, be also possible to smart phone, tablet computer, e-book The terminal devices such as reader, portable computer.
As shown in figure 5, the vulnerability information collecting device may include: processor 1001, such as CPU, memory 1005 leads to Believe bus 1002.Wherein, communication bus 1002 is for realizing the connection communication between processor 1001 and memory 1005.Storage Device 1005 can be high speed RAM memory, be also possible to stable memory (non-volatile memory), such as disk Memory.Memory 1005 optionally can also be the storage device independently of aforementioned processor 1001.
Optionally, which can also include user interface, network interface, camera, RF (Radio Frequency, radio frequency) circuit, sensor, voicefrequency circuit, WiFi module etc..User interface may include display screen (Display), input unit such as keyboard (Keyboard), optional user interface can also include wireline interface, the nothing of standard Line interface.Network interface optionally may include standard wireline interface and wireless interface (such as WI-FI interface).
It will be understood by those skilled in the art that vulnerability information collecting device structure shown in Fig. 5 is not constituted to loophole The restriction of information collecting device may include perhaps combining certain components or different than illustrating more or fewer components Component layout.
As shown in figure 5, as may include that operating system, network are logical in a kind of memory 1005 of computer storage medium Believe module and vulnerability information collection procedure.Operating system is to manage and control vulnerability information collecting device hardware and software resource Program, support the operation of vulnerability information collection procedure and other softwares and/or program.Network communication module is for realizing depositing Communication between the 1005 each component in inside of reservoir, and communicated between other hardware and softwares in vulnerability information collecting device.
In vulnerability information collecting device shown in Fig. 5, processor 1001 is for executing the leakage stored in memory 1005 Hole information collection procedure, performs the steps of
According to the pace of change of vulnerability database website orientation loophole, the current collection node of vulnerability database website is determined;
According to the redaction rule of vulnerability database website, the HTML extracting rule and URL extracting rule of vulnerability database website are determined;
When reaching prefixed time interval, according to HTML extracting rule and URL extracting rule, current agent IP access is controlled Vulnerability database website extracts collection to the vulnerability information of each current collection node.
Further, the speed according to vulnerability database website orientation loophole determines the current collection section of vulnerability database website Point the step of include:
Loophole quantity collected by according to prefixed time interval and in prefixed time interval, determines vulnerability database website orientation The pace of change of loophole;
When the speed of vulnerability database website orientation loophole becomes larger, increase the current collection node of vulnerability database website;
When the speed of vulnerability database website orientation loophole becomes smaller, the current collection node of vulnerability database website is reduced.
Further, described when reaching prefixed time interval, according to HTML extracting rule and URL extracting rule, control Before current agent IP accesses the step of vulnerability database website, processor 1001 is used to execute the loophole letter stored in memory 1005 Collection procedure is ceased, is performed the steps of
Judge whether current agent IP is invalid, if current agent IP is invalid, other generations are randomly choosed from Agent IP library IP is managed as current agent IP.
Further, the step for judging whether current agent IP is invalid includes:
It controls current agent IP and accesses vulnerability database website, and receive the feedback information that access vulnerability database website is returned;
When the identifier for carrying access failure in feedback information, then determine that current agent IP is invalid.
Further, after the step of vulnerability information to each current collection node extracts collection, processor 1001, for executing the vulnerability information collection procedure stored in memory 1005, perform the steps of
Vulnerability information is resolved into loophole reference information, and the loophole reference information is inserted into each of predetermined system Data source, to be monitored to each data source.
Further, described the step of being monitored to each data source, includes:
By in each data source data and loophole reference information comparison, with judge in data source with the presence or absence of with it is any one The item consistent data of loophole reference information;
When there are data consistent with any one loophole reference information in data source, to using making for the data source Prompt information is sent with side.
Vulnerability information collecting device specific embodiment of the present invention and above-mentioned each embodiment of vulnerability information collection method are basic Identical, details are not described herein.
The present invention provides a kind of readable storage medium storing program for executing, the readable storage medium storing program for executing is stored with one or more than one journey Sequence, the one or more programs can also be executed by one or more than one processor to be used for:
According to the pace of change of vulnerability database website orientation loophole, the current collection node of vulnerability database website is determined;
According to the redaction rule of vulnerability database website, the HTML extracting rule and URL extracting rule of vulnerability database website are determined;
When reaching prefixed time interval, according to HTML extracting rule and URL extracting rule, current agent IP access is controlled Vulnerability database website extracts collection to the vulnerability information of each current collection node.
Further, the speed according to vulnerability database website orientation loophole determines the current collection section of vulnerability database website Point the step of include:
Loophole quantity collected by according to prefixed time interval and in prefixed time interval, determines vulnerability database website orientation The pace of change of loophole;
When the speed of vulnerability database website orientation loophole becomes larger, increase the current collection node of vulnerability database website;
When the speed of vulnerability database website orientation loophole becomes smaller, the current collection node of vulnerability database website is reduced.
Further, described when reaching prefixed time interval, according to HTML extracting rule and URL extracting rule, control Before current agent IP accesses the step of vulnerability database website, the one or more programs can also be by one or one Above processor is executed to be used for:
Judge whether current agent IP is invalid, if current agent IP is invalid, other generations are randomly choosed from Agent IP library IP is managed as current agent IP.
Further, the step for judging whether current agent IP is invalid includes:
It controls current agent IP and accesses vulnerability database website, and receive the feedback information that access vulnerability database website is returned;
When the identifier for carrying access failure in feedback information, then determine that current agent IP is invalid.
Further, after the step of vulnerability information to each current collection node extracts collection, described one A or more than one program can also be executed by one or more than one processor to be used for:
Vulnerability information is resolved into loophole reference information, and the loophole reference information is inserted into each of predetermined system Data source, to be monitored to each data source.
Further, described the step of being monitored to each data source, includes:
By in each data source data and loophole reference information comparison, with judge in data source with the presence or absence of with it is any one The item consistent data of loophole reference information;
When there are data consistent with any one loophole reference information in data source, to using making for the data source Prompt information is sent with side.
Readable storage medium storing program for executing specific embodiment of the present invention and above-mentioned each embodiment of vulnerability information collection method are essentially identical, Details are not described herein.
It should also be noted that, herein, the terms "include", "comprise" or its any other variant are intended to non- It is exclusive to include, so that the process, method, article or the device that include a series of elements not only include those elements, It but also including other elements that are not explicitly listed, or further include solid by this process, method, article or device Some elements.In the absence of more restrictions, the element limited by sentence "including a ...", it is not excluded that including There is also other identical elements in the process, method of the element, article or device.
The serial number of the above embodiments of the invention is only for description, does not represent the advantages or disadvantages of the embodiments.
Through the above description of the embodiments, those skilled in the art can be understood that above-described embodiment side Method can be realized by means of software and necessary general hardware platform, naturally it is also possible to by hardware, but in many cases The former is more preferably embodiment.Based on this understanding, technical solution of the present invention substantially in other words does the prior art The part contributed out can be embodied in the form of software products, which is stored in one as described above In storage medium (such as ROM/RAM, magnetic disk, CD), including some instructions are used so that terminal device (it can be mobile phone, Computer, server or network equipment etc.) execute method described in each embodiment of the present invention.
The above description is only a preferred embodiment of the present invention, is not intended to limit the scope of the invention, all at this Under the design of invention, using equivalent structure transformation made by description of the invention and accompanying drawing content, or directly/it is used in it indirectly He is included in scope of patent protection of the invention relevant technical field.

Claims (10)

1. a kind of vulnerability information collection method, which is characterized in that the vulnerability information collection method the following steps are included:
According to the pace of change of vulnerability database website orientation loophole, the current collection node of vulnerability database website is determined;
According to the redaction rule of vulnerability database website, the HTML extracting rule and URL extracting rule of vulnerability database website are determined;
When reaching prefixed time interval, according to HTML extracting rule and URL extracting rule, controls current agent IP and access loophole Library website extracts collection to the vulnerability information of each current collection node.
2. vulnerability information collection method as described in claim 1, which is characterized in that described according to vulnerability database website orientation loophole Speed, the step of determining the current collection node of vulnerability database website includes:
Loophole quantity collected by according to prefixed time interval and in prefixed time interval determines vulnerability database website orientation loophole Pace of change;
When the speed of vulnerability database website orientation loophole becomes larger, increase the current collection node of vulnerability database website;
When the speed of vulnerability database website orientation loophole becomes smaller, the current collection node of vulnerability database website is reduced.
3. vulnerability information collection method as described in claim 1, which is characterized in that it is described when reaching prefixed time interval, According to HTML extracting rule and URL extracting rule, include: before controlling the step of current agent IP accesses vulnerability database website
Judge whether current agent IP is invalid, if current agent IP is invalid, other Agent IPs are randomly choosed from Agent IP library As current agent IP.
4. vulnerability information collection method as claimed in claim 3, which is characterized in that described to judge whether current agent IP is invalid The step of include:
It controls current agent IP and accesses vulnerability database website, and receive the feedback information that access vulnerability database website is returned;
When the identifier for carrying access failure in feedback information, then determine that current agent IP is invalid.
5. vulnerability information collection method according to any one of claims 1-4, which is characterized in that described to each current collection section Include: after the step of vulnerability information of point extracts collection
Vulnerability information is resolved into loophole reference information, and the loophole reference information is inserted into each data of predetermined system Source, to be monitored to each data source.
6. vulnerability information collection method as claimed in claim 5, which is characterized in that described to be monitored to each data source Step includes:
By the data and the comparison of loophole reference information in each data source, to judge to leak in data source with the presence or absence of with any one The consistent data of hole reference information;
When there are data consistent with any one loophole reference information in data source, to the user for using the data source Send prompt information.
7. a kind of vulnerability information collection device, which is characterized in that the vulnerability information collection device includes:
First determining module determines the current receipts of vulnerability database website for the pace of change according to vulnerability database website orientation loophole Collect node;
Second determining module, for the redaction rule according to vulnerability database website, determine vulnerability database website HTML extracting rule and URL extracting rule;
Collection module, for when reaching prefixed time interval, according to HTML extracting rule and URL extracting rule, control to be current Agent IP accesses vulnerability database website, extracts collection to the vulnerability information of each current collection node.
8. vulnerability information collection device as claimed in claim 7, which is characterized in that first determining module includes:
Determination unit determines loophole for loophole quantity collected by according to prefixed time interval and in prefixed time interval The pace of change of library website orientation loophole;
Adding unit, for increasing the current collection section of vulnerability database website when the speed of vulnerability database website orientation loophole becomes larger Point;
Unit is reduced, for reducing the current collection section of vulnerability database website when the speed of vulnerability database website orientation loophole becomes smaller Point.
9. a kind of vulnerability information collecting device, which is characterized in that the vulnerability information collecting device include: memory, processor, Communication bus and the vulnerability information collection procedure being stored on the memory;
The communication bus is for realizing the connection communication between processor and memory;
The processor is for executing the vulnerability information collection procedure, to realize as of any of claims 1-6 The step of vulnerability information collection method.
10. a kind of readable storage medium storing program for executing, which is characterized in that the computer class is read to be stored with vulnerability information collection on storage medium Program realizes that loophole of any of claims 1-6 such as is believed when the vulnerability information collection procedure is executed by processor The step of ceasing collection method.
CN201711468775.1A 2017-12-28 2017-12-28 Vulnerability information collection method, device, equipment and readable storage medium storing program for executing Pending CN109977677A (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201711468775.1A CN109977677A (en) 2017-12-28 2017-12-28 Vulnerability information collection method, device, equipment and readable storage medium storing program for executing
PCT/CN2018/076396 WO2019127869A1 (en) 2017-12-28 2018-02-12 Vulnerability information collection method, apparatus and device, and readable storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201711468775.1A CN109977677A (en) 2017-12-28 2017-12-28 Vulnerability information collection method, device, equipment and readable storage medium storing program for executing

Publications (1)

Publication Number Publication Date
CN109977677A true CN109977677A (en) 2019-07-05

Family

ID=67064976

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201711468775.1A Pending CN109977677A (en) 2017-12-28 2017-12-28 Vulnerability information collection method, device, equipment and readable storage medium storing program for executing

Country Status (2)

Country Link
CN (1) CN109977677A (en)
WO (1) WO2019127869A1 (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102231722B1 (en) * 2019-03-28 2021-03-25 네이버클라우드 주식회사 Apparatus and method for determining duplication of vulnerability
CN110958243A (en) * 2019-11-28 2020-04-03 米哈游科技(上海)有限公司 Network vulnerability submitting method and device, storage medium and electronic equipment
TWI728637B (en) * 2020-01-02 2021-05-21 中華電信股份有限公司 Information security protection method and computer-readable medium
CN114996718A (en) * 2022-06-30 2022-09-02 浙江网商银行股份有限公司 Data processing method and device

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120005147A1 (en) * 2010-06-30 2012-01-05 Hitachi Information Systems, Ltd. Information leak file detection apparatus and method and program thereof
CN103124223A (en) * 2011-12-21 2013-05-29 中国科学院软件研究所 Method for automatically judging security situation of IT (information technology) system in real time
CN104065645A (en) * 2014-05-28 2014-09-24 北京知道创宇信息技术有限公司 Web vulnerability protection method and apparatus
CN104125197A (en) * 2013-04-24 2014-10-29 阿里巴巴集团控股有限公司 Security baseline system and method thereof for implementing security checks
CN105991554A (en) * 2015-02-04 2016-10-05 阿里巴巴集团控股有限公司 Vulnerability detection method and equipment
CN106096406A (en) * 2016-05-30 2016-11-09 北京启明星辰信息安全技术有限公司 Method and device is analyzed in a kind of security breaches backtracking
CN106357635A (en) * 2016-09-09 2017-01-25 浪潮软件集团有限公司 Vulnerability comparison analysis method based on homologous framework
CN106656924A (en) * 2015-10-30 2017-05-10 北京神州泰岳软件股份有限公司 Method and device for processing security vulnerabilities of device
CN106911694A (en) * 2017-02-28 2017-06-30 广东电网有限责任公司信息中心 A kind of method automatically updated based on ANDROID leak knowledge bases

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101808093B (en) * 2010-03-15 2013-08-07 北京安天电子设备有限公司 System and method for automatically detecting WEB security
CN105678170B (en) * 2016-01-05 2018-05-29 广东工业大学 A kind of method of dynamic detection XSS loopholes

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120005147A1 (en) * 2010-06-30 2012-01-05 Hitachi Information Systems, Ltd. Information leak file detection apparatus and method and program thereof
CN103124223A (en) * 2011-12-21 2013-05-29 中国科学院软件研究所 Method for automatically judging security situation of IT (information technology) system in real time
CN104125197A (en) * 2013-04-24 2014-10-29 阿里巴巴集团控股有限公司 Security baseline system and method thereof for implementing security checks
CN104065645A (en) * 2014-05-28 2014-09-24 北京知道创宇信息技术有限公司 Web vulnerability protection method and apparatus
CN105991554A (en) * 2015-02-04 2016-10-05 阿里巴巴集团控股有限公司 Vulnerability detection method and equipment
CN106656924A (en) * 2015-10-30 2017-05-10 北京神州泰岳软件股份有限公司 Method and device for processing security vulnerabilities of device
CN106096406A (en) * 2016-05-30 2016-11-09 北京启明星辰信息安全技术有限公司 Method and device is analyzed in a kind of security breaches backtracking
CN106357635A (en) * 2016-09-09 2017-01-25 浪潮软件集团有限公司 Vulnerability comparison analysis method based on homologous framework
CN106911694A (en) * 2017-02-28 2017-06-30 广东电网有限责任公司信息中心 A kind of method automatically updated based on ANDROID leak knowledge bases

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
李慧等: "基于聚焦爬虫的农业信息服务平台设计与实现", 《农业经济与信息技术》 *

Also Published As

Publication number Publication date
WO2019127869A1 (en) 2019-07-04

Similar Documents

Publication Publication Date Title
EP2989543B1 (en) Method and device for updating client
CN109600408B (en) Resource sharing method and device, storage medium and computer equipment
CN109977677A (en) Vulnerability information collection method, device, equipment and readable storage medium storing program for executing
CN109743315A (en) For Activity recognition method, apparatus, equipment and the readable storage medium storing program for executing of website
US8983935B2 (en) Methods for utilizing a javascript emulator in a web content proxy server and devices thereof
CN101523393B (en) Locally storing web-based database data
CN111414407A (en) Data query method and device of database, computer equipment and storage medium
US8365188B2 (en) Content management
US20110113090A1 (en) Dynamic mobile client
CN102446105B (en) The method and system of on-demand customized application program
CN106708899A (en) Automatic point burying device
CN111651468B (en) SQL analysis-based data updating method and device, electronic equipment and storage medium
CN102651856A (en) Method, system and device for improving Internet surfing security of terminal
CN103443781A (en) Data delivery
CN103268319A (en) Cloud browser based on webpages
CN108200132B (en) Resource acquisition method, device, equipment and computer readable storage medium
CN106060004A (en) Database access method and database agent node
RU2316131C2 (en) Method for storing pages in memory of mobile device (variants) and mobile device for realization of the method
CN110225087A (en) Cloud access method, device and storage medium based on global load balancing
CN106817391A (en) Document breakpoint transmission method and apparatus
CN110022354A (en) Mobile application integrated approach, device, computer equipment and storage medium based on http protocol
CN113536185B (en) Application page loading method, storage medium and related equipment
CN111444455A (en) Browser compatible method, system, computer device and storage medium
CN112346944B (en) Method, equipment and storage medium for monitoring operation experience of webpage user
CN108900627B (en) Network request method, terminal device and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20190705

RJ01 Rejection of invention patent application after publication