CN112804192A - Method, apparatus, electronic device, program, and medium for monitoring hidden network leakage - Google Patents
Method, apparatus, electronic device, program, and medium for monitoring hidden network leakage Download PDFInfo
- Publication number
- CN112804192A CN112804192A CN202011522318.8A CN202011522318A CN112804192A CN 112804192 A CN112804192 A CN 112804192A CN 202011522318 A CN202011522318 A CN 202011522318A CN 112804192 A CN112804192 A CN 112804192A
- Authority
- CN
- China
- Prior art keywords
- data
- verified
- monitored
- leakage
- darknet
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 91
- 238000000034 method Methods 0.000 title claims abstract description 73
- 238000012795 verification Methods 0.000 claims abstract description 114
- 238000012216 screening Methods 0.000 claims abstract description 37
- 230000009193 crawling Effects 0.000 claims description 21
- 230000006399 behavior Effects 0.000 claims description 14
- 238000004590 computer program Methods 0.000 claims description 12
- 238000013075 data extraction Methods 0.000 claims description 9
- 238000012806 monitoring device Methods 0.000 claims description 8
- 239000003550 marker Substances 0.000 claims description 4
- 230000004044 response Effects 0.000 abstract description 8
- 230000008569 process Effects 0.000 description 18
- 238000010586 diagram Methods 0.000 description 13
- 238000001914 filtration Methods 0.000 description 8
- 230000007246 mechanism Effects 0.000 description 6
- 238000013024 troubleshooting Methods 0.000 description 6
- 238000004891 communication Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 241000234282 Allium Species 0.000 description 4
- 235000002732 Allium cepa var. cepa Nutrition 0.000 description 4
- BUGBHKTXTAQXES-UHFFFAOYSA-N Selenium Chemical compound [Se] BUGBHKTXTAQXES-UHFFFAOYSA-N 0.000 description 4
- 238000001514 detection method Methods 0.000 description 4
- 229910052711 selenium Inorganic materials 0.000 description 4
- 239000011669 selenium Substances 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 239000003795 chemical substances by application Substances 0.000 description 2
- 235000014510 cooky Nutrition 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000011835 investigation Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 238000013475 authorization Methods 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1441—Countermeasures against malicious traffic
Landscapes
- Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Computer Hardware Design (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention provides a dark net leakage monitoring method, a device, electronic equipment, a program and a medium. Suspicious data to be verified are actively extracted from the hidden network through the screening rules, verification of hidden network leakage is achieved through authenticity verification, monitoring of hidden network leakage is achieved aiming at specific detected data, the requirement of a user for actively monitoring the hidden network leakage is met, response measures can be timely taken based on a monitoring result, and data safety is improved.
Description
Technical Field
The present invention relates to the field of internet technologies, and in particular, to a method, an apparatus, an electronic device, a program, and a medium for monitoring darknet leakage.
Background
The darknet (Dark Net) is a relatively small part of a deep network that can only be accessed with special software, special authorization, or special settings on the computer, including small peer-to-peer networks of F2F and large popular networks operated by public organizations and individuals, such as Tor, liberty, I2P, rifle, etc. The Tor darknet may also be referred to as an onion area, using top level domain suffixes the onion and "onion routing" traffic anonymization techniques, require access using a Tor browser or Tor proxy.
The existing dark net data acquisition and use technology mainly focuses on the collection and storage of dark net information, and when a small amount of dark net data is used, the correlation and the supplementary information of the existing events are found out from the acquisition results.
Disclosure of Invention
The invention provides a method, a device, an electronic device, a program and a medium for monitoring hidden network leakage, which are used for solving the defects that whether the hidden network leakage exists in data cannot be actively monitored by utilizing the hidden network data and the requirements of users cannot be met in the prior art, and realize the active monitoring of the hidden network leakage.
The invention provides a hidden network leakage monitoring method, which comprises the following steps:
acquiring monitored target data from the dark net according to a screening rule; the screening rule is based on monitored data extraction to be subjected to the dark net leakage monitoring;
determining to-be-verified data to be subjected to authenticity verification based on the target data; wherein the authenticity verification is based on comparing the data to be verified and the monitored data;
and determining whether the monitored data has dark net leakage or not according to a verification result of performing authenticity verification on the data to be verified.
The invention provides a method for monitoring hidden network leakage, which is characterized in that on the basis, the data to be verified of which the authenticity is to be verified is determined based on the target data, and the method comprises the following steps:
sample data searched from a dark net according to the target data is used as the data to be verified;
and/or searching data transaction information from the dark web according to the target data, and taking full commodity data acquired according to the data transaction information as the data to be verified.
On the basis, the method for monitoring the hidden network leakage determines whether the monitored data has the hidden network leakage according to the verification result of the authenticity verification of the data to be verified, and comprises the following steps:
if the number of the data verified as real data is larger than the preset number in the verification result of the authenticity verification, dark net leakage exists in the monitored data;
and/or if the ratio of the number of the real data to the total number of the data to be verified is larger than a preset ratio in the verification result of the authenticity verification, the monitored data has dark network leakage.
On the basis, before determining whether the monitored data has the dark network leakage according to the verification result of the authenticity verification of the data to be verified, the method for monitoring the dark network leakage further comprises the following steps:
obtaining a verification result of authenticity verification of the data to be verified;
for any piece of data to be verified, if data matched with any piece of data to be verified exists in the monitored data, the verification result of any piece of data to be verified is real data.
On the basis, the method for monitoring the leakage of the hidden network, which is provided by the invention, is used for acquiring monitored target data from the hidden network according to a screening rule and comprises the following steps:
acquiring dark web data from a dark web through a crawler, and acquiring data matched with the screening rule from the dark web data as the target data;
wherein the crawler crawls the darknet data in at least one of the following ways: the method comprises the steps of crawling the dark web data by simulating a user login behavior, forwarding an access request to the dark web through different links, crawling the dark web data in parallel through different crawlers, and shielding a predetermined monitoring type path node for monitoring the access behavior.
The invention provides a hidden network leakage monitoring method, which further comprises the following steps on the basis of the above steps:
the method comprises the steps of storing dark web data acquired from a dark web through a crawler, and generating mark information for the dark web data stored each time; wherein the marker information includes a time at which the darknet data is stored;
wherein the darknet data comprises at least one of: and newly added data in the darknet and data with changed darknet.
According to the invention, on the basis, after determining that the monitored data has the dark network leakage, the method further comprises the following steps:
and for any data interface in the system for providing the monitored data, determining whether the risk of leaking the monitored data exists in any data interface according to the data content output by any data interface and the data content contained in the monitored data.
The invention also provides a hidden net leakage monitoring device, which comprises:
the acquisition unit is used for acquiring monitored target data from the dark net according to the screening rule; the screening rule is based on monitored data extraction to be subjected to the dark net leakage monitoring;
a first determination unit configured to determine data to be verified, which is to be subjected to authenticity verification, based on the target data; wherein the authenticity verification is based on comparing the data to be verified and the monitored data;
and the second determining unit is used for determining whether the monitored data has dark net leakage or not according to a verification result of performing authenticity verification on the data to be verified.
The invention also provides an electronic device, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor executes the program to realize the steps of the dark web leakage monitoring method.
The present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the darknet leakage monitoring method as any one of the above.
The present invention also provides a computer program which, when being executed by a processor, carries out the steps of the darknet leakage monitoring method according to any of the above.
According to the dark net leakage monitoring method, the device, the electronic equipment, the program and the medium, the monitored target data are obtained from the dark net through the screening rules extracted from the monitored data, the data to be verified, which may be leakage data, are further determined through the target data, and whether the dark net leakage exists in the monitored data is further determined through the authenticity verification of the data to be verified. Suspicious data to be verified are actively extracted from the hidden network through the screening rules, verification of hidden network leakage is achieved through authenticity verification, monitoring of hidden network leakage is achieved aiming at specific detected data, the requirement of a user for actively monitoring the hidden network leakage is met, response measures can be timely taken based on a monitoring result, and data safety is improved.
Drawings
In order to more clearly illustrate the technical solutions of the present invention or the prior art, the drawings needed for the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.
FIG. 1 is a schematic diagram of a functional module structure using darknet data for comparison provided by the present invention;
FIG. 2 is a schematic diagram of a process for using darknet data as a comparison provided by the present invention;
FIG. 3 is a schematic flow chart of a hidden network leakage monitoring method provided by the present invention;
FIG. 4 is a schematic diagram of crawler crawling darknet data provided by the present invention;
FIG. 5 is a block diagram of the hidden network leakage monitoring and interface troubleshooting provided by the present invention;
FIG. 6 is a block diagram of a hidden network leakage monitoring device according to the present invention;
FIG. 7 is a schematic block diagram of an electronic device according to the present invention;
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Fig. 1 is a schematic structural diagram of a functional module for using the dark web data for comparison provided in this embodiment, and fig. 2 is a schematic diagram of a process for using the dark web data for comparison provided in this embodiment, referring to fig. 1 and fig. 2, the existing dark web monitoring and acquisition technical scheme mainly focuses on the problem of capturing and storing the dark web data, and the related analysis and the early warning mainly focuses on certain relevance analysis and hotspot analysis, rather than mainly performing a dedicated dark web data leakage monitoring response based on a monitoring target.
For the problem that data leakage monitoring based on a monitoring target cannot be performed on dark network data in the prior art, fig. 3 is a schematic flow diagram of the dark network leakage monitoring method provided in this embodiment, and referring to fig. 3, the dark network leakage monitoring method includes:
step 301: acquiring monitored target data from the dark net according to a screening rule; the screening rule is based on monitored data extraction to be monitored for dark net leakage monitoring.
The monitored data is data to be monitored whether hidden network leakage exists, for example, the monitored data may be user information inside an enterprise (such as user information stored in a bank system).
The filtering rule is extracted from the monitored data, for example, the filtering rule is a regular expression for filtering the target data extracted from the monitored data, or a rule for filtering the commodity name and associated data of the target data, or a behavior feature description for filtering the target data, or the like.
The target data may be directly obtained from real-time data circulated in the darknet according to the filtering rule, or may be obtained from stored data captured by the darknet according to the filtering rule, which is not specifically limited in this embodiment. For example, in order to realize real-time monitoring on whether hidden network leakage exists in monitored data, the real-time data of the hidden network can be crawled through a crawler, and target data is obtained from the crawled real-time data through a screening rule.
Step 302: determining to-be-verified data to be subjected to authenticity verification based on the target data; wherein the authenticity verification is based on comparing the data to be verified with the monitored data.
The target data may be a web page link obtained according to the filtering rule, or may be information related to a certain product in the web page, for example, product introduction information, data transaction information, and the like of the certain product.
The authenticity verification is used for verifying the authenticity of the data to be verified, for example, the authenticity of the data to be verified can be determined by comparing the data to be verified with the monitored data.
Step 303: and determining whether the monitored data has dark net leakage or not according to a verification result of performing authenticity verification on the data to be verified.
Understandably, target data can be monitored from real-time data of dark net circulation according to a screening rule, and then authenticity verification is combined to determine whether the monitored data has dark net leakage or not, active monitoring of the data dark net leakage is achieved through real-time monitoring and authenticity verification of the screening rule, accordingly, loss can be timely stopped, data leakage is comprehensively monitored, and data safety is improved.
The embodiment provides a dark network leakage monitoring method, which includes acquiring monitored target data from a dark network through a screening rule extracted from monitored data, further determining to-be-verified data which may be leakage data according to the target data, and further determining whether the monitored data has dark network leakage or not through authenticity verification of the to-be-verified data. Suspicious data to be verified are actively extracted from the hidden network through the screening rules, verification of hidden network leakage is achieved through authenticity verification, monitoring of hidden network leakage is achieved aiming at specific detected data, the requirement of a user for actively monitoring the hidden network leakage is met, response measures can be timely taken based on a monitoring result, and data safety is improved.
Further, on the basis of the foregoing embodiment, the determining, based on the target data, to-be-verified data to be subjected to authenticity verification includes:
sample data searched from a dark net according to the target data is used as the data to be verified;
and/or searching data transaction information from the dark web according to the target data, and taking full commodity data acquired according to the data transaction information as the data to be verified.
For example, the target data may be links of a dark web page which is searched from the dark web and matched with the screening rule, and according to sample data which can be searched for trading in a dark web trading market by the links, whether dark web leakage exists or not can be determined through authenticity verification of the sample data. Data transaction information (namely information of personnel or transaction institutions buying and selling the data) can be searched through the target data, for example, seller information of the selling data is obtained, full-volume commodity data is further purchased through the data transaction information, and whether hidden network leakage exists or not is determined through authenticity verification of the full-volume commodity data.
In this embodiment, sample data or full-volume commodity data suspected to be leaked is acquired according to the target data, and the monitoring of the dark web leakage is realized through authenticity verification of the sample data or the full-volume commodity data.
Further, on the basis of the foregoing embodiments, determining whether the monitored data has dark network leakage according to a verification result of performing authenticity verification on the data to be verified includes:
if the number of the data verified as real data is larger than the preset number in the verification result of the authenticity verification, dark net leakage exists in the monitored data;
and/or if the ratio of the number of the real data to the total number of the data to be verified is larger than a preset ratio in the verification result of the authenticity verification, the monitored data has dark network leakage.
The process of performing the authenticity verification may determine whether the dark net leakage exists according to the number of pieces of data verified as the real data, or a proportion of data to be verified as the real data in the data to be verified.
In the embodiment, the judgment on whether the hidden network leakage exists is realized through authenticity verification, and the method is favorable for taking corresponding measures in time when the hidden network leakage exists.
Further, on the basis of the foregoing embodiments, before determining whether the monitored data has dark network leakage according to a verification result of performing authenticity verification on the data to be verified, the method further includes:
obtaining a verification result of authenticity verification of the data to be verified;
for any piece of data to be verified, if data matched with any piece of data to be verified exists in the monitored data, the verification result of any piece of data to be verified is real data.
The process of authenticity verification is to match any piece of data to be verified with each piece of data in the monitored data, if the monitored data has data matched with any piece of data to be verified, the data to be verified is the real data, otherwise, the data to be verified is not the real data.
In this embodiment, the data to be verified is compared with the monitored data, so as to verify the authenticity of the data to be verified.
Further, if the monitored data has the dark net leakage, alarm information is sent out.
Further, if the monitored data has dark net leakage, generating alarm information corresponding to the secret-related level of the monitored data; wherein, the secret-related level is determined according to at least one of the following information: the security level of the monitored data and the influence range of the leaked monitored data.
Specifically, the data to be verified can be sent to the banking system, so that the banking system performs authenticity verification on the data to be verified according to the monitored data stored in the banking system.
The authenticity verification may include the following:
the authenticity verification may employ sample content match verification or whole content match verification. Sample data contents exist in a partial data commodity home page, and by capturing the sample data contents, a target unit can quickly inquire whether the data is in a database of the target unit through the modes of BloomFilter, batch index inquiry and the like; the occurrence of a piece of data considers that the piece of data is true. And if the real proportion is higher, informing a target unit whether to purchase the commodity for further verification, and performing full-scale matching verification on the purchased data commodity to obtain the integral real proportion. If the monitored data has leakage, alarms of corresponding levels can be generated according to various factors such as data security level, influence range and the like.
Further, on the basis of the foregoing embodiments, the acquiring, according to the screening rule, the monitored target data from the darknet includes:
acquiring dark web data from a dark web through a crawler, and acquiring data matched with the screening rule from the dark web data as the target data;
wherein the crawler crawls the darknet data in at least one of the following ways: the method comprises the steps of crawling the dark web data by simulating a user login behavior, forwarding an access request to the dark web through different links, crawling the dark web data in parallel through different crawlers, and shielding a predetermined monitoring type path node for monitoring the access behavior.
The process of crawling darknet data with respect to crawlers includes:
the crawler can be made to simulate the user login behavior by being based on Python Selenium and using Firefox to drive the Tor browser to access the target site. The Selenium is an automated testing tool, and can drive a browser to execute a specific action, such as clicking, pulling down, and the like. And simultaneously, the source code of the current presented page of the browser can be obtained and can be seen and crawled. The Python Selenium and the corresponding Firefox driver can simulate the login behavior of a real user, so that the crawler module can normally run in a complex login environment (other verification needs to be carried out or verification codes need to be input during login), and the crawler module is more suitable for crawling in the Tor-intranet trading market (a strong anti-crawler mechanism).
And forwarding an access request to the hidden network through different links, and crawling the data of the hidden network, wherein specifically, the access flow is sequentially switched to a Privoxy process, a Tor process, a network agent (overseas) and a Tor network by a crawler process, and finally reaches an onion service site. The Privoxy process and the Tor process are used for flow conversion: privoxy is used as a transit proxy to convert HTTP traffic sent by a Python crawler process into SOCKS5 traffic, and then SOCKS5 traffic is sent out after being encapsulated by a Tor protocol by a Tor client. The network agent sends the Tor process flow to the Tor network node through the overseas server, and the node forwards the flow to the target website. The anti-crawler mechanism of the darknet can be resisted through forwarding between different links.
The hidden web data are crawled in parallel by different crawlers, so that the access efficiency and the access speed of the crawlers are improved.
The monitoring type path node is usually a website provided with honeypots, and in the embodiment, the crawler shields Tor nodes in a high-density honeypot area, so that the concealment of the crawler is improved.
Fig. 4 is a schematic diagram of the principle of crawling the dark web data by the crawler according to this embodiment, and referring to fig. 4, in the process of crawling the dark web data, the crawling process may be performed by a plurality of parallel crawler processes at the same time. Each crawler realizes the forwarding of access flow through the forwarding of multilink and multiple nodes, and the resistance to a hidden network anti-crawler mechanism is improved.
In the embodiment, the crawler crawling the hidden network improves the concealment of the crawler and the access efficiency by simulating the user login behavior, the multilink multiprocess operation and the shielding of the monitoring type path node.
Further, on the basis of the above embodiments, the method further includes:
the method comprises the steps of storing dark web data acquired from a dark web through a crawler, and generating mark information for the dark web data stored each time; wherein the marker information includes a time at which the darknet data is stored;
wherein the darknet data comprises at least one of: and newly added data in the darknet and data with changed darknet.
It can be understood that the data directly crawled by the crawler is usually web page codes, and therefore, the data crawled by the crawler can be stored after being analyzed.
Specifically, the parsing process may include:
and after the crawler module successfully logs in, acquiring the cookie from the browser driver, and updating the cookie into an HTMLSession object of requests _ html. The requests _ html library not only comprises a basic network request function, but also has page parsing and asynchronous page processing functions. The page analysis module processes the acquired website page by using the regular and key words, and the extracted fields mainly comprise commodity detailed information such as commodity names, commodity descriptions, commodity types and the like and supplier attribute information. To improve the efficiency of the crawler system, subsequent page requests are given control of the HTMLSession object.
Storing the parsed darknet data may include:
the data storage module uses a MongoDB database to store the data collected by the crawler module each day. Each item in the database corresponds to a data commodity, and the field in the item represents the commodity attribute acquired by the page analysis module from the darknet market. The data storage module always keeps the commodity attribute in each item to be latest, and records all historical data and the items changed each time in the items by using two fields of 'historical data' and 'change'. When the commodity data is changed, the original data of the changed fields are added into the 'change' fields, and meanwhile, the complete original data are stored in the 'historical data' fields. The list items in the "history data" and "change" fields may increase as the merchandise data is continually updated. The storage module can directly record the basic situation of all commodities in the market, and the historical data field is helpful for analyzing specified types of commodities or suppliers thereof at a later stage and better observing various changes of the commodities and the attributes of the suppliers thereof in a time dimension.
In this embodiment, the crawled dark web data is stored and the generated tag information is used, so that the required dark web data can be rapidly and accurately acquired according to the tag information, and whether leakage exists in the monitored data is monitored according to the acquired dark web data.
Further, on the basis of the foregoing embodiments, after determining that there is a darknet leakage in the monitored data, the method further includes:
and for any data interface in the system for providing the monitored data, determining whether the risk of leaking the monitored data exists in any data interface according to the data content output by any data interface and the data content contained in the monitored data.
For example, the data content included in the monitored data includes information such as name, telephone number, total amount of deposit, and consumption status, and if the overlapping rate of the data content output by any data interface and the data content included in the monitored data reaches a preset overlapping rate threshold value, the data interface may leak the monitored data. And further corresponding measures can be taken for any data interface, and loss can be stopped in time.
Specifically, the troubleshooting on the leaky interface includes:
the leakage data is normalized into different information dimension elements (normalized according to data content, such as name, telephone and the like), and is associated and mapped with interface input and output elements of an interface detection function library, so that the risk leakage point existing in the system interface is discovered. The interface detection function library is a system interface which is suspected of having data leakage possibility due to information overflow, weak verification and the like caused by service requirements, related field information of the system interface is systematically arranged, input and output are normalized, and a judgment function for receiving designated parameters and judging whether data leakage exists in an output interface is constructed. For example, if an online interface accepts name input and outputs a mobile phone number, the interface is judged to have a possible leakage risk. The normalized online interface is divided into parameters of 1 parameter, 2 parameters, 3 parameters and other dimensions according to different input information dimensions, and the input parameters are output as a result for judging whether the interface has information cross leakage risks.
In the embodiment, the investigation of the leakage interface with leakage risk is realized through the data content of the monitored data and the input and output data content of the data interface, so that the leakage of the interface to the monitored data is prevented in time, and the data safety is improved.
To further illustrate the technical solution provided by this embodiment, fig. 5 is a schematic diagram of a module architecture of the darknet leakage monitoring and interface troubleshooting provided by this embodiment, wherein each module may implement the leakage monitoring and interface troubleshooting on the monitored data through the following processes:
1. collecting a hidden net market page by a crawler through a crawling module;
2. the page analysis module analyzes the acquired dark net market page;
3. the analyzed structured data is stored in a storage module;
4. the target detection module simultaneously analyzes the analyzed page data in real time and finds out a monitoring target, for example, the target detection module analyzes the key words, the regular patterns, the characteristics and other factors of the monitoring target and finds out an object meeting the standard; the method comprises the steps that a keyword regular dictionary of specific related darknet commodity data of an enterprise and an object is customized according to monitoring requirements, the dictionary mainly comprises common full names, short names and the like of related products and mechanisms of a target enterprise, and an alarm is given once related privacy data of the enterprise are found;
5. the authenticity judgment module is used for judging the authenticity of the monitored darknet data by combining with other source information, and the authenticity judgment can adopt sample content matching verification or whole content matching verification;
6. if the judgment result is true, the alarm module generates an alarm of a corresponding level according to various factors such as data security level, influence range and the like;
7. and (4) performing risk leakage point investigation on a system interface based on the monitored real data while alarming.
This embodiment has the following improvements:
crawling for crawling the dark net data to avoid honeypot, and collecting time and link personifying anti-crawler mechanism;
judging the authenticity of the monitored leakage data;
detecting a system interface risk leakage point based on leakage data: the leakage data is normalized into different information dimension elements, the system interface extracts input and output elements, and correlation mapping is carried out on the leakage data elements and the interface elements to find the risk leakage points of the system interface;
monitoring response closed loop: data leakage-monitoring discovery-authenticity determination-early warning/leakage point detection-reduction of data leakage possibility.
The following technical effects are achieved:
aiming at the conditions that a dark net market trading market has a better anti-crawler mechanism and the dark net crawling is easily captured by honeypots, a data crawling module simulates real user login behaviors together through Selenium and Firefox drive and random time according with a certain rule, and honeypot collection is carried out based on international honeypot deployment situations to avoid capturing and collecting behaviors by overseas honeypots;
the method comprises the steps that purposiveness monitoring is conducted on dark net market data commodities, an alarm is adopted after the authenticity of a monitoring result is judged, and compared with a mode of crawling-storing-associating existing events for dark net data, purposiveness active monitoring can be conducted to find that data of the dark net market data commodities are leaked out in time mainly, so that a response is made;
meanwhile, on the basis of the monitored real data, the system interface risk leakage point troubleshooting response is carried out, and a closed loop of data leakage, monitoring discovery, authenticity judgment, early warning/leakage point troubleshooting and reduction of the possibility of data leakage is formed.
Fig. 6 is a block diagram of a dark net leakage monitoring device provided in this embodiment, and referring to fig. 6, the dark net leakage monitoring device includes an obtaining unit 601, a first determining unit 602, and a second determining unit 603;
an obtaining unit 601, configured to obtain monitored target data from the darknet according to a screening rule; the screening rule is based on monitored data extraction to be subjected to the dark net leakage monitoring;
a first determining unit 602, configured to determine, based on the target data, to-be-verified data to be subjected to authenticity verification; wherein the authenticity verification is based on comparing the data to be verified and the monitored data;
a second determining unit 603, configured to determine whether dark net leakage exists in the monitored data according to a verification result obtained by performing authenticity verification on the data to be verified.
The dark net leakage monitoring device provided in this embodiment is suitable for the dark net leakage monitoring methods provided in the above embodiments, and details are not described here.
The embodiment provides a device for monitoring dark net leakage, which acquires monitored target data from a dark net through a screening rule extracted from monitored data, further determines to-be-verified data which may be leakage data according to the target data, and further determines whether the monitored data has dark net leakage through authenticity verification of the to-be-verified data. Suspicious data to be verified are actively extracted from the hidden network through the screening rules, verification of hidden network leakage is achieved through authenticity verification, monitoring of hidden network leakage is achieved aiming at specific detected data, the requirement of a user for actively monitoring the hidden network leakage is met, response measures can be timely taken based on a monitoring result, and data safety is improved.
According to the invention, on the basis, the device for monitoring the leakage of the hidden network, which determines the data to be verified of which the authenticity is to be verified based on the target data, comprises the following steps:
sample data searched from a dark net according to the target data is used as the data to be verified;
and/or searching data transaction information from the dark web according to the target data, and taking full commodity data acquired according to the data transaction information as the data to be verified.
On the basis, the invention provides a device for monitoring the hidden network leakage, which determines whether the monitored data has the hidden network leakage according to the verification result of the authenticity verification of the data to be verified, and comprises the following steps:
if the number of the data verified as real data is larger than the preset number in the verification result of the authenticity verification, dark net leakage exists in the monitored data;
and/or if the ratio of the number of the real data to the total number of the data to be verified is larger than a preset ratio in the verification result of the authenticity verification, the monitored data has dark network leakage.
On the basis, before determining whether the monitored data has the dark net leakage according to the verification result of the authenticity verification of the data to be verified, the device for monitoring the dark net leakage further comprises:
obtaining a verification result of authenticity verification of the data to be verified;
for any piece of data to be verified, if data matched with any piece of data to be verified exists in the monitored data, the verification result of any piece of data to be verified is real data.
On the basis, the device for monitoring the leakage of the hidden network, which is provided by the invention, acquires the monitored target data from the hidden network according to the screening rule, comprises the following steps:
acquiring dark web data from a dark web through a crawler, and acquiring data matched with the screening rule from the dark web data as the target data;
wherein the crawler crawls the darknet data in at least one of the following ways: the method comprises the steps of crawling the dark web data by simulating a user login behavior, forwarding an access request to the dark web through different links, crawling the dark web data in parallel through different crawlers, and shielding a predetermined monitoring type path node for monitoring the access behavior.
The invention provides a hidden network leakage monitoring device, which further comprises the following components on the basis:
the method comprises the steps of storing dark web data acquired from a dark web through a crawler, and generating mark information for the dark web data stored each time; wherein the marker information includes a time at which the darknet data is stored;
wherein the darknet data comprises at least one of: and newly added data in the darknet and data with changed darknet.
According to the invention, on the basis, after determining that the monitored data has the dark net leakage, the dark net leakage monitoring device further comprises:
and for any data interface in the system for providing the monitored data, determining whether the risk of leaking the monitored data exists in any data interface according to the data content output by any data interface and the data content contained in the monitored data.
Fig. 7 illustrates a physical structure diagram of an electronic device, and as shown in fig. 7, the electronic device may include: a processor (processor)710, a communication Interface (Communications Interface)720, a memory (memory)730, and a communication bus 740, wherein the processor 710, the communication Interface 720, and the memory 730 communicate with each other via the communication bus 740. Processor 710 may invoke logic instructions in memory 730 to perform a darknet leak monitoring method comprising: acquiring monitored target data from the dark net according to a screening rule; the screening rule is based on monitored data extraction to be subjected to the dark net leakage monitoring; determining to-be-verified data to be subjected to authenticity verification based on the target data; wherein the authenticity verification is based on comparing the data to be verified and the monitored data; and determining whether the monitored data has dark net leakage or not according to a verification result of performing authenticity verification on the data to be verified.
In addition, the logic instructions in the memory 730 can be implemented in the form of software functional units and stored in a computer readable storage medium when the software functional units are sold or used as independent products. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions, which when executed by a computer, enable the computer to perform the darknet leakage monitoring method provided by the above methods, the method comprising: acquiring monitored target data from the dark net according to a screening rule; the screening rule is based on monitored data extraction to be subjected to the dark net leakage monitoring; determining to-be-verified data to be subjected to authenticity verification based on the target data; wherein the authenticity verification is based on comparing the data to be verified and the monitored data; and determining whether the monitored data has dark net leakage or not according to a verification result of performing authenticity verification on the data to be verified.
In yet another aspect, the present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program, which when executed by a processor is implemented to perform the darknet leakage monitoring method provided above, the method comprising: acquiring monitored target data from the dark net according to a screening rule; the screening rule is based on monitored data extraction to be subjected to the dark net leakage monitoring; determining to-be-verified data to be subjected to authenticity verification based on the target data; wherein the authenticity verification is based on comparing the data to be verified and the monitored data; and determining whether the monitored data has dark net leakage or not according to a verification result of performing authenticity verification on the data to be verified.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (11)
1. A hidden network leakage monitoring method is characterized by comprising the following steps:
acquiring monitored target data from the dark net according to a screening rule; the screening rule is based on monitored data extraction to be subjected to the dark net leakage monitoring;
determining to-be-verified data to be subjected to authenticity verification based on the target data; wherein the authenticity verification is based on comparing the data to be verified and the monitored data;
and determining whether the monitored data has dark net leakage or not according to a verification result of performing authenticity verification on the data to be verified.
2. The darknet leakage monitoring method according to claim 1, wherein the determining data to be verified for authenticity verification based on the target data comprises:
sample data searched from a dark net according to the target data is used as the data to be verified;
and/or searching data transaction information from the dark web according to the target data, and taking full commodity data acquired according to the data transaction information as the data to be verified.
3. The darknet leakage monitoring method according to claim 1, wherein said determining whether there is darknet leakage in the monitored data according to the verification result of the authenticity verification of the data to be verified comprises:
if the number of the data verified as real data is larger than the preset number in the verification result of the authenticity verification, dark net leakage exists in the monitored data;
and/or if the ratio of the number of the real data to the total number of the data to be verified is larger than a preset ratio in the verification result of the authenticity verification, the monitored data has dark network leakage.
4. The darknet leakage monitoring method according to claim 1, wherein before determining whether there is a darknet leakage in the monitored data according to the verification result of the authenticity verification of the data to be verified, further comprising:
obtaining a verification result of authenticity verification of the data to be verified;
for any piece of data to be verified, if data matched with any piece of data to be verified exists in the monitored data, the verification result of any piece of data to be verified is real data.
5. The darknet leakage monitoring method of claim 1, wherein the obtaining of the monitored target data from the darknet according to the screening rules comprises:
acquiring dark web data from a dark web through a crawler, and acquiring data matched with the screening rule from the dark web data as the target data;
wherein the crawler crawls the darknet data in at least one of the following ways: the method comprises the steps of crawling the dark web data by simulating a user login behavior, forwarding an access request to the dark web through different links, crawling the dark web data in parallel through different crawlers, and shielding a predetermined monitoring type path node for monitoring the access behavior.
6. The darknet leak monitoring method of claim 1, further comprising:
the method comprises the steps of storing dark web data acquired from a dark web through a crawler, and generating mark information for the dark web data stored each time; wherein the marker information includes a time at which the darknet data is stored;
wherein the darknet data comprises at least one of: and newly added data in the darknet and data with changed darknet.
7. The darknet leak monitoring method of claim 1, further comprising, after determining that a darknet leak exists in the monitored data:
and for any data interface in the system for providing the monitored data, determining whether the risk of leaking the monitored data exists in any data interface according to the data content output by any data interface and the data content contained in the monitored data.
8. A darknet leak monitoring device, comprising:
the acquisition unit is used for acquiring monitored target data from the dark net according to the screening rule; the screening rule is based on monitored data extraction to be subjected to the dark net leakage monitoring;
a first determination unit configured to determine data to be verified, which is to be subjected to authenticity verification, based on the target data; wherein the authenticity verification is based on comparing the data to be verified and the monitored data;
and the second determining unit is used for determining whether the monitored data has dark net leakage or not according to a verification result of performing authenticity verification on the data to be verified.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the darknet leakage monitoring method according to any of claims 1 to 7 when executing the program.
10. A non-transitory readable storage medium having stored thereon a computer program, characterized in that the computer program, when being executed by a processor, carries out the steps of the darknet leakage monitoring method according to any one of claims 1 to 7.
11. A computer program, characterized in that the computer program, when being executed by a processor, carries out the steps of the darknet leakage monitoring method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011522318.8A CN112804192A (en) | 2020-12-21 | 2020-12-21 | Method, apparatus, electronic device, program, and medium for monitoring hidden network leakage |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011522318.8A CN112804192A (en) | 2020-12-21 | 2020-12-21 | Method, apparatus, electronic device, program, and medium for monitoring hidden network leakage |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112804192A true CN112804192A (en) | 2021-05-14 |
Family
ID=75807153
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011522318.8A Pending CN112804192A (en) | 2020-12-21 | 2020-12-21 | Method, apparatus, electronic device, program, and medium for monitoring hidden network leakage |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112804192A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114928532A (en) * | 2022-05-17 | 2022-08-19 | 北京达佳互联信息技术有限公司 | Method, device, equipment and storage medium for generating alarm message |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104486320A (en) * | 2014-12-10 | 2015-04-01 | 国家电网公司 | Intranet sensitive information disclosure evidence collection system and method based on honeynet technology |
| US20170331839A1 (en) * | 2016-05-10 | 2017-11-16 | Allstate Insurance Company | Cyber-security presence monitoring and assessment |
| CN107944019A (en) * | 2017-12-11 | 2018-04-20 | 中广在线（北京）文化传媒有限公司 | A kind of monitoring device of public sentiment overseas based on crawler technology, system and method |
| US20190007440A1 (en) * | 2015-12-28 | 2019-01-03 | Sixgill Ltd. | Dark web monitoring, analysis and alert system and method |
| CN110119469A (en) * | 2019-05-22 | 2019-08-13 | 北京计算机技术及应用研究所 | A kind of data collection and transmission and method towards darknet |
| US20190317968A1 (en) * | 2016-12-16 | 2019-10-17 | Telefonica Digital España, S.L.U. | Method, system and computer program products for recognising, validating and correlating entities in a communications darknet |
| CN111492363A (en) * | 2017-03-17 | 2020-08-04 | 克劳尼克公司 | Detecting data leaks |
| CN112052470A (en) * | 2020-09-14 | 2020-12-08 | 国电南瑞南京控制系统有限公司 | Server file protection method and protection system |
| CN112100655A (en) * | 2020-09-09 | 2020-12-18 | 北京明朝万达科技股份有限公司 | Data detection method and device, electronic equipment and readable storage medium |
-
2020
- 2020-12-21 CN CN202011522318.8A patent/CN112804192A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104486320A (en) * | 2014-12-10 | 2015-04-01 | 国家电网公司 | Intranet sensitive information disclosure evidence collection system and method based on honeynet technology |
| US20190007440A1 (en) * | 2015-12-28 | 2019-01-03 | Sixgill Ltd. | Dark web monitoring, analysis and alert system and method |
| US20170331839A1 (en) * | 2016-05-10 | 2017-11-16 | Allstate Insurance Company | Cyber-security presence monitoring and assessment |
| US20190317968A1 (en) * | 2016-12-16 | 2019-10-17 | Telefonica Digital España, S.L.U. | Method, system and computer program products for recognising, validating and correlating entities in a communications darknet |
| CN111492363A (en) * | 2017-03-17 | 2020-08-04 | 克劳尼克公司 | Detecting data leaks |
| CN107944019A (en) * | 2017-12-11 | 2018-04-20 | 中广在线（北京）文化传媒有限公司 | A kind of monitoring device of public sentiment overseas based on crawler technology, system and method |
| CN110119469A (en) * | 2019-05-22 | 2019-08-13 | 北京计算机技术及应用研究所 | A kind of data collection and transmission and method towards darknet |
| CN112100655A (en) * | 2020-09-09 | 2020-12-18 | 北京明朝万达科技股份有限公司 | Data detection method and device, electronic equipment and readable storage medium |
| CN112052470A (en) * | 2020-09-14 | 2020-12-08 | 国电南瑞南京控制系统有限公司 | Server file protection method and protection system |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114928532A (en) * | 2022-05-17 | 2022-08-19 | 北京达佳互联信息技术有限公司 | Method, device, equipment and storage medium for generating alarm message |
| CN114928532B (en) * | 2022-05-17 | 2023-12-12 | 北京达佳互联信息技术有限公司 | Alarm message generation method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112131882B (en) | Multi-source heterogeneous network security knowledge graph construction method and device | |
| US20210382949A1 (en) | Systems and methods for web content inspection | |
| US10019744B2 (en) | Multi-dimensional behavior device ID | |
| US9213990B2 (en) | Method of reducing financial fraud by user devices patronizing commercial websites | |
| CN109347801B (en) | Vulnerability exploitation risk assessment method based on multi-source word embedding and knowledge graph | |
| Patil et al. | A methodical overview on phishing detection along with an organized way to construct an anti-phishing framework | |
| CN108156131A (en) | Webshell detection methods, electronic equipment and computer storage media | |
| CN109831459B (en) | Method, device, storage medium and terminal equipment for secure access | |
| CN113098887A (en) | Phishing website detection method based on website joint characteristics | |
| CN107800686A (en) | A kind of fishing website recognition methods and device | |
| Singh et al. | A survey on different phases of web usage mining for anomaly user behavior investigation | |
| CN107231364A (en) | A kind of website vulnerability detection method and device, computer installation and storage medium | |
| Wang et al. | Phishing scams detection via temporal graph attention network in Ethereum | |
| Sampat et al. | Detection of phishing website using machine learning | |
| KR101464736B1 (en) | Security Assurance Management System and Web Page Monitoring Method | |
| CN112804192A (en) | Method, apparatus, electronic device, program, and medium for monitoring hidden network leakage | |
| Zaman et al. | Phishing website detection using effective classifiers and feature selection techniques | |
| CN113923037B (en) | Anomaly detection optimization device, method and system based on trusted computing | |
| Njoku et al. | URL Based Phishing Website Detection Using Machine Learning. | |
| US20210203691A1 (en) | Malware and phishing detection and mediation platform | |
| Fonseka et al. | Detecting Tabnabbing Attacks Via An RL-Based Agent | |
| Almazrouei et al. | The Internet of Things Network Penetration Testing Model Using Attack Graph Analysis | |
| Hong et al. | Client-Based Web Attacks Detection Using Artificial Intelligence | |
| Yamsani et al. | Estimate and prevention of malicious URL using logistic regression ML techniques | |
| CN116627466B (en) | Service path extraction method, system, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 100044 2nd floor, building 1, yard 26, Xizhimenwai South Road, Xicheng District, Beijing Applicant after: Qianxin Wangshen information technology (Beijing) Co.,Ltd. Applicant after: Qianxin Technology Group Co.,Ltd. Address before: 100044 2nd floor, building 1, yard 26, Xizhimenwai South Road, Xicheng District, Beijing Applicant before: LEGENDSEC INFORMATION TECHNOLOGY (BEIJING) Inc. Applicant before: Qianxin Technology Group Co.,Ltd. |
|
| CB02 | Change of applicant information |