CN113660264A - Data processing method, device, equipment and storage medium - Google Patents
Data processing method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN113660264A CN113660264A CN202110936917.2A CN202110936917A CN113660264A CN 113660264 A CN113660264 A CN 113660264A CN 202110936917 A CN202110936917 A CN 202110936917A CN 113660264 A CN113660264 A CN 113660264A
- Authority
- CN
- China
- Prior art keywords
- data
- dns
- filtering
- flink
- processing method
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/02—Network architectures or network communication protocols for network security for separating internal from external traffic, e.g. firewalls
- H04L63/0227—Filtering policies
- H04L63/0236—Filtering by address, protocol, port number or service, e.g. IP-address or URL
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L61/00—Network arrangements, protocols or services for addressing or naming
- H04L61/45—Network directories; Name-to-address mapping
- H04L61/4505—Network directories; Name-to-address mapping using standardised directories; using standardised directory access protocols
- H04L61/4511—Network directories; Name-to-address mapping using standardised directories; using standardised directory access protocols using domain name system [DNS]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/02—Network architectures or network communication protocols for network security for separating internal from external traffic, e.g. firewalls
- H04L63/0227—Filtering policies
- H04L63/0245—Filtering by information in the payload
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Hardware Design (AREA)
- Computer Security & Cryptography (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The application discloses a data processing method, a device, equipment and a storage medium, wherein the method comprises the following steps: collecting DNS flow data, and analyzing the DNS flow data to obtain DNS analysis data; performing data filtering on DNS resolution data by using kafka-flink; and mapping the analytic IP in the data after the flink filtering to a Bitmap for data re-filtering. According to the method, firstly, the data are subjected to deduplication processing by utilizing kafka-flink, then the Bitmap is introduced to perform deduplication processing again, the storage compression is high, the memory consumption is small, the multiple filtration of large-flow data is facilitated, the efficiency is high, and the query is fast.
Description
Technical Field
The present invention relates to the field of data processing, and in particular, to a data processing method, apparatus, device, and storage medium.
Background
With the explosive growth of digital information, data in a storage system is increased, and the storage management of the data is more complicated. The existing data deduplication mode generally adopts redis and hashet; when the reds is used for carrying out duplication elimination, the reds is used as a service to be deployed, and the problems of network communication, service downtime and the like exist when the reds is connected, so that the service performance is greatly influenced; the hashet has no problem in network communication, but for large data flow, tens of millions and hundreds of millions of data, the hash set has too low storage compression ratio and large memory consumption, and cannot meet the requirement of large data flow.
Therefore, how to solve the problem of deduplication and filtering of large-flow data is a technical problem that needs to be solved urgently by those skilled in the art.
Disclosure of Invention
In view of this, the present invention provides a data processing method, apparatus, device and storage medium, which have high storage compression and small memory consumption and are helpful for multiple filtering of large-flow data. The specific scheme is as follows:
a method of data processing, comprising:
collecting DNS flow data, and analyzing the DNS flow data to obtain DNS analysis data;
performing data filtering on the DNS resolution data by utilizing kafka-flink;
and mapping the analytic IP in the data after the flink filtering to a Bitmap for data re-filtering.
Preferably, in the data processing method provided in the embodiment of the present invention, the acquiring DNS traffic data and analyzing the DNS traffic data to obtain DNS analysis data includes:
accessing a flow data packet through a DNS acquisition server, acquiring DNS flow data, processing, converting and analyzing the DNS flow data, and analyzing basic parameter information including a domain name, a type, an analysis IP and time as DNS analysis data.
Preferably, in the data processing method provided in the embodiment of the present invention, after obtaining the DNS resolution data, before performing data filtering on the DNS resolution data by using kafka-flink, the method further includes:
and carrying out white list filtering on the DNS analysis data by utilizing the DNS acquisition server.
Preferably, in the above data processing method provided in the embodiment of the present invention, the performing data filtering on the DNS resolution data by using kafka-flink includes:
accessing the data filtered by the white list to topic of kafka;
and reading the data of kafka by using flink, and filtering the data according to a time window set by the flink.
Preferably, in the data processing method provided in the embodiment of the present invention, mapping an analytic IP in the data after the flink filtering to a Bitmap for data re-filtering includes:
mapping the analytic IP in the data after the flink filtering to a bit array of the Bitmap;
judging whether the bit array has current mapping data or not;
if yes, directly filtering out the DNS analysis data corresponding to the mapping data;
if not, establishing related mapping, and storing the DNS analysis data corresponding to the mapping data into a database.
Preferably, in the data processing method provided in the embodiment of the present invention, the method further includes:
clearing Bitmap data by a daily timing task;
the Bitmap data is updated for the same IP with a fixed time point of day as a boundary.
Preferably, in the data processing method provided in the embodiment of the present invention, while performing data re-filtering, the method further includes:
white list filtering is carried out on the data;
while the daily timing task clears Bitmap data, the method further comprises the following steps:
and accessing or updating the white list data.
An embodiment of the present invention further provides a data processing apparatus, including:
the system comprises a collecting and analyzing module, a DNS server and a DNS server, wherein the collecting and analyzing module is used for collecting DNS traffic data and analyzing the DNS traffic data to obtain DNS analysis data;
the data filtering module is used for performing data filtering on the DNS resolution data by utilizing kafka-flink;
and the data re-filtering module is used for mapping the analytic IP in the data after the flink filtering to the Bitmap for data re-filtering.
The embodiment of the present invention further provides a data processing device, which includes a processor and a memory, wherein the processor implements the above data processing method provided in the embodiment of the present invention when executing the computer program stored in the memory.
The embodiment of the present invention further provides a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the above data processing method provided in the embodiment of the present invention.
As can be seen from the foregoing technical solutions, a data processing method provided by the present invention includes: collecting DNS flow data, and analyzing the DNS flow data to obtain DNS analysis data; performing data filtering on DNS resolution data by using kafka-flink; and mapping the analytic IP in the data after the flink filtering to a Bitmap for data re-filtering.
According to the invention, firstly, the data is subjected to deduplication processing by utilizing kafka-flink, then the Bitmap is introduced to perform deduplication processing again, the storage compression is high, the memory consumption is small, the multiple filtration of large-flow data is facilitated, the efficiency is high, and the query is fast. In addition, the invention also provides a corresponding device, equipment and a computer readable storage medium aiming at the data processing method, so that the method has higher practicability, and the device, the equipment and the computer readable storage medium have corresponding advantages.
Drawings
In order to more clearly illustrate the embodiments of the present invention or technical solutions in related arts, the drawings used in the description of the embodiments or related arts will be briefly introduced below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
FIG. 1 is a flow chart of a data processing method according to an embodiment of the present invention;
fig. 2 is a schematic diagram of DNS traffic data collection and resolution according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of data filtering performed by kafka-flink according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of data re-filtering performed by a Bitmap according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The invention provides a data processing method, as shown in fig. 1, comprising the following steps:
s101, collecting DNS (Domain Name System) flow data, and analyzing the DNS flow data to obtain DNS analysis data;
in a specific implementation, step S101 acquires DNS traffic data, and analyzes the DNS traffic data to obtain DNS analysis data, which may specifically include, as shown in fig. 2: and accessing the flow data packet through the DNS acquisition server to acquire DNS flow data, processing, converting and analyzing the DNS flow data, and analyzing basic parameter information including a domain name, a type, an analysis IP and time to serve as DNS analysis data.
It should be noted that the DNS traffic is a DNS request packet acquired by the DNS collection server within a time node. The DNS collection server can analyze the DNS request to obtain the required basic parameter information such as IP (Internet protocol) and domain name. And the IP analysis is IP information acquired by analyzing the DNS request.
S102, performing data filtering on DNS analysis data by utilizing kafka-flink;
in specific implementation, the step S102 of performing data filtering on the DNS resolution data by using kafka-flink may specifically include: firstly, accessing white list filtered data to topic of kafka (message system based on publish/subscribe); then, the data of kafka is read by using flink, and data filtering is performed according to a time window set by the flink.
Specifically, as shown in fig. 3, before the white-list filtered data is accessed to topic of Kafka, Kafka configuration is performed: kafka configures a copy and a partition according to data flow and a server, and starts a zookeeper as a registration center; also flink configuration is performed: flink sets the time window and state backend configuration.
It should be noted that flink (big data stream computing engine) can perform data deduplication through a time window, but only can ensure that there is no duplicated data in the current time window, and the time window cannot be set for a long time, otherwise the performance of flink is extremely affected, data storage is performed directly through deduplication data of the time window, there are still a high proportion of duplicate requests for data of a single day, and the storage of these duplicate data into the database greatly affects the performance of the database. For DNS data, if real-time deduplication of single-day data is to be achieved, it is obvious that Flink cannot meet daily needs of people. Therefore, the invention introduces Bitmap to carry out the re-duplication removal processing on the data in the next step, and has the advantages of high storage compression ratio, quick query and capability of meeting the required technical requirements.
S103, mapping the analytic IP in the data after the flink filtering to a Bitmap for data re-filtering.
In practical application, a Bitmap is a data structure, according to the Bitmap principle, a bit array is used for recording two states of 0 and 1, then specific data are mapped to a specific position of the bit array, the bit is set to be 0 to indicate that the data do not exist, and set to be 1 to indicate that the data exist, and the algorithm has a high space utilization rate and is suitable for large-scale data, but the data states are not many.
By means of Bitmap mapping storage, for data with the data level of 10 hundred million/8/1024/1024-119 MB, storage consumption memory is small, single-day data can be filtered quickly, the data level is reduced sharply, and usability of the database can be guaranteed.
In specific implementation, step S103 maps the parsing IP in the data after the flink filtering to the Bitmap for data re-filtering, which may specifically include: mapping the analytic IP in the data after the flink filtering to a bit array of the Bitmap; judging whether the current mapping data exist in the bit array or not; if yes, directly filtering DNS analysis data corresponding to the mapping data; if not, establishing related mapping, and storing DNS analysis data corresponding to the mapping data into a database. That is, according to the Bitmap characteristics, the analytic IP is mapped into a bit array of the Bitmap, if mapping data exists, the analytic IP is judged to be repeated data, corresponding DNS analytic data are directly discarded, if mapping data does not exist, mapping is newly established, and corresponding DNS flow data are stored in a database.
In the data processing method provided by the embodiment of the invention, firstly, the data is subjected to deduplication processing by utilizing kafka-flink, and then the Bitmap is introduced to perform deduplication processing again, so that the storage compression is high, the memory consumption is small, the multiple filtration of large-flow data is facilitated, the efficiency is high, and the query is fast.
In addition, in a specific implementation, in the data processing method provided in the embodiment of the present invention, after obtaining the DNS resolution data, before performing data filtering on the DNS resolution data by using the kafka-flink, the method may further include: and the DNS analysis data is subjected to white list filtering by using the DNS acquisition server, so that the traffic magnitude can be reduced. The white list may be updated incrementally in the DNS collection server and may be overwritten in full. Specifically, as shown in fig. 2, DNS traffic data collected by the DNS server is resolved into required data, and data filtering is performed by setting a white list, and then the data after white list filtering is accessed to topic of kafka.
Further, in a specific implementation, in the data processing method provided in the embodiment of the present invention, the method may further include: removing Bitmap data by a daily timing task according to the requirement of single-day flow filtration of the data; and for the same IP, updating Bitmap data by taking a fixed time point every day (such as 24 points every day) as a boundary, and warehousing the data inquired again as new data. Therefore, single-day data information can be stored according to the Bitmap, single-day data can be rapidly filtered through data comparison, the data magnitude is sharply reduced, the full record of single-day IP analysis is achieved, single-day repeated data are saved in the database, and the use performance of the database is guaranteed.
Further, in a specific implementation, in the data processing method provided in the embodiment of the present invention, while performing data re-filtering, as shown in fig. 4, the method may further include: and white list filtering is carried out on the data. That is, the Bitmap filtering can also set a white list to further reduce the data magnitude. While the daily timing task clears the Bitmap data, the method further comprises the following steps: and accessing or updating the white list data. Therefore, the multi-level white list can be further preset, and the risk of saving the repeated data to the database is reduced.
Based on the same inventive concept, embodiments of the present invention further provide a data processing apparatus, and since the principle of the apparatus for solving the problem is similar to that of the foregoing data processing method, the implementation of the apparatus may refer to the implementation of the data processing method, and repeated details are not repeated.
In specific implementation, as shown in fig. 5, the data processing apparatus provided in the embodiment of the present invention specifically includes:
the acquisition and analysis module 11 is used for acquiring DNS traffic data and analyzing the DNS traffic data to obtain DNS analysis data;
the data filtering module 12 is configured to perform data filtering on the DNS resolution data by using kafka-flink;
and the data re-filtering module 13 is configured to map the parsing IP in the data after the flink filtering to the Bitmap for data re-filtering.
In the data processing device provided by the embodiment of the invention, data deduplication can be performed through the interaction of the three modules, the storage compression is high, the memory consumption is small, the multiple filtration of large-flow data is facilitated, the efficiency is high, and the query is fast.
In a specific implementation, in the data processing apparatus provided in the embodiment of the present invention, the acquisition and analysis module 11 may be a DNS acquisition server, and is specifically configured to access a traffic data packet, acquire DNS traffic data, perform processing, conversion and analysis on the DNS traffic data, and analyze basic parameter information including a domain name, a type, an analysis IP, and time as DNS analysis data. In addition, the collection and analysis module 11 may be further configured to perform white list filtering on the DNS analysis data.
In specific implementation, in the data processing apparatus provided in the embodiment of the present invention, the data filtering module 12 is specifically configured to access the data after being subjected to white list filtering to topic of kafka; and reading the data of kafka by using flink, and filtering the data according to a time window set by the flink.
In specific implementation, in the data processing apparatus provided in the embodiment of the present invention, the data re-filtering module 13 is specifically configured to map an analytic IP in the data after the flink filtering to a bit array of the Bitmap; judging whether the current mapping data exist in the bit array or not; if yes, directly filtering DNS analysis data corresponding to the mapping data; if not, establishing related mapping, and storing DNS analysis data corresponding to the mapping data into a database. In addition, the data re-filtering module 13 may be further configured to clear Bitmap data for a daily timing task; the Bitmap data is updated for the same IP with a fixed time point of day as a boundary.
For more specific working processes of the modules, reference may be made to corresponding contents disclosed in the foregoing embodiments, and details are not repeated here.
Correspondingly, the embodiment of the invention also discloses data processing equipment, which comprises a processor and a memory; wherein the processor implements the data processing method disclosed in the foregoing embodiments when executing the computer program stored in the memory.
For more specific processes of the above method, reference may be made to corresponding contents disclosed in the foregoing embodiments, and details are not repeated here.
Further, the present invention also discloses a computer readable storage medium for storing a computer program; the computer program, when executed by a processor, implements the data processing method disclosed previously.
For more specific processes of the above method, reference may be made to corresponding contents disclosed in the foregoing embodiments, and details are not repeated here.
The embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same or similar parts among the embodiments are referred to each other. The device, the equipment and the storage medium disclosed by the embodiment correspond to the method disclosed by the embodiment, so that the description is relatively simple, and the relevant points can be referred to the method part for description.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
To sum up, a data processing method provided by the embodiment of the present invention includes: collecting DNS flow data, and analyzing the DNS flow data to obtain DNS analysis data; performing data filtering on DNS resolution data by using kafka-flink; and mapping the analytic IP in the data after the flink filtering to a Bitmap for data re-filtering. According to the invention, firstly, the data is subjected to deduplication processing by utilizing kafka-flink, then the Bitmap is introduced to perform deduplication processing again, the storage compression is high, the memory consumption is small, the multiple filtration of large-flow data is facilitated, the efficiency is high, and the query is fast. In addition, the invention also provides a corresponding device, equipment and a computer readable storage medium aiming at the data processing method, so that the method has higher practicability, and the device, the equipment and the computer readable storage medium have corresponding advantages.
Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The data processing method, apparatus, device and storage medium provided by the present invention are described in detail above, and the principle and implementation of the present invention are explained herein by applying specific examples, and the description of the above examples is only used to help understanding the method and core ideas of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.
Claims (10)
1. A data processing method, comprising:
collecting DNS flow data, and analyzing the DNS flow data to obtain DNS analysis data;
performing data filtering on the DNS resolution data by utilizing kafka-flink;
and mapping the analytic IP in the data after the flink filtering to a Bitmap for data re-filtering.
2. The data processing method according to claim 1, wherein the acquiring DNS traffic data and analyzing the DNS traffic data to obtain DNS analysis data includes:
accessing a flow data packet through a DNS acquisition server, acquiring DNS flow data, processing, converting and analyzing the DNS flow data, and analyzing basic parameter information including a domain name, a type, an analysis IP and time as DNS analysis data.
3. The data processing method according to claim 2, wherein after the obtaining the DNS resolution data, before the performing data filtering on the DNS resolution data by using kafka-flink, the method further comprises:
and carrying out white list filtering on the DNS analysis data by utilizing the DNS acquisition server.
4. The data processing method according to claim 3, wherein the data filtering the DNS resolution data by using kafka-flink comprises:
accessing the data filtered by the white list to topic of kafka;
and reading the data of kafka by using flink, and filtering the data according to a time window set by the flink.
5. The data processing method of claim 4, wherein the mapping the parsing IP in the flink filtered data into a Bitmap for data re-filtering comprises:
mapping the analytic IP in the data after the flink filtering to a bit array of the Bitmap;
judging whether the bit array has current mapping data or not;
if yes, directly filtering out the DNS analysis data corresponding to the mapping data;
if not, establishing related mapping, and storing the DNS analysis data corresponding to the mapping data into a database.
6. The data processing method of claim 5, further comprising:
clearing Bitmap data by a daily timing task;
the Bitmap data is updated for the same IP with a fixed time point of day as a boundary.
7. The data processing method of claim 6, further comprising, while the performing data re-filtering:
white list filtering is carried out on the data;
while the daily timing task clears Bitmap data, the method further comprises the following steps:
and accessing or updating the white list data.
8. A data processing apparatus, comprising:
the system comprises a collecting and analyzing module, a DNS server and a DNS server, wherein the collecting and analyzing module is used for collecting DNS traffic data and analyzing the DNS traffic data to obtain DNS analysis data;
the data filtering module is used for performing data filtering on the DNS resolution data by utilizing kafka-flink;
and the data re-filtering module is used for mapping the analytic IP in the data after the flink filtering to the Bitmap for data re-filtering.
9. A data processing apparatus comprising a processor and a memory, wherein the processor implements the data processing method of any one of claims 1 to 7 when executing a computer program stored in the memory.
10. A computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the data processing method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110936917.2A CN113660264B (en) | 2021-08-16 | 2021-08-16 | Data processing method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110936917.2A CN113660264B (en) | 2021-08-16 | 2021-08-16 | Data processing method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113660264A true CN113660264A (en) | 2021-11-16 |
| CN113660264B CN113660264B (en) | 2023-04-18 |
Family
ID=78479235
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110936917.2A Active CN113660264B (en) | 2021-08-16 | 2021-08-16 | Data processing method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113660264B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117112302A (en) * | 2023-08-30 | 2023-11-24 | 广州经传多赢投资咨询有限公司 | Abnormal disaster recovery method, system, equipment and medium for financial data |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108121706A (en) * | 2016-11-28 | 2018-06-05 | 央视国际网络无锡有限公司 | A kind of optimization method of distributed reptile |
| CN110648185A (en) * | 2019-11-28 | 2020-01-03 | 苏宁云计算有限公司 | Target crowd circling method and device and computer equipment |
| CN111177137A (en) * | 2019-12-30 | 2020-05-19 | 广州酷狗计算机科技有限公司 | Data deduplication method, device, equipment and storage medium |
| CN113206797A (en) * | 2021-05-07 | 2021-08-03 | 上海微盟企业发展有限公司 | Flow control method and device, electronic equipment and storage medium |
-
2021
- 2021-08-16 CN CN202110936917.2A patent/CN113660264B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108121706A (en) * | 2016-11-28 | 2018-06-05 | 央视国际网络无锡有限公司 | A kind of optimization method of distributed reptile |
| CN110648185A (en) * | 2019-11-28 | 2020-01-03 | 苏宁云计算有限公司 | Target crowd circling method and device and computer equipment |
| CN111177137A (en) * | 2019-12-30 | 2020-05-19 | 广州酷狗计算机科技有限公司 | Data deduplication method, device, equipment and storage medium |
| CN113206797A (en) * | 2021-05-07 | 2021-08-03 | 上海微盟企业发展有限公司 | Flow control method and device, electronic equipment and storage medium |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117112302A (en) * | 2023-08-30 | 2023-11-24 | 广州经传多赢投资咨询有限公司 | Abnormal disaster recovery method, system, equipment and medium for financial data |
| CN117112302B (en) * | 2023-08-30 | 2024-03-12 | 广州经传多赢投资咨询有限公司 | Abnormal disaster recovery method, system, equipment and medium for financial data |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113660264B (en) | 2023-04-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10652265B2 (en) | Method and apparatus for network forensics compression and storage | |
| CN100555954C (en) | A kind of method and system that realize the audit of user's internet behavior | |
| CN101754253B (en) | General packet radio service (GPRS) end-to-end performance analysis method and system | |
| CN107579874B (en) | Method and device for detecting data collection missing report of flow collection equipment | |
| CN107528872A (en) | A kind of data reconstruction method, device and cloud storage system | |
| CN103957282B (en) | Terminal user's domain name mapping acceleration system and its method in a kind of domain | |
| CN104572976B (en) | Website data update method and system | |
| CN107087006B (en) | A kind of agreement shunt method, system and server | |
| CN106326280B (en) | Data processing method, device and system | |
| US20150188879A1 (en) | Apparatus for grouping servers, a method for grouping servers and a recording medium | |
| CN105791213A (en) | Strategy optimization device and method | |
| CN113660264B (en) | Data processing method, device, equipment and storage medium | |
| CN115333966B (en) | Topology-based Nginx log analysis method, system and equipment | |
| CN112632129A (en) | Code stream data management method, device and storage medium | |
| CN110661807A (en) | Automatic acquisition method and device for IPv6 address | |
| Gouel et al. | IP geolocation database stability and implications for network research | |
| CN113051460A (en) | Elasticissearch-based data retrieval method and system, electronic device and storage medium | |
| CN105554181A (en) | DNS log compression method and device | |
| CN103647774A (en) | Web content information filtering method based on cloud computing | |
| CN109510857A (en) | Date storage method, device, server and system | |
| CN108989484A (en) | A kind of compression and storage method and device of domain name system DNS log | |
| CN109302406B (en) | Distributed webpage evidence obtaining method and system | |
| CN113839952B (en) | Threat tracking method and device for log access relationship and electronic equipment | |
| CN111756869B (en) | DNS resolution server address collection method and system | |
| WO2023029066A1 (en) | Feature extraction method and apparatus for streaming data, and storage medium and computer device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |