CN111431821A - Method for rapidly detecting and identifying specific information in network large flow - Google Patents

Method for rapidly detecting and identifying specific information in network large flow Download PDF

Info

Publication number
CN111431821A
CN111431821A CN202010231603.8A CN202010231603A CN111431821A CN 111431821 A CN111431821 A CN 111431821A CN 202010231603 A CN202010231603 A CN 202010231603A CN 111431821 A CN111431821 A CN 111431821A
Authority
CN
China
Prior art keywords
module
information
data
receiving
network
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010231603.8A
Other languages
Chinese (zh)
Inventor
马旸
蔡冰
罗雅琼
尹魏昕
仲思超
王祥
赵云
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Jiangsu Branch Center National Computer Network And Information Security Management Center
Original Assignee
Jiangsu Branch Center National Computer Network And Information Security Management Center
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Jiangsu Branch Center National Computer Network And Information Security Management Center filed Critical Jiangsu Branch Center National Computer Network And Information Security Management Center
Priority to CN202010231603.8A priority Critical patent/CN111431821A/en
Publication of CN111431821A publication Critical patent/CN111431821A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00—Traffic control in data switching networks
    • H04L47/10—Flow control; Congestion control
    • H04L47/24—Traffic characterised by specific attributes, e.g. priority or QoS
    • H04L47/2441—Traffic characterised by specific attributes, e.g. priority or QoS relying on flow classification, e.g. using integrated services [IntServ]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00—Traffic control in data switching networks
    • H04L47/10—Flow control; Congestion control
    • H04L47/24—Traffic characterised by specific attributes, e.g. priority or QoS
    • H04L47/2483—Traffic characterised by specific attributes, e.g. priority or QoS involving identification of individual flows
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00—Traffic control in data switching networks
    • H04L47/50—Queue scheduling
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/02—Network architectures or network communication protocols for network security for separating internal from external traffic, e.g. firewalls
    • H04L63/0227—Filtering policies
    • H04L63/0236—Filtering by address, protocol, port number or service, e.g. IP-address or URL
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
    • H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
    • H04L63/1416—Event detection, e.g. attack signature detection
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
    • H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
    • H04L63/1425—Traffic logging, e.g. anomaly detection
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Computer Security & Cryptography (AREA)
  • Computer Hardware Design (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

The invention discloses a method for quickly detecting and identifying specific information in large flow of a network, which comprises a processor, wherein the processor is connected with a database module, an information identification module, an information storage module and a protection module, wherein the database module comprises a data acquisition module, a sending module, a receiving module and a central control module; the information identification module comprises an acquisition module and a screening module; the information storage module comprises a data conversion unit, a communication unit, a cloud storage and a feedback system, and the protection module comprises a wireless intrusion protection system and a firewall system. The invention can rapidly identify the specific information content from the large flow of the network by using the information fingerprint technology on the basis of classifying and identifying each application and protocol, and can greatly improve the accuracy of identifying and extracting the specific information of the internet; and can protect information security, avoid information to lose.

Description

Method for rapidly detecting and identifying specific information in network large flow
Technical Field
The invention relates to the technical field of network information processing, in particular to a method for quickly detecting and identifying specific information in network large flow.
Background
With the rapid development of the internet and the increase of WEB information, quasi-real-time data acquisition, protocol classification, application identification and feature extraction are carried out on massive internet traffic, and an information processing technology is developed as a sea fishing needle, wherein the information processing technology is that page information of a large number of websites on the network is collected locally and processed to establish an information database and an index database, so that various searches proposed by users are responded, and information or related pointers required by the users are provided. The user's search approach mainly includes free word full text search, keyword search, classified search and other specific information search. Therefore, a method for rapidly detecting and identifying specific information in large flow of a network is provided.
Disclosure of Invention
The invention aims to solve the problems in the background art and provides a method for quickly detecting and identifying specific information in network large flow.
In order to achieve the purpose, the invention adopts the following technical scheme: a method for rapidly detecting and identifying specific information in network large flow comprises a processor, wherein the processor is connected with a database module, an information identification module, an information storage module and a protection module, wherein the database module comprises a data acquisition module, a sending module, a receiving module and a central control module; the information identification module comprises an acquisition module and a screening module; the information storage module comprises a data conversion unit, a communication unit, a cloud storage and a feedback system, and the protection module comprises a wireless intrusion protection system and a firewall system module.
In the method for rapidly detecting and identifying the specific information in the network large flow, in the database module, the data acquisition module is used for acquiring data on the internet, preliminarily classifying the data, compressing the data of the same category and transmitting the data to the sending server; after the sending module receives the data sent by the data acquisition module, the received data are stored, and a sending queue list is established according to the data type and the receiving time; the receiving modules are grouped according to data types, the receiving module groups are numbered, the receiving module groups are classified according to the data, different receiving module groups only receive the data of the same type, and the received data are stored; the central control module collects the states of the receiving modules, establishes a receiving queue list according to the collected states, and stores the working states and the busy degree of each receiving module in the receiving queue list.
In the above method for rapidly detecting and identifying specific information in a network large flow, in the information identification module, the obtaining module is configured to obtain a network information set; the screening module is used for screening the release information matched with the specified information number in the network information set according to the specified information number.
In the method for rapidly detecting and identifying specific information in network mass flow, in the information storage module, the data conversion unit can compress the stored information into an electric signal and send a request signal to the communication unit, the communication unit sends the request signal to the cloud storage, the cloud storage receives the request signal, the cloud storage receives an uploading request and feeds back a signal for allowing information to be uploaded to the communication unit through the feedback system, and the communication unit starts to upload information after receiving the feedback signal, so that the effect of data storage is achieved.
In the method for quickly detecting and identifying the specific information in the network large flow, the database module is also provided with a data redundancy judgment module; the data redundancy judgment module is connected with the receiving module and the data acquisition module, redundancy judgment is carried out on data acquired by the data acquisition module, and if the data stored in the receiving module is the same as the data acquired by the data acquisition module, the same data is discarded.
In the method for quickly detecting and identifying the specific information in the network large flow, the central control module is further provided with an electric power control module, and the electric power control module controls the corresponding idle receiving server to be in a standby state according to the state of the receiving module stored in the receiving queue list and the data type of the sending queue list.
In the above method for rapidly detecting and identifying specific information in a large flow rate of a network, the method comprises the following steps:
s1, information acquisition and construction of the database: collecting IP addresses of different computers or intelligent equipment and an attribute data set of the computers or the intelligent equipment, converting all elements in the attribute data set of the computers or the intelligent equipment into a digital format after processing, and classifying and storing the attribute data set of the computers or the intelligent equipment in the digital format so as to construct a database;
s2, rapid detection and identification of information: inputting the acquired network information set into a database module to screen release information matched with the specified information number in the network information set;
s3, information storage: the data conversion unit can compress the stored information into an electric signal and send a request signal to the communication unit, the communication unit sends the request signal to the cloud storage, the cloud storage receives the request signal, the cloud storage receives the uploading request and feeds back a signal allowing information to be uploaded to the communication unit through the feedback system, the communication unit starts to upload the information after receiving the feedback signal and stores the data to prevent information loss;
s4, information protection: the protection module can protect network safety in real time, pre-warns the IP address detected by the firewall in real time by controlling the IP address configured on the network card, and performs log record backup on the pre-warning information in an intranet Web server.
Compared with the prior art, the method for rapidly detecting and identifying the specific information in the large flow of the network has the advantages that: by classifying data in the early stage, establishing a sending queue list and then circularly detecting the sending queue list, the information of the user has good identification, and meanwhile, the readability and the naturalness of the user information are not damaged or are hardly damaged, the information belonging to a specific user can be quickly found out from many kinds of similar information, so that the information of the specific user can be accurately found out by utilizing information fingerprints when a large amount of similar information is obtained, and the accuracy of identifying and extracting the specific information of the internet can be greatly improved; and the information safety is protected by arranging the protection module, so that the information loss is avoided.
Drawings
Fig. 1 is a block diagram of a method for rapidly detecting and identifying specific information in a large flow rate of a network according to the present invention.
Detailed Description
The following examples are for illustrative purposes only and are not intended to limit the scope of the present invention.
Examples
Referring to fig. 1, a method for rapidly detecting and identifying specific information in a large flow rate of a network includes a processor, the processor is connected with a database module, an information identification module, an information storage module and a protection module; the database module comprises a data acquisition module, a sending module, a receiving module and a central control module, wherein the data acquisition module is used for acquiring data on the Internet, preliminarily classifying the data, compressing the data of the same category and transmitting the data to the sending server; after the sending module receives the data sent by the data acquisition module, the received data is stored, and a sending queue list is established according to the data type and the receiving time; the receiving modules are grouped according to the data types, the receiving module groups are numbered, the receiving module groups are classified according to the data, different receiving module groups only receive the data of the same type, and the received data are stored; the central control module collects the state of the receiving module, establishes a receiving queue list according to the collected state, and stores the working state and the busy degree of each receiving module in the receiving queue list; wherein, the database module is also provided with a data redundancy judgment module; the data redundancy judgment module is connected with the receiving module and the data acquisition module, redundancy judgment is carried out on the data acquired by the data acquisition module, and if the data stored in the receiving module is the same as the data acquired by the data acquisition module, the same data is discarded; the central control module is also internally provided with a power control module which controls the corresponding idle receiving server to be in a standby state according to the state of the receiving module stored in the receiving queue list and the data type of the sending queue list;
the information identification module comprises an acquisition module and a screening module, wherein the acquisition module is used for acquiring a network information set; the screening module is used for screening the release information matched with the specified information number in the network information set according to the specified information number;
the information storage module comprises a data conversion unit, a communication unit, a cloud storage and a feedback system, the data conversion unit can compress stored information into electric signals and send request signals to the communication unit, the communication unit sends the request signals to the cloud storage, the cloud storage receives the request signals, the cloud storage receives uploading requests and feeds back signals allowing information to be uploaded to the communication unit through the feedback system, the communication unit starts to upload the information after receiving the feedback signals, and the effect of data storage is achieved.
The invention discloses a method for quickly detecting and identifying specific information in network large flow, which comprises the following steps:
s1, information acquisition and construction of the database: collecting IP addresses of different computers or intelligent equipment and an attribute data set of the computers or the intelligent equipment, converting all elements in the attribute data set of the computers or the intelligent equipment into a digital format after processing, and classifying and storing the attribute data set of the computers or the intelligent equipment in the digital format so as to construct a database;
s2, rapid detection and identification of information: inputting the acquired network information set into a database module to screen release information matched with the specified information number in the network information set;
s3, information storage: the data conversion unit can compress the stored information into an electric signal and send a request signal to the communication unit, the communication unit sends the request signal to the cloud storage, the cloud storage receives the request signal, the cloud storage receives the uploading request and feeds back a signal allowing information to be uploaded to the communication unit through the feedback system, the communication unit starts to upload the information after receiving the feedback signal and stores the data to prevent information loss;
s4, information protection: the protection module can protect network safety in real time, pre-warns the IP address detected by the firewall in real time by controlling the IP address configured on the network card, and performs log record backup on the pre-warning information in an intranet Web server.
The present invention is not limited to the above preferred embodiments, and any modifications, equivalent replacements, improvements, etc. within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (7)

1. A method for rapidly detecting and identifying specific information in network large flow comprises a processor, and is characterized in that the processor is connected with a database module, an information identification module, an information storage module and a protection module, wherein the database module comprises a data acquisition module, a sending module, a receiving module and a central control module; the information identification module comprises an acquisition module and a screening module; the information storage module comprises a data conversion unit, a communication unit, a cloud storage and a feedback system, and the protection module comprises a wireless intrusion protection system and a firewall system.
2. The method for rapidly detecting and identifying the specific information in the large flow rate of the network according to claim 1, wherein in the database module, the data acquisition module is used for acquiring data on the internet, preliminarily classifying the data, compressing the data of the same category and transmitting the data to the sending server; after the sending module receives the data sent by the data acquisition module, the received data are stored, and a sending queue list is established according to the data type and the receiving time; the receiving modules are grouped according to data types, the receiving module groups are numbered, the receiving module groups are classified according to the data, different receiving module groups only receive the data of the same type, and the received data are stored; the central control module collects the states of the receiving modules, establishes a receiving queue list according to the collected states, and stores the working states and the busy degree of each receiving module in the receiving queue list.
3. The method for rapidly detecting and identifying specific information in large traffic volume of network according to claim 1, wherein in the information identification module, the obtaining module is used for obtaining network information set; the screening module is used for screening the release information matched with the specified information number in the network information set according to the specified information number.
4. The method according to claim 1, wherein in the information storage module, the data conversion unit compresses the stored information into an electrical signal and sends a request signal to the communication unit, the communication unit sends the request signal to a cloud storage, the cloud storage receives the request signal, the cloud storage receives an upload request and feeds back a signal allowing information to be uploaded to the communication unit through a feedback system, and the communication unit starts uploading the information after receiving the feedback signal, so that the effect of data storage is achieved.
5. The method for rapidly detecting and identifying the specific information in the large flow rate of the network according to claim 1, wherein a data redundancy judgment module is further arranged in the database module; the data redundancy judgment module is connected with the receiving module and the data acquisition module, redundancy judgment is carried out on data acquired by the data acquisition module, and if the data stored in the receiving module is the same as the data acquired by the data acquisition module, the same data is discarded.
6. The method according to claim 1, wherein a power control module is further disposed in the central control module, and the power control module controls the idle receiving servers to be in a standby state according to the state of the receiving module stored in the receiving queue table and the data type of the sending queue table.
7. The method for rapidly detecting and identifying the specific information in the large flow of the network according to claim 1, comprising the following steps:
s1, information acquisition and construction of the database: collecting IP addresses of different computers or intelligent equipment and an attribute data set of the computers or the intelligent equipment, converting all elements in the attribute data set of the computers or the intelligent equipment into a digital format after processing, and classifying and storing the attribute data set of the computers or the intelligent equipment in the digital format so as to construct a database;
s2, rapid detection and identification of information: inputting the acquired network information set into a database module to screen release information matched with the specified information number in the network information set;
s3, information storage: the data conversion unit can compress the stored information into an electric signal and send a request signal to the communication unit, the communication unit sends the request signal to the cloud storage, the cloud storage receives the request signal, the cloud storage receives the uploading request and feeds back a signal allowing information to be uploaded to the communication unit through the feedback system, the communication unit starts to upload the information after receiving the feedback signal and stores the data to prevent information loss;
s4, information protection: the protection module can protect network safety in real time, pre-warns the IP address detected by the firewall in real time by controlling the IP address configured on the network card, and performs log record backup on the pre-warning information in an intranet Web server.
CN202010231603.8A 2020-03-27 2020-03-27 Method for rapidly detecting and identifying specific information in network large flow Pending CN111431821A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010231603.8A CN111431821A (en) 2020-03-27 2020-03-27 Method for rapidly detecting and identifying specific information in network large flow

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010231603.8A CN111431821A (en) 2020-03-27 2020-03-27 Method for rapidly detecting and identifying specific information in network large flow

Publications (1)

Publication Number Publication Date
CN111431821A true CN111431821A (en) 2020-07-17

Family

ID=71549507

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010231603.8A Pending CN111431821A (en) 2020-03-27 2020-03-27 Method for rapidly detecting and identifying specific information in network large flow

Country Status (1)

Country Link
CN (1) CN111431821A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113835877A (en) * 2021-08-19 2021-12-24 重庆恩谷信息科技有限公司 Remote data information storage system based on big data
CN114745609A (en) * 2022-03-12 2022-07-12 广东绿建联能源环境科技有限公司 Energy consumption monitoring system

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113835877A (en) * 2021-08-19 2021-12-24 重庆恩谷信息科技有限公司 Remote data information storage system based on big data
CN114745609A (en) * 2022-03-12 2022-07-12 广东绿建联能源环境科技有限公司 Energy consumption monitoring system
CN114745609B (en) * 2022-03-12 2023-01-24 广东绿建联能源环境科技有限公司 Energy consumption monitoring system

Similar Documents

Publication Publication Date Title
CN109033387B (en) Internet of things searching system and method fusing multi-source data and storage medium
CN102750326A (en) Log management optimization method of cluster system based on downsizing strategy
CN108769255A (en) The acquisition of business data and administering method
CN101304426A (en) Method and device for recognizing and reporting questionable document
CN110198303A (en) Threaten the generation method and device, storage medium, electronic device of information
CN111431821A (en) Method for rapidly detecting and identifying specific information in network large flow
CN104462096B (en) Public sentiment method for monitoring and analyzing and device
CN109254957A (en) A kind of archive management system based on big data
CN115103157A (en) Video analysis method and device based on edge cloud cooperation, electronic equipment and medium
CN117312098B (en) Log abnormity alarm method and device
CN117130870B (en) Transparent request tracking and sampling method and device for Java architecture micro-service system
CN110909380B (en) Abnormal file access behavior monitoring method and device
CN106533728A (en) Server information collecting method and apparatus
CN116192607A (en) Fault alarm method and device
CN112506886B (en) Multi-source service operation log acquisition method and system
CN114338346A (en) Alarm message processing method and device and electronic equipment
CN101510211A (en) Multimedia data processing system and method
CN112487082B (en) Biological feature recognition method and related equipment
CN103544476A (en) Flow point face recognition monitoring method, flow point face recognition monitoring system and mobile terminal
CN113204529A (en) Music score collecting and sharing system based on Internet
CN112118265A (en) User information data collection authentication system based on Internet of things
CN210804423U (en) Website information acquisition and release platform system
CN104980750A (en) Collection method, device and system for video transcoding logs
CN116166472B (en) Data recovery method and system for stored data
CN117909295A (en) Data processing method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
WD01 Invention patent application deemed withdrawn after publication
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20200717