CN110955900A - Vulnerability detection method for big data platform - Google Patents
Vulnerability detection method for big data platform Download PDFInfo
- Publication number
- CN110955900A CN110955900A CN201911302348.5A CN201911302348A CN110955900A CN 110955900 A CN110955900 A CN 110955900A CN 201911302348 A CN201911302348 A CN 201911302348A CN 110955900 A CN110955900 A CN 110955900A
- Authority
- CN
- China
- Prior art keywords
- big data
- vulnerability
- data platform
- detection method
- vulnerability detection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/50—Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems
- G06F21/57—Certifying or maintaining trusted computer platforms, e.g. secure boots or power-downs, version controls, system software checks, secure updates or assessing vulnerabilities
- G06F21/577—Assessing vulnerabilities and evaluating computer system security
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2221/00—Indexing scheme relating to security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F2221/03—Indexing scheme relating to G06F21/50, monitoring users, programs or devices to maintain the integrity of platforms
- G06F2221/034—Test or assess a computer or a system
Abstract
The invention discloses a vulnerability detection method for a big data platform, which comprises the following specific steps: firstly, preparing a basic environment, and then deploying debugging in the basic environment; the invention carries out automatic node discovery and the issuing of an unattended scanning task on a big data platform through the tool, and the problem existing in the final report of the scanning result, and pushes the scanning report to related personnel in a mail mode, and the invention can improve the safety of the big data platform: deep baseline and vulnerability detection scanning are carried out on the big data/cloud platform, professional rectification and improvement protection suggestions can be provided according to safety problems of enterprises, safety and compliance of components are guaranteed, and accordingly safety of the big data platform is improved.
Description
Technical Field
The invention relates to the technical field of data processing, in particular to a vulnerability detection method for a big data platform.
Background
A big data platform in an enterprise needs to perform distributed storage and calculation on mass data, the number of related nodes is large, how to protect bottom security vulnerabilities and configuration compliance become the most important factor of current security work, and the current vulnerability checking method mainly comprises the following steps: and automatically identifying node services, collecting information, performing vulnerability scanning and baseline inspection, and if the services are different from the standard, determining that hidden troubles exist in configuration.
With the wide application of big data technology, based on the reasons in the aspects of trusted network and trusted personnel management, the security risks of identity authentication, authority control, data encryption transmission, integrity check of cluster node components and the like of multiple tenants are faced; because the big data platform is built by more clusters, the accuracy and efficiency of manual inspection cannot meet the rapid discovery requirement of the service, and the problems of errors, false detection and the like are easily caused.
Disclosure of Invention
The present invention is directed to a vulnerability detection method for a big data platform, so as to solve the problems mentioned in the background art.
In order to achieve the purpose, the invention provides the following technical scheme:
a vulnerability detection method for a big data platform comprises the following specific steps: firstly, preparing a basic environment, and then deploying debugging in the basic environment; the automatic node discovery and the issuing of the unattended scanning task are carried out on the big data platform through the tool, the problem existing in the final report of the scanning result is solved, and the scanning report is pushed to related personnel through a mail mode.
As a further scheme of the invention: the base environment is a CentOS or Redhat operating system.
As a further scheme of the invention: the network of deployment servers needs to be reachable with large data platforms.
As a further scheme of the invention: vulnerability checking of the big data platform comprises vulnerability information acquisition and big data baseline scanning.
As a further scheme of the invention: and the vulnerability information acquisition automatically carries out periodic synchronization on the vulnerability information related to the big data by taking the CVE and the CNVD as standard vulnerability libraries. And forming a vulnerability knowledge base by taking the synchronous vulnerability information as a reference.
As a further scheme of the invention: the big data baseline scanning mainly comprises the steps of reading the safety requirements of a network security method on big data, forming an inspection standard, automatically collecting big data cluster information, managing the distribution condition of nodes and data nodes, the service distribution condition of each node and the core configuration file information of a big data assembly, and comparing the collected big data cluster information with a standard library.
As a further scheme of the invention: and the vulnerability information acquisition also carries out information acquisition script issuing on the host through the infrastructure automation, and carries out intelligent analysis with the vulnerability library through the acquired information.
As a further scheme of the invention: the analysis engine adopts a machine learning algorithm to improve the accuracy of the algorithm.
Compared with the prior art, the invention has the beneficial effects that:
1. automatically identifying the big data type and the component category: the method has the automatic capability of operating the components on the big data platform, finds the type and the version of the big data platform, identifies each node and the operated components in the cluster, provides node probing, and eliminates abnormal nodes.
2. Automatic acquisition of component configuration information data: according to the components operated by the existing nodes, the installation path of the components is automatically found, and safety related configuration information is automatically acquired, so that the accuracy and the execution efficiency of information acquisition are improved.
3. Automatic detection of baseline configuration: and performing basic configuration detection on various components of the big data platform, and enhancing the safety compliance of the components of the big data platform through the automatic acquisition result and the inspection item.
4. The security of the big data platform is improved: deep baseline and vulnerability detection scanning are carried out on the big data/cloud platform, professional rectification and improvement protection suggestions can be provided according to safety problems of enterprises, safety and compliance of components are guaranteed, and accordingly safety of the big data platform is improved.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In the embodiment of the invention, a vulnerability detection method for a big data platform comprises the following specific steps: firstly, preparing basic environments such as operating systems (CentOS, Redhat) and the like, and then deploying debugging in the operating systems; the network of the deployment server needs to be reachable with a big data platform, automatic node discovery and the issuing of an unattended scanning task are carried out on the big data platform through the tool, the problem existing in the final report of a scanning result is solved, the scanning report is pushed to related personnel through a mail mode, the deployment is carried out in a software mode, industrial personal computer equipment and a physical bypass are not needed to be deployed, the operation process cannot affect big data services, and the overall performance can be improved through transverse expansion. Independent of big data and cloud computing platform design.
Example 2: on the basis of the embodiment 1, the vulnerability check of the big data platform mainly comprises 2 parts:
1. and the vulnerability information acquisition automatically carries out periodic synchronization on the vulnerability information related to the big data by taking the CVE and the CNVD as standard vulnerability libraries. And forming a vulnerability knowledge base by taking the synchronous vulnerability information as a reference. Issuing an information acquisition script to the host computer through the infrastructure automation, and carrying out intelligent analysis on the acquired information and the leak library; the accuracy of a training algorithm is continuously improved by adopting a high-efficiency and stable scanning engine and machine learning in an analysis means;
2. the big data baseline scanning is mainly characterized in that a check standard is formed by reading the safety requirements of the big data in national specifications such as a network security method and a guarantee 2.0, information such as big data cluster information, management nodes, data node distribution conditions, service distribution conditions of all nodes, core configuration files of big data components and the like is automatically acquired, and the acquired information is compared with a standard library. The analysis engine adopts algorithms such as machine learning and the like, and the accuracy of the algorithms is continuously improved.
The invention has the beneficial effects that:
1. automatically identifying the big data type and the component category: the method has the automatic capability of operating the components on the big data platform, finds the type and the version of the big data platform, identifies each node and the operated components in the cluster, provides node probing, and eliminates abnormal nodes.
2. Automatic acquisition of component configuration information data: according to the components operated by the existing nodes, the installation path of the components is automatically found, and safety related configuration information is automatically acquired, so that the accuracy and the execution efficiency of information acquisition are improved.
3. Automatic detection of baseline configuration: and performing basic configuration detection on various components of the big data platform, and enhancing the safety compliance of the components of the big data platform through the automatic acquisition result and the inspection item.
4. The security of the big data platform is improved: deep baseline and vulnerability detection scanning are carried out on the big data/cloud platform, professional rectification and improvement protection suggestions can be provided according to safety problems of enterprises, safety and compliance of components are guaranteed, and accordingly safety of the big data platform is improved.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein.
Furthermore, it should be understood that although the present description refers to embodiments, not every embodiment may contain only a single embodiment, and such description is for clarity only, and those skilled in the art should integrate the description, and the embodiments may be combined as appropriate to form other embodiments understood by those skilled in the art.
Claims (8)
1. A vulnerability detection method for a big data platform is characterized by comprising the following specific steps: firstly, preparing a basic environment, and then deploying debugging in the basic environment; the automatic node discovery and the issuing of the unattended scanning task are carried out on the big data platform through the tool, the problem existing in the final report of the scanning result is solved, and the scanning report is pushed to related personnel through a mail mode.
2. The vulnerability detection method for big data platforms according to claim 1, characterized in that the base environment is CentOS or Redhat operating system.
3. The vulnerability detection method for big data platform of claim 2, characterized in that the network of deployment servers needs to be reachable with big data platform.
4. The vulnerability detection method for big data platform of claim 2, wherein the vulnerability examination of big data platform comprises vulnerability information acquisition and big data baseline scan.
5. The vulnerability detection method for big data platform of claim 4, wherein the vulnerability information acquisition automatically performs periodic synchronization to big data related vulnerability information by using CVE, CNVD as standard vulnerability library,
and forming a vulnerability knowledge base by taking the synchronous vulnerability information as a reference.
6. The vulnerability detection method for big data platforms according to claim 4, characterized in that the big data baseline scan mainly reads the security requirements of the network security method on big data to form inspection standards, automatically collects big data cluster information, manages the distribution of nodes and data nodes, the service distribution of each node, and the core configuration file information of big data components, and compares the collected information with the standard library.
7. The vulnerability detection method for big data platforms according to claim 4, wherein the vulnerability information acquisition further issues an information acquisition script to the host through an alarm automation, and performs intelligent analysis with the vulnerability database through the acquired information.
8. The vulnerability detection method for big data platforms according to claim 6, characterized in that the analysis engine adopts machine learning algorithm to improve the accuracy of the algorithm.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911302348.5A CN110955900A (en) | 2019-12-17 | 2019-12-17 | Vulnerability detection method for big data platform |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911302348.5A CN110955900A (en) | 2019-12-17 | 2019-12-17 | Vulnerability detection method for big data platform |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110955900A true CN110955900A (en) | 2020-04-03 |
Family
ID=69982201
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911302348.5A Pending CN110955900A (en) | 2019-12-17 | 2019-12-17 | Vulnerability detection method for big data platform |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110955900A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111611592A (en) * | 2020-05-27 | 2020-09-01 | 中国信息安全测评中心 | Big data platform security assessment method and device |
| CN115550306A (en) * | 2021-11-10 | 2022-12-30 | 苏州蓝驰网络科技有限公司 | Cloud intelligent mail system |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130125245A1 (en) * | 2011-11-15 | 2013-05-16 | Micron Technology, Inc. | Apparatuses, integrated circuits, and methods for testmode security systems |
| CN108876152A (en) * | 2018-06-21 | 2018-11-23 | 王飞 | A kind of big data security baseline inspection method |
| CN109446817A (en) * | 2018-10-29 | 2019-03-08 | 成都思维世纪科技有限责任公司 | A kind of detection of big data and auditing system |
-
2019
- 2019-12-17 CN CN201911302348.5A patent/CN110955900A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130125245A1 (en) * | 2011-11-15 | 2013-05-16 | Micron Technology, Inc. | Apparatuses, integrated circuits, and methods for testmode security systems |
| CN108876152A (en) * | 2018-06-21 | 2018-11-23 | 王飞 | A kind of big data security baseline inspection method |
| CN109446817A (en) * | 2018-10-29 | 2019-03-08 | 成都思维世纪科技有限责任公司 | A kind of detection of big data and auditing system |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111611592A (en) * | 2020-05-27 | 2020-09-01 | 中国信息安全测评中心 | Big data platform security assessment method and device |
| CN115550306A (en) * | 2021-11-10 | 2022-12-30 | 苏州蓝驰网络科技有限公司 | Cloud intelligent mail system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112102111B (en) | Intelligent processing system for power plant data | |
| KR102017756B1 (en) | Apparatus and method for detecting abnormal behavior | |
| CN109583711B (en) | Safety risk assessment overall process management system | |
| CN111881452B (en) | Safety test system for industrial control equipment and working method thereof | |
| US10140453B1 (en) | Vulnerability management using taxonomy-based normalization | |
| CN114095273A (en) | Deep learning-based internet vulnerability mining method and big data mining system | |
| CN106874159A (en) | A kind of concentrating type automated testing method | |
| CN110088744B (en) | Database maintenance method and system | |
| CN110955900A (en) | Vulnerability detection method for big data platform | |
| CN110971464A (en) | Operation and maintenance automatic system suitable for disaster recovery center | |
| Zhang et al. | A survey on quality assurance techniques for big data applications | |
| CN105260286A (en) | Method for monitoring CPU working state in real time | |
| CN114329498A (en) | Data center operation and maintenance safety management and control method and device | |
| CN113965355B (en) | Illegal IP (Internet protocol) intra-provincial network plugging method and device based on SOC (system on chip) | |
| KR101741108B1 (en) | Apparatus and method for analyzing system fault detection | |
| CN117240594A (en) | Multi-dimensional network security operation and maintenance protection management system and method | |
| CN115618353B (en) | Industrial production safety identification system and method | |
| CN110888949A (en) | Equipment alarm shielding method, device, equipment and medium based on three-dimensional map | |
| CN108616383A (en) | A kind of network and the security process of information manage system | |
| Najafian et al. | Signature-based method and stream data mining technique performance evaluation for security and intrusion detection in advanced metering infrastructures (ami) | |
| CN114116904A (en) | Asset account chain storage system and method for information security | |
| CN113364592A (en) | Engineering system file management system and method based on credit value union chain | |
| CN114268460B (en) | Network security anomaly detection method and device, storage medium and computing equipment | |
| CN110321130A (en) | The not reproducible compiling localization method of log is called based on system | |
| CN111932706B (en) | Informationized inspection method and device, storage medium and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200403 |
|
| RJ01 | Rejection of invention patent application after publication |