CN117251863A - Log data protection method, device, equipment and storage medium - Google Patents
Log data protection method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN117251863A CN117251863A CN202311076171.8A CN202311076171A CN117251863A CN 117251863 A CN117251863 A CN 117251863A CN 202311076171 A CN202311076171 A CN 202311076171A CN 117251863 A CN117251863 A CN 117251863A
- Authority
- CN
- China
- Prior art keywords
- log data
- storage system
- firmware
- log
- acquiring
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 57
- 238000004458 analytical method Methods 0.000 claims abstract description 43
- 238000007726 management method Methods 0.000 claims description 31
- 238000005192 partition Methods 0.000 claims description 16
- 238000004891 communication Methods 0.000 claims description 13
- 238000012423 maintenance Methods 0.000 claims description 12
- 238000012360 testing method Methods 0.000 claims description 10
- 238000012545 processing Methods 0.000 claims description 8
- 230000004044 response Effects 0.000 claims description 5
- 238000011835 investigation Methods 0.000 abstract description 4
- 238000005516 engineering process Methods 0.000 description 7
- 230000008569 process Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 4
- 238000012544 monitoring process Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 238000013475 authorization Methods 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000000802 evaporation-induced self-assembly Methods 0.000 description 1
- 230000014759 maintenance of location Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000005693 optoelectronics Effects 0.000 description 1
- 230000000630 rising effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/602—Providing cryptographic facilities or services
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3466—Performance evaluation by tracing or monitoring
- G06F11/3476—Data logging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3466—Performance evaluation by tracing or monitoring
- G06F11/3485—Performance evaluation by tracing or monitoring for I/O devices
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
- G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
Abstract
The embodiment of the invention provides a log data protection method, a device, equipment and a storage medium, wherein the method comprises the following steps: by retrieving multiple types of log data from multiple tiers of a storage system; storing the log data of various types into a designated directory according to a storage strategy; determining fault information corresponding to the log data under the appointed directory through an intelligent analysis engine; determining safety protection information corresponding to log data under a specified directory through a safety protection engine; and determining protection strategies for various types of log data through the fault information and the safety protection information. The embodiment of the invention solves the problems of difficult management and lack of protection of the log of the storage system, greatly improves the log collection and management capacity of the storage system, ensures the safety of log data, and is convenient for better log analysis and fault investigation under various complex business scenes.
Description
Technical Field
The present invention relates to the field of data storage technologies, and in particular, to a method, an apparatus, a device, and a storage medium for protecting log data.
Background
Along with the development of technologies such as cloud computing, internet of things and mobile computing and the continuous expansion of application fields, the leading effect of modern information technology in social production and life is more and more prominent, and the supporting function of information technology means in various fields of society is more and more important. Meanwhile, whether the information provider or the visitor has higher and higher security requirements and importance, the requirements of people on the information security comprise five major elements of confidentiality, integrity, reliability, usability and non-repudiation. Of these five major elements, non-repudiation is achieved mainly by means of log data.
The data of the business system is a core asset of the data center, the business data contains a plurality of sensitive information, and after a hacker obtains the sensitive information, the hacker can easily obtain benefits, so that the business data is focused by an external hacker, and in recent years, the illegal acquisition of the business data is in the rising trend year by year; in addition, after a hacker obtains the data, evidence is destroyed, namely log data on the storage system is deleted or destroyed, which leads to the data being destroyed and even causes serious economic loss.
At present, service data is mainly stored on a storage system, the storage system becomes a core infrastructure of a current data center, so that management and protection of log data of the storage system are increasingly important, and various log data on the storage system include host side service access log data, operation log data generated by configuration management of storage system management software, operation log data generated by an operating system where the storage system is located, log data generated by software/firmware update and upgrade of the storage system, log data generated during various hardware operations and faults of the storage system, data IO log data, metadata IO log data, state log data generated by operation of the storage system and the like. Since various log data of the storage system are respectively generated by hardware, firmware, an operating system and software, and the generated time and module are different, the management and protection of the log data of the storage system are quite challenging, and no mature scheme exists in the industry at present.
Disclosure of Invention
The embodiment of the invention aims to provide a log data protection method, device, equipment and storage medium, which solve the problems that various log data of the existing storage system are respectively generated by hardware, firmware, an operating system and software, and the generated time and modules are different, so that the management and protection of the log data of the storage system are quite challenging, and no mature scheme exists in the industry at present, and the specific technical scheme is as follows:
In a first aspect of the present invention, there is provided a log data protection method, which is characterized in that the method is applied to a log management center, and the method includes:
acquiring log data of multiple types from multiple levels of a storage system;
storing the log data of multiple types into a designated directory according to a storage strategy;
determining fault information corresponding to the log data under the appointed directory through an intelligent analysis engine, wherein the intelligent analysis engine is pre-deployed in the log data management center;
determining safety protection information corresponding to the log data under the appointed directory through a safety protection engine, wherein the safety protection engine is deployed in the log data management center in advance;
and determining protection strategies for various types of log data through the fault information and the safety protection information.
Optionally, before the log data of multiple types is stored in a designated directory according to a storage policy, the method further includes:
dividing a plurality of disk partitions from the log data management center;
generating a unique corresponding specified directory for each disk partition;
and allocating a storage strategy for each specified directory, wherein the storage strategy is used for determining the specified directory to be stored of the log data according to the source and the purpose of the log data.
Optionally, the plurality of types of log data include: hardware running log data, medium running log data, firmware running log data, operating system running log data, software dump log data, software state log data, configuration management log data, upgrade/update/maintenance log data, service access log data, data input/output IO log data and metadata IO log data;
the obtaining log data of multiple types from multiple tiers of the storage system includes:
acquiring the hardware running log data from a plurality of pieces of hardware of a storage system;
acquiring the medium operation log data from various mediums of a storage system;
acquiring the firmware operation log data from a plurality of firmware of a storage system;
acquiring operation log data of an operation system from the operation system of a storage system;
acquiring the software dump log data from software running in a central processing unit of a storage system;
acquiring the software state log data from a plurality of software of a storage system;
under the condition that the storage system is detected to have configuration management operation, acquiring relevant configuration management log data;
Acquiring relevant upgrade/update/maintenance log data under the condition that upgrade and/or update and/or maintenance operations of a storage system are detected;
under the condition that the service program of the front-end host is detected to be accessed into a storage system, acquiring relevant service access log data;
under the condition that the existence of data IO in the storage system is detected, acquiring related data input/output IO log data;
and acquiring the metadata IO log data from a storage system.
Optionally, the firmware includes: the self-grinding firmware and the third-party firmware, and the firmware operation log data comprises: self-grinding firmware log data, third-party firmware log data and firmware dump log data;
the obtaining the firmware operation log data from a plurality of firmware of a storage system includes:
receiving self-grinding firmware log data uploaded by self-grinding firmware in a storage system according to a stipulated mode;
collecting third-party firmware log data about third-party firmware in a storage system;
and acquiring the firmware dump log data when the existence of the firmware dump log data in the plurality of firmware is detected.
Optionally, the acquiring the hardware running log data from a plurality of hardware of the storage system includes:
Transmitting a test data packet according to a preset period;
and acquiring the hardware operation log data through a response result of the test data packet within a target time range.
Optionally, the determining, by the intelligent analysis engine, fault information corresponding to the log data under the specified directory includes:
determining source information of log data through the specified directory;
determining fault type information corresponding to target field information of the log data under the appointed directory through an intelligent analysis engine;
and determining fault information corresponding to the log data under the appointed directory through the source information and the fault type information.
Optionally, the determining a protection policy for the log data of multiple types through the fault information and the safety protection information includes:
sending an alarm prompt through the fault information;
and performing security control on access control, storage validity period, export and forwarding of log data through the security protection information.
In a second aspect of the present invention, there is also provided a log data protection device, which is applied to a log management center, including:
a first acquisition module for acquiring log data of a plurality of types from a plurality of tiers of the storage system;
The storage module is used for storing the log data of various types into a designated directory according to a storage strategy;
the first determining module is used for determining fault information corresponding to the log data under the appointed directory through an intelligent analysis engine, and the intelligent analysis engine is pre-deployed in the log data management center;
the second determining module is used for determining safety protection information corresponding to the log data under the appointed directory through a safety protection engine, and the safety protection engine is deployed in the log data management center in advance;
and the third determining module is used for determining protection strategies for the log data of multiple types through the fault information and the safety protection information.
In a third aspect of the present invention, there is also provided a communication device comprising: a transceiver, a memory, a processor, and a program stored on the memory and executable on the processor;
the processor is used for reading the program in the memory to realize and execute the log data protection method.
In a fourth aspect of the present invention, there is also provided a computer readable storage medium having instructions stored therein which, when run on a computer, cause the computer to perform any of the above-described log data protection methods.
The log data protection method provided by the embodiment of the invention is applied to a log management center, and various types of log data are obtained from a plurality of levels of a storage system; storing the log data of various types into a designated directory according to a storage strategy; determining fault information corresponding to the log data under the appointed directory through an intelligent analysis engine, wherein the intelligent analysis engine is deployed in a log data management center in advance; determining safety protection information corresponding to the log data under the appointed directory through a safety protection engine, wherein the safety protection engine is deployed in a log data management center in advance; according to the embodiment of the invention, the log data of different levels are collected and stored in the log management center, so that the log management center is convenient to store in a classified mode and manage in a unified mode, the fault information corresponding to the log data is determined through the pre-deployed intelligent analysis engine, the timely alarm is convenient to be carried out according to the fault information, the safety protection information of the log data is determined through the pre-deployed safety protection engine, the safety control on the log data according to the safety protection information is convenient, and the log is ensured to be free from loss and leakage. In summary, the method solves the problems of difficult management and lack of protection of the log of the storage system based on the unified management of the log data, the intelligent analysis engine and the safety protection engine technology, greatly improves the log collection and management capacity of the storage system, ensures the safety of the log data, and is convenient for better log analysis and fault investigation under various complex business scenes.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below.
FIG. 1 is a flowchart illustrating a log data protection method according to an embodiment of the present invention;
FIG. 2 is a second flowchart illustrating a log data protection method according to an embodiment of the present invention;
FIG. 3 is a flowchart of step 103 of the log data protection method provided by the embodiment of the present invention;
FIG. 4 is a schematic diagram of a log management center in the log data protection method according to the embodiment of the present invention shown in FIG. 1;
fig. 5 is a schematic structural diagram of a log data protection device according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of a communication device according to an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the following detailed description of the embodiments of the present invention will be given with reference to the accompanying drawings. However, those of ordinary skill in the art will understand that in various embodiments of the present invention, numerous technical details have been set forth in order to provide a better understanding of the present application. However, the technical solutions claimed in the present application can be implemented without these technical details and with various changes and modifications based on the following embodiments. The following embodiments are divided for convenience of description, and should not be construed as limiting the specific implementation of the present invention, and the embodiments can be mutually combined and referred to without contradiction.
Referring to fig. 1, one of the step flowcharts of the log data protection method provided by the embodiment of the present invention is shown and applied to a log management center, where the method may include:
step 101, acquiring log data of multiple types from multiple tiers of a storage system.
The storage system in the embodiment of the present invention is a dedicated storage device connected to a server through a network based on different protocols, and is a generic term for storage hardware devices and storage software systems that provide storage services through a network, where acquiring log data from multiple levels of the storage system refers to performing targeted data acquisition on relevant points of hardware, a storage medium, firmware, an operating system and software of the storage system, where the hardware is various physical devices in a computer system, and is composed of electronic, mechanical, and optoelectronic elements, such as a graphics card, a network card, a CPU, a hard disk, a memory bank, and the like, where the storage medium refers to a carrier of data on the storage system, including a hard disk, a magnetic tape, a cloud resource, a virtual storage resource, and the like, and the firmware refers to a program written in an EPROM (eras-rom) or an EEPROM (Electrically Erasable Programmable read only memory), and the acquisition mode may be that each module actively reports, or may load a log management center agent.
In addition, since the collected plurality of types of log data are collected through a plurality of hierarchies, the corresponding plurality of types of log data include: hardware running log data, medium running log data, firmware running log data, operating system running log data, software dump log data, software state log data, configuration management log data, upgrade/update/maintenance log data, service access log data, data input/output IO log data and metadata IO log data;
further, obtaining log data of multiple types from multiple tiers of the storage system, including:
acquiring hardware operation log data from a plurality of hardware of a storage system;
acquiring medium operation log data from various media of a storage system;
acquiring firmware operation log data from a plurality of firmware of a storage system;
acquiring operation system operation log data from an operation system of a storage system;
acquiring software dump log data from software running in a central processing unit of a storage system;
acquiring software state log data from a plurality of software of a storage system;
under the condition that the storage system is detected to have configuration management operation, acquiring relevant configuration management log data;
Acquiring relevant upgrade/update/maintenance log data under the condition that upgrade and/or update and/or maintenance operations of the storage system are detected;
under the condition that the service program of the front-end host is detected to be accessed into the storage system, acquiring related service access log data;
under the condition that the existence of data IO in the storage system is detected, acquiring related data input/output IO log data;
metadata IO log data is obtained from the storage system.
From the above, it can be known that the log management center of the storage system may collect various types of log data from different levels of the storage system, and it should be noted that, by way of example, when the log data is obtained from hardware by obtaining the log data through different levels (channels), the normal state and the abnormal state of the hardware operation obtained by monitoring are monitored, so as to generate the log data of the hardware operation, and different monitoring modes are adopted for different hardware, for example, for a network card or an FC card, a mode of sending test data packets at regular time and judging packet response is adopted, and the specific implementation steps include:
transmitting a test data packet according to a preset period;
And obtaining the hardware operation log data through the response result of the test data packet in the target time range.
Further, the method for obtaining the media operation log data from various media also includes monitoring the operation states of various media of the storage system, generating the media operation log by the monitored normal state and abnormal state of the media, and adopting different monitoring modes for different media, for example: the method is performed by adopting an IO test mode aiming at the HDD hard disk, but is different from the method when firmware operation log data are acquired from a plurality of firmware, because the operation log of each firmware and dump information of the firmware are acquired when the firmware operation log data are acquired in the embodiment of the invention, and the firmware in the embodiment of the invention comprises: the self-grinding firmware and the third-party firmware, so the firmware operation log data in the embodiment of the invention comprises: the self-grinding firmware log data, the third-party firmware log data and the firmware dump p log data are different in the mode of acquiring running log data of different firmware, and for example, because the code of the self-grinding firmware is independently developed, the running state generated by the self-grinding firmware can be reported to the log management center in a contracted mode, for example: the self-grinding BMC can report the running state generated by the self-grinding BMC to the log management center in a contracted mode, and the third-party firmware can only acquire the running state of the third-party firmware in a mode of collecting the running log of the third-party firmware because the third-party firmware is an outsourced mature product and report the running state to the log management center, and when the dump log file is generated, the firmware cannot normally run, and at the moment, the dump log file must be actively collected and reported to the log management center, and the implementation steps comprise:
Receiving self-grinding firmware log data uploaded by self-grinding firmware in a storage system according to a stipulated mode;
collecting third-party firmware log data about third-party firmware in a storage system;
and acquiring the firmware dump p log data when the existence of the firmware dump log data in the plurality of firmware is detected.
By designing different modes for collecting log data, the log data is more reasonable and complete, and further, when the operating system running log data is obtained from the operating system, as the storage service system is a software system running on the operating system, the normal running of the storage service system can be ensured only by the normal running of the operating system, a large number of logs can be generated in the running process of the operating system, the storage system actively collects the logs concerned by itself according to the own needs, generates the operating system running log and reports the operating system running log to a log management center, wherein the storage system service comprises the data access types provided by the storage system to a server, such as block data access, file data access, object data access and the like; when the software dump log data is obtained, dump information generated by storage system software running on a CPU is collected to generate the software dump log data, when the software state log data is obtained, the software state log is generated by the running state information of the software under the condition of normal running of a storage service system, the software state log is reported to a log management center at any time, after any configuration management operation is carried out by a user in the running process of the storage service system, the storage system software immediately generates configuration management log by the configuration management operation information and reports the configuration management log to the log management center, when the storage system software is upgraded, updated, and maintained, the upgrading, updated and maintained related information is reported to the log management center by an upgrading, updated, maintained program, and when the service access log data is accessed to the storage system by a service program on a front-end host, the storage system software captures related access information, wherein different access information depends on a specific access protocol, for example: SMB protocol, NFS protocol, etc. for providing file storage service; iSCSI protocol to provide block storage services, etc.; the method comprises the steps of providing an S3 protocol of an object storage service and the like, acquiring data IO log data, wherein the core function of a storage system is to write and read data, so that after the storage system generates data IO, storage system software generates data IO logs by related information generated by IO and reports the data IO logs to a log management center, and acquiring metadata IO log data, wherein the metadata is data describing the data and is the basis of storage system service processing, so that for each IO of the metadata, the storage system software records the related information to generate metadata IO logs and reports the metadata IO logs to the log management center, and unified management is realized by acquiring the log data from a plurality of layers, thereby effectively solving the problems of various types of the log of the storage system, different time of various log generation and difficult log management caused by different module generated by various log generation.
And 102, storing the log data of various types into a designated directory according to a storage strategy.
In the embodiment of the invention, the log data are obtained from different levels, so that the obtained log data have different sources and types, and in order to better manage the log data and facilitate the subsequent analysis of the log data, a storage strategy is preset in the embodiment of the invention, the log data are respectively stored in a designated directory according to the storage strategy, wherein the designated directory can have a plurality of corresponding log data sources under each designated directory, for example, the designated directory A stores hardware operation log data, the designated directory B stores medium operation log data and the like, and the designated directory A can also be provided with a subdirectory, for example, the designated directory A stores hardware operation log data, and the hardware can be provided with a display card, a CPU, a hard disk and the like, so that the subdirectory A1 is used for storing the display card operation log data, the subdirectory A2 is used for storing the CPU operation log data, and the subdirectory A3 is used for storing the hard disk operation log data, and the invention is not particularly limited.
And step 103, determining fault information corresponding to the log data under the specified directory through the intelligent analysis engine.
According to the embodiment of the invention, the intelligent analysis engine is pre-deployed in the log data management center, can continuously run in the background, judges or predicts certain fault information of the storage system at present or about to occur by analyzing various collected log data, for example, the intelligent analysis engine judges the residual service life of the SSD hard disk by means of disk SMART data in a medium running log, and predicts how long the current SSD hard disk can be used by combining the data IO log and the metadata IO log.
Step 104, determining the security protection information corresponding to the log data under the specified directory through the security protection engine.
In the embodiment of the invention, the security protection engine is pre-deployed in the log data management center, and continuously operates in the background, so that various protection strategies configured for log data by an administrator can be obtained, for example: setting a certain log retention period to be 5 years, so as to ensure that the log cannot be tampered and deleted within 5 years; if a certain log check needs to input a password, no wrong password is input or input, so that the content of the log cannot be seen at all.
Step 105, determining protection strategies for various types of log data through the fault information and the safety protection information.
According to the embodiment of the invention, the intelligent analysis engine and the safety protection engine can determine the fault information and the safety protection information corresponding to the log data, and the alarm prompt can be sent according to the fault information, so that the staff can process in time, and the alarm prompt can comprise: sound, words, pictures, video and other forms, according to the security protection information, subsequent staff can perform security control on access control, storage validity period, export and forwarding of log data according to the security protection information, and the method specifically comprises the following implementation steps:
sending an alarm prompt through fault information;
and performing security control on access control, storage expiration date, export and forwarding of log data through the security protection information.
The confidentiality, the integrity and the availability of the log data can be ensured through the protection strategy determined by the two information.
As shown in fig. 4, in the embodiment of the present invention, different data acquisition modules are set in the log management center to acquire log data from different levels, and then the log data are sent to the log classification storage module, and then the log data of the log classification storage module are analyzed by the intelligent analysis engine and the security protection engine to determine the protection policy of the log data.
The log data protection method provided by the embodiment of the invention is applied to a log management center, and various types of log data are obtained from a plurality of levels of a storage system; storing the log data of various types into a designated directory according to a storage strategy; determining fault information corresponding to the log data under the appointed directory through an intelligent analysis engine, wherein the intelligent analysis engine is deployed in a log data management center in advance; determining safety protection information corresponding to the log data under the appointed directory through a safety protection engine, wherein the safety protection engine is deployed in a log data management center in advance; according to the embodiment of the invention, the log data of different levels are collected and stored in the log management center, so that the log management center is convenient to store in a classified mode and manage in a unified mode, the fault information corresponding to the log data is determined through the pre-deployed intelligent analysis engine, the timely alarm is convenient to be carried out according to the fault information, the safety protection information of the log data is determined through the pre-deployed safety protection engine, the safety control on the log data according to the safety protection information is convenient, and the log is ensured to be free from loss and leakage. In summary, the method solves the problems of difficult management and lack of protection of the log of the storage system based on the unified management of the log data, the intelligent analysis engine and the safety protection engine technology, greatly improves the log collection and management capacity of the storage system, ensures the safety of the log data, and is convenient for better log analysis and fault investigation under various complex business scenes.
Referring to fig. 2, a second step flowchart of a log data protection method provided by an embodiment of the present invention is shown, which specifically includes:
in step 201, multiple types of log data are obtained from multiple tiers of a storage system.
The above step 201 refers to the content of the previous step 101, and the present invention is not repeated here.
Step 202, dividing a plurality of disk partitions from a log data management center.
In the embodiment of the invention, the log data management center acquires various types of log data from a plurality of levels of the storage system, and in order to store the log data in a classified manner, the embodiment of the invention divides different disk partitions in the log management center, wherein the types of the log data can be referred to when the disk partitions are divided, for example, 10 types of data are acquired, and then 10 disk partitions can be divided in the log management center.
In step 203, a unique corresponding specified directory is generated for each disk partition.
After the disk partitions are divided, the embodiment of the invention creates different catalogs in each disk partition, namely, generates a unique corresponding appointed catalogs for each disk partition, and can be realized by setting different and unique names, and the appointed catalogs set for 5 disk partitions are named A, B, C, D and E respectively by way of example.
In step 204, a storage policy is assigned to each specified directory.
The storage strategy of the embodiment of the invention is used for determining the appointed catalogue to be stored of the log data according to the source and the purpose of the log data, so that the corresponding appointed catalogue can be conveniently distributed for the collected log data according to the storage strategy.
And step 205, storing the log data of various types into a designated directory according to a storage strategy.
And 206, determining fault information corresponding to the log data under the specified directory through the intelligent analysis engine.
Step 207, determining, by the security engine, security protection information corresponding to the log data under the specified directory.
In step 208, protection policies for various types of log data are determined from the fault information and the security protection information.
The steps 205-208 are referred to in the foregoing description of the steps 102-105, and the present invention is not repeated here.
According to the embodiment of the invention, the disk of the log management center is partitioned, and the storage strategy is distributed for the appointed catalogue arranged in the disk partition, so that the collected log data can be conveniently classified and stored, and the protection strategy can be conveniently and rapidly determined later.
Referring to fig. 3, a flowchart illustrating a step 103 of a log data protection method according to an embodiment of the present invention specifically includes:
In step 301, source information of log data is determined by designating a directory.
In the embodiment of the invention, the specified directory is provided with the storage strategy, and the storage strategy is used for determining the specified directory to be stored of the log data according to the source and the purpose of the log data, so that the source information of the log data can be determined from the specified directory.
Step 302, determining fault type information corresponding to target field information of log data under a specified directory through an intelligent analysis engine.
In the embodiment of the invention, the intelligent analysis engine determines the fault type information by analyzing the target field information of the log data under the appointed directory, the target field information can be obtained by determining a keyword, such as setting the keyword error, and then extracting the field information with the error as the target field information.
And step 303, determining fault information corresponding to the log data under the specified directory through the source information and the fault type information.
The fault information of the embodiment of the invention can be certain fault information of the current or to-be-generated fault information of the storage system, the position of the fault is determined through the source information, and the reason of the fault can be determined through the fault type information.
Referring to fig. 5, a schematic structural diagram of a log data protection device according to an embodiment of the present invention is shown, where, as shown in fig. 5, the device may include:
the first obtaining module 401 is configured to obtain log data of multiple types from multiple tiers of the storage system.
And the storage module 402 is used for storing the log data of various types into a specified directory according to a storage strategy.
The first determining module 403 is configured to determine, by using an intelligent analysis engine, fault information corresponding to the log data in the specified directory, where the intelligent analysis engine is pre-deployed in the log data management center.
The second determining module 404 is configured to determine, by using a security protection engine, security protection information corresponding to the log data in the specified directory, where the security protection engine is pre-deployed in the log data management center.
A third determining module 405, configured to determine protection policies for multiple types of log data according to the fault information and the security protection information.
Optionally, the log data protection device further includes:
the dividing module is used for dividing a plurality of disk partitions from the log data management center.
The first generation module is used for generating a unique corresponding specified directory for each disk partition.
And the allocation module is used for allocating a storage strategy for each specified directory, wherein the storage strategy is used for determining the specified directory to be stored of the log data according to the source and the purpose of the log data.
Optionally, the plurality of types of log data include: hardware running log data, medium running log data, firmware running log data, operating system running log data, software dump log data, software state log data, configuration management log data, upgrade/update/maintenance log data, service access log data, data input/output IO log data, and metadata IO log data.
The first acquisition module 401 further includes:
the first acquisition sub-module is used for acquiring hardware operation log data from a plurality of pieces of hardware of the storage system.
And the second acquisition sub-module is used for acquiring the medium operation log data from various mediums of the storage system.
And the third acquisition sub-module is used for acquiring the firmware running log data from a plurality of firmware of the storage system.
And the fourth acquisition sub-module is used for acquiring the operation system operation log data from the operation system of the storage system.
And the fifth acquisition sub-module is used for acquiring the software dump log data from the software running by the central processing unit of the storage system.
And a sixth acquisition sub-module, configured to acquire software status log data from a plurality of software of the storage system.
And a seventh acquisition sub-module, configured to acquire relevant configuration management log data when detecting that the storage system has a configuration management operation.
And the eighth acquisition sub-module is used for acquiring relevant upgrade/update/maintenance log data under the condition that the upgrade and/or update and/or maintenance operation of the storage system are detected.
And the ninth acquisition sub-module is used for acquiring related service access log data under the condition that the service program of the front-end host is detected to be accessed to the storage system.
And the tenth acquisition sub-module is used for acquiring related data input/output IO log data under the condition that the data IO exists in the storage system.
An eleventh acquisition sub-module is configured to acquire the metadata IO log data from the storage system.
Optionally, the firmware includes: the self-grinding firmware and the third-party firmware, the firmware operation log data comprises: self-grinding firmware log data, third party firmware log data and firmware dump log data.
The third acquisition sub-module further includes:
the receiving unit is used for receiving the self-grinding firmware log data uploaded by the self-grinding firmware in the storage system according to the appointed mode.
And the acquisition unit is used for acquiring the third-party firmware log data about the third-party firmware in the storage system.
And the first acquisition unit is used for acquiring the firmware dump log data when detecting that the firmware dump log data exist in the plurality of firmware.
The first acquisition sub-module further includes:
and the transmitting unit is used for transmitting the test data packet according to a preset period.
And the second acquisition unit is used for acquiring the hardware operation log data according to the response result of the test data packet in the target time range.
The first determination module 403 further includes:
and the first determining submodule is used for determining source information of the log data through the specified catalogue.
And the second determining submodule is used for determining fault type information corresponding to the target field information of the log data under the appointed directory through the intelligent analysis engine.
And the third determining submodule is used for determining fault information corresponding to the log data under the appointed directory through the source information and the fault type information.
The third determination module 405 further includes:
and the alarm sub-module is used for sending an alarm prompt through the fault information.
And the security control sub-module is used for performing security control on access control, storage expiration date, export and forwarding of the log data through the security protection information.
The log data protection method provided by the embodiment of the invention is applied to a log management center, and various types of log data are obtained from a plurality of levels of a storage system; storing the log data of various types into a designated directory according to a storage strategy; determining fault information corresponding to the log data under the appointed directory through an intelligent analysis engine, wherein the intelligent analysis engine is deployed in a log data management center in advance; determining safety protection information corresponding to the log data under the appointed directory through a safety protection engine, wherein the safety protection engine is deployed in a log data management center in advance; according to the embodiment of the invention, the log data of different levels are collected and stored in the log management center, so that the log management center is convenient to store in a classified mode and manage in a unified mode, the fault information corresponding to the log data is determined through the pre-deployed intelligent analysis engine, the timely alarm is convenient to be carried out according to the fault information, the safety protection information of the log data is determined through the pre-deployed safety protection engine, the safety control on the log data according to the safety protection information is convenient, and the log is ensured to be free from loss and leakage. In summary, the method solves the problems of difficult management and lack of protection of the log of the storage system based on the unified management of the log data, the intelligent analysis engine and the safety protection engine technology, greatly improves the log collection and management capacity of the storage system, ensures the safety of the log data, and is convenient for better log analysis and fault investigation under various complex business scenes.
The embodiment of the invention also provides a communication device, as shown in fig. 6, comprising a processor 501, a communication interface 502, a memory 503 and a communication bus 504, wherein the processor 501, the communication interface 502 and the memory 503 complete communication with each other through the communication bus 504,
a memory 503 for storing a computer program;
the processor 501 is configured to execute the program stored in the memory 503, and implement the following steps:
acquiring log data of multiple types from multiple levels of a storage system;
storing the log data of multiple types into a designated directory according to a storage strategy;
determining fault information corresponding to the log data under the appointed directory through an intelligent analysis engine, wherein the intelligent analysis engine is pre-deployed in the log data management center;
determining safety protection information corresponding to the log data under the appointed directory through a safety protection engine, wherein the safety protection engine is deployed in the log data management center in advance;
and determining protection strategies for various types of log data through the fault information and the safety protection information.
The communication bus mentioned by the above terminal may be a peripheral component interconnect standard (Peripheral Component Interconnect, abbreviated as PCI) bus or an extended industry standard architecture (Extended Industry Standard Architecture, abbreviated as EISA) bus, etc. The communication bus may be classified as an address bus, a data bus, a control bus, or the like. For ease of illustration, the figures are shown with only one bold line, but not with only one bus or one type of bus.
The communication interface is used for communication between the terminal and other devices.
The memory may include random access memory (Random Access Memory, RAM) or non-volatile memory (non-volatile memory), such as at least one disk memory. Optionally, the memory may also be at least one memory device located remotely from the aforementioned processor.
The processor may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU for short), a network processor (Network Processor, NP for short), etc.; but also digital signal processors (Digital Signal Processing, DSP for short), application specific integrated circuits (Application Specific Integrated Circuit, ASIC for short), field-programmable gate arrays (Field-Programmable Gate Array, FPGA for short) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
The present invention also provides a readable storage medium which, when executed by a processor of an electronic device, enables the electronic device to perform the log data protection method of the foregoing embodiments.
For the device embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, and reference is made to the description of the method embodiments for relevant points.
The algorithms and displays presented herein are not inherently related to any particular computer, virtual machine, or other apparatus. The required structure for the construction of such devices is apparent from the description above. In addition, the present invention is not directed to any particular programming language. It will be appreciated that the teachings of the present invention described herein may be implemented in a variety of programming languages, and the above description of specific languages is provided for disclosure of enablement and best mode of the present invention.
In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
Similarly, it should be appreciated that in the above description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be construed as reflecting the intention that: i.e., the claimed invention requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this invention.
Those skilled in the art will appreciate that the modules in the apparatus of the embodiments may be adaptively changed and disposed in one or more apparatuses different from the embodiments. The modules or units or components of the embodiments may be combined into one module or unit or component and, furthermore, they may be divided into a plurality of sub-modules or sub-units or sub-components. Any combination of all features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or units of any method or apparatus so disclosed, may be used in combination, except insofar as at least some of such features and/or processes or units are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings), may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.
Various component embodiments of the invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art will appreciate that some or all of the functions of some or all of the components in a sorting device according to the present invention may be implemented in practice using a microprocessor or Digital Signal Processor (DSP). The present invention may also be implemented as an apparatus or device program for performing part or all of the methods described herein. Such a program embodying the present invention may be stored on a computer readable medium, or may have the form of one or more signals. Such signals may be downloaded from an internet website, provided on a carrier signal, or provided in any other form.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The use of the words first, second, third, etc. do not denote any order. These words may be interpreted as names.
It will be clearly understood by those skilled in the art that, for convenience and brevity of description, specific working procedures of the apparatus, the apparatus and the units described above may refer to corresponding procedures in the foregoing method embodiments, which are not repeated herein.
The foregoing description of the preferred embodiments of the invention is not intended to be limiting, but rather is intended to cover all modifications, equivalents, and alternatives falling within the spirit and principles of the invention.
The foregoing is merely illustrative of the present invention, and the present invention is not limited thereto, and any person skilled in the art will readily recognize that variations or substitutions are within the scope of the present invention. Therefore, the protection scope of the invention is subject to the protection scope of the claims.
It should be noted that, in the embodiment of the present application, the various data-related processes are all performed under the condition of conforming to the corresponding data protection rule policy of the country of the location and obtaining the authorization given by the owner of the corresponding device.
Claims (10)
1. A log data protection method, applied to a log management center, comprising:
acquiring log data of multiple types from multiple levels of a storage system;
storing the log data of multiple types into a designated directory according to a storage strategy;
Determining fault information corresponding to the log data under the appointed directory through an intelligent analysis engine, wherein the intelligent analysis engine is pre-deployed in the log data management center;
determining safety protection information corresponding to the log data under the appointed directory through a safety protection engine, wherein the safety protection engine is deployed in the log data management center in advance;
and determining protection strategies for various types of log data through the fault information and the safety protection information.
2. The method of claim 1, wherein before storing the plurality of types of log data in the specified directory according to the storage policy, further comprising:
dividing a plurality of disk partitions from the log data management center;
generating a unique corresponding specified directory for each disk partition;
and allocating a storage strategy for each specified directory, wherein the storage strategy is used for determining the specified directory to be stored of the log data according to the source and the purpose of the log data.
3. The method of claim 1, wherein the plurality of types of log data comprises: hardware running log data, medium running log data, firmware running log data, operating system running log data, software dump log data, software state log data, configuration management log data, upgrade/update/maintenance log data, service access log data, data input/output IO log data and metadata IO log data;
The obtaining log data of multiple types from multiple tiers of the storage system includes:
acquiring the hardware running log data from a plurality of pieces of hardware of a storage system;
acquiring the medium operation log data from various mediums of a storage system;
acquiring the firmware operation log data from a plurality of firmware of a storage system;
acquiring operation log data of an operation system from the operation system of a storage system;
acquiring the software dump log data from software running in a central processing unit of a storage system;
acquiring the software state log data from a plurality of software of a storage system;
under the condition that the storage system is detected to have configuration management operation, acquiring relevant configuration management log data;
acquiring relevant upgrade/update/maintenance log data under the condition that upgrade and/or update and/or maintenance operations of a storage system are detected;
under the condition that the service program of the front-end host is detected to be accessed into a storage system, acquiring relevant service access log data;
under the condition that the existence of data IO in the storage system is detected, acquiring related data input/output IO log data;
And acquiring the metadata IO log data from a storage system.
4. A method according to claim 3, wherein the firmware comprises: the self-grinding firmware and the third-party firmware, and the firmware operation log data comprises: self-grinding firmware log data, third-party firmware log data and firmware dump log data;
the obtaining the firmware operation log data from a plurality of firmware of a storage system includes:
receiving self-grinding firmware log data uploaded by self-grinding firmware in a storage system according to a stipulated mode;
collecting third-party firmware log data about third-party firmware in a storage system;
and acquiring the firmware dump log data when the existence of the firmware dump log data in the plurality of firmware is detected.
5. The method of claim 3, wherein the retrieving the hardware log data from the plurality of hardware of the storage system comprises:
transmitting a test data packet according to a preset period;
and acquiring the hardware operation log data through a response result of the test data packet within a target time range.
6. The method of claim 1, wherein the determining, by the intelligent analysis engine, fault information corresponding to the log data under the specified directory, comprises:
Determining source information of log data through the specified directory;
determining fault type information corresponding to target field information of the log data under the appointed directory through an intelligent analysis engine;
and determining fault information corresponding to the log data under the appointed directory through the source information and the fault type information.
7. The method of claim 1, wherein the determining a protection policy for the log data of a plurality of types by the fault information and the safety protection information comprises:
sending an alarm prompt through the fault information;
and performing security control on access control, storage validity period, export and forwarding of log data through the security protection information.
8. A log data protection device, applied to a log management center, comprising:
a first acquisition module for acquiring log data of a plurality of types from a plurality of tiers of the storage system;
the storage module is used for storing the log data of various types into a designated directory according to a storage strategy;
the first determining module is used for determining fault information corresponding to the log data under the appointed directory through an intelligent analysis engine, and the intelligent analysis engine is pre-deployed in the log data management center;
The second determining module is used for determining safety protection information corresponding to the log data under the appointed directory through a safety protection engine, and the safety protection engine is deployed in the log data management center in advance;
and the third determining module is used for determining protection strategies for the log data of multiple types through the fault information and the safety protection information.
9. A communication device, comprising: a transceiver, a memory, a processor, and a program stored on the memory and executable on the processor;
the processor being configured to read a program in a memory to implement the steps of the log data protection method according to any one of claims 1 to 7.
10. A readable storage medium storing a program, wherein the program, when executed by a processor, implements the steps of the log data protection method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311076171.8A CN117251863A (en) | 2023-08-24 | 2023-08-24 | Log data protection method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311076171.8A CN117251863A (en) | 2023-08-24 | 2023-08-24 | Log data protection method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117251863A true CN117251863A (en) | 2023-12-19 |
Family
ID=89135933
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311076171.8A Pending CN117251863A (en) | 2023-08-24 | 2023-08-24 | Log data protection method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117251863A (en) |
-
2023
- 2023-08-24 CN CN202311076171.8A patent/CN117251863A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8661291B2 (en) | Diagnosing a fault incident in a data center | |
| CA3141329A1 (en) | Request link tracking method and service request processing method | |
| CN110826111B (en) | Test supervision method, device, equipment and storage medium | |
| US10936386B2 (en) | Method, device and computer program product for monitoring access request | |
| CN113312241A (en) | Abnormal alarm method, access log generation method and operation and maintenance system | |
| CN114077525A (en) | Abnormal log processing method and device, terminal equipment, cloud server and system | |
| CN109492406A (en) | Monitor the methods, devices and systems of kernel loophole attack | |
| CN111258832B (en) | Interface parameter verification method, device, equipment and medium | |
| CN105183799B (en) | Authority management method and client | |
| CN110941530A (en) | Method and device for acquiring monitoring data, computer equipment and storage medium | |
| CN105573872A (en) | Hardware maintenance method and device of data storage system | |
| CN109582454A (en) | Permission releasing control method, device and equipment in a kind of distributed storage cluster | |
| CN117251863A (en) | Log data protection method, device, equipment and storage medium | |
| US10310948B2 (en) | Evaluation of risk of data loss and backup procedures | |
| CN113934758A (en) | Vehicle fault repairing method and device, vehicle-mounted terminal, server and storage medium | |
| CN113806119A (en) | Memory card processing method, device, equipment and storage medium | |
| CN115994353B (en) | Host intrusion detection system and method based on container deployment and electronic equipment | |
| CN112530139B (en) | Monitoring system, method, device, collector and storage medium | |
| CN109446166B (en) | Method for detecting file directory, computer readable storage medium and terminal device | |
| KR101377408B1 (en) | Apparatus and method for managing error areas | |
| CN117668861A (en) | Object operation method and device, electronic equipment and computer readable storage medium | |
| CN116738097A (en) | Multi-system page access method, device, processor and storage medium | |
| CN117834265A (en) | Abnormal network request testing method and system | |
| CN116975852A (en) | Risk detection method, system, electronic equipment and storage medium | |
| CN117806930A (en) | Service link inspection method and device, storage medium and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |