CN110377479B - Sensitive field monitoring method and device of log file and computer equipment - Google Patents
Sensitive field monitoring method and device of log file and computer equipment Download PDFInfo
- Publication number
- CN110377479B CN110377479B CN201910440468.5A CN201910440468A CN110377479B CN 110377479 B CN110377479 B CN 110377479B CN 201910440468 A CN201910440468 A CN 201910440468A CN 110377479 B CN110377479 B CN 110377479B
- Authority
- CN
- China
- Prior art keywords
- field
- row
- sensitive field
- sensitive
- logs
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3065—Monitoring arrangements determined by the means or processing involved in reporting the monitored data
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Debugging And Monitoring (AREA)
Abstract
The application provides a method and a device for monitoring sensitive fields of log files and computer equipment, which are applied to the technical field of intelligent decision making. The method comprises the following steps: acquiring the latest log file of the system; respectively determining the read initial position and the end position of each row of logs in the latest log file, and forming a candidate sensitive field set of each row of logs by each field between the initial position and the end position of each row of logs; the total length of the candidate sensitive field sets of each row of log is less than that of the corresponding row of log; detecting whether each field in a candidate sensitive field set of each row of logs is a sensitive field or not according to a preset sensitive field identification rule; and if the field is the sensitive field, recording the row log where the sensitive field is located and the sensitive field. According to the embodiment of the application, the monitoring efficiency of the sensitive field in the log file is improved.
Description
Technical Field
The application relates to the technical field of intelligent decision making, in particular to a method and a device for monitoring sensitive fields of log files and computer equipment.
Background
The system can generate an event record called log (log file) when in operation; in the log file, each row of log records the description of the date, time, user and action. Sensitive fields inevitably appear in the log files, and the sensitive fields are also called private data, and commonly include names, identification numbers, addresses, telephones, bank accounts, mailboxes, passwords, medical information, educational backgrounds and the like. If these sensitive fields cannot be effectively protected, they will cause significant loss to the user once utilized by an illegal user. Therefore, sensitive fields of the log file are effectively detected, and the sensitive fields are further processed, so that the safety of sensitive information is improved. But at present, a user needs to manually monitor sensitive fields in log files, and the efficiency is low.
Disclosure of Invention
Aiming at the defects of the existing mode, the application provides a method and a device for monitoring the sensitive field of the log file and computer equipment, so as to improve the monitoring efficiency of the sensitive field in the log file.
According to a first aspect, an embodiment of the present application provides a method for monitoring sensitive fields of a log file, including:
acquiring the latest log file of the system;
respectively determining the read initial position and the end position of each row of logs in the latest log file, and forming a candidate sensitive field set of each row of logs by each field between the initial position and the end position of each row of logs; the total length of the candidate sensitive field sets of each row of log is less than that of the corresponding row of log;
detecting whether each field in a candidate sensitive field set of each row of logs is a sensitive field or not according to a preset sensitive field identification rule;
and if the field is the sensitive field, recording the row log where the sensitive field is located and the sensitive field.
In one embodiment, the determining the starting position and the ending position of each line of logs read in the latest log file respectively includes:
acquiring each historical log file of the system and the position of a sensitive field in each row of log of each historical log file;
respectively calculating a union set of the positions of the sensitive fields of the same row of logs in each history log file to obtain the sensitive field areas of the rows of logs;
and determining the read starting position and the read ending position of each row of logs in the latest log file according to the sensitive field area of each row of logs.
In one embodiment, if the field is a sensitive field, the sensitive field monitoring method further includes:
determining a security level of the sensitive field;
if the security level is higher than a preset threshold value, encrypting the sensitive field to obtain an encrypted field, and replacing the sensitive field in the log file with the encrypted field;
and if the security level is less than or equal to a preset threshold value, generating a random password with the same data type as the sensitive field, and inserting the random password into the sensitive field of the log file.
In one embodiment, said encrypting said sensitive field to obtain an encrypted field includes:
performing double hash calculation on the sensitive field to obtain a hash value;
and generating an encryption field by the hash value and a plurality of setting characters.
In one embodiment, the sensitive field identification rule comprises a plurality of rules of each field and the weight of each rule, and the weight of a rule is in direct proportion to the probability of detecting the sensitive field by the rule;
the detecting whether each field in the candidate sensitive field set of each row of logs is a sensitive field according to a preset sensitive field identification rule includes:
selecting a row of logs from the rows of logs;
selecting a field from the selected candidate sensitive field set of the row of the log;
detecting whether the rules of the fields in the sensitive field identification rules are multiple or not;
if the number of the fields is multiple, sequentially detecting whether the fields meet all the rules according to the sequence of the weights of all the rules of the fields from large to small;
if the field meets any one rule, determining the field as a sensitive field, otherwise, determining the field not as a sensitive field;
selecting another field from the selected candidate sensitive field set of the journal, and returning to the step of detecting whether the field rule in the sensitive field identification rule is multiple or not until all the fields in the candidate sensitive field set of the journal are selected to obtain the sensitive field of the journal;
and selecting another row of log from the logs in each row, and returning to the step of selecting one field from the candidate sensitive field set of the selected row of log until all the rows of logs are selected.
In one embodiment, the weight of each rule of the field is obtained by:
counting the times of detecting the field as a sensitive field by each rule of the field according to the historical log file of the system in a preset time period;
and setting the corresponding times of each rule of the field as corresponding weights.
In an embodiment, after acquiring the latest log file of the system, before respectively determining the start position and the end position of each line of logs read in the latest log file, the method further includes:
detecting whether a system to which the latest log file belongs is consistent with a system which does not need to carry out sensitive field monitoring on the preset log file;
if not, the step of respectively determining the read starting position and the read ending position of each row of logs in the latest log file is carried out, otherwise, the step of obtaining the latest log file of the system is returned.
According to a second aspect, an embodiment of the present application further provides a device for monitoring sensitive fields of a log file, including:
the log file acquisition module is used for acquiring the latest log file of the system;
a candidate sensitive field set determining module, configured to determine a start position and an end position where each row of logs in the latest log file is read, respectively, and form a candidate sensitive field set of each row of logs by each field between the start position and the end position of each row of logs; the total length of the candidate sensitive field sets of each row of the logs is smaller than the total length of the corresponding row of the logs;
the sensitive field detection module is used for detecting whether each field in the candidate sensitive field set of each row of logs is a sensitive field according to a preset sensitive field identification rule;
and the sensitive field recording module is used for recording the line log where the sensitive field is located and the sensitive field when the field is the sensitive field.
The embodiment of the present application further provides a computer readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the computer program implements the method for monitoring the sensitive field of the log file described in any one of the above.
Embodiments of the present application also provide, according to a fourth aspect, a computer device, including:
one or more processors;
a storage device to store one or more programs,
when the one or more programs are executed by the one or more processors, the one or more processors implement any one of the above-mentioned methods for monitoring sensitive fields of a log file.
According to the method and the device for monitoring the sensitive fields of the log file and the computer equipment, the automatic monitoring of the sensitive fields is realized according to the sensitive field identification rule, so that a user does not need to monitor a large number of sensitive logs manually, the monitoring efficiency of the sensitive fields in the log file is improved, and a large amount of manpower is saved. In addition, considering that the probability that some fields in the log file are sensitive fields is almost zero, such as time fields, etc., only the fields in the log file which may be sensitive fields are monitored, and the monitoring efficiency of the sensitive fields is greatly improved.
Additional aspects and advantages of the present application will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the present application.
Drawings
The above and/or additional aspects and advantages of the present application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
FIG. 1 is a diagram illustrating a method for monitoring sensitive fields of a log file according to an embodiment of the present application;
FIG. 2 is a diagram illustrating an apparatus for monitoring sensitive fields of a log file according to an embodiment of the present application;
FIG. 3 is a schematic diagram of a computer device according to an embodiment of the present application.
Detailed Description
Reference will now be made in detail to the embodiments of the present application, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the drawings are exemplary only for the purpose of explaining the present application and are not to be construed as limiting the present application.
As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It will be understood by those within the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
Fig. 1 is a schematic diagram of a sensitive field monitoring method of a log file according to an embodiment, where the method includes:
and S110, acquiring the latest log file of the system.
The system can generate a large amount of log files in the running process, the latest log files of each system can be obtained from each computer device in real time, the latest log files of each system can also be pulled from each computer device at regular time, for example, a job can be configured based on jenkins, and the latest log files of each system are pulled from each computer device at regular time through the job to monitor sensitive logs in the log files. The latest log file generally includes a plurality of log lines, each of which records a description of the date, time, user and action related operations.
S120, respectively determining the read initial position and the end position of each row of logs in the latest log file, and forming a candidate sensitive field set of each row of logs by each field between the initial position and the end position of each row of logs; and the total length of the candidate sensitive field set of each row of log is less than that of the corresponding row of log.
Considering that a field which cannot be a sensitive field may exist in each row of logs in the latest log file, in order to improve the monitoring efficiency of the sensitive field, the starting position and the ending position of each row of logs, where the row of logs are read, are respectively determined. The starting position and the ending position of each line correspond to each other, and the number of the lines can be more than one or one. The starting position and the ending position of each line may be the same or different. When the sensitive field is monitored, only the fields between the starting position and the ending position are monitored, and because the fields are extracted partial fields which may be sensitive fields, namely the total length of the candidate sensitive field set of the nth row of logs is less than the total length of the nth row of logs, compared with a mode of monitoring all the fields in one row, the monitoring efficiency is effectively improved.
S130, detecting whether each field in the candidate sensitive field set of each row of logs is a sensitive field or not according to a preset sensitive field identification rule.
A rule file (i.e. a sensitive field identification rule) is preset, and the rule file is used for setting a corresponding rule for a field to be monitored, i.e. a condition that the field needs to meet. The rule file can be a file in any format, for example, the rule file can be an excel, and rules can be added or reduced as required. The layout manner of each rule in the rule file may be set according to actual needs, for example, each rule is a column, the first row is a column name (i.e., a name of a field to be compared), and the second row or subsequent rows are rules (which may be a regular expression, a specific field value, etc.), for example, aa, and a field value indicating that the field must be a value including aa.
After the latest log file is pulled down, a script is automatically executed, and the principle of the script is as follows: traversing each row of logs of the log file, and respectively executing the following operations on each row of logs: and judging whether each field to be monitored of the log in the row conforms to the set rule of the corresponding field or not according to the rule of each field set by the rule file.
And S140, if the field is the sensitive field, recording the row log where the sensitive field is located and the sensitive field.
If the field value of the field does not accord with the rule set for the field, the field is judged to be a sensitive field, the row log where the field is located is recorded into a result file, and the name and the field value of the field are recorded in the result file. Optionally, the name of the sensitive field not meeting the rule is separately recorded in the next row of the log record in the row in the result file, and the field value not meeting the rule is also recorded in the next row, so that the user can conveniently view the expected result and the actual result. If the field value of a field complies with the rules set for that field, it is determined that the field is not a sensitive field.
According to the embodiment, the automatic monitoring of the sensitive fields is realized according to the sensitive field identification rule, so that a user does not need to monitor a large number of sensitive logs manually, the monitoring efficiency of the sensitive fields in the log files is improved, and a large amount of manpower is saved. In addition, considering that the probability that some fields in the log file are sensitive fields is almost zero, only the fields which are possibly sensitive fields in the log file are monitored, and the monitoring efficiency of the sensitive fields is greatly improved.
In one embodiment, the determining the starting position and the ending position of each line of logs read in the latest log file respectively includes:
s1201, obtaining each historical log file of the system and the position of the sensitive field in each row of log of each historical log file.
The historical log file refers to a log file prior to the latest log file of the same system. After the history log file is obtained, the position of the sensitive field in each row of log of the history log file is determined, for example, in the nth row, the 3 rd character to the 8 th character of the nth row appear in the sensitive field.
And S1202, respectively calculating a union set of positions of the sensitive fields of the logs in the same row in each history log file to obtain the sensitive field areas of the logs in different rows.
The same row of logs are logs with the same row number in each history log file. The union refers to the union of the locations of the sensitive fields of the log in the same row. If the sensitive field of the log of the 1 st row of the history log file A is located at the 3 rd character to the 8 th character, and the sensitive field of the log of the 1 st row of the history log file B is located at the 5 th character to the 16 th character, the union of the logs of the 1 st row is the 3 rd character to the 16 th character, namely the sensitive field area of the 1 st row is the 3 rd character to the 16 th character.
S1203, determining the read starting position and the read ending position of each row of logs in the latest log file according to the sensitive field area of each row of logs.
After the sensitive field area of each row of log is obtained, the starting position of the sensitive field area is the starting position of the corresponding row of log in the latest log file which is read, and the ending position of the sensitive area is the ending position of the corresponding row of log in the latest log file which is read.
Considering that the number of lines of the log file may not be consistent, optionally, if the number of lines of the log corresponding to the determined sensitive field area is greater than or equal to the number of lines of the log in the latest log file, determining a start position and an end position of each line of the log read in the latest log file according to a one-to-one correspondence relationship of the number of lines, if the sensitive field area of the lines 1 to 9 is determined and the number of lines of the latest log file is 9, determining the start position and the end position of each line of the log read in the line 1 of the latest log file according to the sensitive field area of the line 1, \\ 8230; \\ 8230, determining the start position and the end position of each line of the log read in the line 9 of the latest log file according to the sensitive field area of the line 9; if the line number of the log corresponding to the determined sensitive field area is less than the line number of the log in the latest log file, the read start position and the read end position of the log of the remaining lines in the latest log file can be determined according to the sensitive field area of the last line, if the sensitive field areas of the 1 st line to the 9 th line are determined and the line number of the latest log file is 10 lines, the read start position and the read end position of the log of the 1 st line of the latest log file are determined according to the sensitive field area of the 1 st line, 8230, the read start position and the read end position of the log of the 9 th line of the latest log file are determined according to the sensitive field area of the 9 th line, and the read start position and the read end position of the log of the 10 th line of the latest log file are determined according to the sensitive field area of the 9 th line.
In view of the fact that the log file of some system has a low probability of having a sensitive field, in order to improve the monitoring efficiency, in an embodiment, after the obtaining the latest log file of the system, before the determining the start position and the end position of each line of log in the latest log file, respectively, further includes: detecting whether a system to which the latest log file belongs is consistent with a system which does not need to carry out sensitive field monitoring on the preset log file; if not, the step of respectively determining the read starting position and the read ending position of each row of logs in the latest log file is carried out, otherwise, the step of obtaining the latest log file of the system is returned. In this embodiment, the system to be skipped is configured in the rule file, for example, some log files are logs belonging to a certain system, and after configuration in the rule file, the log file of the system does not need to be scanned.
Optionally, the sensitive field identification rule includes a plurality of rules of each field and a weight of each rule, and a weight of a rule is in a direct proportion relationship with a probability that the sensitive field is detected by the rule. The rules may be weighted in a number of ways, for example, in one embodiment, the weight of each rule of the field is obtained by: counting the times of detecting the field as a sensitive field by each rule of the field according to the historical log file of the system in a preset time period; and setting the corresponding times of each rule of the field as corresponding weights. And acquiring a historical log file from the same source (namely the same system) as the latest log file, counting the times of detecting the sensitive fields by each rule of each field according to the historical log file for each field, and setting the weight of each rule according to the times, wherein the higher the times is, the higher the weight is. For example, if the number of times that the rule 1 of a certain field a in the rule file monitors the sensitive field is 6, and the number of times that the rule 2 of the field a monitors the sensitive field is 3, the weight of the rule 1 is 6, and the weight of the rule 2 is 3.
If there are more rules in a field in the rule file, in order to improve monitoring efficiency, in an embodiment, the detecting, according to a preset sensitive field identification rule, whether each field in a candidate sensitive field set of each row of logs is a sensitive field includes:
s1301, selecting one row of logs from the logs in each row;
s1302, selecting a field from the selected candidate sensitive field set of the row of logs;
s1303, detecting whether the number of the field rules in the sensitive field identification rules is multiple;
s1304, if the number of the fields is multiple, sequentially detecting whether the fields meet all the rules according to the sequence from large to small of the weight of all the rules of the fields;
s1305, if the field meets any one rule, determining the field to be a sensitive field, otherwise, determining the field not to be a sensitive field;
s1306, another field is selected from the selected candidate sensitive field set of the journal, the step S1303 is returned until all fields in the candidate sensitive field set of the journal are selected, and the sensitive field of the journal is obtained;
s1307, selecting another row of logs from the rows of logs, and returning to step S1302 until all the rows of logs are selected.
In the embodiment, whether the field value of the log file accords with the rule with the highest weight or not is preferentially judged in actual judgment, if so, the rule with the second highest weight is continuously used for judging, and by analogy, the sensitive field is easier to monitor by the rule with the higher weight, so that the monitoring efficiency is improved by the method.
For example, if the field value of a field in the rule file is a Chinese character, the field value of the field in the latest log file is matched with the field value of the field in the rule file, if the field value of the field in the rule file is a Chinese character, the field value of the field in the latest log file is matched with the field value of the field in the rule file, and if the field value of the field in the rule file is inconsistent with the field value of the field in the rule file, the field value of the field is matched with a synonym of the field value in the rule file.
In order to improve the security of data, after the sensitive field is detected, different shielding modes are adopted according to the security level preset for the sensitive field. For example, in one embodiment, if the field is a sensitive field, the sensitive field monitoring method further includes:
s150, determining the security level of the sensitive field.
Various types of security levels may be preset, for example, the security level of the password is level 1, the security level of the identification number is level 2, and the security level of level 1 is higher than that of level 2. And judging the type of the sensitive field, such as judging whether the sensitive field is a password or an identity card number, and obtaining the security level of the sensitive field according to the security level of the preset type.
S160, if the security level is higher than a preset threshold value, encrypting the sensitive field to obtain an encrypted field, and replacing the sensitive field in the log file with the encrypted field.
There are many ways of encryption. For example, in one embodiment, the encrypting the sensitive field to obtain an encrypted field includes: performing double hash calculation on the sensitive field to obtain a hash value; and generating an encryption field by the hash value and a plurality of setting characters. For another example, in another embodiment, the encrypting the sensitive field to obtain an encrypted field includes: and carrying out encryption calculation on the sensitive field by adopting an encryption algorithm to obtain each encrypted alphabetic character, and remapping each encrypted alphabetic character into each digital character according to the mapping relation between the alphabetic character and the digital character, wherein each digital character forms an encrypted field.
S170, if the security level is less than or equal to a preset threshold value, generating a random password with the same data type as the sensitive field, and inserting the random password into the sensitive field of the log file.
For better protection, the random password is of the same data type as the sensitive field, for example, if the sensitive field is a string of numbers, the random password is also a string of numbers. The random password insertion mode can be set according to actual needs, for example, a random password is inserted every fixed number of characters, and the like.
Based on the same inventive concept, the application also provides a sensitive field monitoring device of the log file, and the following describes the specific implementation of the device in detail with reference to the accompanying drawings.
Fig. 2 is a schematic diagram of a sensitive field monitoring apparatus of a log file according to an embodiment, where the apparatus includes:
a log file obtaining module 210, configured to obtain a latest log file of the system;
a candidate sensitive field set determining module 220, configured to determine a start position and an end position where each row of logs in the latest log file is read, respectively, and form a candidate sensitive field set of each row of logs by each field between the start position and the end position of each row of logs; the total length of the candidate sensitive field sets of each row of the logs is smaller than the total length of the corresponding row of the logs;
a sensitive field detection module 230, configured to detect whether each field in a candidate sensitive field set of each row of logs is a sensitive field according to a preset sensitive field identification rule;
and a sensitive field recording module 240, configured to record, when a field is a sensitive field, a row log in which the sensitive field is located and the sensitive field.
In one embodiment, the candidate sensitive field set determination module 220 includes:
the position acquisition unit is used for acquiring each historical log file of the system and the position of a sensitive field in each row of log of each historical log file;
the sensitive field area determining unit is used for respectively calculating a union set of the positions of the sensitive fields of the same row of logs in each history log file to obtain the sensitive field areas of the rows of logs;
and the position determining unit is used for determining the read starting position and the read ending position of each row of logs in the latest log file according to the sensitive field area of each row of logs.
In one embodiment, if the field is a sensitive field, the sensitive field monitoring apparatus further includes:
the security level determining module is used for determining the security level of the sensitive field;
the first processing module is used for encrypting the sensitive field to obtain an encrypted field and replacing the sensitive field in the log file with the encrypted field when the security level is higher than a preset threshold value;
and the second processing module is used for generating a random password with the same data type as the sensitive field when the security level is less than or equal to a preset threshold value, and inserting the random password into the sensitive field of the log file.
In one embodiment, the first processing module comprises:
the Hash calculation unit is used for carrying out double Hash calculation on the sensitive field to obtain a Hash value;
and the encrypted field obtaining unit is used for generating an encrypted field by the hash value and a plurality of setting characters.
In one embodiment, the sensitive field identification rule comprises a plurality of rules of each field and the weight of each rule, and the weight of a rule is in direct proportion to the probability of detecting the sensitive field by the rule; the sensitive field detection module 230 includes:
the log selecting unit is used for selecting one row of logs from the rows of logs;
the field selection unit is used for selecting a field from the selected candidate sensitive field set of the row of the log;
the quantity detection unit is used for detecting whether the field rules in the sensitive field identification rules are multiple or not;
a field detection unit, configured to, when there are multiple rules, sequentially detect whether the field satisfies each rule according to a descending order of the weight of each rule of the field;
the judging unit is used for determining the field as a sensitive field when the field meets any rule, and otherwise, determining the field not as a sensitive field;
the field selecting unit selects another field from the selected candidate sensitive field set of the log row, the field enters the quantity detecting unit to execute the function of detecting whether the field rule in the sensitive field identification rule is multiple or not until all the fields in the candidate sensitive field set of the log row are selected to obtain the sensitive field of the log row;
and the log selection unit selects another row of logs from the logs in each row, and the log enters the field selection unit to execute the function of selecting one field from the selected candidate sensitive field set of the log in the row until all the logs in the row are selected.
In one embodiment, the weight of each rule of the field is obtained by:
counting the times of detecting the field as a sensitive field by each rule of the field according to the historical log file of the system in a preset time period;
and setting the corresponding times of each rule of the field as corresponding weights.
In an embodiment, the sensitive field monitoring apparatus further includes a system detection module connected between the log file obtaining module 210 and the candidate sensitive field set determining module 220, where the system detection module is configured to detect whether a system to which the latest log file belongs is consistent with a system that is set in advance and does not need to perform sensitive field monitoring on the log file; when the log files are inconsistent, the entry candidate sensitive field set determining module 220 performs a function of determining the read start position and the read end position of each row of logs in the latest log file respectively, and when the log files are consistent, the entry log file acquiring module 210 performs a function of acquiring the latest log file of the system.
Other technical features of the sensitive field monitoring apparatus of the log file are the same as those of the sensitive field monitoring method of the log file, and are not described herein again.
An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements any one of the above-mentioned sensitive field monitoring methods for log files. The storage medium includes, but is not limited to, any type of disk including floppy disks, hard disks, optical disks, CD-ROMs, and magneto-optical disks, ROMs (Read-Only memories), RAMs (Random AcceSS memories), EPROMs (EraSable Programmable Read-Only memories), EEPROMs (Electrically EraSable Programmable Read-Only memories), flash memories, magnetic cards, or optical cards. That is, a storage medium includes any medium that can store or transmit information in a form readable by a device (e.g., a computer). Which may be a read-only memory, magnetic or optical disk, or the like.
An embodiment of the present application further provides a computer device, where the computer device includes:
one or more processors;
a storage device for storing one or more programs,
when the one or more programs are executed by the one or more processors, the one or more processors implement the sensitive field monitoring method of the log file.
Fig. 3 is a schematic structural diagram of a computer apparatus according to the present application, which includes a processor 320, a storage device 330, an input unit 340, a display unit 350, and the like. Those skilled in the art will appreciate that the structural elements shown in fig. 3 do not constitute a limitation of all computer devices and may include more or fewer components than those shown, or some of the components may be combined. The storage 330 may be used to store the application 310 and various functional modules, and the processor 320 executes the application 310 stored in the storage 330, thereby performing various functional applications of the device and data processing. The storage 330 may be an internal memory or an external memory, or include both internal and external memories. The internal memory may include read-only memory, programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), flash memory, or random access memory. The external memory may include a hard disk, a floppy disk, a ZIP disk, a usb-disk, a magnetic tape, etc. The memory devices disclosed herein include, but are not limited to, these types of memory devices. The memory device 330 disclosed herein is provided by way of example only and not by way of limitation.
The input unit 340 is used to receive input of signals, and to receive log files of the system and the like. The input unit 340 may include a touch panel and other input devices. The touch panel can collect touch operations of a user on or near the touch panel (for example, operations of the user on or near the touch panel by using any suitable object or accessory such as a finger, a stylus and the like) and drive the corresponding connecting device according to a preset program; other input devices may include, but are not limited to, one or more of a physical keyboard, function keys (e.g., play control keys, switch keys, etc.), a trackball, a mouse, a joystick, and the like. The display unit 350 may be used to display information input by a user or information provided to the user and various menus of the computer device. The display unit 350 may take the form of a liquid crystal display, an organic light emitting diode, or the like. The processor 320 is a control center of the computer device, connects various parts of the entire computer using various interfaces and lines, and performs various functions and processes data by operating or executing software programs and/or modules stored in the storage device 330 and calling data stored in the storage device.
In one embodiment, the computer device includes one or more processors 320, and one or more storage 330, one or more applications 310, wherein the one or more applications 310 are stored in the storage 330 and configured to be executed by the one or more processors 320, and the one or more applications 310 are configured to perform the sensitive field monitoring method of the log file described in the above embodiment.
It should be understood that, although the steps in the flowcharts of the figures are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and may be performed in other orders unless otherwise indicated herein. Moreover, at least a portion of the steps in the flow chart of the figure may include multiple sub-steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of execution is not necessarily sequential, but may be performed alternately or alternately with other steps or at least a portion of the sub-steps or stages of other steps.
It should be understood that each functional unit in the embodiments of the present application may be integrated into one processing module, each unit may also exist alone physically, or two or more units may also be integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode.
The foregoing is only a partial embodiment of the present application, and it should be noted that, for those skilled in the art, several modifications and decorations can be made without departing from the principle of the present application, and these modifications and decorations should also be regarded as the protection scope of the present application.
Claims (9)
1. A method for monitoring sensitive fields of log files is characterized by comprising the following steps:
acquiring the latest log file of the system;
respectively determining the read starting position and the end position of each row of logs in the latest log file, and forming a candidate sensitive field set of each row of logs by each field between the starting position and the end position of each row of logs; the total length of the candidate sensitive field sets of each row of log is less than that of the corresponding row of log;
detecting whether each field in a candidate sensitive field set of each row of logs is a sensitive field or not according to a preset sensitive field identification rule;
if the field is a sensitive field, recording a row log where the sensitive field is located and the sensitive field; wherein:
the determining the read start position and the end position of each row of logs in the latest log file respectively comprises:
acquiring each historical log file of the system and the position of a sensitive field in each row of log of each historical log file;
respectively calculating a union set of the positions of the sensitive fields of the logs in the same row in each history log file to obtain the sensitive field areas of the logs in each row;
and determining the read starting position and the read ending position of each row of logs in the latest log file according to the sensitive field area of each row of logs, and determining the read starting position and the read ending position of the rest rows of logs, which are greater than the log row number corresponding to the sensitive field area, in the latest log according to the sensitive field area of the last row of logs when the row number of the logs corresponding to the sensitive field area is less than the log row number of the latest log file.
2. The method of claim 1, wherein if the field is a sensitive field, the method further comprises:
determining a security level of the sensitive field;
if the security level is higher than a preset threshold value, encrypting the sensitive field to obtain an encrypted field, and replacing the sensitive field in the log file with the encrypted field;
and if the security level is less than or equal to a preset threshold value, generating a random password with the same data type as the sensitive field, and inserting the random password into the sensitive field of the log file.
3. The method for monitoring the sensitive field of the log file according to claim 2, wherein the encrypting the sensitive field to obtain the encrypted field comprises:
performing double hash calculation on the sensitive field to obtain a hash value;
and generating an encryption field by the hash value and a plurality of setting characters.
4. The method for monitoring the sensitive fields of the log file according to claim 1, wherein the sensitive field identification rules comprise a plurality of rules of each field and a weight of each rule, and the weight of a rule is in a direct proportion relation with the probability of detecting the sensitive field by the rule;
the detecting whether each field in the candidate sensitive field set of each row of logs is a sensitive field according to the preset sensitive field identification rule includes:
selecting a row of logs from the rows of logs;
selecting a field from the selected candidate sensitive field set of the row of the log;
detecting whether the rules of the fields in the sensitive field identification rules are multiple or not;
if the number of the fields is multiple, sequentially detecting whether the fields meet all the rules or not according to the sequence from large to small of the weight of all the rules of the fields;
if the field meets any one rule, determining the field as a sensitive field, otherwise, determining the field not as a sensitive field;
selecting another field from the selected candidate sensitive field set of the journal, and returning to the step of detecting whether the field rule in the sensitive field identification rule is multiple or not until all the fields in the candidate sensitive field set of the journal are selected to obtain the sensitive field of the journal;
and selecting another row of log from the logs in each row, and returning to the step of selecting one field from the candidate sensitive field set of the selected row of log until all the rows of logs are selected.
5. The method for monitoring the sensitive fields of the log file according to claim 4, wherein the weight of each rule of the fields is obtained by the following steps:
counting the times of detecting the field as a sensitive field by each rule of the field according to the historical log file of the system in a preset time period;
and setting the corresponding times of each rule of the field as corresponding weights.
6. The method for monitoring the sensitive fields of the log file according to any of the claims 1 to 5, wherein after the latest log file of the system is obtained, before the respectively determining the start position and the end position of each row of logs in the latest log file is read, further comprising:
detecting whether a system to which the latest log file belongs is consistent with a system which does not need to carry out sensitive field monitoring on the preset log file;
if not, the step of respectively determining the read starting position and the read ending position of each row of logs in the latest log file is carried out, otherwise, the step of obtaining the latest log file of the system is returned.
7. A log file sensitive field monitoring device is characterized by comprising:
the log file acquisition module is used for acquiring the latest log file of the system;
a candidate sensitive field set determining module, configured to determine a start position and an end position where each row of logs in the latest log file is read, respectively, and form a candidate sensitive field set of each row of logs by each field between the start position and the end position of each row of logs; the total length of the candidate sensitive field sets of each row of log is less than that of the corresponding row of log;
the sensitive field detection module is used for detecting whether each field in the candidate sensitive field set of each row of logs is a sensitive field according to a preset sensitive field identification rule;
the sensitive field recording module is used for recording a line log where the sensitive field is located and the sensitive field when the field is the sensitive field; wherein:
the determining the read start position and the end position of each row of logs in the latest log file respectively comprises the following steps:
acquiring each historical log file of the system and the position of a sensitive field in each row of log of each historical log file;
respectively calculating a union set of the positions of the sensitive fields of the logs in the same row in each history log file to obtain the sensitive field areas of the logs in each row;
and determining the read starting position and the read ending position of each row of logs in the latest log file according to the sensitive field area of each row of logs, and determining the read starting position and the read ending position of the rest rows of logs, which are greater than the log row number corresponding to the sensitive field area, in the latest log according to the sensitive field area of the last row of logs when the row number of the logs corresponding to the sensitive field area is less than the log row number of the latest log file.
8. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out a method of monitoring sensitive fields of a log file according to any one of claims 1 to 6.
9. A computer device, characterized in that the computer device comprises:
one or more processors;
a storage device to store one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the sensitive field monitoring method of the log file of any of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910440468.5A CN110377479B (en) | 2019-05-24 | 2019-05-24 | Sensitive field monitoring method and device of log file and computer equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910440468.5A CN110377479B (en) | 2019-05-24 | 2019-05-24 | Sensitive field monitoring method and device of log file and computer equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110377479A CN110377479A (en) | 2019-10-25 |
| CN110377479B true CN110377479B (en) | 2022-12-09 |
Family
ID=68248734
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910440468.5A Active CN110377479B (en) | 2019-05-24 | 2019-05-24 | Sensitive field monitoring method and device of log file and computer equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110377479B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112733188B (en) * | 2021-01-13 | 2023-09-22 | 航天晨光股份有限公司 | Sensitive file management method |
| CN113343699B (en) * | 2021-06-22 | 2023-10-20 | 湖北华中电力科技开发有限责任公司 | Log security risk monitoring method and device, electronic equipment and medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB201220817D0 (en) * | 2011-11-28 | 2013-01-02 | Ibm | Data transformation by replacement of sensitive information in a log |
| CN106547658A (en) * | 2016-10-28 | 2017-03-29 | 合网络技术（北京）有限公司 | A kind of automated testing method and device |
| CN106598827A (en) * | 2016-12-19 | 2017-04-26 | 东软集团股份有限公司 | Method and device for extracting log data |
| CN108829789A (en) * | 2018-06-01 | 2018-11-16 | 平安普惠企业管理有限公司 | Log processing method, device, computer equipment and storage medium |
| CN109525608A (en) * | 2019-01-07 | 2019-03-26 | Oppo广东移动通信有限公司 | Log reporting method and device, blog management method and device and terminal device |
| CN109614814A (en) * | 2018-10-31 | 2019-04-12 | 平安普惠企业管理有限公司 | The method, apparatus and computer equipment of the sensitive log of scanning based on log monitoring |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10158149B2 (en) * | 2014-11-20 | 2018-12-18 | Motorola Solutions, Inc. | Method and apparatus to detect and manage battery pack cell swell |
| CN105825137B (en) * | 2015-01-05 | 2018-10-02 | 中国移动通信集团江苏有限公司 | A kind of method and device of determining sensitive data dispersal behavior |
| CN105528535A (en) * | 2015-12-25 | 2016-04-27 | 北京奇虎科技有限公司 | Log information based user behavior analysis method and apparatus |
| EP3620923B1 (en) * | 2016-09-19 | 2022-08-17 | Elmos Semiconductor SE | Watchdog for monitoring a processor |
| CN107423190B (en) * | 2017-04-19 | 2020-09-01 | 国家电网公司 | Method and device for identifying abnormal direction of log data |
| CN108229154A (en) * | 2017-12-12 | 2018-06-29 | 顺丰科技有限公司 | Sensitive data operation log recording method, device, storage medium and equipment |
-
2019
- 2019-05-24 CN CN201910440468.5A patent/CN110377479B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB201220817D0 (en) * | 2011-11-28 | 2013-01-02 | Ibm | Data transformation by replacement of sensitive information in a log |
| CN106547658A (en) * | 2016-10-28 | 2017-03-29 | 合网络技术（北京）有限公司 | A kind of automated testing method and device |
| CN106598827A (en) * | 2016-12-19 | 2017-04-26 | 东软集团股份有限公司 | Method and device for extracting log data |
| CN108829789A (en) * | 2018-06-01 | 2018-11-16 | 平安普惠企业管理有限公司 | Log processing method, device, computer equipment and storage medium |
| CN109614814A (en) * | 2018-10-31 | 2019-04-12 | 平安普惠企业管理有限公司 | The method, apparatus and computer equipment of the sensitive log of scanning based on log monitoring |
| CN109525608A (en) * | 2019-01-07 | 2019-03-26 | Oppo广东移动通信有限公司 | Log reporting method and device, blog management method and device and terminal device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110377479A (en) | 2019-10-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US6301699B1 (en) | Method for detecting buffer overflow for computer security | |
| US9135427B2 (en) | Authentication using a subset of a user-known code sequence | |
| CN110377479B (en) | Sensitive field monitoring method and device of log file and computer equipment | |
| US20080148375A1 (en) | Authentication system, authentication device, and authentication method | |
| CN107515716B (en) | Billboard card management method, apparatus, computer equipment and storage medium | |
| CN110348193A (en) | Verification method, device, equipment and storage medium | |
| CN106098069B (en) | Identity authentication method and terminal equipment | |
| US8656279B2 (en) | Global settings for the enablement of culture-based gestures | |
| CN105159475B (en) | A kind of characters input method and device | |
| CN110222243A (en) | Determine the method, apparatus and storage medium of abnormal behaviour | |
| CN107871079A (en) | A kind of suspicious process detection method, device, equipment and storage medium | |
| US20150310199A1 (en) | Secure data entry | |
| JP2010079562A (en) | Information processing apparatus, information processing method and program | |
| US10372890B2 (en) | Apparatus and method for verifying an identity of a user | |
| CN111414728B (en) | Numerical data display method, device, computer equipment and storage medium | |
| CN106843727B (en) | Method and system for preventing character from being deleted by mistake | |
| CN110502890B (en) | Verification code processing method and device, electronic equipment and storage medium | |
| JP2008040961A (en) | Personal identification system and personal identification method | |
| CN107247558A (en) | A kind of terminal control method, device, computer installation and readable storage medium storing program for executing | |
| CN108734014A (en) | Cryptographic data authentication method and apparatus, code data guard method and device | |
| CN106529296A (en) | Method for attacking software protection virtual machine based on fuzzy clustering | |
| CN103853573B (en) | A kind of information processing method and electronic equipment | |
| CN111556339A (en) | Video information privacy protection system and method based on sensitive information measurement | |
| JP2008181231A (en) | System, method and program for preventing use of computer by spoofing | |
| CN110688645A (en) | Big data analysis system based on computer verification code technology |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |