CN109542341B - Read-write IO monitoring method, device, terminal and computer readable storage medium - Google Patents
Read-write IO monitoring method, device, terminal and computer readable storage medium Download PDFInfo
- Publication number
- CN109542341B CN109542341B CN201811314005.6A CN201811314005A CN109542341B CN 109542341 B CN109542341 B CN 109542341B CN 201811314005 A CN201811314005 A CN 201811314005A CN 109542341 B CN109542341 B CN 109542341B
- Authority
- CN
- China
- Prior art keywords
- read
- write operation
- information
- write
- monitoring
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0604—Improving or facilitating administration, e.g. storage management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0653—Monitoring storage devices or systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0673—Single storage device
- G06F3/0674—Disk device
- G06F3/0676—Magnetic disk device
Abstract
The embodiment of the invention relates to the technical field of Internet and discloses a read-write IO monitoring method, a read-write IO monitoring device, a read-write IO monitoring terminal and a computer readable storage medium. In some embodiments of the present application, a read/write IO monitoring method includes: monitoring read-write operation through a kernel probe; after the monitoring result indicates that the process performs read-write operation on the disk, determining read-write operation information of the process and information of the process; the read-write operation information comprises information of a file read and written by a process; and recording the read-write operation information of the process according to the information of the process. The IO detection method enables a user to master the operation condition of the process on the file of the disk, and to know the read-write operation condition in the disk more macroscopically.
Description
Technical Field
The embodiment of the invention relates to the technical field of internet, in particular to a read-write IO monitoring method, a device, a terminal and a computer readable storage medium.
Background
Currently, there are roughly three types of tools that can monitor the read/write (IO) operation of a system:
1. system read-write state command iostat
The iostat can be used for seeing various performance indexes of the disk, such as: the number of data blocks read and written per second, the number of times of writing and reading per second, the utilization rate of a disk, the average queue length and the like. The information can reflect the activity of a certain disk in a certain time period, so that the performance of the disk can be roughly evaluated according to the performance indexes.
2. System read and write status command block _ dump
The block _ dump is actually a kernel switch, and after the block _ dump is turned on, the kernel feeds back the read-write condition of the data block on the disk to the log file, and the function is changed more deeply than iostat:
a) the condition of IO can be explored by taking the process as an individual (distinguished by using the process number);
b) the performance of IO is based on the fact that a certain data block on a certain disk is written or read, and the disk and the data block can be clearly identified.
This function provides a great help to locate data anomalies or data block errors. If an error occurs in a data block, a check can be made through the log file to find out which process generated the error.
3. System read-write state command IOtop
The iotop reserves a time factor in iostat, adds a process factor, and counts reading and writing of data from the process perspective, wherein the data source of the iotop is/proc/pid/io. The method has the advantages that the IO condition of a certain process can be clearly seen in seconds, and the read-write change of each process to the disk can be fed back in a speed form. The user can clearly know which process consumes the IO resource, and accordingly adjust the process according to the actual service condition.
However, the inventors found that at least the following problems exist in the prior art: the iostat tool is capable of providing data on a relatively macroscopic level, but is limited when it needs further exploration due to non-disk problems. The log content generated by the block _ dump tool is detailed, a certain data block can be recorded in the log as long as the data block changes, and when a certain process modifies multiple positions of a file, the block _ dump can generate a large amount of log information, so that the IO analysis difficulty is increased. The iotop tool cannot know the files operated by the process, and when the process generates a large number of IO, the reason is not known in time. Therefore, a new IO monitoring tool is needed to better monitor and analyze IO.
It is to be noted that the information disclosed in the above background section is only for enhancement of understanding of the background of the present disclosure, and thus may include information that does not constitute prior art known to those of ordinary skill in the art.
Disclosure of Invention
An object of an embodiment of the present invention is to provide a read/write IO monitoring method, apparatus, terminal, and computer-readable storage medium, which enable to record read/write operation information on a disk in units of files.
In order to solve the above technical problem, an embodiment of the present invention provides a read/write IO monitoring method, including the following steps: monitoring read-write operation through a kernel probe; after the monitoring result indicates that the process performs read-write operation on the disk, determining read-write operation information of the process and information of the process; the read-write operation information comprises information of a file read and written by a process; and recording the read-write operation information of the process according to the information of the process.
The embodiment of the present invention further provides a read/write IO monitoring apparatus, including: the device comprises a determining module and a recording module; the determining module is used for monitoring read-write operation through the kernel probe; after the monitoring result indicates that the process performs read-write operation on the disk, determining read-write operation information of the process and information of the process; the read-write operation information comprises information of a file read and written by a process; the recording module is used for recording the read-write operation information of the process according to the information of the process.
An embodiment of the present invention further provides a terminal, including: at least one processor; and a memory communicatively coupled to the at least one processor; the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the read-write IO monitoring method mentioned in the above embodiments.
The embodiment of the invention also provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the read-write IO monitoring method mentioned in the above embodiment is implemented.
Compared with the prior art, the method and the device have the advantages that when the processes read and write the disk, the information of the file read and written by the processes is recorded according to the information of the processes, so that a user can determine which processes read and write the disk according to the recorded data, and each process reads and writes which files in the disk respectively and how many times, so that the user can master the operation condition of the processes on the files of the disk and can know the read and write operation condition in the disk more macroscopically.
In addition, the read-write operation information also includes the size of data read or written by the process in the file, and/or process access path information. In the implementation, a user can determine the data volume modified by the process and/or the access path of the process according to the recorded information, so that the IO operation condition of the process can be more comprehensively known.
In addition, the information of the process comprises information of a sub-process of the process No. 1 corresponding to the process, wherein the information of the corresponding sub-process of the process No. 1 is determined according to the information of the process and a parent-child relationship between the processes. In the realization, the category quantity of the terminal in the process of counting the read-write operation information can be reduced, the recorded information is convenient to combine and analyze in the later period, the reading categories of a user are reduced, and the user can read more conveniently.
In addition, recording read-write operation information of the process according to the information of the process specifically includes: and recording the read-write operation information of the process in a data storage area corresponding to the sub-process of the corresponding process No. 1 according to the information of the sub-process of the corresponding process No. 1.
In addition, before recording read-write operation information of the process according to the information of the process, the IO monitoring method further includes: and determining that the size of the data corresponding to the process is smaller than a first preset value, wherein the data corresponding to the process comprises read-write operation information of the process. In the implementation, the size of the data corresponding to the recorded process is limited, and the situation that the data generated by the read-write operation occupies too much memory is avoided.
In addition, after recording the read-write operation information of the process according to the information of the process, the IO monitoring method further includes: and determining the times of the process for performing the read-write operation on the file according to the recorded read-write operation information.
In addition, the read-write operation information also comprises the time for the progress to carry out the read-write operation; before recording read-write operation information of the process according to the information of the process, the IO monitoring method further includes: and determining that the interval between the time of the process for performing the read-write operation at this time and the recorded time of the process for performing the read-write operation at the last time is greater than a second preset value. In the implementation, the terminal selectively records the read-write operation information of the process, so that the space occupied by the data generated by IO monitoring can be effectively reduced, and the situation that the data generated by the read-write operation occupies too much memory is avoided.
In addition, the process performs read-write operations on the disk through the file system.
In addition, the file system is a fourth-generation extended file system EXT 4; monitoring read-write operation through a kernel probe, which specifically comprises: and monitoring the read-write function of the EXT4 through the kernel probe, and if the read-write function of the EXT4 is determined to be called, determining that the monitoring result indicates that a process exists to carry out read-write operation on the disk.
In addition, monitoring the read-write function of EXT4 through the kernel probe specifically includes: judging whether a preset detection point is triggered or not; wherein, the detection point is set in the read-write function of EXT 4: the addresses corresponding to ext4_ readpages, ext4_ readpages and ext4_ file _ write; if yes, determining that the read-write function of EXT4 is called; otherwise, it is determined that the read-write function of EXT4 is not called.
Drawings
One or more embodiments are illustrated by way of example in the accompanying drawings, which correspond to the figures in which like reference numerals refer to similar elements and which are not to scale unless otherwise specified.
Fig. 1 is a flowchart of an IO monitoring method according to a first embodiment of the present invention;
FIG. 2 is a diagram illustrating the instruction execution process without a kernel probe according to the first embodiment of the present invention;
FIG. 3 is a diagram illustrating the process of executing instructions after a kernel probe is inserted according to the first embodiment of the present invention;
FIG. 4 is a diagram illustrating parent-child relationships between processes of a terminal according to a first embodiment of the present invention;
fig. 5 is a flowchart of an IO monitoring method according to a second embodiment of the present invention;
fig. 6 is a schematic configuration diagram of an IO monitoring device according to a third embodiment of the present invention;
fig. 7 is a schematic configuration diagram of a terminal according to a fourth embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention more apparent, embodiments of the present invention will be described in detail below with reference to the accompanying drawings. However, it will be appreciated by those of ordinary skill in the art that numerous technical details are set forth in order to provide a better understanding of the present application in various embodiments of the present invention. However, the technical solution claimed in the present application can be implemented without these technical details and various changes and modifications based on the following embodiments.
The first embodiment of the invention relates to a read-write IO monitoring method, which is applied to a terminal, wherein the terminal can be a computer, a mobile phone, a robot and other devices. As shown in fig. 1, the IO monitoring method includes the following steps:
step 101: and monitoring the read-write operation through the kernel probe.
Specifically, the core probe may be a core probe kprobe, or a core probe jprobe or other core probes, where the core probe kprobe is taken as an example herein to describe a process of monitoring read/write operations by an IO monitoring device, and reference may be made to the related description of the embodiment of the present invention for a process of monitoring read/write operations by using other core probes, and the embodiment does not limit a specific means for monitoring read/write functions.
The general principle of kprobe is explained below. kprobe can dynamically collect debugging information and performance information, and a user can actively set a probe point (function) and set corresponding exception handling functions, such as an exception handling function pre _ handler and an exception handling function post _ handler. When the program is executed to the probe location, kprobe replaces the probe function instruction with break point, and moves the original function instruction to another location in memory. When BREAKPOINT is triggered, the system is stuck in an "exception". The terminal executes a preset exception handling function pre _ handler, after the execution of the pre _ handler is finished, the kprobe enables the terminal to execute an original function which is originally moved to other positions in a memory, and after the execution of the original function is finished, the terminal falls into 'exception' again, and then executes an exception handling function post _ handler. And after the post _ handler is executed, the kprobe also completes the detection, and the terminal continues to execute the function after the original function. For example, the schematic diagram of the instruction execution process without the kernel probe is shown in fig. 2, and the schematic diagram of the instruction execution process after the kernel probe kprobe is inserted at the position of the instruction 2 is shown in fig. 3. As can be seen from fig. 3, the position of the original instruction 2 is replaced by break point, the terminal executes instruction 1, and after reaching break point, executes pre _ handler, instruction 2, and post _ handler, and after executing post _ handler, executes instruction 3.
Step 102: and after the monitoring result indicates that the process performs read-write operation on the disk, determining read-write operation information of the process and information of the process.
Specifically, the read-write operation information includes information of the file read and written by the process. A plurality of files may be stored in the disk, each file may include a plurality of data blocks, and when a process operates on a certain data block or certain data blocks, information of the file to which the data block belongs is determined. The read-write file information may be a file name of the read-write file or other identification information of the file, and the process information may include a process name and/or a process number of the process.
It should be noted that, since a process may operate on data blocks in different files, the read-write operation information may include information of multiple files.
It should be noted that, as can be understood by those skilled in the art, the read-write operation information may further include size of data read or written by the process in the file, and/or other information such as process access path information, where the process access path information refers to information of a path for the process to access the file, and the present embodiment does not limit the type and the number of types of information included in the read-write operation information.
It should be noted that, when the terminal monitors the size of the data read or written by the process in the file, the user may determine the data amount modified by the process according to the recorded information, and when the terminal monitors the information of the path through which the process accesses the file, the user may know the path through which the process accesses the file, so that the IO operation condition of the process can be known more comprehensively.
In specific implementation, the process performs read-write operation on the disk through the file system. The File System may be any one of a New Technology File System (NTFS), a Second-generation extended File System (EXT 2), a Third-generation extended File System (EXT 3), and the like.
The following takes the file system as the fourth-generation extended file system EXT4 as an example, and illustrates a method for the terminal to determine the read-write operation of the process on the disk.
And the terminal monitors the read-write function of the EXT4 through the kernel probe kprobe, and if the read-write function of the EXT4 is determined to be called, the monitoring result indicates that a process exists to carry out read-write operation on the disk. Specifically, the terminal judges whether a preset detection point is triggered or not; wherein, the detection point is set at the read-write function of EXT 4: ext4_ readpage, ext4_ readpages and ext4_ file _ write; if yes, determining that the read-write function of EXT4 is called; otherwise, it is determined that the read-write function of EXT4 is not called. When a process reads and writes data from and to a disk through EXT4, the following three key functions are required: ext4_ readpages, ext4_ readpages, and ext4_ file _ write, where kprobes are inserted, respectively, which replace the addresses of these functions with BREAKPOINTs (break points). When the process accesses the address of the function and calls the read-write function of the file system, the kprobe is triggered to collect the information of the process and the read-write operation information.
As can be seen from the above, inserting the kernel probe at the position of the three key functions of EXT4 can make the process jump to execute pre _ handler, and the terminal sets the instruction for acquiring the read-write operation information and the process information in pre _ handler, that is, can acquire the read-write operation information and the process information when the process reads and writes the disk. Wherein, the location of the key function of the EXT4 can be obtained through a configuration file. The configuration file may be a kernel symbol table (/ proc/kallsyms). Since the developer can extract the addresses (the positions in the memory) of all functions, global variables and static variables in the kernel code to form the kernel symbol table, the debugging process of the kernel code is facilitated. By querying the kernel symbol table, the location of the key function of EXT4 may be determined.
Step 103: and recording the read-write operation information of the process according to the information of the process.
Specifically, the terminal records read-write operation information of the monitored read-write operation. The method for recording the read-write operation information by the terminal includes, but is not limited to, the following four methods:
mode 1: the information of the process includes a process name of the process and/or a process number of the process. And the terminal records the information of the process and the read-write operation information in a form of a table. For example, the process information includes a process name and a process number of the process, the read-write operation information includes a read-write file name, and the terminal acquires the process name, the process number, and the read-write file name of the process after monitoring that the process performs the read-write operation on the disk, and fills the information in table 1.
It should be noted that the terminal may also record the process information and the read-write operation information of the process in the memory in a one-to-one correspondence manner in other forms, which is not described herein again.
TABLE 1
| Process number of a process | Process name of a process | Filename of read-write |
Mode 2: the information of the process includes a process name of the process and/or a process number of the process. The terminal allocates a data storage area for each process, determines the read-write operation information of the read-write operation after determining that a certain process performs the read-write operation on the disk, and stores the read-write operation information into the data storage area corresponding to the process. In a specific implementation, the read-write operation information may be stored in the data storage area in a table form or other forms.
Mode 3: the information of the process comprises information of sub-processes of the process No. 1 corresponding to the process, wherein the information of the sub-processes of the process No. 1 corresponding to the process is determined according to the information of the process and the parent-child relationship among the processes. And the terminal records the information of the sub-process of the process No. 1 corresponding to the process and the read-write operation information in a form of a table. The information of the sub-process of the process No. 1 corresponding to the process may include a process name of the sub-process of the process No. 1 corresponding to the process, and/or a process number of the sub-process of the process No. 1 corresponding to the process. The process 1 refers to an init process, which is a first user process started and run by a kernel.
Mode 4: the information of the process includes information of a sub-process of process No. 1 corresponding to the process. And the terminal records the read-write operation information of the process in a data storage area corresponding to the sub-process of the process No. 1 corresponding to the process according to the information of the sub-process of the process No. 1 corresponding to the process.
It should be noted that, in the manner 3 and the manner 4, the information of the process may further include a process name of the process and/or a process number of the process, and the terminal records the process name of the process and/or the process number of the process and the read-write operation information in a one-to-one correspondence manner.
The following exemplifies a procedure of determining a sub-process of process No. 1 corresponding to the process in mode 3 and mode 4. It is assumed that a schematic diagram of parent-child relationships between processes of a terminal is shown in fig. 4. When the process 15 performs read-write operation on the disk through the file system, the terminal determines that the process performing the read-write operation this time is the process 15, and determines read-write operation information of the read-write operation this time. And the terminal determines the sub-process of the process No. 1 corresponding to the process 15 as the process 1 according to the parent-child relationship among the stored processes. Therefore, the terminal puts the read-write operation information of the process 15 of the read-write operation into the process 1 name, that is, records the read-write operation information of the current time in the data storage area corresponding to the process 1, or records the process name of the process 1 and/or the process number of the process 1 and the read-write operation information of the current time in a table in a one-to-one correspondence manner. Similarly, read and write operations for process 5, process 8, process 9, and process 10 will also be under the name of process 1. While the read and write operations of process 6 and process 7 will be under the name of process 4.
It should be noted that, as can be understood by those skilled in the art, in the method 3, the merging method of attributing the read/write operation information of the process to the name of the sub-process of the process No. 1 corresponding to the process is merely an example, and in practical applications, the read/write operation information of the process may also be attributed to the names of processes of other levels. For example, the merging manner is set as: the read/write operations of the process 8, the process 9, the process 10 and the process 15 are named under the process 5, the read/write operations of the process 11 and the process 12 are named under the process 6, and the read/write operations of the process 13 and the process 14 are named under the process 7.
It is worth mentioning that the read-write information of the process is put under the sub-process name of the process No. 1 corresponding to the process, so that the category number of the read-write operation information counted by the terminal can be reduced, the recorded information can be conveniently merged and analyzed at the later stage, the reading category of the user is reduced, and the user can read the information more conveniently.
It should be noted that, after recording the read-write operation information of the process according to the information of the process, the terminal may further determine, according to the recorded read-write operation information, the number of times that the process performs the read-write operation on the file. Specifically, the terminal may determine the total number of times that the process performs read-write operations on all files, that is, the sum of the total number of times that the process performs read-write operations on each file, according to the user's requirements and the recorded read-write operation information. The terminal can also determine the number of times of the process for performing read-write operation on a certain file within a certain period of time according to the requirement of the user and the recorded read-write operation information.
The above description is only for illustrative purposes and does not limit the technical aspects of the present invention.
Compared with the prior art, the read-write IO monitoring method provided in this embodiment records information of files read and written by the processes according to the information of the processes when the processes perform read-write operations on the disk, so that a user can determine which processes perform read-write operations on the disk according to the recorded data, and each process reads and writes which files in the disk respectively, how many times each process reads and writes, so that the user can grasp the operation conditions of the processes on the files of the disk, and can more macroscopically know the read-write operations occurring in the disk. In addition, when the recorded read-write operation data of the terminal also comprises the size of the data read or written by the process in the file, the user can master the data read and written by each process.
A second embodiment of the present invention relates to an IO monitoring method, which is further improved on the basis of the first embodiment, and the specific improvements are as follows: other related steps are added prior to step 102.
Specifically, as shown in fig. 5, step 201 to step 204 are included in this embodiment, where step 201 and step 204 are substantially the same as step 101 and step 102 in the first embodiment, respectively, and are not repeated here. The following mainly introduces the differences:
step 201 and step 202 are performed.
Step 203: and determining that the size of the data corresponding to the process is smaller than a first preset value.
Specifically, the data corresponding to the process includes read-write operation information of the process. Before recording read-write operation information of a process, a terminal judges whether the size of data corresponding to the process is smaller than a first preset value; if yes, recording the read-write operation information of the process at this time, or the read-write operation information and the process information; otherwise, the information generated by the current read-write operation of the process is not recorded. The first preset value may be determined according to the size of the memory of the terminal.
It should be noted that, in practical application, the data corresponding to the process further includes information of the process itself that has been recorded, or the data corresponding to the process further includes read-write operation information of other processes under the sub-process name of the process No. 1 corresponding to the process, and those skilled in the art can understand that the content of the data corresponding to the process can be set as needed.
In specific implementation, in order to avoid that the data obtained by monitoring the IO operation occupies too much memory, a storage space with a fixed size is set for the data corresponding to the process, for example, the storage space is set to 1kb, and when the space occupied by the stored data corresponding to the process is larger than the fixed size, the data cannot be recorded continuously.
It is worth mentioning that the size of the data corresponding to the process is limited, so that the situation that the data generated by the read-write operation occupies too much memory can be avoided.
Step 204: and determining that the interval between the time of the process performing the reading and writing operation at this time and the recorded time of the process performing the reading and writing operation at the last time is greater than a second preset value.
Specifically, the read-write operation information further includes time for the process to perform the read-write operation. Because the disk read-write event may not occur in the terminal system all the time, if the terminal records all the read-write operation information, it may occupy too much memory. In addition, the operation of recording the read/write operation information by the terminal may also affect the use of a Central Processing Unit (CPU) of the terminal. Therefore, the terminal can record the read-write operation condition in the terminal in a manner similar to sampling.
For example, the terminal creates a new process at a certain time, called process 16. The process 16 performs a first read/write operation on the disk at a first time, and since the process 16 performs the first read/write operation, the terminal records read/write operation information of the process 16 performing the first read/write operation and information of the process 16. Then, the process 16 performs the second read-write operation at the second time, the terminal first determines whether the time interval between the second time and the first time is greater than a second preset value, if yes, the read-write operation information of the second read-write operation of the process 16 and the information of the process 16 are recorded, otherwise, the terminal does not record the read-write operation information of the second read-write operation of the process 16 and the information of the process 16. If the process 16 performs the third read-write operation at the third time and the terminal records the read-write operation information of the second read-write operation of the process 16 and the information of the process 16, the terminal determines whether the time interval between the third time and the second time is greater than a second preset value, if so, the read-write operation information of the third read-write operation of the process 16 and the information of the process 16 are recorded, otherwise, the read-write operation information of the third read-write operation of the process 16 and the information of the process 16 are not recorded. If the process 16 performs the third read-write operation at the third time and the terminal does not record the read-write operation information of the second read-write operation of the process 16 and the information of the process 16, the terminal determines whether the time interval between the third time and the first time is greater than a second preset value, if so, the read-write operation information of the third read-write operation of the process 16 and the information of the process 16 are recorded, otherwise, the read-write operation information of the third read-write operation of the process 16 and the information of the process 16 are not recorded.
It should be mentioned that, the terminal selectively records the read-write operation information of the process, so that the space occupied by the data generated by the monitoring IO can be effectively reduced, and the situation that the data generated by the read-write operation occupies too much memory is avoided.
Step 205 is performed.
It should be noted that step 203 and step 204 are not necessarily required to be performed, and step 203 and step 204 may be selectively performed, or either one of step 203 and step 204 may be performed.
In this embodiment, for clarity of description, step 204 is set as a step subsequent to step 203. However, it can be understood by those skilled in the art that in practical applications, step 204 may be executed first, and then step 203 may be executed, which is not limited in the present embodiment.
It should be noted that, in this embodiment, for clarity of explanation, step 203 and step 204 are set as steps executed after determining information of the process and information of the read/write operation, and in practical applications, after determining that the process performs the read/write operation on the disk, step 203 and step 204 may be executed first, and then the information of the process and the information of the read/write operation are determined, which is not limited in this embodiment.
The above description is only for illustrative purposes and does not limit the technical aspects of the present invention.
Compared with the prior art, the read-write IO monitoring method provided in this embodiment records information of files read and written by the processes according to the information of the processes when the processes perform read-write operations on the disk, so that a user can determine which processes perform read-write operations on the disk according to the recorded data, and each process reads and writes which files in the disk respectively, how many times each process reads and writes, so that the user can grasp the operation conditions of the processes on the files of the disk, and can more macroscopically know the read-write operations occurring in the disk. In addition, the terminal limits the size of the data corresponding to the process, and records the read-write operation of the process in a sampling mode, so that the situation that the data generated by the read-write operation occupies too much memory is avoided.
The steps of the above methods are divided for clarity, and the implementation may be combined into one step or split some steps, and the steps are divided into multiple steps, so long as the same logical relationship is included, which are all within the protection scope of the present patent; it is within the scope of this patent to add insignificant modifications or introduce insignificant designs to the algorithms or processes, but not to change the core designs of the algorithms and processes.
A third embodiment of the present invention relates to a read/write IO monitoring apparatus, as shown in fig. 6, including: a determining module 301 and a recording module 302; the determining module 301 is configured to monitor a read-write operation through a kernel probe; after the monitoring result indicates that the process performs read-write operation on the disk, determining read-write operation information of the process and information of the process; the read-write operation information comprises information of a file read and written by a process; the recording module 302 is configured to record read-write operation information of the process according to the information of the process.
In a specific implementation, the read-write operation information may further include a size of data read or written by the process in the file, and/or process access path information.
In a specific implementation, the information of the process may include information of a child process of the process No. 1 corresponding to the process, where the information of the child process of the process No. 1 corresponding to the process is determined according to the information of the process and a parent-child relationship between the processes.
In a specific implementation, the recording module 302 is specifically configured to: and recording the read-write operation information of the process in a data storage area corresponding to the sub-process of the process No. 1 corresponding to the process according to the information of the sub-process of the process No. 1 corresponding to the process.
In specific implementation, the IO monitoring apparatus further includes a first determining module, where the first determining module is configured to determine whether the size of the data corresponding to the process is smaller than a first preset value, and after it is determined that the size of the data corresponding to the process is smaller than the first preset value, control the recording module 302 to record read and write operation information of the process according to information of the process.
In a specific implementation, the IO monitoring apparatus further includes a statistics module, and the statistics module is configured to determine, according to the recorded read-write operation information, the number of times that the process performs the read-write operation on the file.
In a specific implementation, the IO monitoring apparatus further includes a second determining module, where the second determining module is configured to determine whether an interval between the time of performing the read-write operation of the process this time and the time of performing the read-write operation of the process recorded last time is greater than a second preset value, and after determining that the interval between the time of performing the read-write operation of the process this time and the time of performing the read-write operation of the process recorded last time is greater than the second preset value, control the recording module 302 to record the read-write operation information of the process according to the information of the process.
In specific implementation, the process performs read-write operation on the disk through the file system. The file system is a fourth-generation extended file system EXT4, and the method for determining the read-write operation of the process on the disk comprises the following steps: and monitoring whether the read-write function of the EXT4 is called or not by using the kernel probe, and if the read-write function of the EXT4 is determined to be called, determining that a process performs read-write operation on the disk.
It should be noted that the present embodiment is a system example corresponding to the first embodiment and the second embodiment, and may be implemented in cooperation with the first embodiment and the second embodiment. The related technical details mentioned in the first embodiment and the second embodiment are still valid in this embodiment, and are not described herein again in order to reduce repetition. Accordingly, the related-art details mentioned in the present embodiment can also be applied to the first embodiment and the second embodiment.
It should be noted that each module referred to in this embodiment is a logical module, and in practical applications, one logical unit may be one physical unit, may be a part of one physical unit, and may be implemented by a combination of multiple physical units. In addition, in order to highlight the innovative part of the present invention, a unit which is less closely related to solving the technical problem proposed by the present invention is not introduced in the present embodiment, but it does not indicate that no other unit exists in the present embodiment.
A fourth embodiment of the present invention relates to a terminal, as shown in fig. 7, including: at least one processor 401; and a memory 402 communicatively coupled to the at least one processor 401; the memory 402 stores instructions executable by the at least one processor 401, and the instructions are executed by the at least one processor 401, so that the at least one processor 401 can execute the read-write IO monitoring method according to the above embodiment.
The terminal includes: one or more processors 401 and a memory 402, one processor 401 being exemplified in fig. 7. The processor 401 and the memory 402 may be connected by a bus or other means, and fig. 7 illustrates the connection by a bus as an example. Memory 402, which is a non-volatile computer-readable storage medium, may be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. The processor 401 executes various functional applications and data processing of the device by running the nonvolatile software programs, instructions, and modules stored in the memory 402, that is, the read/write IO monitoring method is implemented.
The memory 402 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store a list of options, etc. Further, the memory 402 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some embodiments, memory 402 may optionally include memory located remotely from processor 401, which may be connected to an external device via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
One or more modules are stored in the memory 402 and when executed by the one or more processors 401, perform the read-write IO monitoring method in any of the above method embodiments.
The product can execute the method provided by the embodiment of the application, has corresponding functional modules and beneficial effects of the execution method, and can refer to the method provided by the embodiment of the application without detailed technical details in the embodiment.
A fifth embodiment of the present invention relates to a computer-readable storage medium storing a computer program. The computer program realizes the above-described method embodiments when executed by a processor.
That is, as can be understood by those skilled in the art, all or part of the steps in the method for implementing the embodiments described above may be implemented by a program instructing related hardware, where the program is stored in a storage medium and includes several instructions to enable a device (which may be a single chip, a chip, or the like) or a processor (processor) to execute all or part of the steps of the method described in the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
It will be understood by those of ordinary skill in the art that the foregoing embodiments are specific examples of practicing the invention, and that various changes in form and detail may be made therein without departing from the spirit and scope of the invention in practice.
Claims (11)
1. A read-write IO monitoring method is characterized by comprising the following steps:
monitoring read-write operation through a kernel probe;
after the monitoring result indicates that a process carries out read-write operation on the disk, determining read-write operation information of the process and information of the process; the read-write operation information comprises information of a file read and written by the process;
according to the process information, recording read-write operation information of the process by taking the file as a unit;
before the recording, according to the information of the process, the read-write operation information of the process in the unit of the file, the IO monitoring method further includes:
and determining that the size of the data corresponding to the process is smaller than a first preset value, wherein the data corresponding to the process comprises read-write operation information of the process, and/or determining that the interval between the time of performing the read-write operation of the process this time and the recorded time of performing the read-write operation of the process last time is larger than a second preset value, and the read-write operation information also comprises the time of performing the read-write operation of the process.
2. The IO monitoring method according to claim 1, wherein the read-write operation information further includes a size of data read or written by the process in the file, and/or process access path information.
3. The IO monitoring method according to claim 1, wherein the information of the process includes information of a child process of the process No. 1 corresponding to the process, and wherein the information of the child process of the corresponding process No. 1 is determined according to the information of the process and a parent-child relationship between the processes.
4. The IO monitoring method according to claim 3, wherein the recording read-write operation information of the process according to the information of the process specifically includes:
and recording the read-write operation information of the process in a data storage area corresponding to the sub-process of the corresponding process No. 1 according to the information of the sub-process of the corresponding process No. 1.
5. The IO monitoring method according to any one of claims 1 to 4, wherein after the recording of the read-write operation information of the process according to the information of the process, the IO monitoring method further includes:
and determining the times of the process performing read-write operation on the file according to the recorded read-write operation information.
6. The IO monitoring method according to any one of claims 1 to 4, wherein the process performs read-write operations on the disk through a file system.
7. The IO monitoring method according to claim 6, wherein the file system is a fourth-generation extended file system EXT 4;
the monitoring of the read-write operation through the kernel probe specifically includes:
and monitoring a read-write function of the EXT4 through the kernel probe, and if the read-write function of the EXT4 is determined to be called, determining that the monitoring result indicates that a process exists to perform read-write operation on the disk.
8. The IO monitoring method according to claim 7, wherein the monitoring of the read-write function of EXT4 through the kernel probe specifically includes:
judging whether a preset detection point is triggered or not; wherein the detection point is set in the read-write function of EXT 4: ext4_ readpage, ext4_ readpages and ext4_ file _ write;
if yes, determining that the read-write function of the EXT4 is called;
otherwise, it is determined that the read-write function of the EXT4 is not called.
9. A read-write IO monitoring device, comprising: a determining module and a recording module;
the determining module is used for monitoring read-write operation through the kernel probe; after the monitoring result indicates that a process carries out read-write operation on the disk, determining read-write operation information of the process and information of the process; the read-write operation information comprises information of a file read and written by the process;
the recording module is used for recording the read-write operation information of the process by taking the file as a unit according to the information of the process;
the read-write IO monitoring device also comprises a first judging module and/or a second judging module;
the first judging module is used for judging whether the size of the data corresponding to the process is smaller than a first preset value or not, and controlling the recording module to record the read-write operation information of the process by taking the file as a unit according to the information of the process after determining that the size of the data corresponding to the process is smaller than the first preset value;
the second determination module is configured to determine whether an interval between the time of performing the read-write operation of the process this time and the time of performing the read-write operation of the process recorded last time is greater than a second preset value, and control the recording module to record the read-write operation information of the process in units of the file according to the information of the process after determining that the interval between the time of performing the read-write operation of the process this time and the time of performing the read-write operation of the process recorded last time is greater than the second preset value.
10. A terminal, comprising: at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the read-write IO monitoring method of any one of claims 1 to 8.
11. A computer-readable storage medium storing a computer program, wherein the computer program is configured to implement the read/write IO monitoring method according to any one of claims 1 to 8 when executed by a processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811314005.6A CN109542341B (en) | 2018-11-06 | 2018-11-06 | Read-write IO monitoring method, device, terminal and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811314005.6A CN109542341B (en) | 2018-11-06 | 2018-11-06 | Read-write IO monitoring method, device, terminal and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109542341A CN109542341A (en) | 2019-03-29 |
| CN109542341B true CN109542341B (en) | 2022-07-19 |
Family
ID=65846149
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811314005.6A Active CN109542341B (en) | 2018-11-06 | 2018-11-06 | Read-write IO monitoring method, device, terminal and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109542341B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111159117B (en) * | 2019-12-17 | 2023-07-04 | 复旦大学 | Low-overhead file operation log acquisition method |
| CN113076233B (en) * | 2021-03-30 | 2024-02-27 | 中国建设银行股份有限公司 | IO performance detection method, device, equipment and storage medium |
| CN115098017B (en) * | 2022-05-12 | 2023-04-11 | 北京卡普拉科技有限公司 | Data processing method and device, electronic equipment and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102393829A (en) * | 2011-10-31 | 2012-03-28 | 华南理工大学 | Disk I/O (Input/Output) test system oriented to embedded system |
| CN103761175A (en) * | 2013-11-25 | 2014-04-30 | 中国科学院计算技术研究所 | System and method for monitoring program execution paths under Linux system |
| CN104331324A (en) * | 2014-11-04 | 2015-02-04 | 哈尔滨工业大学 | MapReduce fault injection external member |
| CN104462955A (en) * | 2014-12-25 | 2015-03-25 | 中国科学院信息工程研究所 | Host behavior active detection system and method based on virtualization |
| US9183111B2 (en) * | 2011-05-10 | 2015-11-10 | Microsoft Technology Licensing, Llc | Methods and computer program products for collecting storage resource performance data using file system hooks |
| CN105354050A (en) * | 2015-09-30 | 2016-02-24 | 深圳市九洲电器有限公司 | Application software calling method for intelligent terminal |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102810092B (en) * | 2011-05-31 | 2017-07-07 | 腾讯科技（深圳）有限公司 | data read-write method and system |
| CN103177080B (en) * | 2013-02-07 | 2016-03-30 | 华为技术有限公司 | The method and apparatus that file pre-reads |
| CN108399118B (en) * | 2018-03-20 | 2021-11-26 | Oppo广东移动通信有限公司 | File system test data processing method and device, storage medium and terminal |
-
2018
- 2018-11-06 CN CN201811314005.6A patent/CN109542341B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9183111B2 (en) * | 2011-05-10 | 2015-11-10 | Microsoft Technology Licensing, Llc | Methods and computer program products for collecting storage resource performance data using file system hooks |
| CN102393829A (en) * | 2011-10-31 | 2012-03-28 | 华南理工大学 | Disk I/O (Input/Output) test system oriented to embedded system |
| CN103761175A (en) * | 2013-11-25 | 2014-04-30 | 中国科学院计算技术研究所 | System and method for monitoring program execution paths under Linux system |
| CN104331324A (en) * | 2014-11-04 | 2015-02-04 | 哈尔滨工业大学 | MapReduce fault injection external member |
| CN104462955A (en) * | 2014-12-25 | 2015-03-25 | 中国科学院信息工程研究所 | Host behavior active detection system and method based on virtualization |
| CN105354050A (en) * | 2015-09-30 | 2016-02-24 | 深圳市九洲电器有限公司 | Application software calling method for intelligent terminal |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109542341A (en) | 2019-03-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8141053B2 (en) | Call stack sampling using a virtual machine | |
| JP5933762B2 (en) | Code coverage rate determination method and system | |
| US7603589B2 (en) | Method and system for debugging a software program | |
| US20100017583A1 (en) | Call Stack Sampling for a Multi-Processor System | |
| US20090287729A1 (en) | Source code coverage testing | |
| US8132170B2 (en) | Call stack sampling in a data processing system | |
| CN110221983B (en) | Test method, test device, computer readable storage medium and computer equipment | |
| US7698690B2 (en) | Identifying code that wastes time performing redundant computation | |
| CN109542341B (en) | Read-write IO monitoring method, device, terminal and computer readable storage medium | |
| US20080276129A1 (en) | Software tracing | |
| CN110018914B (en) | Shared memory based message acquisition method and device | |
| US9355003B2 (en) | Capturing trace information using annotated trace output | |
| CN110718264A (en) | Method and device for testing information of solid state disk, computer equipment and storage medium | |
| CN103578568A (en) | Method and apparatus for testing performances of solid state disks | |
| US20030145255A1 (en) | Hierarchical multi-component trace facility using multiple buffers per component | |
| US7490269B2 (en) | Noise accommodation in hardware and software testing | |
| CN109471697A (en) | The method, apparatus and storage medium that system is called in a kind of monitoring virtual machine | |
| CN110597704B (en) | Pressure test method, device, server and medium for application program | |
| EP3274839B1 (en) | Technologies for root cause identification of use-after-free memory corruption bugs | |
| CN113268427B (en) | Crash analysis method and system for binary program | |
| US9916220B2 (en) | Smart logging of trace data for storage systems | |
| CN116820610A (en) | Method and device for dynamically tracking user state program in observable field | |
| US8762960B2 (en) | Software probe minimization | |
| CN113127314A (en) | Method and device for detecting program performance bottleneck and computer equipment | |
| CN106095631B (en) | Multi-cycle non-pipeline CPU dynamic debugging method based on finite state machine |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |