CN110750432A - IO performance analysis method and system of distributed storage system and related components - Google Patents
IO performance analysis method and system of distributed storage system and related components Download PDFInfo
- Publication number
- CN110750432A CN110750432A CN201911026065.2A CN201911026065A CN110750432A CN 110750432 A CN110750432 A CN 110750432A CN 201911026065 A CN201911026065 A CN 201911026065A CN 110750432 A CN110750432 A CN 110750432A
- Authority
- CN
- China
- Prior art keywords
- traffic
- flow
- storage system
- distributed storage
- category
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3466—Performance evaluation by tracing or monitoring
- G06F11/3485—Performance evaluation by tracing or monitoring for I/O devices
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3452—Performance evaluation by statistical analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
Abstract
The invention discloses an IO performance analysis method of a distributed storage system, which comprises the following steps: setting a monitoring point in each process of an IO stack of the distributed storage system; the method comprises the steps of obtaining IO flow sent by a client within a preset first duration; and determining the time-consuming information of the IO flow in each flow through each set monitoring point, and performing IO performance analysis on the distributed storage system through each determined time-consuming information. By applying the technical scheme provided by the invention, the staff can be effectively assisted to analyze the performance bottleneck point of the distributed storage system. The invention also discloses an IO performance analysis system of the distributed storage system and related components, and the system has corresponding technical effects.
Description
Technical Field
The invention relates to the technical field of storage, in particular to an IO performance analysis method and system of a distributed storage system and related components.
Background
As the amount of data for applications continues to expand, the performance requirements of clients for distributed storage systems also become higher and higher. Performance is regarded as an important index for measuring a distributed storage system, and is paid more and more attention by more and more manufacturers.
At present, IOPS, bandwidth and time consumption statistics of the distributed storage cluster can be conveniently obtained through a performance testing tool based on distributed storage, and then workers can know basic performance indexes of the distributed storage system according to the data. However, in some cases, the current performance testing tool still cannot analyze the performance bottleneck point of the distributed storage system.
In summary, how to effectively assist the staff to analyze the performance bottleneck point of the distributed storage system is a technical problem that needs to be solved urgently by those skilled in the art.
Disclosure of Invention
The invention aims to provide an IO performance analysis method, an IO performance analysis system and related components of a distributed storage system, so as to effectively assist workers in analyzing performance bottleneck points of the distributed storage system.
In order to solve the technical problems, the invention provides the following technical scheme:
an IO performance analysis method of a distributed storage system comprises the following steps:
setting a monitoring point in each process of an IO stack of the distributed storage system;
the method comprises the steps of obtaining IO flow sent by a client within a preset first duration;
and determining the time consumption information of the IO flow in each process through each set monitoring point, and performing IO performance analysis on the distributed storage system through each determined time consumption information.
Preferably, after the obtaining of the IO traffic sent by the client within the preset first duration, the method further includes:
according to the size of each IO data in the obtained IO traffic, shunting the IO traffic to obtain IO traffic of at least 2 categories;
correspondingly, the determining the time consumption information of the IO traffic in each process includes:
and determining the time consumption information of the IO traffic of each category in each flow aiming at the IO traffic of each category after the shunting.
Preferably, after the obtaining of the IO traffic sent by the client within the preset first duration, the method further includes:
according to the type of each IO data in the obtained IO flow, the IO flow is shunted to obtain a writing operation type, the operation type is deleted, the operation type is modified and the operation type is read, and the total number of the IO flows is 4;
correspondingly, the determining the time consumption information of the IO traffic in each process includes:
and determining the time consumption information of the IO traffic of each category in each flow aiming at the IO traffic of each category after the shunting.
Preferably, after the obtaining of the IO traffic sent by the client within the preset first duration, the method further includes:
according to the size of each IO data in the obtained IO traffic and the type of each IO data, shunting the IO traffic to obtain multiple classes of IO traffic;
correspondingly, the determining the time consumption information of the IO traffic in each process includes:
and determining the time consumption information of the IO traffic of each category in each flow aiming at the IO traffic of each category after the shunting.
Preferably, the IO flow acquired in the first time period is greater than a preset peak flow value;
the peak flow value is determined by the following steps:
and counting IO (input/output) flow of the distributed storage system in the flow peak period of each day when the distributed storage system operates for n days, and determining the flow value in the peak period by taking an average value.
An IO performance analysis system of a distributed storage system, comprising:
the monitoring point setting module is used for setting monitoring points in each process of an IO stack of the distributed storage system;
the IO flow acquisition module is used for acquiring IO flows sent by the client within a preset first duration;
and the segmented time-consuming information determining module is used for determining the time-consuming information of the IO flow in each flow through each set monitoring point, and carrying out IO performance analysis on the distributed storage system through each determined time-consuming information.
Preferably, the method further comprises the following steps:
the first shunting module is used for shunting the IO traffic according to the size of each IO data in the obtained IO traffic to obtain IO traffic of at least 2 categories;
correspondingly, the segmented time-consuming information determining module is specifically configured to:
and determining the time-consuming information of the IO flow of each category in each flow aiming at the IO flow of each category after the shunting through each set monitoring point.
Preferably, the method further comprises the following steps:
the second shunting module is used for shunting the IO traffic according to the type of each IO data in the obtained IO traffic to obtain the IO traffic of 4 categories, namely a writing operation category, a deleting operation category, a modifying operation category and a reading operation category;
correspondingly, the segmented time-consuming information determining module is specifically configured to:
and determining the time-consuming information of the IO flow of each category in each flow aiming at the IO flow of each category after the shunting through each set monitoring point.
An IO performance analysis device of a distributed storage system, comprising:
a memory for storing a computer program;
a processor for executing the computer program to implement the steps of the IO performance analysis method of the distributed storage system described in any one of the above.
A computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the IO performance analysis method of the distributed storage system of any one of the above.
By applying the technical scheme provided by the embodiment of the invention, considering that the flow of the IO stack of the current distributed storage system is usually longer, the traditional scheme can be integrally analyzed, but the bottleneck point of the performance cannot be easily and accurately analyzed in some occasions. Therefore, in the scheme of the application, the monitoring points are arranged in each flow of the IO stack of the distributed storage system, so that after the IO traffic sent by the client within the preset first time length is obtained, the time consumption information of the IO traffic in each flow can be determined through the arranged monitoring points. Because the time-consuming information of the IO flow in each flow is determined, the method is also beneficial to assisting the staff to more conveniently and effectively analyze the distributed storage system, and further finds the bottleneck point in the distributed storage system, therefore, the scheme of the application can effectively assist the staff to analyze the performance bottleneck point of the distributed storage system.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a flowchart illustrating an embodiment of an IO performance analysis method of a distributed storage system according to the present invention;
FIG. 2 is a schematic structural diagram of an IO performance analysis system of a distributed storage system according to the present invention;
fig. 3 is a schematic structural diagram of an IO performance analysis device of a distributed storage system according to the present invention.
Detailed Description
The core of the invention is to provide an IO performance analysis method of a distributed storage system, which can effectively assist workers to analyze the performance bottleneck point of the distributed storage system.
In order that those skilled in the art will better understand the disclosure, the invention will be described in further detail with reference to the accompanying drawings and specific embodiments. It is to be understood that the described embodiments are merely exemplary of the invention, and not restrictive of the full scope of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, fig. 1 is a flowchart illustrating an IO performance analysis method of a distributed storage system according to an embodiment of the present invention, where the IO performance analysis method of the distributed storage system may include the following steps:
step S101: and setting a monitoring point in each process of the IO stack of the distributed storage system.
In the scheme of the application, considering that the flow of an IO stack of the current distributed storage system is usually long, although the traditional scheme can perform analysis from the whole, it is not easy to accurately analyze the bottleneck point of the performance in some occasions. Therefore, the scheme of the application adopts a sectional analysis mode.
Firstly, monitoring points need to be set in each flow of an IO stack of the distributed storage system, so that each flow can be counted independently when time consumption information is counted in subsequent steps.
Step S102: and obtaining IO flow sent by the client within a preset first time length.
The specific value of the first duration can be set and adjusted as required, and the scheme of the application can be generally applied to a performance testing tool, so that the type, the size and the quantity of the IO data in the IO traffic can be set and adjusted by using the performance testing tool.
Step S103: and determining the time-consuming information of the IO flow in each flow through each set monitoring point, and performing IO performance analysis on the distributed storage system through each determined time-consuming information.
The time-consuming information usually includes three items, i.e., an average delay, a maximum delay, and a minimum delay, and of course, in a few cases, the time-consuming information may also be adjusted as needed, for example, the time-consuming information further includes information such as a total delay.
Due to the fact that each monitoring point is preset, time-consuming information of IO flow in each process can be determined according to the scheme. For example, the IO traffic includes 100 IO data, the IO stack of the distributed storage system includes 7 flows, for example, when the 1 st flow is executed, the minimum delay in the 100 IO data is, for example, 3 seconds consumed by the 5 th IO data, the maximum delay is, for example, 7.5 seconds consumed by the 62 th IO data, and the average delay is, for example, 4 seconds. Correspondingly, when the 2 nd flow is executed, the minimum delay, the maximum delay, and the average delay of the 100 IO data may also be determined. Similarly, the time consumption information of the remaining 5 processes can be determined, that is, in this embodiment, 7 time consumption information can be determined, which corresponds to 7 processes, and each time consumption information includes three items. After the determination is completed, the determined time-consuming information can be recorded, and a worker can subsequently perform IO performance analysis of the distributed storage system based on the time-consuming information, and certainly, the user can also view the recorded data at any time.
It should be noted that, for convenience of description in this embodiment, it is assumed that the IO traffic includes 100 IO data, and the IO stack of the distributed storage system includes 7 flows, and in practical application, these data may be other values, which does not affect the implementation of the present invention. The data in the following examples are illustrated for the same reason.
By applying the technical scheme provided by the embodiment of the invention, considering that the flow of the IO stack of the current distributed storage system is usually longer, the traditional scheme can be integrally analyzed, but the bottleneck point of the performance cannot be easily and accurately analyzed in some occasions. Therefore, in the scheme of the application, the monitoring points are arranged in each flow of the IO stack of the distributed storage system, so that after the IO traffic sent by the client within the preset first time length is obtained, the time consumption information of the IO traffic in each flow can be determined through the arranged monitoring points. Because the time-consuming information of the IO flow in each flow is determined, the method is also beneficial to assisting the staff to more conveniently and effectively analyze the distributed storage system, and further finds the bottleneck point in the distributed storage system, therefore, the scheme of the application can effectively assist the staff to analyze the performance bottleneck point of the distributed storage system.
Further, in an embodiment of the present invention, after step S102, the method may further include:
according to the size of each IO data in the obtained IO traffic, shunting the IO traffic to obtain IO traffic of at least 2 categories;
correspondingly, the determining of the time consumption information of the IO traffic in each flow described in step S103 may specifically be:
and determining the time consumption information of the IO traffic of each category in each flow aiming at the IO traffic of each category after the shunting.
In this embodiment, it is considered that although the foregoing is based on each flow, a segmented analysis manner is beneficial to assist the staff to analyze the performance bottleneck point of the distributed storage system. However, the analysis result is only applied to the specific IO traffic acquired in step S102. In practical application, the service of the customer site varies greatly, the performance effect generated by different IO streams is different, and the fixed IO stream is effective in the foregoing embodiment, but the actual site IO stream condition cannot be simulated well.
In the embodiment, one of the obvious characteristics of the IO traffic of the client site is that the data size is uncertain, so that the applicant considers that classification can be performed according to the size, and time consumption statistics of the IO traffic of different sizes is performed respectively, which is also beneficial for a worker to check time consumption information of each type respectively, thereby better assisting the worker to perform analysis on the performance bottleneck point of the distributed storage system.
Specifically, the IO traffic is shunted according to the size of each IO data in the obtained IO traffic, so that IO traffic of at least 2 categories is obtained. For example, in a specific embodiment, the IO traffic includes 100 IO data, and of the 100 IO data, IO data smaller than 4K is taken as a class, for example, 35 IO data, data from 4K to 4M is taken as a class, for example, 20 IO data, and data larger than 4M is taken as a class, for example, 45 IO data.
After the classification, time-consuming information of each class of IO traffic in each flow is determined subsequently for each class of IO traffic after the shunting. For example, for the class of IO traffic smaller than 4K after offloading, the class of IO traffic is composed of 35 IO data, and it is still assumed that the IO stack of the distributed storage system includes 7 flows, and when the 35 IO data executes the 1 st flow, for example, the minimum delay is 3 seconds, the maximum delay is 7.5 seconds, and the average delay is 4 seconds, the time consumption information of the 35 IO data in the 1 st flow is determined to be completed. In other categories, the time-consuming information can be obtained in 3 × 7 — 21 in this embodiment.
It should be noted that, in this embodiment, 0 to 4K, 4K to 4M, and greater than 4M constitute 3 splitting sections, that is, each IO data in the IO traffic is divided into 3 categories, and in other embodiments, the IO data may be divided into other categories, and the boundary value of the splitting section may be set and adjusted as needed, without affecting the implementation of the present invention.
In an embodiment of the present invention, after step S102, the method may further include:
according to the type of each IO data in the obtained IO flow, the IO flow is shunted to obtain a writing operation type, the operation type is deleted, the operation type and the reading operation type are modified, and the total number of the IO flow of 4 types is obtained;
correspondingly, the determining of the time consumption information of the IO traffic in each flow described in step S103 may specifically be:
and determining the time consumption information of the IO traffic of each category in each flow aiming at the IO traffic of each category after the shunting.
In the foregoing, classification is performed based on the size of the IO data, in this embodiment, considering that another characteristic of the IO traffic of the customer site is that the type of the IO data is not fixed, that is, in practical application, the proportions of the write operation, the delete operation, the modify operation, and the read operation in different periods are uncertain, and therefore, in this embodiment, classification is performed based on these four types of IO operations. Namely, the staff can independently analyze the time-consuming information of the IO data of a certain type, and the staff is further assisted to analyze the performance bottleneck point of the distributed storage system.
Still assuming that the IO stack of the distributed storage system includes 7 flows, in this embodiment, IO traffic is shunted to obtain IO traffic of 4 categories, namely, a write operation category, a delete operation category, a modify operation category, and a read operation category, so in this embodiment, 7 × 4 — 28 pieces of time-consuming information can be obtained, and each piece of time-consuming information may generally include three items, namely, a minimum delay, a maximum delay, and an average delay.
In an embodiment of the present invention, after step S102, the method may further include:
according to the size of each IO data in the obtained IO flow and the type of each IO data, the IO flow is shunted to obtain multiple classes of IO flows;
correspondingly, determining the time consumption information of the IO flow in each process includes:
and determining the time consumption information of the IO traffic of each category in each flow aiming at the IO traffic of each category after the shunting.
In this embodiment, the shunting may be performed according to the size of each IO data in the IO traffic and the type of each IO data at the same time. The method realizes more detailed classification, and is beneficial to assisting workers in analyzing the performance bottleneck points of the distributed storage system.
For example, in 100 IO data, IO data smaller than 4K, for example, 35 IO data are distinguished, for example, 20 IO data from 4K to 4M, and for example, 45 IO data larger than 4M are distinguished. Of the 35 IO data smaller than 4K, there are, for example, 6 write operations, 8 delete operations, 8 modify operations, and 13 read operations. Similarly, 20 IO data from 4K to 4M and 45 IO data greater than 4M are also continuously divided according to the types of the IO data, for example, 4 categories are continuously divided. A total of 12 classes of IO traffic are partitioned in this example. Still assuming that 7 processes are included in the IO stack of the distributed storage system, this embodiment may obtain 7 × 12 — 84 pieces of time-consuming information, where each time-consuming information may generally include three items, i.e., a minimum delay, a maximum delay, and an average delay.
It should be further noted that, in the foregoing embodiment, the obtained IO traffic is shunted by using the size of the IO data, the type of the IO data, and the size and the type are considered at the same time, so that statistics on the time consumption information is performed respectively.
In a specific embodiment of the present invention, the IO traffic obtained within the first time period is greater than a preset peak traffic value;
the peak flow value is determined by the following steps: and when the distributed storage system runs for n days, counting IO (input/output) flow in a flow peak period of each day, and determining a flow value in the peak period by taking an average value.
In this embodiment, considering that another characteristic of the customer service is that there are a peak time and a peak time, and the performance bottleneck of the distributed storage system sometimes only becomes prominent when the IO traffic is particularly large, therefore, in this embodiment, the IO traffic acquired in S102 for subsequent IO performance analysis is set to be large, that is, needs to be larger than the preset peak time traffic value. The peak flow value may be set by an average value of IO flows in the flow peak period of n days, and of course, the determined average value may be adjusted to a certain extent according to actual needs, for example, the average value is multiplied by a coefficient greater than 1 and then is used as the preset peak flow value, which does not affect the implementation of the present invention.
Corresponding to the above method embodiment, an embodiment of the present invention further provides an IO performance analysis system of a distributed storage system, which can be referred to in correspondence with the above.
Referring to fig. 2, a schematic structural diagram of an IO performance analysis system of a distributed storage system according to the present invention includes:
a monitoring point setting module 201, configured to set a monitoring point in each flow of an IO stack of the distributed storage system;
an IO traffic obtaining module 202, configured to obtain IO traffic sent by a client within a preset first duration;
the segmented time-consuming information determining module 203 is configured to determine time-consuming information of the IO traffic in each flow through each set monitoring point, so as to perform IO performance analysis of the distributed storage system through each determined time-consuming information.
In one embodiment of the present invention, the method further comprises:
the first shunting module is used for shunting the IO traffic according to the size of each IO data in the obtained IO traffic to obtain IO traffic of at least 2 categories;
accordingly, the segmented time-consuming information determining module 203 is specifically configured to:
and determining the time-consuming information of the IO flow of each category in each flow aiming at the IO flow of each category after the shunting through each set monitoring point.
In one embodiment of the present invention, the method further comprises:
the second shunting module is used for shunting the IO traffic according to the type of each IO data in the obtained IO traffic to obtain the IO traffic of 4 categories, namely a writing operation category, a deleting operation category, a modifying operation category and a reading operation category;
accordingly, the segmented time-consuming information determining module 203 is specifically configured to:
and determining the time-consuming information of the IO flow of each category in each flow aiming at the IO flow of each category after the shunting through each set monitoring point.
In one embodiment of the present invention, the method further comprises:
the third shunting module is used for shunting the IO traffic according to the size of each IO data in the obtained IO traffic and the type of each IO data to obtain multiple categories of IO traffic;
accordingly, the segmented time-consuming information determining module 203 is specifically configured to:
and determining the time consumption information of the IO traffic of each category in each flow aiming at the IO traffic of each category after the shunting.
In a specific embodiment of the present invention, the IO traffic obtained within the first time period is greater than a preset peak traffic value;
the peak flow value is determined by the following peak flow value calculation module:
the peak period flow value calculation module is used for: and when the distributed storage system runs for n days, counting IO (input/output) flow in a flow peak period of each day, and determining a flow value in the peak period by taking an average value.
Corresponding to the above method and system embodiments, the embodiments of the present invention further provide an IO performance analysis device of a distributed storage system and a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the IO performance analysis method of the distributed storage system according to any of the above embodiments are implemented. A computer-readable storage medium as referred to herein may include Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
Referring to fig. 3, a schematic structural diagram of an IO performance analysis device of a distributed storage system is shown, including:
a memory 301 for storing a computer program;
a processor 302 for executing a computer program to implement the steps of the IO performance analysis method of the distributed storage system according to any of the above embodiments.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The principle and the implementation of the present invention are explained in the present application by using specific examples, and the above description of the embodiments is only used to help understanding the technical solution and the core idea of the present invention. It should be noted that, for those skilled in the art, it is possible to make various improvements and modifications to the present invention without departing from the principle of the present invention, and those improvements and modifications also fall within the scope of the claims of the present invention.
Claims (10)
1. An IO performance analysis method of a distributed storage system is characterized by comprising the following steps:
setting a monitoring point in each process of an IO stack of the distributed storage system;
the method comprises the steps of obtaining IO flow sent by a client within a preset first duration;
and determining the time consumption information of the IO flow in each process through each set monitoring point, and performing IO performance analysis on the distributed storage system through each determined time consumption information.
2. The IO performance analysis method of the distributed storage system according to claim 1, wherein after the obtaining of the IO traffic sent by the client within the preset first duration, the method further includes:
according to the size of each IO data in the obtained IO traffic, shunting the IO traffic to obtain IO traffic of at least 2 categories;
correspondingly, the determining the time consumption information of the IO traffic in each process includes:
and determining the time consumption information of the IO traffic of each category in each flow aiming at the IO traffic of each category after the shunting.
3. The IO performance analysis method of the distributed storage system according to claim 1, wherein after the obtaining of the IO traffic sent by the client within the preset first duration, the method further includes:
according to the type of each IO data in the obtained IO flow, the IO flow is shunted to obtain a writing operation type, the operation type is deleted, the operation type is modified and the operation type is read, and the total number of the IO flows is 4;
correspondingly, the determining the time consumption information of the IO traffic in each process includes:
and determining the time consumption information of the IO traffic of each category in each flow aiming at the IO traffic of each category after the shunting.
4. The IO performance analysis method of the distributed storage system according to claim 1, wherein after the obtaining of the IO traffic sent by the client within the preset first duration, the method further includes:
according to the size of each IO data in the obtained IO traffic and the type of each IO data, shunting the IO traffic to obtain multiple classes of IO traffic;
correspondingly, the determining the time consumption information of the IO traffic in each process includes:
and determining the time consumption information of the IO traffic of each category in each flow aiming at the IO traffic of each category after the shunting.
5. The IO performance analysis method of a distributed storage system according to claim 1, wherein IO traffic acquired in the first duration is greater than a preset peak traffic value;
the peak flow value is determined by the following steps:
and counting IO (input/output) flow of the distributed storage system in the flow peak period of each day when the distributed storage system operates for n days, and determining the flow value in the peak period by taking an average value.
6. An IO performance analysis system of a distributed storage system, comprising:
the monitoring point setting module is used for setting monitoring points in each process of an IO stack of the distributed storage system;
the IO flow acquisition module is used for acquiring IO flows sent by the client within a preset first duration;
and the segmented time-consuming information determining module is used for determining the time-consuming information of the IO flow in each flow through each set monitoring point, and carrying out IO performance analysis on the distributed storage system through each determined time-consuming information.
7. The IO performance analysis system of the distributed storage system according to claim 6, further comprising:
the first shunting module is used for shunting the IO traffic according to the size of each IO data in the obtained IO traffic to obtain IO traffic of at least 2 categories;
correspondingly, the segmented time-consuming information determining module is specifically configured to:
and determining the time-consuming information of the IO flow of each category in each flow aiming at the IO flow of each category after the shunting through each set monitoring point.
8. The IO performance analysis system of the distributed storage system according to claim 6, further comprising:
the second shunting module is used for shunting the IO traffic according to the type of each IO data in the obtained IO traffic to obtain the IO traffic of 4 categories, namely a writing operation category, a deleting operation category, a modifying operation category and a reading operation category;
correspondingly, the segmented time-consuming information determining module is specifically configured to:
and determining the time-consuming information of the IO flow of each category in each flow aiming at the IO flow of each category after the shunting through each set monitoring point.
9. An IO performance analysis device of a distributed storage system, comprising:
a memory for storing a computer program;
a processor for executing the computer program to implement the steps of the IO performance analysis method of the distributed storage system according to any one of claims 1 to 5.
10. A computer-readable storage medium, having stored thereon a computer program which, when being executed by a processor, carries out the steps of the IO performance analysis method of the distributed storage system according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911026065.2A CN110750432B (en) | 2019-10-25 | 2019-10-25 | IO performance analysis method and system of distributed storage system and related components |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911026065.2A CN110750432B (en) | 2019-10-25 | 2019-10-25 | IO performance analysis method and system of distributed storage system and related components |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110750432A true CN110750432A (en) | 2020-02-04 |
| CN110750432B CN110750432B (en) | 2022-04-22 |
Family
ID=69280137
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911026065.2A Active CN110750432B (en) | 2019-10-25 | 2019-10-25 | IO performance analysis method and system of distributed storage system and related components |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110750432B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108132885A (en) * | 2018-01-02 | 2018-06-08 | 北京奇虎科技有限公司 | Data capture method, device and electronic equipment |
| US10013283B1 (en) * | 2016-08-15 | 2018-07-03 | Datacore Software Corporation | Methods and apparatus for data request scheduling in performing parallel IO operations |
| US10048874B1 (en) * | 2016-06-29 | 2018-08-14 | EMC IP Holding Company LLC | Flow control with a dynamic window in a storage system with latency guarantees |
| CN109992481A (en) * | 2019-04-11 | 2019-07-09 | 苏州浪潮智能科技有限公司 | A kind of performance analysis tool, method and storage medium for distributed block storage |
| CN110287158A (en) * | 2019-05-05 | 2019-09-27 | 平安科技（深圳）有限公司 | Monitor the method, apparatus and storage medium of distributed file system IO time delay |
-
2019
- 2019-10-25 CN CN201911026065.2A patent/CN110750432B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10048874B1 (en) * | 2016-06-29 | 2018-08-14 | EMC IP Holding Company LLC | Flow control with a dynamic window in a storage system with latency guarantees |
| US10013283B1 (en) * | 2016-08-15 | 2018-07-03 | Datacore Software Corporation | Methods and apparatus for data request scheduling in performing parallel IO operations |
| CN108132885A (en) * | 2018-01-02 | 2018-06-08 | 北京奇虎科技有限公司 | Data capture method, device and electronic equipment |
| CN109992481A (en) * | 2019-04-11 | 2019-07-09 | 苏州浪潮智能科技有限公司 | A kind of performance analysis tool, method and storage medium for distributed block storage |
| CN110287158A (en) * | 2019-05-05 | 2019-09-27 | 平安科技（深圳）有限公司 | Monitor the method, apparatus and storage medium of distributed file system IO time delay |
Non-Patent Citations (2)
| Title |
|---|
| HUADONG DAI ET AL.: ""Research and implementation of big data preprocessing system based on Hadoop"", 《2016 IEEE INTERNATIONAL CONFERENCE ON BIG DATA ANALYSIS (ICBDA)》 * |
| 王小平 等: ""采用云计算技术的网络流量检测"", 《电讯技术》 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110750432B (en) | 2022-04-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109828721B (en) | Data deleting method and device, computer equipment and storage medium | |
| US8321479B2 (en) | Efficient processing of time series data | |
| US8270410B2 (en) | Sampling techniques | |
| CN110806954A (en) | Method, device and equipment for evaluating cloud host resources and storage medium | |
| CN109407984B (en) | Method, device and equipment for monitoring performance of storage system | |
| CN110647447B (en) | Abnormal instance detection method, device, equipment and medium for distributed system | |
| US20120303325A1 (en) | Inferring effects of configuration on performance | |
| WO2020233347A1 (en) | Test method and apparatus for workflow management system, storage medium and terminal device | |
| WO2018122890A1 (en) | Log analysis method, system, and program | |
| CN111680085A (en) | Data processing task analysis method and device, electronic equipment and readable storage medium | |
| US8903696B2 (en) | System and method for controlling granularity of transaction recording in discrete event simulation | |
| CN112346962B (en) | Control data testing method and device applied to control testing system | |
| CN109388550B (en) | Cache hit rate determination method, device, equipment and readable storage medium | |
| CN103177080A (en) | File pre-reading method and file pre-reading device | |
| CN108932241A (en) | Daily record data statistical method, device and node | |
| US8306953B2 (en) | Online management of historical data for efficient reporting and analytics | |
| CN110750432B (en) | IO performance analysis method and system of distributed storage system and related components | |
| WO2023246391A1 (en) | Extraction of risk feature description | |
| CN110287158B (en) | Method, device and storage medium for monitoring IO time delay of distributed file system | |
| Loboz | Cloud resource usage: extreme distributions invalidating traditional capacity planning models | |
| CN105227410A (en) | Based on the method and system that the server load of adaptive neural network detects | |
| EP1622309A2 (en) | Method and system for treating events and data uniformly | |
| US10877972B1 (en) | Systems and methods for high efficiency data logging | |
| US9633061B2 (en) | Methods for determining event counts based on time-sampled data | |
| US8949862B2 (en) | Rate of operation progress reporting |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |