CN114546264A - Method, system and computer storage medium for identifying IO model - Google Patents

Method, system and computer storage medium for identifying IO model Download PDF

Info

Publication number
CN114546264A
CN114546264A CN202210076754.XA CN202210076754A CN114546264A CN 114546264 A CN114546264 A CN 114546264A CN 202210076754 A CN202210076754 A CN 202210076754A CN 114546264 A CN114546264 A CN 114546264A
Authority
CN
China
Prior art keywords
model
request
identifying
type
identification
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
CN202210076754.XA
Other languages
Chinese (zh)
Inventor
刘杰
孟祥瑞
安祥文
罗浩
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Suzhou Inspur Intelligent Technology Co Ltd
Original Assignee
Suzhou Inspur Intelligent Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Suzhou Inspur Intelligent Technology Co Ltd filed Critical Suzhou Inspur Intelligent Technology Co Ltd
Priority to CN202210076754.XA priority Critical patent/CN114546264A/en
Publication of CN114546264A publication Critical patent/CN114546264A/en
Withdrawn legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0604—Improving or facilitating administration, e.g. storage management
    • G06F3/0607—Improving or facilitating administration, e.g. storage management by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/061—Improving I/O performance

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention provides a method, a system and a computer storage medium for identifying an IO model, wherein the method comprises the steps of obtaining a processing request of object storage equipment OSD; identifying a read-write request in the processing request and an identifier of the read-write request; counting the request length with the identification; and when the statistical quantity reaches a preset value, judging the type of the IO model according to the statistical request length. The invention takes the object prefix as the unique identification for the OSD processing request, counts the read-write request and the length thereof to obtain the type of the current IO model, realizes the accurate identification of the IO model, is convenient for the use of the subsequent processing flow and improves the storage performance.

Description

Method, system and computer storage medium for identifying IO model
Technical Field
The invention relates to the technical field of storage, in particular to a method and a system for identifying an IO model and a computer storage medium.
Background
With the continuous development of information technology, data is gradually valued as a precious resource, and how to quickly process data resources and obtain expected results becomes one of the key problems of resource-to-asset transition. Data are generated by various activities of people in work and life, useful information can be obtained by collecting the data and analyzing and processing the data, and the conversion from resources to assets is realized, so that the rapid development of big data and high-performance calculation is catalyzed.
Data storage, one of the core elements of data resources, has also been in the period of rapid development. The traditional network storage system adopts a centralized storage server to store all data, the storage server becomes the bottleneck of the system performance, is also the focus of reliability and safety, and cannot meet the requirement of large-scale storage application. The distributed network storage system adopts an expandable system structure, not only improves the reliability, the availability and the access efficiency of the system, but also is easy to expand, thereby being accepted by more and more enterprise units. Distributed storage systems are typically constructed with multiple nodes to provide high performance, mass data storage.
At present, in the use process of a block scene, some volumes are read and written by large IO (IO request larger than 1M) and some volumes are read and written by small IO (IO request smaller than 1M). After the distributed storage is optimized for the large IO and the small IO, the large IO and the small IO cannot be accurately distinguished, and the storage performance is affected.
Disclosure of Invention
The invention provides a method, a system and a computer storage medium for identifying an IO model, which are used for solving the problem that the storage performance is influenced because the large IO and the small IO cannot be accurately distinguished after the large IO and the small IO are optimized in the conventional distributed storage.
In order to achieve the purpose, the invention adopts the following technical scheme:
in a first aspect, the present invention provides a method for identifying an IO model, including the following steps:
s1, acquiring the OSD processing request of the object storage device;
s2, recognizing the read-write request in the processing request and the identification of the read-write request;
s3, counting the request length with the identification;
and S4, judging the type of the IO model according to the counted request length when the counted number reaches a preset value.
Further, the processing request for obtaining the object storage device OSD further comprises:
and setting identification parameters, wherein the identification parameters comprise preset values of the statistical quantity, preset threshold values of the request length, preset proportions, judgment conditions of the IO model and default IO model types.
Further, the identification is a prefix of the object name.
Further, the specific step of determining the type of the IO model according to the counted request length is:
if the number of the counted request lengths is larger than the preset threshold and exceeds the preset proportion, the IO model is of a default type.
Further, the types of the IO models include a large IO model and a small IO model, and the default IO type is the large IO model.
Further, after the type of the IO model is obtained, the current statistical data is deleted, and the steps S1 to S4 are executed again.
A second aspect of the present invention provides a system for identifying an IO model, the system comprising:
the information acquisition unit is used for acquiring the OSD processing request of the object storage device;
the request identification unit is used for identifying the read-write request in the processing request and the identification of the read-write request;
a counting unit for counting the request length with the identifier;
and the analysis and judgment unit is used for judging the type of the IO model according to the counted request length when the counted number reaches a preset value.
Further, the system further comprises a parameter configuration unit, wherein the parameter configuration unit is configured to set identification parameters, and the identification parameters include preset values of statistical quantity, preset thresholds of request length, preset proportions, IO model determination conditions, and default IO model types.
Further, the system further comprises a data clearing unit, and the data clearing unit deletes the current statistical data and calls the information acquisition unit after obtaining the type of the IO model.
A third aspect of the invention provides a computer storage medium having stored thereon computer instructions which, when run on the system, cause the system to perform the steps of the method.
The system for identifying an IO model according to the second aspect of the present invention can implement the methods according to the first aspect and the implementation manners of the first aspect, and achieve the same effects.
The effects provided in the summary of the invention are only the effects of the embodiments, not all of the effects of the invention, and one of the above technical solutions has the following advantages or beneficial effects:
the invention takes the object prefix as the unique identification for the OSD processing request, counts the read-write request and the length thereof to obtain the type of the current IO model, realizes the accurate identification of the IO model, is convenient for the use of the subsequent processing flow and improves the storage performance.
Drawings
In order to more clearly illustrate the embodiments or technical solutions in the prior art of the present invention, the drawings used in the description of the embodiments or prior art will be briefly described below, and it is obvious for those skilled in the art that other drawings can be obtained based on these drawings without creative efforts.
FIG. 1 is a schematic flow diagram of an embodiment of the method of the present invention;
FIG. 2 is a schematic flow chart of one implementation of the method embodiment of the present invention;
fig. 3 is a schematic structural diagram of an embodiment of the system of the present invention.
Detailed Description
In order to clearly explain the technical features of the present invention, the following detailed description of the present invention is provided with reference to the accompanying drawings. The following disclosure provides many different embodiments, or examples, for implementing different features of the invention. To simplify the disclosure of the present invention, the components and arrangements of specific examples are described below. Furthermore, the present invention may repeat reference numerals and/or letters in the various examples. This repetition is for the purpose of simplicity and clarity and does not in itself dictate a relationship between the various embodiments and/or configurations discussed. It should be noted that the components illustrated in the figures are not necessarily drawn to scale. Descriptions of well-known components and processing techniques and procedures are omitted so as to not unnecessarily limit the invention.
As shown in fig. 1, a method for identifying an IO model provided in an embodiment of the present invention includes the following steps:
s1, acquiring the OSD processing request of the object storage device;
s2, recognizing the read-write request in the processing request and the identification of the read-write request;
s3, counting the request length with the identification;
and S4, judging the type of the IO model according to the counted request length when the counted number reaches a preset value.
In step S1, at the entry of the OSD processing request of the target processing device, an OP (operation) request is acquired.
In step S2, in the obtained OP request, the read-write request is identified, and the identifier of the current read-write request is obtained.
In this embodiment, the object name prefix is used as the unique identifier of the volume.
In step S3, the read-write request length with the prefix of the current object name is counted, and the counted number is recorded.
In step S4, the specific determination of the type of the IO model according to the counted request length is:
if the number of the counted request lengths is larger than the preset threshold and exceeds the preset proportion, the IO model is of a default type.
The types of the IO models comprise a large IO model and a small IO model, and the default IO type is the large IO model.
In an implementation manner of the embodiment of the present invention, before the obtaining the processing request of the OSD of the object storage device, the method further includes:
and setting identification parameters, wherein the identification parameters comprise preset values of statistical quantity, preset threshold values of request length, preset proportions, judgment conditions of IO models and default IO model types. The default IO model type is the big IO model.
In the embodiment of the present invention, the preset value of the statistical number is 64, the preset threshold of the request length is 1M, and the preset proportion is 60%, in step S4, when the type of the IO model is determined, if the number of the request lengths greater than 1M exceeds 60%, the model is a large IO model.
In one implementation process of the embodiment of the invention, after the type of the IO model is obtained, the current statistical data is deleted, and the steps S1-S4 are executed again.
As shown in fig. 2, after the OSD is initialized, detecting whether the OP is a read-write request, if so, analyzing the prefix of the object name in the OP, and counting the length of the request by using the prefix of the object name as a key, otherwise, repeating the step of detecting the OP; and judging whether the statistical quantity reaches a preset value 64, if not, repeating the step of detecting the OP, if so, judging the IO model according to the statistical information, emptying the statistical information for statistics again, and repeating the step of detecting the OP.
As shown in fig. 3, an embodiment of the present invention provides a system for identifying an IO model, where the system includes an information acquisition unit 1, a request identification unit 2, a statistical counting unit 3, an analysis and determination unit 5, a parameter configuration unit 5, and a data clearing unit 6.
The information acquisition unit 1 is used for acquiring a processing request of an object storage device OSD; the request identification unit 2 is configured to identify a read-write request in the processing request and an identifier of the read-write request; the counting unit 3 is used for counting the request length with the identification; and the analysis and judgment unit 4 is used for judging the type of the IO model according to the counted request length when the counted number reaches a preset value.
The parameter configuration unit 5 is configured to set identification parameters, where the identification parameters include preset values of statistical quantities, preset thresholds of request lengths, preset proportions, IO model determination conditions, and default IO model types.
And after the data clearing unit 6 obtains the type of the IO model, deleting the current statistical data and calling the information acquisition unit.
In addition, the embodiment of the invention also provides an interface for inquiring the IO model type for subsequent processing flows.
The embodiment of the invention also provides a computer storage medium, wherein a computer instruction is stored in the computer storage medium, and when the computer instruction runs on the system, the system executes the steps of the method.
Although the embodiments of the present invention have been described with reference to the accompanying drawings, it is not intended to limit the scope of the present invention, and it should be understood by those skilled in the art that various modifications and variations can be made without inventive efforts by those skilled in the art based on the technical solution of the present invention.

Claims (10)

1. A method of identifying an IO model, the method comprising the steps of:
s1, acquiring the OSD processing request of the object storage device;
s2, recognizing the read-write request in the processing request and the identification of the read-write request;
s3, counting the request length with the identification;
and S4, judging the type of the IO model according to the counted request length when the counted number reaches a preset value.
2. The method for identifying IO models as claimed in claim 1, wherein the processing request for obtaining the OSD of the object storage device further comprises:
and setting identification parameters, wherein the identification parameters comprise preset values of the statistical quantity, preset threshold values of the request length, preset proportions, judgment conditions of the IO model and default IO model types.
3. The method of identifying an IO model as claimed in claim 1, wherein the identifier is a prefix of an object name.
4. The method for identifying IO models according to claim 1, wherein the determining the type of IO model according to the counted request length is specifically:
if the number of the counted request lengths is larger than the preset threshold and exceeds the preset proportion, the IO model is of a default type.
5. The method of identifying an IO model as claimed in claim 4, wherein the types of the IO model include a large IO model and a small IO model, and the default IO type is the large IO model.
6. The method for identifying IO models as claimed in claim 1, wherein after obtaining the type of IO model, the current statistics are deleted and the steps S1-S4 are re-executed.
7. A system for identifying an IO model, the system comprising:
the information acquisition unit is used for acquiring the OSD processing request of the object storage device;
the request identification unit is used for identifying the read-write request in the processing request and the identification of the read-write request;
a counting unit for counting the request length with the identifier;
and the analysis and judgment unit is used for judging the type of the IO model according to the counted request length when the counted number reaches a preset value.
8. The system for identifying IO models as claimed in claim 7, further comprising a parameter configuration unit, wherein the parameter configuration unit is configured to set identification parameters, and the identification parameters include a preset value of the statistical number, a preset threshold value of the request length, a preset ratio, a determination condition of the IO model, and a default IO model type.
9. The IO model identification system according to claim 7, further comprising a data removal unit, wherein the data removal unit deletes current statistical data and invokes the information collection unit after obtaining the type of the IO model.
10. A computer storage medium having computer instructions stored thereon, which, when run on a system according to any of claims 7-9, cause the system to perform the steps of the method according to any of claims 1-6.
CN202210076754.XA 2022-01-21 2022-01-21 Method, system and computer storage medium for identifying IO model Withdrawn CN114546264A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210076754.XA CN114546264A (en) 2022-01-21 2022-01-21 Method, system and computer storage medium for identifying IO model

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210076754.XA CN114546264A (en) 2022-01-21 2022-01-21 Method, system and computer storage medium for identifying IO model

Publications (1)

Publication Number Publication Date
CN114546264A true CN114546264A (en) 2022-05-27

Family

ID=81671561

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210076754.XA Withdrawn CN114546264A (en) 2022-01-21 2022-01-21 Method, system and computer storage medium for identifying IO model

Country Status (1)

Country Link
CN (1) CN114546264A (en)

Similar Documents

Publication Publication Date Title
EP3425524A1 (en) Cloud platform-based client application data calculation method and device
CN111563014B (en) Interface service performance test method, device, equipment and storage medium
CN110147470B (en) Cross-machine-room data comparison system and method
WO2021093365A1 (en) Gpu video memory management control method and related device
CN111078513A (en) Log processing method, device, equipment, storage medium and log alarm system
CN112749013B (en) Thread load detection method and device, electronic equipment and storage medium
CN112269534A (en) Data reading method, device and equipment and computer readable storage medium
CN113032157A (en) Automatic intelligent server capacity expansion and reduction method and system
CN112235262A (en) Message analysis method and device, electronic equipment and computer readable storage medium
CN110781166B (en) Data management method and device in storage system
CN107291743B (en) Data calling method and device
CN111049684B (en) Data analysis method, device, equipment and storage medium
CN114546264A (en) Method, system and computer storage medium for identifying IO model
CN115878914A (en) Domain name matching method, device, equipment and storage medium
CN110232007A (en) A kind of electric power enterprise information service monitoring method based on APM technology
CN111324583A (en) Method and device for classifying service logs
CN113297148B (en) Method, device and equipment for collecting service log data and readable storage medium
CN115016890A (en) Virtual machine resource allocation method and device, electronic equipment and storage medium
CN114741266A (en) Page white screen duration determining method and device, electronic equipment and storage medium
CN110191026B (en) Distributed service link monitoring method and device
CN113645585A (en) Internet of things data acquisition method
CN112860469A (en) Method, device, equipment and storage medium for collecting information of katon log
CN114115751B (en) Automatic operation time consumption monitoring method and system for large-scale distributed storage
CN118138483B (en) Data stream processing method, device, computer equipment and readable storage medium
CN117539840B (en) Log acquisition method, device, equipment and medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WW01 Invention patent application withdrawn after publication

Application publication date: 20220527

WW01 Invention patent application withdrawn after publication