CN107943416A - A kind of mixing storage system for improving data loading speed and mixing storage method - Google Patents
A kind of mixing storage system for improving data loading speed and mixing storage method Download PDFInfo
- Publication number
- CN107943416A CN107943416A CN201711146834.3A CN201711146834A CN107943416A CN 107943416 A CN107943416 A CN 107943416A CN 201711146834 A CN201711146834 A CN 201711146834A CN 107943416 A CN107943416 A CN 107943416A
- Authority
- CN
- China
- Prior art keywords
- storage
- data
- rotating speed
- performance
- host computer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
- G06F3/0613—Improving I/O performance in relation to throughput
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
- G06F3/0647—Migration mechanisms
- G06F3/0649—Lifecycle management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0683—Plurality of storage devices
- G06F3/0685—Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Signal Processing For Digital Recording And Reproducing (AREA)
Abstract
A kind of mixing storage system for improving data loading speed and mixing storage method, belong to storage system field, including：Hard disk storage units, connect host computer, for using multi gear rotating speed output storage data；High-performance storage unit, connects hard disk storage units and host computer；Statistic unit, connects hard disk storage units and high-performance storage unit；Control unit, connection hard disk storage units, high-performance storage unit and statistic unit；The cycle each transmitted includes self study stage, practice stage and calibration phase three phases.Beneficial effects of the present invention：Use the hard disk storage units of adjustable rotating speed instead, different phase in a transmission cycle uses different transmission method and control method, solve the problems, such as to cause preextraction result inaccurate because the self-learning algorithm in user behavior change or statistic unit is inaccurate in existing mixing storage system, effectively improve data loading speed.
Description
Technical field
The present invention relates to memory system technologies field, more particularly to a kind of mixing storage system for improving data loading speed
And mixing storage method.
Background technology
High-revolving hard disk drive (Hard Disk Drive, HDD) is used in many storage systems at present, is made
It can be remained operating under high performance state.However, time of the system I/O real works under high speed throughput rate state is only
It is small part.There is analysis to show, in existing mass-storage system, system I/O in 99% time
33% of actual throughput rate less than its highest throughput rate, and the actual throughput rate of system I/O is not in 70% time
To the 5% of its highest throughput rate, this has resulted in the waste of high rotating speed HDD a big chunk performances, also so that the power consumption of HDD
It is excessive, considerably increase use cost.That is HDD is that need not keep high-revolving in most times.
In order to reduce the waste of power consumption, the HDD using the slow-speed of revolution is a kind of solution, but on condition that does not influence storage system
The performance of system, or even the performance of the system can be further lifted on the contrary.Therefore, it is quick to occur a kind of I/O in the prior art
The mixing storage system of sense, as shown in Figure 1, the system mainly includes high-performance storage unit, slow-speed of revolution HDD units, statistics list
Member and four parts of control unit.Statistic unit has self-learning function, that is, can be in a regular time section
Inside record the operating condition of each program, and statistical analysis is carried out according to the behavior of the operating condition of program to user, and then
To different user different application in data transmission characteristics in different time periods, then control unit can be according to these data transfers
Feature, be multiplexed those memory bandwidth utilization rates low time needs what is be transmitted with high bandwidth and high I/O speed by host computer
Data are extracted in high-performance storage unit from slow-speed of revolution HDD units in advance, when host computer needs these data, it is possible to straight
It is connected in high-performance storage unit and reads without by slow-speed of revolution HDD units, just because of the presence of preextraction strategy, because
This mixing storage system can use the HDD of the slow-speed of revolution, and the HDD of the slow-speed of revolution is enough to tackle host computer and obtains other and (be not required to
High bandwidth and high I/O speed is utilized to be transmitted) operations of data.Therefore, which is just reached using the HDD of the slow-speed of revolution
Reduce power consumption purpose, and due to high-performance storage unit read or write speed than HDD (even high rotating speed) much faster, institute
With the system, performance can also have been lifted for all storage systems using high rotating speed HDD.
However, user is not the different application journey of unalterable, different user to the usage behavior of application program
Data transmission characteristics of the sequence in the different periods are likely to change, and self study used in statistic unit is calculated
Error also occurs in method sometimes, so that it cannot the data characteristics of user is obtained exactly, so as to cause self study result to be not allowed
Really, these all can bring influence to the result of preextraction.If by certain time (this time is more than the time of self study)
Afterwards, the behavior of user is changed, or there are certain error for self-learning algorithm, then will result in host computer needs
Partial data not by preextraction into high-performance storage unit, i.e., this partial data is lost in high-performance storage unit
(Miss), HDD thus when host computer needs this partial data must be accessed again, and is deposited in mixing as shown in Figure 1
HDD is the slow-speed of revolution in storage system, and host computer, which will obtain these data, will spend the long time so that system performance drops
It is low, and power consumption is likely to increase.If the data that this part is lost in high-performance memory are relatively more, then this
Mixing storage system with self study and preextraction technology cannot not only reduce power consumption, lifting compared to traditional storage system
Performance, can also increase power consumption, reduce performance, effect is run counter to desire on the contrary.
Here a parameter is first defined：Lose ratio data (miss_rate_ratio), i.e., should be by preextraction to high property
The data not being extracted actually in energy memory account for the ratio of all data that should be extracted.It is assumed that all should be carried in advance
The data taken represent should actually not had by preextraction into high-performance memory by the data of preextraction B tables with A
Show, then miss_rate_ratio is：
Assuming that in the case of conventional store framework (include the use of the storage system of single high rotating speed HDD as shown in Figure 2 or
The mixing storage system of addition caching (Cache) is as shown in Figure 3 on high-revolving HDD), when host computer needs data A, this
When HDD rotating speed should be high rotating speed X, it is assumed that be at this time Δ T1 from the time of HDD transmission data A to host computer, then Δ T1 is：
Wherein, IOPSX is I/O transmission rates when HDD rotating speeds are X.
The mixing storage architecture proposed according to Fig. 1, when user changes the usage behavior of application program, either
Due to the existing error of self-learning algorithm used in statistic unit in itself, the rotating speed of HDD is slow-speed of revolution Y at this time, then this
When from the time of HDD transmission data B (A*miss_rate_ratio) to host computer be Δ T2, then Δ T2 is：
Therefore, if do not caused damages to system performance, then Δ T2 is necessarily less than Δ T1, that is to say, that for Fig. 1 institutes
For the mixing storage architecture shown, on the premise of not causing damages to system performance, there is an admissible miss_ of maximum
Rate_ratio, it is：
When user behavior changes or since the inaccuracy of self-learning algorithm causes miss_rate_ratio ratios
When miss_rate_ratiomax is small, the mixing storage system shown in Fig. 1 can't bring infringement to system performance at this time, but work as
User behavior change or
When causing miss_rate_ratio bigger than miss_rate_ratiomax due to the inaccuracy of self-learning algorithm,
Infringement will be brought to the performance of system.From formula (4) we can see that corresponding different rotating speed, system can allow most
Big miss_rate_ratio is different.
In conclusion it is likely to occur in existing mixing storage system because of the self-study in user behavior change or statistic unit
Practise algorithm inaccurate and cause preextraction result inaccurate.
The content of the invention
For it is existing in the prior art mixing storage system in be likely to occur because user behavior change or statistic unit
In self-learning algorithm it is inaccurate and the problem of cause preextraction result inaccurate, the present invention provides one kind to improve system data
The mixing storage system and method for loading speed, it is intended to retain its low-power consumption and high performance excellent while solving the above problems
Gesture.The present invention adopts the following technical scheme that：
A kind of mixing storage system for improving data loading speed, the mixing storage system connection host computer are described mixed
Storage system is closed to be used to by the cycle of a preset time period export the storage data to prestore to the host computer, the cycle bag
Self study stage, time phase and the calibration phase set gradually is included, the storage data include the first storage data and second
Store data, it is described first storage data transmission broadband and I/O speed less than described second storage data transmission bandwidth and
I/O speed, the working status of the mixing storage system include idle condition and busy state, the system during idle condition
The memory bandwidth utilization rate of system when memory bandwidth utilization rate is less than the busy state；The mixing storage system includes：
Hard disk storage units, the hard disk storage units connect the host computer, for using described in the output of multi gear rotating speed
Data are stored, the multi gear rotating speed includes the first rotating speed and the second rotating speed more than first rotating speed；
High-performance storage unit, the high-performance storage unit connect the hard disk storage units and the host computer, use
In obtaining the storage data from the hard disk storage units and being exported using the 3rd rotating speed, the 3rd rotating speed is more than described more
Shelves rotating speed；
Statistic unit, the statistic unit connects the hard disk storage units and the high-performance storage unit, for leading to
Cross self study obtain it is described mixing storage system in the first data transmission characteristic in the self study stage, in the practice stage
The second Data Transmission Feature and the 3rd Data Transmission Feature in the calibration phase；
Control unit, described control unit connect hard disk storage units, the high-performance storage unit and the system
Unit is counted, for exporting institute using first rotating speed in an idle state in hard disk storage units described in self study stage control
The first storage data are stated to the host computer, and using second rotating speed output the second storage data under busy state
To the host computer, and for controlling the statistic unit to obtain the first data transmission characteristic；And
For the practice stage according to the first data transmission Characteristics Control hard disk storage units in idle condition
It is lower that data are stored to the host computer using first rotating speed output described first and export the second storage data to institute
High-performance storage unit is stated, and controls the high-performance storage unit to export second storage received under busy state
Data to the host computer then controls the hard disk storage units to export the high-performance storage list using first rotating speed
The second storage data that member does not receive are used to control the statistic unit to obtain second data to the host computer
Transmission characteristic；And
For controlling the hard disk storage units in idle condition according to second Data Transmission Feature in calibration phase
It is lower that data are stored to the host computer using first rotating speed output described first and export the second storage data to institute
High-performance storage unit is stated, and controls the high-performance storage unit to export second storage received under busy state
Data to the host computer then controls the hard disk storage units to export the high-performance storage list using second rotating speed
The second storage data that member does not receive are used to control the statistic unit to obtain the 3rd data to the host computer
Transmission characteristic；
3rd output characteristics is applied to the practice stage as first data transmission characteristic.
Preferably, the hard disk storage units are to be made of using large-scale inactive disk array technology multiple disks
Disk array.
Preferably, first rotating speed is the minimum speed of the disk array.
Preferably, the high-performance storage unit is hard disk drive；Or
The high-performance storage unit is solid state hard disc；Or
The high-performance storage unit is phase transition storage；Or
The high-performance storage unit is resistive random access memory；Or
The high-performance storage unit is ferroelectric random access memory.
Preferably, the first data transmission characteristic for multiple users and multiple applies journey for the mixing storage system
Data transfer rule of the sequence in multiple periods；
Second Data Transmission Feature is for the mixing storage system for multiple users and multiple application programs more
The data transfer rule of a period；
3rd Data Transmission Feature is for the mixing storage system for multiple users and multiple application programs more
The data transfer rule of a period.
Preferably, the data transfer rule includes frequency, the transmission speed and memory bandwidth of system I/O of reading and writing data
Utilization rate.
A kind of mixing storage method for improving data loading speed, using above-mentioned mixing storage system, including：
Step S1, the hard disk storage units are controlled to use in an idle state in self study stage, described control unit
First rotating speed output the first storage data are defeated using second rotating speed to the host computer, and under busy state
Go out the second storage data to the host computer, and described control unit controls the statistic unit to obtain institute by self study
State first data transmission characteristic of the mixing storage system in the self study stage；
Step S2, deposited in practice stage, described control unit hard disk according to the first data transmission Characteristics Control
Storage unit is in an idle state using first rotating speed output, the first storage data to described in the host computer and output
Second storage data control the high-performance storage unit output to connect to the high-performance storage unit under busy state
The second storage data to the host computer received then controls the hard disk storage units to be exported using first rotating speed
The second storage data that the high-performance storage unit does not receive are to the host computer, and described in described control unit control
Statistic unit obtains second Data Transmission Feature of the mixing storage system in the practice stage by self study；
Step S3, the hard disk is controlled to deposit according to second Data Transmission Feature in calibration phase, described control unit
Storage unit is in an idle state using first rotating speed output, the first storage data to described in the host computer and output
Second storage data control the high-performance storage unit output to connect to the high-performance storage unit under busy state
The second storage data to the host computer received then controls the hard disk storage units to be exported using second rotating speed
The second storage data that the high-performance storage unit does not receive are to the host computer, and described in described control unit control
Statistic unit obtains threeth Data Transmission Feature of the mixing storage system in the calibration phase by self study；
Step S4, described control unit judges whether to receive stop signal：
If the determination result is YES, the mixing storage system is out of service, with backed off after random；
If judging result is no, described control unit passes the 3rd Data Transmission Feature as first data
Defeated characteristic, then goes to step S2.
Preferably, in the step S2, the second storage data are being delivered to the process of the high-performance storage unit
It is middle that there are one first Loss Rate；
In the step S3, there are one during the high-performance storage unit is delivered to for the second storage data
Second Loss Rate；
First Loss Rate is more than second Loss Rate.
Beneficial effects of the present invention：Use the hard disk storage units of adjustable rotating speed instead, the not same order in a transmission cycle
Transmission methods different Duan Caiyong and control method, solve in existing mixing storage system because user behavior changes or counts
Self-learning algorithm in unit is inaccurate and the problem of cause preextraction result inaccurate, effectively improve data loading speed.
Brief description of the drawings
Fig. 1 is a kind of mixing memory system architecture figure of I/O sensitivities；
Fig. 2 is the storage system in the case of conventional store framework using single high rotating speed HDD；
Fig. 3 is the mixing storage system for adding Cache in the case of conventional store framework on high-revolving HDD；
Fig. 4 is in a preferred embodiment of the present invention, mixes memory system architecture figure；
Fig. 5 is in a preferred embodiment of the present invention, mixes storage method flow chart；
Fig. 6 is the time shaft for the mixing storage method that data loading speed is improved in a preferred embodiment of the present invention
One of；
Fig. 7 is the time shaft for the mixing storage method that data loading speed is improved in a preferred embodiment of the present invention
Two；
Detailed operation when Fig. 8 is in the prior art, mixes in one section of course of work of storage system using conventional art is shown
It is intended to；
Fig. 9 is in a preferred embodiment of the present invention, mixes in one section of course of work of storage system and utilizes institute of the present invention
State detailed operation schematic diagram during technology.
Embodiment
It should be noted that in the case where there is no conflict, following technical proposals, can be mutually combined between technical characteristic.
The embodiment of the present invention is further described below in conjunction with the accompanying drawings：
As shown in figure 4, a kind of mixing storage system for improving data loading speed, the mixing storage system connection is upper
Machine, the mixing storage system are used to by the cycle of a preset time period export the storage data to prestore to the host computer,
The cycle includes self study stage, time phase and the calibration phase set gradually, and the storage data include the first storage
Data and the second storage data, the transmission broadband of the first storage data and I/O speed are less than the described second storage data
Transmission bandwidth and I/O speed, the working status of the mixing storage system include idle condition and busy state, the free time shape
The memory bandwidth utilization rate of system when the memory bandwidth utilization rate of system is less than the busy state during state；The mixing storage system
System includes：
Hard disk storage units, the hard disk storage units connect the host computer, for using described in the output of multi gear rotating speed
Data are stored, the multi gear rotating speed includes the first rotating speed and the second rotating speed more than first rotating speed；
High-performance storage unit, the high-performance storage unit connect the hard disk storage units and the host computer, use
In obtaining the storage data from the hard disk storage units and being exported using the 3rd rotating speed, the 3rd rotating speed is more than described more
Shelves rotating speed；
Statistic unit, the statistic unit connects the hard disk storage units and the high-performance storage unit, for leading to
Cross self study obtain it is described mixing storage system in the first data transmission characteristic in the self study stage, in the practice stage
The second Data Transmission Feature and the 3rd Data Transmission Feature in the calibration phase；
Control unit, described control unit connect hard disk storage units, the high-performance storage unit and the system
Unit is counted, for exporting institute using first rotating speed in an idle state in hard disk storage units described in self study stage control
The first storage data are stated to the host computer, and using second rotating speed output the second storage data under busy state
To the host computer, and for controlling the statistic unit to obtain the first data transmission characteristic；And
For the practice stage according to the first data transmission Characteristics Control hard disk storage units in idle condition
It is lower that data are stored to the host computer using first rotating speed output described first and export the second storage data to institute
High-performance storage unit is stated, and controls the high-performance storage unit to export second storage received under busy state
Data to the host computer then controls the hard disk storage units to export the high-performance storage list using first rotating speed
The second storage data that member does not receive are used to control the statistic unit to obtain second data to the host computer
Transmission characteristic；And
For controlling the hard disk storage units in idle condition according to second Data Transmission Feature in calibration phase
It is lower that data are stored to the host computer using first rotating speed output described first and export the second storage data to institute
High-performance storage unit is stated, and controls the high-performance storage unit to export second storage received under busy state
Data to the host computer then controls the hard disk storage units to export the high-performance storage list using second rotating speed
The second storage data that member does not receive are used to control the statistic unit to obtain the 3rd data to the host computer
Transmission characteristic；
3rd output characteristics is applied to the practice stage as first data transmission characteristic.
In the present embodiment, the hard disk storage units of adjustable rotating speed are used instead, the different phase in a transmission cycle is adopted
With different transmission method and control method, solve in existing mixing storage system because user behavior changes or statistic unit
In self-learning algorithm it is inaccurate and the problem of cause preextraction result inaccurate, effectively improve data loading speed.
With continued reference to Fig. 4, in preferred embodiments of the present invention, the hard disk storage units are the extensive inactive magnetic of application
The disk array being made of multiple disks of disk array technology.
With continued reference to Fig. 4, in preferred embodiments of the present invention, first rotating speed is minimum turn of the disk array
Speed.
With continued reference to Fig. 4, in preferred embodiments of the present invention, the high-performance storage unit is hard disk drive (Hard
Disk Drive, HDD)；Or
The high-performance storage unit is solid state hard disc (Solid State Drives, SSD)；Or
The high-performance storage unit is phase transition storage (Phase Change Memory, PCM)；Or
The high-performance storage unit is resistive random access memory (Resistor RAM, ReRAM)；Or
The high-performance storage unit is ferroelectric random access memory (Ferroelectric RAM, FeRAM)；Or
Other high-performance memories.
With continued reference to Fig. 4, in preferred embodiments of the present invention, the first data transmission characteristic is the mixing storage system
System is directed to the data transfer rule of multiple users and multiple application programs in multiple periods；
Second Data Transmission Feature is for the mixing storage system for multiple users and multiple application programs more
The data transfer rule of a period；
3rd Data Transmission Feature is for the mixing storage system for multiple users and multiple application programs more
The data transfer rule of a period.
With continued reference to Fig. 4, in preferred embodiments of the present invention, the data transfer rule include reading and writing data frequency,
The transmission speed and memory bandwidth utilization rate of system I/O.
As shown in figures 4-9, a kind of mixing storage method for improving data loading speed, system is stored using above-mentioned mixing
System, including：
Step S1, the first rotating speed is used in an idle state in self study stage, control unit control hard disk storage units
The storage data of output first to host computer, and under busy state using the second rotating speed output the second storage data to host computer,
And control unit control statistic unit obtains first data transmission spy of the mixing storage system in the self study stage by self study
Property；
Step S2, in the practice stage, control unit is according to first data transmission Characteristics Control hard disk storage units in the free time
Using the first storage data of the first rotating speed output to host computer and the storage data of output second to high performance memory location under state,
And the second storage data to the host computer for controlling the output of high-performance storage unit to receive under busy state then controls hard disk
The second storage data that storage unit is not received using the first rotating speed output high-performance storage unit are to host computer, and control unit
Statistic unit is controlled to obtain second Data Transmission Feature of the mixing storage system in the practice stage by self study；
Step S3, hard disk storage units are controlled in the free time according to the second Data Transmission Feature in calibration phase, control unit
Using the first storage data of the first rotating speed output to host computer and the storage data of output second to high performance memory location under state,
And the second storage data to the host computer for controlling the output of high-performance storage unit to receive under busy state then controls hard disk
The second storage data that storage unit is not received using the second rotating speed output high-performance storage unit are to host computer, and control unit
Statistic unit is controlled to obtain threeth Data Transmission Feature of the mixing storage system in calibration phase by self study；
Step S4, control unit judges whether to receive stop signal：
If the determination result is YES, it is out of service to mix storage system, with backed off after random；
If judging result is no, control unit is using the 3rd Data Transmission Feature as first data transmission characteristic, then
Go to step S2.
With continued reference to Fig. 4-9, in preferred embodiments of the present invention, in the step S2, the second storage data are defeated
There are one first Loss Rate during sending to the high-performance storage unit；
In the step S3, there are one during the high-performance storage unit is delivered to for the second storage data
Second Loss Rate；
First Loss Rate is more than second Loss Rate.
Embodiment one：
Hard disk storage units are the HDD of adjustable rotating speed.
The present invention uses the HDD of adjustable rotating speed instead, it is assumed that the alternative rotating speeds of the HDD of the adjustable rotating speed have
Slow-speed of revolution R1 (the first rotating speed) and two grades of high rotating speed R2 (the second rotating speed)；Certainly, actual capabilities have more multi gear rotating speed, specific rotating speed
The number of gear number is determined by HDD.Control unit can make corresponding change according to the working status of system to the rotating speed of HDD, such as
By the adjustment of rotational speed of HDD it is high rotating speed R2 at busy state (Active), by the adjustment of rotational speed of HDD during idle condition (Idle)
For slow-speed of revolution R1.It will be apparent that HDD rotating speeds are higher, its performance is better, but power consumption is also higher.
The present invention still uses preextraction strategy, the number obtained afterwards by self study according to statistic unit by control unit
According to transmission feature (such as first data transmission feature, the second data transmission characteristics, the 3rd data transmission characteristics), in memory bandwidth profit
With rate than the second data for needing to be transmitted with high bandwidth and high I/O speed by host computer in the relatively low time in advance from HDD
In extract in the middle of high-performance memory.The transmission feature of data includes but is not limited to the frequency of reading and writing data, system I/O
Transmission speed and memory bandwidth utilization rate etc..During statistic unit carries out self study, the working status of HDD is from it
He influences factor, its rotating speed is still adjusted according to the working status of system.Such as shown in fig. 6, T0-T1 is self study rank
Section, when a length of X, self study stage HDD rotating speeds according to its busy extent switch；T1-T2 is the practice stage, when a length of Y, HDD is solid
It is set to slow-speed of revolution R1, using preextraction strategy, during which there is a situation where that user behavior changes or self-learning algorithm has error；T2-
T3 is calibration phase, when a length of X, open new round self study, HDD rotating speeds switch according to its busy extent, still using pre-
Extraction strategy.Statistic unit proceeds by self study from the T0 moment, and by duration X, (X represents one time for being not fixed length
Section) after, it is assumed that arrive the T1 moment and terminated self study and different user different application has been obtained according to the result of self study
In first data transmission feature in different time periods.In T0 to T1 this periods, the rotating speed of HDD can't be because of statistic unit
The addition of self study and change, its rotating speed is adjusted still according to working state of system, if system is at busy state,
The rotating speed of HDD will be brought to high rotating speed R2, if system is in idle condition, the rotating speed of HDD will be brought to slow-speed of revolution R1.
After completing self study in T1 moment statistic unit, control unit can be according to the first data transmission of statistic unit acquisition
Feature the memory bandwidth utilization rate relatively low time by data preextraction into high-performance storage unit, host computer needs these
Can directly it be read during data from high-performance memory, HDD avoids the need for higher performance at this time, therefore, the T1 moment
HDD can be fixed to slow-speed of revolution R1 afterwards., should be by the rotating speed of HDD specifically, if HDD has that multi gear rotating speed is adjustable in practice
It is adjusted to not influence system worked well performance and alap rotating speed is to be reduced as far as power consumption.
If a period of time Y (same, Y also illustrates that one time for being not fixed length) is crossed since the T1 moment, at this
User changes the usage behavior of application program in the section time, or since self study used in statistic unit is calculated
Method existing error in itself, so that use row of the user to application program can accurately not reflected by causing the result of self study
For, that is to say, that the data transmission characteristics and mismatch that user obtains the usage behavior of application program with self-learning algorithm.This
Also mean that high bandwidth and high I/O speed need not be used by containing a part by the data in preextraction to high-performance memory
The data being transmitted, at the same also some should be extracted on the contrary by the data of preextraction high-performance memory work as
In.
Assuming that after certain time, the behavior of user is changed or the inaccuracy due to self-learning algorithm
Property cause miss_rate_ratio than under slow-speed of revolution R1 receptible maximum miss_rate_ratiomax it is also big
When, it will cause damage at this time to the performance of system, because HDD at this time has been fixed to slow-speed of revolution R1, and host computer is also
It must go to read the data for needing to be transmitted using high bandwidth and high I/O speed from the HDD of this slow-speed of revolution R1, taking must
So can be long.
For this reason, since the T2 moment, HDD is no longer fixed to slow-speed of revolution R1, its rotating speed is again according to the work shape of system
State switches over, i.e., when system is in busy state, the rotating speed for adjusting HDD is high rotating speed R2, when system is in idle condition
When, the rotating speed for adjusting HDD is slow-speed of revolution R1.
In addition, since the T2 moment, after discovery user behavior changes or self-learning algorithm has error, statistics is single
The self study that member starts a new round learns user's to obtain the behavioural habits of user or the improved self-learning algorithm of utilization
Behavioural habits are used to obtain newest most accurate 3rd data transmission characteristics.At the same time since the T2 moment, control unit still needs
To be carried in advance according to the second data transmission characteristics multiplexing memory bandwidth utilization rate that self study before obtains relatively low time
Extract operation.Because while user behavior to cause to be deposited to high-performance by preextraction there occurs either self-learning algorithm inaccuracy is changed
Data in reservoir contain the data that a part need not be transmitted with high bandwidth and high I/O speed, while also have one
Point should be extracted on the contrary among high-performance memory by the data of preextraction, but preextraction result still have it is certain
Hit rate, i.e. control unit according to inaccurate self study result come perform preextraction operation still can be a part of needs height
The data preextraction of memory bandwidth utilization rate and high I/O transmission rates is into high-performance storage unit.That is due to being carried
Getting the data in high-performance memory, some meets the requirements, so host computer also can only be from high property when needing data
It can be obtained in memory when the part in the data needed for subtask, the data of required remainder still must be from HDD
Go to read.Since HDD at this time is no longer fixed as slow-speed of revolution R1, in order to not cause damage to system performance, work as host computer
The rotating speed of HDD just is heightened (such as being adjusted to high rotating speed R2) when reading that remaining partial data from HDD to read to reduce data
The time gone out.Use the advantages of still using preextraction strategy under our labors after the T2 moment below, although at this time
The result of preextraction may be inaccurate.
It is assumed that the total amount of data that a certain moment after the T2 moment needs to transmit is DATA, preextraction strategy is not being used
In the case of, the rotating speed of HDD should be high rotating speed R2 at this time, then from HDD transmit DATA to the host computer required time be Δ T1,
So Δ T1 is：
It is assumed that since the behavior of user is changed or since data caused by the inaccuracy of self-learning algorithm are lost
Mistake ratio is miss_rate_ratio, then the hit rate hit_ratio of preextraction result is：
Hit_ratio=1-miss_rate_ratio (6)
So in the case of using preextraction strategy, control unit performs preextraction according to inaccurate self study result
Operation still can need the data preextraction with high memory bandwidth utilization rate and high I/O transmission rates to be deposited to high-performance a part
In storage unit, this partial data amount DATA_1 is：
DATA_1=DATA* (1-miss_rate_ratio) (7)
So in the case of using preextraction strategy, the rotating speed of HDD is also high rotating speed R2 at this time, then is transmitted from HDD
Lose data (DATA-DATA_1) to the host computer required time be Δ T2：
From formula (5) and formula (8) as can be seen that Δ T2 will be significantly less than Δ T1, that is to say, that adopted after the T2 moment
With performance higher of the preextraction strategy than not using preextraction strategy.
Assuming that again after X after a while, if being the T3 moment at this time, statistic unit completes the self study of a new round,
So the rotating speed of HDD can be fixed as slow-speed of revolution R1 again, at the same update at the T3 moment self study as a result, control unit according to
The usage behavior and data transmission characteristics of the result renewal user of a newest self study, still in those memory bandwidth utilization rates
The low time is by data preextraction into high-performance memory.Hereafter, when host computer needs data, those with high bandwidth and
The data that high I/O speed is transmitted just are read from high-performance memory, and it is remaining those only need low bandwidth and low I/O
The data of speed rates are just obtained from HDD.Simply, also it can be understood that for the T3 moment equivalent to the T1 moment has been returned to, so
Afterwards it is ensuing operation just as it is above-mentioned it is described as always circulating repetition go down.In this way, complete new round self study and carry in advance
Mixing storage system after extract operation just overcomes problem present in background technology, and performance is improved again, is also reduced
Power consumption.
Further, it is assumed that HDD units have used MAID (Massive Array in mixing storage system shown in Fig. 4
Of Idle Disks, large-scale inactive disk array) technology, and the rotating speed of HDD is under busy state (active)
15000rpm, under idle condition (idle), the rotating speed of HDD is 7200rpm.
As shown in fig. 7, T0-T1 is the self study stage, when a length of X, self study stage HDD rotating speeds cut according to its busy extent
Change, using MAID technologies；T1-T2 is the practice stage, when a length of Y, HDD is fixed as 7200rpm, using preextraction strategy, during which
There is a situation where that user behavior changes or self-learning algorithm has error；T2-T3 is calibration phase, when a length of X, open a new round from
Study, using MAID technologies (being different from tradition MAID), HDD rotating speeds switch according to its busy extent, still using preextraction
Strategy.Statistic unit proceeds by self study at the T0 moment, after one section of duration X self study terminate.As previously described, exist
In this X period, influence that the rotating speed of HDD operates from self study, but voluntarily controlled by system.Herein, because we
The HDD using MAID technologies is used, its rotating speed is adjusted according to MAID technologies, therefore when self study is performed, when
When HDD is in active states, the rotating speed of HDD is 15000rpm, and when HDD is in idle states, the rotating speed of HDD is
7200rpm.After self study, statistic unit has obtained different user different application when different by statistical analysis
Between section data transmission characteristics, then at the T1 moment, control unit can stored according to this data transmission characteristics
The bandwidth availability ratio relatively low time, preextraction was into high-performance storage unit from HDD by data, so that the rotating speed of HDD can be with
It is lowered, the tachometer value that can be specifically reduced to is determined by the permitted minimum speed of MAID technologies, for example we make herein
For HDD under active states, the rotating speed of HDD is 15000rpm, and under idle states, the rotating speed of HDD is 7200rpm.Cause
This is this using under the HDD of MAID technologies, and since the T1 moment, the rotating speed of HDD is just fixed as 7200rpm.
Assuming that system, during a period of time Y is run, the behavior of user is changed or since self study is calculated
The inaccuracy of method causes miss_rate_ratio bigger, then after it have updated self-learning algorithm, since the T2 moment
Just need the self study of a progress new round.Meanwhile in order to solve because miss_rate_ratio is excessive and so that employing preextraction
The problem of causing damage after technology to system performance, while new round self study is carried out, the rotating speed of HDD is no longer fixed
For 7200rpm, but according to rotation speed operation as defined in MAID.Implementation and traditional MAID technologies used here as MAID are not
Together, since the T2 moment, although HDD is still the MAID technologies used, pre- carry is added on the basis of MAID here
The technology taken, i.e., since the T2 moment, at memory bandwidth utilization rate not high (HDD is in idle states), still according to before
Self study result preextraction data, although the data of preextraction are inaccurate, it is still comprising certain accuracy rate,
That is current preextraction operation still can be the useful data preextraction of a part of user into high-performance storage unit.
For from the T2 moment to the concrete condition during the T3 moment, the present invention, that is, by preextraction technology and MAID skills
Art is combined, and is contrasted using there is obvious advantage for traditional MAID technologies, while during T1 to the T2 moment, HDD is solid
It is fixed and be low rotating speed, thus method proposed by the present invention can also reduce it is reliable caused by rotating speed frequent switching in MAID
Sex chromosome mosaicism.
When using traditional MAID technologies, as shown in Figure 8, it is assumed that to T21 HDD this period at moment since the T2 moment
In idle states, therefore the rotating speed of this period is 7200rpm, when host computer is needing data at the T21 moment, HDD increases
Rotating speed is to 15000rpm, for quickly by Data Migration to host computer, the migrating data required time to be Δ T1, i.e., from T21
Moment to T22 moment.
For preextraction technology and MAID technologies are combined, as shown in Figure 9, it is assumed that the T2 moment to T21 moment
HDD is in idle states, therefore the rotating speed of this period is 7200rpm, while is during this period of time obtained according to above self study
To data transmission characteristics by data preextraction into high-performance storage unit.Assuming that the miss_ of data is found at the T2 moment
Rate_ratio is miss_rate_ratio_1, it is assumed that the data that user needs at the T21 moment are DATA_1, then can have：
DATA_1=IOPS15000*ΔT1(9)
Wherein, IOPS15000The I/O speed for being HDD rotating speeds in 15000rpm.
Preextraction is actually that user exists into the data in high-performance storage unit so within T2 to T21 this periods
What the T21 moment needed is：
DATA_1*(1-miss_rate_ratio_1)(10)
And the data lost in high-performance storage unit are：
DATA_1*miss_rate_ratio_1(11)
At the T21 moment, the adjustment of rotational speed of HDD is 15000rpm, and the data at this time needing to transmit are to be deposited in high-performance
The data lost in storage unit, it is assumed that the time used in transmission loss data is Δ T2, then：
ΔT2=Δ T1*miss_rate_ratio(12)
Since miss_rate_ratio must be between 0 and 1, it is known that necessarily there is Δ T2Less than Δ T1, so, it will carry in advance
Technology is taken to be applied to that in MAID hard disks performance of storage system can be lifted certainly.And if miss_rate_ratio is very big, that
Δ T2Will be towards Δ T1It is close, still, recover rotating speed can be heightened to 15000rpm using the HDD of MAID technologies, it is passed
Defeated performance is certain to more far better than the HDD for being fixed as slow-speed of revolution 7200rpm.That is when most data are in high property
Can in memory during miss, host computer read from the HDD of 15000rpm the speed of these data certainly than from 7200rpm when it is fast
Much, therefore can solve to bring to system performance because HDD is fixed as the slow-speed of revolution when miss_rate_ratio is very big
Infringement problem.
New round self study by the T2 moment to T3 this periods, statistic unit is recorded and to analyze user new
Data transfer is accustomed to, and improves new self-learning algorithm to reduce the probability of loss, therefore since the T3 moment, is continuing with pre-
The technology of extraction, and the rotating speed of HDD is set to fixed 7200rpm, that is to say, that it is identical with the T1 moment.If in system operation
During a period of time Y, user behavior is changed again, then just still according to it is above-mentioned it is described as according to the T1 moment
Later step continues to execute, and so circulation is gone down always.During self study new every time, HDD is still according to MAID's
The rotating speed rotation of technology defined, simply when HDD is in idle, continues with preextraction strategy and prefetches data, at the same time
Increase speed when host computer needs data, and then solve due to fixing the slow-speed of revolution to system performance in mixing storage system
The loss brought.By the self study of this period, the use habit of user is updated, new self-learning algorithm is improved and is lost with reducing
The probability of mistake, then proceedes to using the fixed slow-speed of revolution and engages the method transmission data of preextraction.
By explanation and attached drawing, the exemplary embodiments of the specific structure of embodiment are given, it is smart based on the present invention
God, can also make other conversions.Although foregoing invention proposes existing preferred embodiment, however, these contents are not intended as
Limitation.
For a person skilled in the art, after reading described above, various changes and modifications undoubtedly will be evident.
Therefore, appended claims should regard whole variations and modifications of the true intention and scope that cover the present invention as.Weighing
Any and all scope and content of equal value, are all considered as still belonging to the intent and scope of the invention in the range of sharp claim.
Claims (8)
1. a kind of mixing storage system for improving data loading speed, the mixing storage system connection host computer, its feature exist
In, the mixing storage system is used to by the cycle of a preset time period export the storage data to prestore to the host computer,
The cycle includes self study stage, time phase and the calibration phase set gradually, and the storage data include the first storage
Data and the second storage data, the transmission broadband of the first storage data and I/O speed are less than the described second storage data
Transmission bandwidth and I/O speed, the working status of the mixing storage system include idle condition and busy state, the free time shape
The memory bandwidth utilization rate of system when the memory bandwidth utilization rate of system is less than the busy state during state；The mixing storage system
System includes：
Hard disk storage units, the hard disk storage units connect the host computer, for exporting the storage using multi gear rotating speed
Data, the multi gear rotating speed include the first rotating speed and the second rotating speed more than first rotating speed；
High-performance storage unit, the high-performance storage unit connect the hard disk storage units and the host computer, for from
The hard disk storage units are obtained the storage data and are exported using the 3rd rotating speed, and the 3rd rotating speed turns more than the multi gear
Speed；
Statistic unit, the statistic unit connect the hard disk storage units and the high-performance storage unit, for by certainly
Study obtains the first data transmission characteristic of the mixing storage system in the self study stage, the in the practice stage
Two Data Transmission Features and the 3rd Data Transmission Feature in the calibration phase；
Control unit, it is single that described control unit connects the hard disk storage units, the high-performance storage unit and the statistics
Member, in hard disk storage units described in self study stage control in an idle state using first rotating speed output described the
One storage data store data to institute under busy state to the host computer using second rotating speed output described second
Host computer is stated, and for controlling the statistic unit to obtain the first data transmission characteristic；And
For being adopted in an idle state in practice stage hard disk storage units according to the first data transmission Characteristics Control
With first rotating speed output, the first storage data to the host computer and the second storage data are exported to the height
Performance memory location, and control the high-performance storage unit to export the second storage data received under busy state
The hard disk storage units are then controlled to export the high-performance storage unit not using first rotating speed to the host computer
The the second storage data received are used to control the statistic unit to obtain second data transfer to the host computer
Characteristic；And
For controlling the hard disk storage units to adopt in an idle state according to second Data Transmission Feature in calibration phase
With first rotating speed output, the first storage data to the host computer and the second storage data are exported to the height
Performance memory location, and control the high-performance storage unit to export the second storage data received under busy state
The hard disk storage units are then controlled to export the high-performance storage unit not using second rotating speed to the host computer
The the second storage data received are used to control the statistic unit to obtain the 3rd data transfer to the host computer
Characteristic；
3rd output characteristics is applied to the practice stage as first data transmission characteristic.
2. mixing storage method as claimed in claim 1, it is characterised in that the hard disk storage units are that application is extensive non-
The disk array being made of multiple disks of removable disk array technique.
3. as claimed in claim 2 mixing storage method, it is characterised in that first rotating speed for the disk array most
The slow-speed of revolution.
4. mixing storage method as claimed in claim 1, it is characterised in that the high-performance storage unit is hard drive
Device；Or
The high-performance storage unit is solid state hard disc；Or
The high-performance storage unit is phase transition storage；Or
The high-performance storage unit is resistive random access memory；Or
The high-performance storage unit is ferroelectric random access memory.
5. mixing storage method as claimed in claim 1, it is characterised in that the first data transmission characteristic is the mixing
Storage system is directed to the data transfer rule of multiple users and multiple application programs in multiple periods；
Second Data Transmission Feature for the mixing storage system for multiple users and multiple application programs when multiple
Between section data transfer rule；
3rd Data Transmission Feature for the mixing storage system for multiple users and multiple application programs when multiple
Between section data transfer rule.
6. mixing storage method as claimed in claim 4, it is characterised in that the data transfer rule includes reading and writing data
Frequency, the transmission speed of system I/O and memory bandwidth utilization rate.
A kind of 7. mixing storage method for improving data loading speed, using mixed as described in any one in claim 1-6
Close storage system, it is characterised in that including：
Step S1, exported in an idle state using the first rotating speed in self study stage, control unit control hard disk storage units
First storage data store data to host computer under busy state to host computer using the second rotating speed output second, and control
Unit control statistic unit processed obtains first data transmission characteristic of the mixing storage system in the self study stage by self study；
Step S2, in the practice stage, control unit is according to first data transmission Characteristics Control hard disk storage units in idle condition
It is lower that data are stored to host computer and the storage data of output second to high performance memory location using the first rotating speed output first, and
The second storage data to host computer that the output of high-performance storage unit has received is controlled then to control hard-disc storage under busy state
The second storage data that unit is not received using the first rotating speed output high-performance storage unit are to host computer, and control unit controls
Statistic unit obtains second Data Transmission Feature of the mixing storage system in the practice stage by self study；
Step S3, hard disk storage units are controlled in idle condition according to the second Data Transmission Feature in calibration phase, control unit
It is lower that data are stored to host computer and the storage data of output second to high performance memory location using the first rotating speed output first, and
The second storage data to host computer that the output of high-performance storage unit has received is controlled then to control hard-disc storage under busy state
The second storage data that unit is not received using the second rotating speed output high-performance storage unit are to host computer, and control unit controls
Statistic unit obtains threeth Data Transmission Feature of the mixing storage system in calibration phase by self study；
Step S4, control unit judges whether to receive stop signal：
If the determination result is YES, it is out of service to mix storage system, with backed off after random；
If judging result is no, control unit then turns step using the 3rd Data Transmission Feature as first data transmission characteristic
Rapid S2.
8. mixing storage method as claimed in claim 7, it is characterised in that in the step S2, the second storage data
There are one first Loss Rate during the high-performance storage unit is delivered to；
In the step S3, there are one second during the high-performance storage unit is delivered to for the second storage data
Loss Rate；
First Loss Rate is more than second Loss Rate.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711146834.3A CN107943416A (en) | 2017-11-17 | 2017-11-17 | A kind of mixing storage system for improving data loading speed and mixing storage method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711146834.3A CN107943416A (en) | 2017-11-17 | 2017-11-17 | A kind of mixing storage system for improving data loading speed and mixing storage method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107943416A true CN107943416A (en) | 2018-04-20 |
Family
ID=61931733
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711146834.3A Pending CN107943416A (en) | 2017-11-17 | 2017-11-17 | A kind of mixing storage system for improving data loading speed and mixing storage method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107943416A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030196031A1 (en) * | 2000-10-30 | 2003-10-16 | Chen Jack Yajie | Storage controller with the disk drive and the RAM in a hybrid architecture |
| CN101777028A (en) * | 2010-01-21 | 2010-07-14 | 北京北大众志微系统科技有限责任公司 | Realization method and device of mixed secondary storage system |
| CN102662459A (en) * | 2012-04-22 | 2012-09-12 | 复旦大学 | Method for reducing energy consumption of server by using mixed storage of solid-state drive and mechanical hard disk |
| CN104461389A (en) * | 2014-12-03 | 2015-03-25 | 上海新储集成电路有限公司 | Automatically learning method for data migration in mixing memory |
| CN106569577A (en) * | 2016-10-18 | 2017-04-19 | 上海新储集成电路有限公司 | Heterogeneous storage system and data storage center |
-
2017
- 2017-11-17 CN CN201711146834.3A patent/CN107943416A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030196031A1 (en) * | 2000-10-30 | 2003-10-16 | Chen Jack Yajie | Storage controller with the disk drive and the RAM in a hybrid architecture |
| CN101777028A (en) * | 2010-01-21 | 2010-07-14 | 北京北大众志微系统科技有限责任公司 | Realization method and device of mixed secondary storage system |
| CN102662459A (en) * | 2012-04-22 | 2012-09-12 | 复旦大学 | Method for reducing energy consumption of server by using mixed storage of solid-state drive and mechanical hard disk |
| CN104461389A (en) * | 2014-12-03 | 2015-03-25 | 上海新储集成电路有限公司 | Automatically learning method for data migration in mixing memory |
| CN106569577A (en) * | 2016-10-18 | 2017-04-19 | 上海新储集成电路有限公司 | Heterogeneous storage system and data storage center |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11669260B2 (en) | Predictive data orchestration in multi-tier memory systems | |
| US11977787B2 (en) | Remote direct memory access in multi-tier memory systems | |
| DE102014111990B4 (en) | Heterogeneous memory access | |
| CN111831219B (en) | Method, medium, and storage system for storage system | |
| JP3933027B2 (en) | Cache memory partition management method in disk array system | |
| US20200174938A1 (en) | Bypass storage class memory read cache based on a queue depth threshold | |
| US8380928B1 (en) | Applying data access activity measurements | |
| US11494311B2 (en) | Page table hooks to memory types | |
| JP5719013B2 (en) | Host read command return reordering based on flash read command completion time estimation | |
| TWI525433B (en) | Adaptive address mapping with dynamic runtime memory mapping selection | |
| US20090144347A1 (en) | Storage volume spanning with intelligent file placement and/or rearrangement | |
| US11204705B2 (en) | Retention-aware data tiering algorithm for hybrid storage arrays | |
| WO2019152224A1 (en) | Memory virtualization for accessing heterogeneous memory components | |
| EP2250585A1 (en) | Selecting storage location for file storage based on storage longevity and speed | |
| CN103186350A (en) | Hybrid storage system and hot spot data block migration method | |
| EP2350840B1 (en) | Method for controlling performance aspects of a data storage and access routine | |
| JP4699837B2 (en) | Storage system, management computer and data migration method | |
| CN102117248A (en) | Caching system and method for caching data in caching system | |
| CN103765397A (en) | Method, apparatus and system for determining an identifier of a volume of memory | |
| EP2156281A1 (en) | Virtualized storage performance controller | |
| CN104460941B (en) | A kind of method for reducing main store memory oepration at full load power consumption | |
| US20170039002A1 (en) | Memory device that changes execution order of commands | |
| CN102981971A (en) | Quick-response phase change memory wear-leveling method | |
| CN1928804A (en) | Method and system for power management in a distributed file system | |
| CN104834478B (en) | A kind of data write-in and read method based on isomery mixing storage device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20180420 |
|
| WD01 | Invention patent application deemed withdrawn after publication |