CN108549525B - Data storage and access method and device, electronic equipment and storage medium - Google Patents

Data storage and access method and device, electronic equipment and storage medium Download PDF

Info

Publication number
CN108549525B
CN108549525B CN201810301994.9A CN201810301994A CN108549525B CN 108549525 B CN108549525 B CN 108549525B CN 201810301994 A CN201810301994 A CN 201810301994A CN 108549525 B CN108549525 B CN 108549525B
Authority
CN
China
Prior art keywords
data
storage device
storage
data block
preset
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201810301994.9A
Other languages
Chinese (zh)
Other versions
CN108549525A (en
Inventor
田志鹤
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tianjin Branch Of Beijing Lanqi Data Technology Co ltd
Original Assignee
Tianjin Branch Of Beijing Lanqi Data Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tianjin Branch Of Beijing Lanqi Data Technology Co ltd filed Critical Tianjin Branch Of Beijing Lanqi Data Technology Co ltd
Priority to CN201810301994.9A priority Critical patent/CN108549525B/en
Publication of CN108549525A publication Critical patent/CN108549525A/en
Application granted granted Critical
Publication of CN108549525B publication Critical patent/CN108549525B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/061—Improving I/O performance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671—In-line storage system
    • G06F3/0683—Plurality of storage devices

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The embodiment of the disclosure discloses a data storage and access method, a data storage and access device, an electronic device and a storage medium. The method comprises the following steps: when the access frequency of preset storage data is lower than a first preset threshold value, dividing the preset storage data into a plurality of data blocks; storing at least one of the plurality of data chunks in a first storage device and other of the plurality of data chunks in a second storage device; wherein the first storage device has a higher access speed than the second storage device. According to the embodiment of the disclosure, the reading efficiency of the temperature/cold data can be greatly improved, the requirement that a user can timely display the temperature/cold data when accessing the temperature/cold data is met, the time of the user is saved, and the user experience is improved.

Description

Data storage and access method and device, electronic equipment and storage medium
Technical Field
The present disclosure relates to the field of computer technologies, and in particular, to a data storage and access method and apparatus, an electronic device, and a storage medium.
Background
With the popularization of cloud computing and the internet of things, the data volume is increasing at a high speed, so that more and more files or data need to be processed by enterprises are generated, and how to reasonably and efficiently manage the files is important. Although the existing file system can meet the basic functions of file uploading, downloading, inquiring and the like, the operations are basically centered on the file, and the data are not convenient for business personnel to efficiently manage. Therefore, it is important to classify data properly, and some data used by the system in real time is called "hot data", some data which is not used too frequently but is called by the system frequently is called "warm data", and some data which is used rarely by the system is called "cold data" according to the difference of data.
Disclosure of Invention
The embodiment of the disclosure provides a data storage and access method, a data storage and access device, an electronic device and a storage medium.
In a first aspect, an embodiment of the present disclosure provides a data storage method, including:
when the access frequency of preset storage data is lower than a first preset threshold value, dividing the preset storage data into a plurality of data blocks;
storing at least one of the plurality of data chunks in a first storage device and other of the plurality of data chunks in a second storage device; wherein the first storage device has a higher access speed than the second storage device.
Optionally, dividing the preset storage data into a plurality of data blocks includes:
and dividing the preset storage data into a plurality of data blocks according to the size of the preset storage data.
Optionally, dividing the preset storage data into a plurality of data blocks includes:
determining the sizes of the data blocks according to the unloading speed and the browsing speed of the user on the preset storage data; the unloading speed is the speed of unloading other data blocks stored on the second storage device to the first storage device;
Dividing the preset storage data into the plurality of data blocks according to the sizes of the plurality of data blocks.
Optionally, storing at least one of the plurality of data chunks in a first storage device and other of the plurality of data chunks in a second storage device, comprising:
determining at least one data block which is accessed first when the user accesses the preset storage data;
storing the first accessed at least one data block in the first storage device and other data blocks of the plurality of data blocks in a second storage device.
Optionally, the determining that the at least one data block accessed first by the user when accessing the preset storage data includes at least one of:
when the preset storage data is audio or video, determining a data block where initial content is located when the audio or video is played in sequence as the at least one data block which is accessed firstly;
when the preset storage data is a picture, determining a data block where a thumbnail of the picture is located as the at least one data block which is accessed firstly;
and when the preset storage data is a text, determining a data block where the initial content of the text is located as the at least one data block which is accessed firstly.
In a second aspect, an embodiment of the present disclosure provides a data access method, including:
in response to an event of reading preset storage data, outputting at least one first data block stored on a first storage device, and unloading at least one second data block stored on a second storage device onto the first storage device;
the preset storage data comprises at least one first data block and at least one second data block, and the access speed of the first storage device is higher than that of the second storage device.
Optionally, outputting at least one first data block stored on a first storage device and unloading at least one second data block stored on a second storage device onto the first storage device, comprising:
outputting the first data block where the content accessed first in the preset storage data is located, and transferring the second data block to the first storage device; wherein, in the preset storage data, the content of the second data block is next to the content of the first data block;
outputting the second data block from the first storage device in response to an event that the user has finished browsing the first data block.
Optionally, a time from outputting the first data block to receiving an event that a user browses the first data block is greater than or equal to a time for transferring the second data block from the second storage device to the first storage device.
In a third aspect, an embodiment of the present disclosure provides a data storage device, including:
the device comprises a dividing module, a storing module and a processing module, wherein the dividing module is configured to divide preset storage data into a plurality of data blocks when the access frequency of the preset storage data is lower than a first preset threshold;
a storage module configured to store at least one of the plurality of data chunks in a first storage device and other of the plurality of data chunks in a second storage device; wherein the first storage device has a higher access speed than the second storage device.
Optionally, the dividing module includes:
a first dividing module configured to divide the preset storage data into a plurality of data blocks according to a size of the preset storage data.
Optionally, the dividing module includes:
the first determining sub-module is configured to determine the sizes of the data blocks according to the unloading speed and the browsing speed of a user on the preset storage data; the unloading speed is the speed of unloading other data blocks stored on the second storage device to the first storage device;
A second division submodule configured to divide the preset storage data into the plurality of data blocks according to sizes of the plurality of data blocks.
Optionally, the storage module includes:
the second determining submodule is configured to determine at least one data block which is accessed first when the user accesses the preset storage data;
a storage submodule configured to store the at least one data block that is accessed first in the first storage device and other data blocks of the plurality of data blocks in a second storage device.
Optionally, the second determining submodule includes at least one of:
a third determining sub-module, configured to determine, when the preset storage data is an audio or a video, a data block in which a start content when the audio or the video is played in sequence is located as the at least one data block that is accessed first;
a fourth determining sub-module, configured to determine, when the preset storage data is a picture, a data block where a thumbnail of the picture is located as the at least one data block accessed first;
a fifth determining sub-module, configured to determine, when the preset storage data is a text, a data block in which starting content of the text is located as the at least one data block that is accessed first.
In a fourth aspect, an embodiment of the present disclosure provides a data access apparatus, including:
a response module configured to output at least one first data block stored on a first storage device in response to an event of reading preset storage data, and to transfer at least one second data block stored on a second storage device onto the first storage device;
the preset storage data comprises at least one first data block and at least one second data block, and the access speed of the first storage device is higher than that of the second storage device.
Optionally, the response module includes:
the output sub-module is configured to output the first data block where the content accessed first in the preset storage data is located, and transfer the second data block to the first storage device; wherein, in the preset storage data, the content of the second data block is next to the content of the first data block;
a response submodule configured to output the second data block from the first storage device in response to an event that a user browses the first data block.
Optionally, a time from outputting the first data block to receiving an event that the user browses the first data block is greater than or equal to a time for transferring the second data block from the second storage device to the first storage device.
The functions can be realized by hardware, and the functions can also be realized by executing corresponding software by hardware. The hardware or software includes one or more modules corresponding to the functions described above.
In one possible design, the data storage device and the data access device each include a memory and a processor, the memory is used for storing one or more computer instructions for supporting the data storage device or the data access device to execute the data storage method or the data access method in the first aspect, and the processor is configured to execute the computer instructions stored in the memory. The data storage device or the data access device may further comprise a communication interface for the data storage device or the data access device to communicate with other devices or a communication network.
In a fifth aspect, embodiments of the present disclosure provide an electronic device, including a memory and a processor; wherein the memory is configured to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method steps of the first or second aspect.
In a sixth aspect, the disclosed embodiments provide a computer-readable storage medium for storing computer instructions for a data storage device or a data access device, which includes computer instructions for performing the data storage method or the data access method in the first aspect or the second aspect.
The technical scheme provided by the embodiment of the disclosure can have the following beneficial effects:
the embodiment of the disclosure provides a data storage and access method, by dividing temperature/cold data with a lower access frequency into a plurality of data blocks, and storing at least one of the data blocks as a first storage device with a higher access speed, such as a memory or an SSD, while the remaining data blocks are stored in a second storage device with a lower speed, when a user needs to access the cold data, the data blocks stored in the first storage device are accessed first, and simultaneously, the remaining data blocks stored in the second storage device are transmitted to the first storage device with a higher access speed. Through the mode, after the user checks the data blocks in the first storage equipment such as the memory or the SSD, other data blocks of the warm/cold data also realize conversion to the warm data, namely the data blocks are transmitted to the first storage equipment with the higher access speed, so that the reading efficiency of the warm/cold data is greatly improved, the requirement that the user can timely display the warm/cold data when accessing the warm/cold data is met, the time of the user is saved, and the user experience is improved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
Other features, objects, and advantages of the present disclosure will become more apparent from the following detailed description of non-limiting embodiments when taken in conjunction with the accompanying drawings. In the drawings:
FIG. 1 illustrates a flow diagram of a data storage method according to an embodiment of the present disclosure;
FIG. 2 shows a flow chart of step S101 according to the embodiment shown in FIG. 1;
FIG. 3 shows a flow chart of step S103 according to the embodiment shown in FIG. 1;
FIG. 4 illustrates a flow diagram of a data access method according to an embodiment of the present disclosure;
FIG. 5 shows a flow chart of steps S401 and S402 according to the embodiment shown in FIG. 4;
FIG. 6 shows a block diagram of a data storage device according to an embodiment of the present disclosure;
fig. 7 is a schematic structural diagram of an electronic device suitable for implementing a data storage method or a data access method according to an embodiment of the present disclosure.
Detailed Description
Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement them. Furthermore, parts that are not relevant to the description of the exemplary embodiments have been omitted from the drawings for the sake of clarity.
In the present disclosure, it is to be understood that terms such as "including" or "having," etc., are intended to indicate the presence of the disclosed features, numbers, steps, behaviors, components, parts, or combinations thereof, and are not intended to preclude the possibility that one or more other features, numbers, steps, behaviors, components, parts, or combinations thereof may be present or added.
It should also be noted that the embodiments and features of the embodiments in the present disclosure may be combined with each other without conflict. The present disclosure will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.
More and more enterprises back up data which is not used frequently and store the data in a low-energy-consumption and high-capacity mode, namely cold data storage in reality. For data which is not accessed frequently, such as health medical data, bank account information, social security data, growth records and the like, the data is accessed frequently but is very effective, and when the data is used by a user, the data may need to be recovered for hours or even longer, which is unacceptable for many users. There is currently no particularly good way to solve this problem, and most businesses use network backup methods, but this is not a preferred solution and requires a large amount of data traffic and expense to support.
In the prior art, "hot data" is stored in a memory or SSD, such storage devices and stored data are online all the time, the storage and access efficiency of data is high, and "warm data" and "cold data" with relatively low access frequency are stored in a relatively cheap and low-speed medium, such as an HDD (Hard Disk Drive), an optical disc, etc., which results in that when a user accesses the warm/cold data, the data reading speed is slow, and the user experience is reduced.
Fig. 1 shows a flow chart of a data storage method according to an embodiment of the present disclosure. As shown in fig. 1, the data storage method includes the following steps S101 to S103:
in step S101, when the access frequency of preset storage data is lower than a first preset threshold, dividing the preset storage data into a plurality of data blocks;
in step S102, storing at least one of the plurality of data blocks in a first storage device and storing other of the plurality of data blocks in a second storage device; wherein the first storage device has a higher access speed than the second storage device.
In the prior art, temperature/cold data may be stored in a low-speed storage medium such as a hard disk, an optical disk, etc., and when a user wants to review some cold data again, the data recovery time is too long, and the user's waiting time is too long, which may bring a bad experience to the user. Therefore, the present disclosure proposes a data storage and access method, by dividing the temperature/cold data with low access frequency into a plurality of data blocks, and storing at least one of the data blocks as a data block in a first storage device with a fast access speed, such as a memory or an SSD, while the rest of the data blocks are stored in a second storage device with a slow speed, when a user needs to access the cold data, the data block stored in the first storage device is accessed first, and at the same time, the rest of the data blocks stored in the second storage device are transferred to the first storage device with a fast access speed. Through the mode, after the user checks the data blocks in the first storage device such as the memory or the SSD, other data blocks of the warm/cold data also realize conversion to the warm data, namely the data blocks are transmitted to the first storage device with the higher access speed, so that the reading efficiency of the warm/cold data is greatly improved, the requirement that the user can timely display the access to the warm/cold data is met, the time of the user is saved, and the user experience is improved.
In this embodiment, the preset storage data may be some pre-specified storage data, for example, some data that is not accessed frequently but has a long validity, such as health and medical data, bank account information, social security data, growth records, and the like, and when the access frequency of these data is low, these data may be used as warm/cold data and stored by using the data storage method provided by the present disclosure. Of course, the preset storage data may also be all data stored in the system, and may be specifically set according to an actual situation, which is not limited herein. The first preset threshold may be preset according to an empirical summary or other conditions, that is, data with an access frequency lower than the first preset threshold is stored as warm/cold data according to the data storage method provided by the embodiment of the present disclosure.
The data block division of the preset storage data may be performed according to a preset division principle according to a data type or other conditions, for example, for video, audio, and other data, the data block division may be performed in an average division manner to form a plurality of data blocks with equal size, and the size of the data block may be determined according to the size of the video or audio, and may also be determined according to the size of the first storage device, the speed of transferring data on the second storage device to the first storage device, the browsing speed of the user on the data, and the like. The specific factors considered for dividing the data block may be set based on practical situations, and are not limited herein. It is to be understood that the sizes of the multiple data blocks may be the same or different, and the data block stored in the first storage device may be one data block or multiple data blocks, which may be specifically set according to an actual situation, and are not limited herein.
The first storage device may be a storage medium with a high access speed, such as a memory or an SSD, and the second storage device may be a storage medium with a low access speed, but a large storage capacity and a low price, such as an HDD. The access speed of the first storage device is greater than the access speed of the second storage device. A storage medium such as a memory or an SSD is suitable for storing data that is frequently accessed because it has a high access speed but is expensive and has a small storage capacity, while a storage medium such as an HDD is suitable for storing data that is not frequently accessed because it has a low price and a large storage capacity but has a busy access speed. Therefore, according to the embodiment of the disclosure, a small part of the temperature/cold data with the access frequency lower than the first preset threshold is stored in the first storage device, and the rest of the data is stored in the second storage device, so that on one hand, the efficiency of accessing the temperature/cold data by the user can be improved, the user experience can be improved, on the other hand, the cost can be reduced as much as possible, and the expenditure can be saved for enterprises.
The preset storage data is subjected to classification processing, the first preset threshold is set according to the access frequency, if the access frequency of the preset storage data is greater than or equal to the first preset threshold, the preset storage data can be regarded as hot data, and if the access frequency is less than the first preset threshold, the preset storage data can be regarded as warm/cold data. For example, for some social software (QQ, wechat, microblog, etc.), a large amount of data is generated at each moment for the circles of friends, spaces, and microblogs of these applications, the information amount in the circles of friends, spaces, and microblogs of some users is particularly large, specifically, several dynamic states may be issued each day, each dynamic state includes a large amount of data, and when the user views the circles of friends, spaces, and microblogs, the user cannot see a certain amount of data, so that for the circles of friends, spaces, and microblogs of the user with particularly large information data amount, the frequency of data access may be lower than a first preset threshold several years ago, and at this time, the part of data is marked as warm/cold data. For another example, for a browser (google, 360, dog search, etc.) and a shopping application (panning, jingdong, suting, etc.), a history record browsed by a user may be frequently browsed in a short time, the history record is marked as hot data when the browsing frequency is greater than a set threshold, but the history record is hardly browsed by the user for a long time ago, and the history record is marked as warm/cold data when the browsing frequency is less than a first preset threshold.
Then, the embodiment of the present disclosure may divide the warm/cold data into a plurality of data blocks according to the content of the warm/cold data, and set at least one of the data blocks as a hot data block to be stored in a first storage device with a higher access speed, such as a memory or an SSD, and store the rest of the data blocks in a second storage device with a lower access speed, such as an HDD, to reduce the data storage cost. For example, for videos and audios with low access frequency, data blocks can be divided according to the lengths of the videos and audios, for example, half-hour-long videos or audios can be divided into 30 data blocks, each data block is one-minute video or audio content, a first data block is stored into a first storage device such as a memory or an SSD as hot data, and the remaining 29 data blocks are stored into a second storage device such as a disk; for a large picture, the thumbnail of the picture may be stored as a hot data block in a first storage device, and the remaining data blocks may be stored as warm/cold data in a second storage device such as an HDD at a low speed.
In an optional implementation manner of this embodiment, in the step S101, the step of dividing the preset storage data into a plurality of data blocks further includes the following steps:
And dividing the preset storage data into a plurality of data blocks according to the size of the preset storage data.
In this embodiment, the plurality of data blocks may be divided according to a size of the preset storage data. For example, when the preset storage data is large, a few data blocks can be divided more, and when the preset storage data is small, a few data blocks can be divided. Of course, in other embodiments, other factors may be considered besides the preset size of the storage data, such as the size of the first storage device, etc.
In an optional implementation manner of this embodiment, as shown in fig. 2, the step of dividing the preset storage data into a plurality of data blocks in the step S101 further includes the following steps S201 to S202:
in step S201, determining the sizes of the data blocks according to the unloading speed and the browsing speed of the user on the preset storage data; the unloading speed is the speed of unloading other data blocks stored on the second storage device to the first storage device;
in step S202, the preset storage data is divided into the plurality of data blocks according to the sizes of the plurality of data blocks.
In this embodiment, for preset storage data of different contents, browsing speeds of users are different, for example, for preset storage data of a document or a picture that occupies a smaller memory, the browsing speed of the user is faster, and for files of video or audio that occupies a larger memory, the browsing speed of the user is slower, for example, a first data block of a large video is stored in a first storage device such as a memory or an SSD, when the user accesses the video segment, the first data block stored on the first storage device is played first, and at the same time, a next data block stored in a second storage device such as an HDD starts to be converted to thermal data, that is, the data block is transferred to the first storage device, at this time, the size of the data block can be calculated according to the transfer speed and the browsing speed of the user, so that after the user has just finished browsing the first data block stored on the first storage device, the second data block just completes the conversion of the hot data, and the third data block just completes the conversion of the hot data just after the second data block is browsed, namely the data are just transferred to the first storage device. By analogy, the storage cost can be reduced and the continuity of reading the content by the user can be ensured until the user browses the whole video clip, so that the efficiency of the whole system is improved and the user experience is enhanced.
In an optional implementation manner of this embodiment, as shown in fig. 3, in the step S102, the step of storing at least one data block of the multiple data blocks in a first storage device, and storing other data blocks of the multiple data blocks in a second storage device further includes the following steps S301 to S302:
in step S301, at least one data block that is accessed first when the user accesses the preset storage data is determined;
in step S302, the at least one data block accessed first is stored in the first storage device, and other data blocks of the plurality of data blocks are stored in a second storage device.
In this embodiment, when a user accesses preset storage data stored as warm/cold data, at least one data block to be accessed first may be stored in the first storage device, so that when the user accesses the preset storage data, the user may access the preset storage data from the first storage device first, and then when the user accesses the at least one data block, the subsequent data block to be accessed on the second storage device is transferred to the first storage device, which may improve the efficiency of reading the warm/cold data by the user, and may also ensure the continuity of reading contents by the user.
In this alternative implementation, the first accessed content may vary from data type to data type, for example, for audio or video data, the first accessed content is typically the starting content in the playing order, for picture data, the first accessed content may be thumbnails of pictures, and for text data, the first accessed content may be the starting content of text. Certainly, the content accessed first is not limited to the above, and may be set according to actual needs, which is not limited herein.
In an optional implementation manner of this embodiment, in step S301, that is, the step of determining at least one data block that is accessed first when the user accesses the preset storage data, the step further includes at least one of the following steps:
when the preset storage data is audio or video, determining a data block where initial content is located when the audio or video is played in sequence as the at least one data block which is accessed firstly;
when the preset storage data is a picture, determining a data block where a thumbnail of the picture is located as the at least one data block which is accessed firstly;
and when the preset storage data is a text, determining a data block where the initial content of the text is located as the at least one data block which is accessed firstly.
In this alternative implementation, the data blocks that may be accessed first are different due to the different types of data content. Therefore, at least one data block accessed first may be selected according to a type of preset storage data, and the at least one data block accessed first may be stored in the first storage device. For example, for audio or video data, a user usually plays video in sequence, and thus a data block where the start content is located when playing in sequence can be determined as at least one data block accessed first; for the picture data, a user usually browses the thumbnail of the picture first, so that the data block where the thumbnail is located can be determined as at least one data block accessed first; for text data, a user may generally browse text starting from the beginning content of the text, and therefore, the data block in which the beginning content of the text is located may also be determined as the at least one data block that is accessed first. It is understood that one or more data blocks stored on the first storage device may be used, and may be specifically set according to the actual situation, which is not limited herein.
FIG. 4 shows a flow diagram of a data access method according to an embodiment of the present disclosure. As shown in fig. 4, the data access method includes the following steps S401 to S402:
In step S401, in response to an event of reading preset storage data, outputting at least one first data block stored on a first storage device; and is provided with
In step S402, at least one second data block stored on a second storage device is transferred to the first storage device;
the preset storage data comprises at least one first data block and at least one second data block, and the access speed of the first storage device is higher than that of the second storage device.
In this embodiment, it should be noted that steps S401 and S402 do not have a sequence therebetween, and are two steps that may be executed in parallel; step S401 may be executed first and then step S402 may be executed, step S402 may be executed first and then step S401 may be executed, or steps S401 and S402 may be executed simultaneously.
The embodiment provides a data access method for data stored by the data storage method disclosed in the embodiment shown in fig. 1 and related contents. According to the method, for preset storage data that a part of data blocks are stored on a first storage device with a high access speed and the rest of data are stored on a second storage device with a low access speed, when a request event for reading the preset storage data is received, at least one data block in the preset storage data stored on the first storage device is output, and the rest of data blocks in the preset storage data are transferred to the first storage device while the at least one data block is output, so that after the output of the at least one data block stored on the first storage device is finished, the rest of data blocks transferred to the first storage device from the second storage device are output.
For other technical details of the embodiment, reference may be made to the embodiment shown in fig. 1 and the data storage method provided in the related content, which are not described herein again.
In an optional implementation manner of this embodiment, as shown in fig. 5, the steps S401 and S402, namely, the steps of outputting at least one first data block stored on a first storage device and transferring at least one second data block stored on a second storage device to the first storage device, further include the following steps S501 to S502:
in step S501, the first data block in which the content that is accessed first in the preset storage data is located is output, and the second data block is transferred to the first storage device; wherein, in the preset storage data, the content of the second data block is next to the content of the first data block;
in step S502, in response to an event that the user finishes browsing the first data block, outputting the second data block from the first storage device.
In this optional implementation manner, when the preset storage data, which is the temperature/cold data, is stored, at least one data block to be accessed first may be stored in the first storage device, so that when a user accesses the preset storage data, the user may access the preset storage data from the first storage device first, and then when the user accesses the at least one data block, the data block to be accessed subsequently on the second storage device is transferred to the first storage device. For example, the preset storage data is divided into three data blocks, which are respectively the first to third data blocks in the browsing order of the user, and at the time of storage, the first data block is stored on the first storage device, and the second and third data blocks are stored on the second storage device. When a user accesses preset storage data, the first data block stored on the first storage device can be accessed firstly, namely the first data block is output to the user after an access request of the user is received, meanwhile, the second data block and the third data block are transferred to the second storage device, and after the user browses the first data block, the second data block and the third data block are continuously output to the user from the first storage device, so that the efficiency of reading the temperature/cold data by the user can be improved, and the continuity of reading contents by the user can be ensured.
In this alternative implementation, the content that is accessed first may be different according to different data types, for example, for audio/video data, the content that is accessed first is usually the starting content in the playing sequence, for picture data, the content that is accessed first may be a thumbnail of a picture, and for text data, the content that is accessed first may be the starting content of text. Certainly, the content accessed first is not limited to the above, and may be set according to actual needs, which is not limited herein.
In an optional implementation manner of this embodiment, a time from when the first data block is output to when an event that the user browses the first data block is received is greater than or equal to a time for transferring the second data block from the second storage device to the first storage device.
In this optional implementation manner, in order to improve the efficiency of accessing the warm/hot data by the user and ensure the persistence of the user when accessing the warm/hot data, the time from when the user finishes browsing the data block on the first storage device to when the user transfers the next data block to the first storage data block from the second storage device, that is, the time from when the first data block is output to when the event that the user finishes browsing the first data block is received is greater than or equal to the time from when the second data block is transferred to the first storage device from the second storage device, so that the user does not break in the process of browsing the data, and the user experience can be improved.
The following are embodiments of the disclosed apparatus that may be used to perform embodiments of the disclosed methods.
Fig. 6 shows a block diagram of a data storage device according to an embodiment of the present disclosure, which may be implemented as part or all of an electronic device by software, hardware, or a combination of both. As shown in fig. 6, the data storage apparatus includes a dividing module 601 and a storage module 602:
the dividing module 601 is configured to divide preset storage data into a plurality of data blocks when the access frequency of the preset storage data is lower than a first preset threshold;
a storage module 602 configured to store at least one of the plurality of data chunks on a first storage device and other of the plurality of data chunks on a second storage device; wherein the first storage device has a higher access speed than the second storage device.
In an optional implementation manner of this embodiment, the dividing module 601 includes:
a first dividing module configured to divide the preset storage data into a plurality of data blocks according to a size of the preset storage data.
In an optional implementation manner of this embodiment, the dividing module 601 includes:
The first determining sub-module is configured to determine the sizes of the data blocks according to the unloading speed and the browsing speed of a user on the preset storage data; the unloading speed is the speed of unloading other data blocks stored on the second storage device to the first storage device;
a second division submodule configured to divide the preset storage data into the plurality of data blocks according to sizes of the plurality of data blocks.
In an optional implementation manner of this embodiment, the storage module 602 includes:
the second determining submodule is configured to determine at least one data block which is accessed first when the user accesses the preset storage data;
a storage submodule configured to store the at least one data block that is accessed first in the first storage device and other data blocks of the plurality of data blocks in a second storage device.
In an optional implementation manner of this embodiment, the second determining sub-module includes at least one of:
a third determining sub-module, configured to determine, when the preset storage data is an audio or a video, a data block in which a start content when the audio or the video is played in sequence is located as the at least one data block that is accessed first;
A fourth determining sub-module, configured to determine, when the preset storage data is a picture, a data block where a thumbnail of the picture is located as the at least one data block accessed first;
a fifth determining sub-module, configured to determine, when the preset storage data is a text, a data block in which starting content of the text is located as the at least one data block that is accessed first.
The data storage apparatus proposed in the embodiment and the related implementation shown in fig. 6 corresponds to the data storage method proposed in the embodiment and the related implementation shown in fig. 1, and for specific details, reference may be made to the description of the data storage method proposed in the embodiment and the related implementation shown in fig. 1, and details are not repeated here.
According to an embodiment of the present disclosure, a data access apparatus is further provided, which may be implemented as part or all of an electronic device through software, hardware or a combination of both. The data access device includes:
the response module is configured to respond to an event of reading preset storage data, output at least one first data block stored on a first storage device, and transfer at least one second data block stored on a second storage device onto the first storage device; the preset storage data comprises at least one first data block and at least one second data block, and the access speed of the first storage device is higher than that of the second storage device.
In an optional implementation manner of this embodiment, the response module includes:
the output sub-module is configured to output the first data block where the content accessed first in the preset storage data is located, and to transfer the second data block to the first storage device; wherein, in the preset storage data, the content of the second data block is next to the content of the first data block;
a response submodule configured to output the second data block from the first storage device in response to an event that a user browses the first data block.
In an optional implementation manner of this embodiment, a time from when the first data block is output to when an event that the user finishes browsing the first data block is received is greater than or equal to a time for transferring the second data block from the second storage device to the first storage device.
The data access device proposed in the above embodiment and related implementation is corresponding to the data access method proposed in the embodiment and related implementation shown in fig. 4, and specific details may refer to the description of the data access method proposed in the embodiment and related implementation shown in fig. 4, and are not described herein again.
Fig. 7 is a schematic structural diagram of an electronic device suitable for implementing a data storage method or a data access method according to an embodiment of the present disclosure.
As shown in fig. 7, the electronic apparatus 700 includes a Central Processing Unit (CPU)701, which can execute various processes in the embodiment shown in fig. 1 described above according to a program stored in a Read Only Memory (ROM)702 or a program loaded from a storage section 708 into a Random Access Memory (RAM) 703. In the RAM703, various programs and data necessary for the operation of the electronic apparatus 700 are also stored. The CPU701, ROM702, and RAM703 are connected to each other via a bus 704. An input/output (I/O) interface 705 is also connected to bus 704.
The following components are connected to the I/O interface 705: an input portion 706 including a keyboard, a mouse, and the like; an output section 707 including components such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and a speaker; a storage section 708 including a hard disk and the like; and a communication section 709 including a network interface card such as a LAN card, a modem, or the like. The communication section 709 performs communication processing via a network such as the internet. A drive 710 is also connected to the I/O interface 705 as needed. A removable medium 711 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 710 as necessary, so that a computer program read out therefrom is mounted into the storage section 708 as necessary.
In particular, according to embodiments of the present disclosure, the method described above with reference to fig. 1 may be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program tangibly embodied on a medium readable thereby, the computer program comprising program code for performing the method of fig. 1 or 4. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 709, and/or installed from the removable medium 711.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units or modules described in the embodiments of the present disclosure may be implemented by software or hardware. The units or modules described may also be provided in a processor, and the names of the units or modules do not in some cases constitute a limitation on the units or modules themselves.
As another aspect, the present disclosure also provides a computer-readable storage medium, which may be the computer-readable storage medium included in the apparatus in the above embodiment; or it may be a separate computer readable storage medium not incorporated into the device. The computer readable storage medium stores one or more programs for use by one or more processors in performing the methods described in the present disclosure.
The foregoing description is only exemplary of the preferred embodiments of the disclosure and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention in the present disclosure is not limited to the specific combination of the above-mentioned features, but also encompasses other embodiments in which any combination of the above-mentioned features or their equivalents is made without departing from the inventive concept. For example, the above features and the technical features disclosed in the present disclosure (but not limited to) having similar functions are replaced with each other to form the technical solution.

Claims (18)

1. A method of storing data, comprising:
when the access frequency of preset storage data is lower than a first preset threshold value, dividing the preset storage data into a plurality of data blocks, wherein the preset storage data refers to data which has the access frequency lower than a certain value and is marked as warm/cold data;
storing at least one of the plurality of data chunks in a first storage device and other of the plurality of data chunks in a second storage device; the access speed of the first storage device is higher than that of the second storage device, the first storage device comprises a Solid State Disk (SSD), and the second storage device comprises a Hard Disk Drive (HDD);
the at least one of the plurality of data blocks refers to at least one of the plurality of data blocks that was accessed first.
2. The data storage method of claim 1, wherein dividing the predetermined storage data into a plurality of data blocks comprises:
and dividing the preset storage data into a plurality of data blocks according to the size of the preset storage data.
3. The data storage method of claim 1, wherein dividing the predetermined storage data into a plurality of data blocks comprises:
Determining the sizes of the data blocks according to the unloading speed and the browsing speed of the user on the preset storage data; the unloading speed is the speed of unloading other data blocks stored on the second storage device to the first storage device;
dividing the preset storage data into the plurality of data blocks according to the sizes of the plurality of data blocks.
4. The data storage method of claim 1, wherein storing at least one of the plurality of data chunks in a first storage device and other of the plurality of data chunks in a second storage device comprises:
determining at least one data block which is accessed first when the user accesses the preset storage data;
storing the first accessed at least one data block in the first storage device and other data blocks of the plurality of data blocks in a second storage device.
5. The data storage method of claim 4, wherein determining at least one data block that is accessed first by a user when accessing the preset storage data comprises at least one of:
When the preset storage data is audio or video, determining a data block where initial content is located when the audio or video is played in sequence as the at least one data block which is accessed firstly;
when the preset storage data is a picture, determining a data block where a thumbnail of the picture is located as the at least one data block which is accessed firstly;
and when the preset storage data is a text, determining a data block where the initial content of the text is located as the at least one data block which is accessed firstly.
6. A method of accessing data, comprising:
responding to an event of reading preset storage data, outputting at least one first data block stored on a first storage device, and transferring at least one second data block stored on a second storage device to the first storage device, wherein the preset storage data refer to data which has an access frequency lower than a certain value and is marked as warm/cold data;
the preset storage data is data with access frequency lower than a first preset threshold value, the preset storage data comprises at least one first data block and at least one second data block, the access speed of the first storage device is higher than that of the second storage device, the first storage device comprises a Solid State Disk (SSD), and the second storage device comprises a Hard Disk Drive (HDD);
The at least one first data block refers to at least one data block accessed first in the preset storage data.
7. The data access method of claim 6, wherein outputting at least one first data block stored on a first storage device and unloading at least one second data block stored on a second storage device onto the first storage device comprises:
outputting the first data block where the content accessed first in the preset storage data is located, and transferring the second data block to the first storage device; wherein, in the preset storage data, the content of the second data block is next to the content of the first data block;
outputting the second data block from the first storage device in response to an event that the user has finished browsing the first data block.
8. The data access method of claim 7, wherein a time from when the first data block is output to when an event that a user browses the first data block is received is greater than or equal to a time for transferring the second data block from the second storage device to the first storage device.
9. A data storage device, comprising:
the device comprises a dividing module, a storage module and a judging module, wherein the dividing module is configured to divide preset storage data into a plurality of data blocks when the access frequency of the preset storage data is lower than a first preset threshold value, and the preset storage data refers to data which has the access frequency lower than a certain value and is marked as warm/cold data;
a storage module configured to store at least one of the plurality of data chunks in a first storage device and other of the plurality of data chunks in a second storage device; the access speed of the first storage device is higher than that of the second storage device, the first storage device comprises a Solid State Disk (SSD), and the second storage device comprises a Hard Disk Drive (HDD);
the at least one of the plurality of data blocks refers to at least one of the plurality of data blocks that was accessed first.
10. The data storage device of claim 9, wherein the partitioning module comprises:
a first dividing module configured to divide the preset storage data into a plurality of data blocks according to a size of the preset storage data.
11. The data storage device of claim 9, wherein the partitioning module comprises:
The first determining sub-module is configured to determine the sizes of the data blocks according to the unloading speed and the browsing speed of a user on the preset storage data; the unloading speed is the speed of unloading other data blocks stored on the second storage device to the first storage device;
a second division submodule configured to divide the preset storage data into the plurality of data blocks according to sizes of the plurality of data blocks.
12. The data storage device of claim 9, wherein the storage module comprises:
the second determining submodule is configured to determine at least one data block which is accessed first when the user accesses the preset storage data;
a storage submodule configured to store the at least one data block that is accessed first in the first storage device and other data blocks of the plurality of data blocks in a second storage device.
13. The data storage device of claim 12, wherein the second determination submodule comprises at least one of:
a third determining sub-module, configured to determine, when the preset storage data is an audio or a video, a data block in which a start content when the audio or the video is played in sequence is located as the at least one data block that is accessed first;
A fourth determining sub-module, configured to determine, when the preset storage data is a picture, a data block where a thumbnail of the picture is located as the at least one data block accessed first;
a fifth determining sub-module, configured to determine, when the preset storage data is a text, a data block in which starting content of the text is located as the at least one data block that is accessed first.
14. A data access device, comprising:
the response module is configured to respond to an event of reading preset storage data, output at least one first data block stored on a first storage device, and transfer at least one second data block stored on a second storage device to the first storage device, wherein the preset storage data refer to data which has an access frequency lower than a certain value and is marked as warm/cold data;
the preset storage data is data with access frequency lower than a first preset threshold value, the preset storage data comprises at least one first data block and at least one second data block, the access speed of the first storage device is higher than that of the second storage device, the first storage device comprises a Solid State Disk (SSD), and the second storage device comprises a Hard Disk Drive (HDD);
The at least one first data block refers to at least one data block accessed first in the preset storage data.
15. The data access device of claim 14, wherein the response module comprises:
the output sub-module is configured to output the first data block where the content accessed first in the preset storage data is located, and to transfer the second data block to the first storage device; wherein, in the preset storage data, the content of the second data block is next to the content of the first data block;
a response submodule configured to output the second data block from the first storage device in response to an event that a user browses the first data block.
16. The data access apparatus of claim 15, wherein a time from when the first data block is output to when an event that a user browses the first data block is received is greater than or equal to a time to dump the second data block from the second storage device to the first storage device.
17. An electronic device comprising a memory and a processor; wherein,
The memory is to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method steps of any of claims 1-8.
18. A computer-readable storage medium having stored thereon computer instructions, which, when executed by a processor, carry out the method steps of any one of claims 1 to 8.
CN201810301994.9A 2018-04-04 2018-04-04 Data storage and access method and device, electronic equipment and storage medium Active CN108549525B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810301994.9A CN108549525B (en) 2018-04-04 2018-04-04 Data storage and access method and device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810301994.9A CN108549525B (en) 2018-04-04 2018-04-04 Data storage and access method and device, electronic equipment and storage medium

Publications (2)

Publication Number Publication Date
CN108549525A CN108549525A (en) 2018-09-18
CN108549525B true CN108549525B (en) 2022-07-15

Family

ID=63514026

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810301994.9A Active CN108549525B (en) 2018-04-04 2018-04-04 Data storage and access method and device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN108549525B (en)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111443865A (en) * 2019-01-16 2020-07-24 香港商希瑞科技股份有限公司 Server and related computer program product
CN110045925A (en) * 2019-03-05 2019-07-23 北京字节跳动网络技术有限公司 Data processing method, device, electronic equipment and computer readable storage medium
CN110083649A (en) * 2019-04-24 2019-08-02 北京电子工程总体研究所 It is a kind of based on cold, warm, dsc data data management system and data managing method
CN113687766A (en) * 2020-05-19 2021-11-23 中移动信息技术有限公司 Data storage method, device, equipment and storage medium
CN111666438A (en) * 2020-05-22 2020-09-15 东华大学 Cloud photo album text keyword fuzzy search system and use method
CN114063880B (en) * 2020-07-31 2024-09-06 伊姆西Ip控股有限责任公司 Method, electronic device and computer program product for processing input/output request
CN116361252A (en) * 2023-03-09 2023-06-30 北京百度网讯科技有限公司 Data storage and access method, device, equipment and medium

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN100550160C (en) * 2005-02-18 2009-10-14 威盛电子股份有限公司 Multimedia reads playing system and method
JP4556789B2 (en) * 2005-07-07 2010-10-06 ソニー株式会社 Playback apparatus, playback method, and playback program
CN100568957C (en) * 2007-10-17 2009-12-09 华中科技大学 A kind of video frequency request program caching method based on the p2p technology
CN101207799A (en) * 2007-11-22 2008-06-25 深圳市同洲电子股份有限公司 System and method for storing program and program ordering system
US9582431B2 (en) * 2010-03-22 2017-02-28 Seagate Technology Llc Storage address space to NVM address, span, and length mapping/converting
US8782340B2 (en) * 2010-12-06 2014-07-15 Xiotech Corporation Hot sheet upgrade facility
US8782369B2 (en) * 2011-11-15 2014-07-15 Lsi Corporation Apparatus to manage efficient data migration between tiers
US9703500B2 (en) * 2012-04-25 2017-07-11 International Business Machines Corporation Reducing power consumption by migration of data within a tiered storage system
CN103631538B (en) * 2013-12-05 2017-04-05 华为技术有限公司 Cold and hot data identification threshold value calculation, device and system
CN104581230A (en) * 2015-01-20 2015-04-29 卡内基投资科技有限公司 Method capable of continuously playing different video clips without regeneration

Also Published As

Publication number Publication date
CN108549525A (en) 2018-09-18

Similar Documents

Publication Publication Date Title
CN108549525B (en) Data storage and access method and device, electronic equipment and storage medium
US11734125B2 (en) Tiered cloud storage for different availability and performance requirements
US8825598B2 (en) Media file synchronization
US11580061B2 (en) System and method for file archiving using machine learning
RU2629448C2 (en) System and method of controlling and organizing web-browser cash
US20140108335A1 (en) Cloud based file system surpassing device storage limits
US9357007B2 (en) Controlling storing of data
WO2017117919A1 (en) Data storage method and device
JP2012526320A (en) Method and system for search using search engine indexing and index
CN107197359B (en) Video file caching method and device
CN102307234A (en) Resource retrieval method based on mobile terminal
CN108829809B (en) Information display method, terminal equipment and network equipment thereof
US11392545B1 (en) Tracking access pattern of inodes and pre-fetching inodes
US20180203636A1 (en) Likelihood of access based object storage in a cloud environment
US20110106815A1 (en) Method and Apparatus for Selectively Re-Indexing a File System
EP3519993A1 (en) Tracking access pattern of inodes and pre-fetching inodes
CN107797758B (en) Date storage method, data access method and device
US9787755B2 (en) Method and device for browsing network data, and storage medium
CN107480269B (en) Object display method and system, medium and computing equipment
US10235293B2 (en) Tracking access pattern of inodes and pre-fetching inodes
EP2960778A1 (en) A storage abstraction layer and a system and a method thereof
CN113377548A (en) Identification generation method and device, computer equipment and storage medium
US11977785B2 (en) Non-volatile memory device-assisted live migration of virtual machine data
JP2015162174A (en) Program, information processor, electronic content display system and display suppression method
US10869089B1 (en) Systems and methods for preventing display of blocked content

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant