CN111176557B - Data reading and storing method and data reading and storing device - Google Patents

Data reading and storing method and data reading and storing device Download PDF

Info

Publication number
CN111176557B
CN111176557B CN201911105040.1A CN201911105040A CN111176557B CN 111176557 B CN111176557 B CN 111176557B CN 201911105040 A CN201911105040 A CN 201911105040A CN 111176557 B CN111176557 B CN 111176557B
Authority
CN
China
Prior art keywords
hard disk
file data
file
data
reading
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201911105040.1A
Other languages
Chinese (zh)
Other versions
CN111176557A (en
Inventor
胡义
马斌
商朝晖
胡柯良
刘强
王威
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
National Astronomical Observatories of CAS
Original Assignee
National Astronomical Observatories of CAS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by National Astronomical Observatories of CAS filed Critical National Astronomical Observatories of CAS
Priority to CN201911105040.1A priority Critical patent/CN111176557B/en
Publication of CN111176557A publication Critical patent/CN111176557A/en
Application granted granted Critical
Publication of CN111176557B publication Critical patent/CN111176557B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0602Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0625Power saving in storage systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0628Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0629Configuration or reconfiguration of storage systems
    • G06F3/0634Configuration or reconfiguration of storage systems by changing the state or mode of one or more devices
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0628Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0655Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
    • G06F3/0659Command handling arrangements, e.g. command buffers, queues, command scheduling

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The application relates to a data reading and storing method and device. The reading method comprises the following steps: receiving a request for reading file data; responding to the request for reading the file data and judging the running state of the hard disk in which the file data is positioned; copying the file data into a read buffer area when the hard disk is in an activated state, and triggering read buffer area cleaning operation in real time according to preset conditions in the copying process; recording the read access time after the copying is finished; judging the interval between the current time and the last read access time; and triggering the reading closing operation when the interval is greater than the first preset time. The method and the device can improve the speed of data reading and storing, and solve the problem of overlarge power consumption in the data reading and storing process.

Description

Data reading and storing method and data reading and storing device
Technical Field
The present application relates to the field of data storage and reading technologies, and in particular, to a data reading method, a data reading apparatus, a data storage method, and a data storage apparatus.
Background
In a hard environment in the field, such as a high-altitude mountain area, the south Pole inland and the like, an unattended scientific instrument device, such as a telescope, which runs on the earth tends to generate a large amount of data, and a large amount of energy is consumed in the process of data transmission storage and reading, while a hard environment in the field tends to have extremely limited power supply, which requires a data reading device and a storage device with low power consumption. At present, commercial storage equipment on the market usually has power consumption over 1000W according to the capacity, and cannot meet the data reading and storage requirements of low power consumption in a field environment.
Disclosure of Invention
In view of the above, it is necessary to provide a low power consumption data reading method, a data storage method, a data reading apparatus and a data storage apparatus to solve the problem of excessive power consumption of the conventional data reading method, data storage method, data reading apparatus and data storage apparatus.
A method of data reading, the method comprising:
receiving a request for reading file data;
responding to the request for reading the file data and judging the running state of the hard disk in which the file data is positioned; copying the file data into a read buffer area when the hard disk is in an activated state, and triggering read buffer area cleaning operation in real time according to preset conditions in the copying process; recording the read access time after the copying is finished; judging the interval between the current time and the last read access time;
and triggering the reading closing operation when the interval is greater than the first preset time.
Preferably, the operation state of the hard disk includes hard disk opening, hard disk mounting, and hard disk closing, the hard disk marking state includes a hard disk writing mark, a hard disk reading access mark, a hard disk damage mark, and a hard disk full mark, and when the operation state of the hard disk where the file data is located is determined, the method includes: and judging whether the hard disk is opened, mounted or closed.
Preferably, if the hard disk is mounted, the hard disk is judged to be in an activated state, the file data is copied into a read buffer area, a read access mark of the file data is recorded, meanwhile, file data generated within a second preset time before and after the file data is also copied into the read buffer area, and the hard disk mark is marked with the read access mark.
Preferably, if the hard disk is not started, a hard disk starting instruction is sent to a program controllable power supply distribution unit (hereinafter abbreviated as PDU), if the hard disk is started successfully, the hard disk is mounted, if the mounting fails, the operation is repeated twice, if the mounting fails, a hard disk array list is updated, the hard disk and file data damage is marked, and a hard disk closing instruction is sent to the PDU to return the file data damage to a user.
Preferably, if the hard disk is not mounted, a hard disk mounting instruction is sent to the PDU, if mounting fails, the mounting is repeated twice, if mounting fails, the hard disk array list is updated, damage of the hard disk and file data is marked, a hard disk closing instruction is sent to the PDU, and the damage of the file data is returned to a user.
Preferably, the preset condition is that the available remaining capacity of the read buffer is smaller than a first threshold; and in the process of copying the file data into the read buffer, when the available residual capacity of the read buffer is smaller than a first threshold value, meeting a preset condition and triggering the read buffer cleaning operation.
Preferably, the read buffer scrubbing operation includes the following steps:
finding out all file data with the last access time being greater than the third preset time, sorting the file data in a reverse order according to the last access time, deleting the file data one by one until the available residual capacity of the read buffer area is greater than a second threshold value, and finishing cleaning; and
preferably, the read buffer scrubbing operation further includes the steps of:
if the available capacity of the read buffer is still smaller than the second threshold after all the file data with the last access time larger than the third preset time are deleted, all the file data in the read buffer are sorted in a reverse order according to the total access times in a range backtracking from the current time to a fourth preset time, and are deleted one by one until the available residual of the read buffer is larger than the second threshold or all the file data are deleted, and the clearing is finished.
Preferably, the read closing operation includes the steps of:
deleting a read access mark of the hard disk;
judging whether the hard disk is marked with a write access mark or not; and
if not, sending a hard disk closing instruction to the PDU.
Preferably, in the step of determining whether the hard disk is marked with the write access flag, the method further includes the steps of:
and if the hard disk is marked with the write access mark, the data reading system keeps the current hard disk opening state unchanged.
A method of data storage comprising the steps of:
acquiring data information of a current file to be stored in a file storage queue, wherein the data information of the file comprises a file name and a file address;
acquiring a writable hard disk number;
submitting a file data storage request according to the file address and the hard disk number;
writing the file data requested by the file data storage request into the hard disk;
after the file is successfully written, updating a file information database and a hard disk array list, and marking a write access mark on the hard disk;
judging the available residual capacity of the hard disk;
and when the available residual capacity of the hard disk is smaller than a third threshold value, marking that the hard disk is full, and triggering write-off operation.
Preferably, the step of acquiring the number of the writable hard disk comprises:
finding out a hard disk with a health state and enough capacity from the hard disk array list;
judging the running state of the hard disk; and
and if the hard disk is in the activated state, returning the disk number of the hard disk.
Preferably, in the step of determining the available remaining capacity of the hard disk, the method further includes the steps of:
and if the available residual capacity of the hard disk is larger than a third threshold value, keeping the opening state of the current hard disk unchanged by the data storage system.
Preferably, the write close operation includes the steps of:
deleting the write access mark of the hard disk;
judging whether the hard disk is marked with a read access mark or not; and
if not, sending a hard disk closing instruction to the PDU.
Preferably, in the step of determining whether the hard disk is marked with the read access flag when the write-off operation is performed, the method further includes the steps of:
and if the hard disk is marked with the read access mark, the data storage system keeps the current hard disk opening state unchanged.
Preferably, the file information database comprises the number of a hard disk where the file is located, a check code of the MD5, the size of the file and the marking state of the file data; the hard disk array list comprises a hard disk number, the running state of the hard disk, the available residual capacity of the hard disk and the marking state of the hard disk.
Preferably, the hard disk marking state comprises a hard disk writing mark, a hard disk reading access mark, a hard disk damage mark and a hard disk full mark; the running state of the hard disk comprises hard disk opening, hard disk mounting and hard disk closing; the file data mark state comprises a file data write mark, a file data read access mark and a file data damage mark.
A data reading apparatus comprising a memory and a processor, the memory storing a computer program, the processor implementing the computer program when executing the computer program:
receiving a request for reading file data;
responding to the request for reading the file data and judging the running state of the hard disk in which the file data is positioned; copying the file data into a read buffer area when the hard disk is in an activated state, and triggering read buffer area cleaning operation in real time according to preset conditions in the copying process; recording the read access time after the copying is finished; judging the interval between the current time and the last read access time;
and triggering the reading closing operation when the interval is greater than the first preset time.
A data storage device comprising a memory and a processor, the memory storing a computer program which when executed by the processor effects the steps of:
acquiring data information of a current file to be stored in a file storage queue, wherein the data information of the file comprises a file name and a file address;
acquiring a writable hard disk number;
submitting a file data storage request according to the file address and the hard disk number;
writing the file data requested by the file data storage request into the hard disk;
after the file is successfully written, updating a file information database and a hard disk array list, and marking a write access mark on the hard disk;
judging the available residual capacity of the hard disk;
and when the available residual capacity of the hard disk is smaller than a third threshold value, marking that the hard disk is full, and triggering write-off operation.
Compared with the prior art, the data reading and storing method and device dynamically manage the hard disk running state (including hard disk opening, hard disk mounting and hard disk closing) and the hard disk marking state (hard disk reading access mark, hard disk writing access mark and hard disk damage mark) in the process of reading and storing the file data, so that the hard disk is opened and closed in real time, and the energy is saved to the maximum extent on the basis of minimizing influence on data storage and reading experience.
Drawings
FIG. 1 is a flow chart illustrating a data reading method according to an embodiment;
FIG. 2 is a flow chart illustrating a data reading method according to another embodiment;
FIG. 3 is a flow diagram illustrating a read buffer scrubbing operation in one embodiment;
FIG. 4 is a flow diagram illustrating a read shutdown operation in one embodiment;
FIG. 5 is a flow diagram illustrating a data storage method according to one embodiment;
FIG. 6 is a flow diagram illustrating a write close operation in one embodiment;
FIG. 7 is a block diagram of a data reading apparatus according to an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
In one embodiment, as shown in fig. 1, there is provided a data reading method including the steps of:
step S101, a file data reading request is received.
And step S102, judging the running state of the hard disk where the file data is located.
Wherein, the running state of the hard disk comprises: the method comprises the steps of starting a hard disk, mounting the hard disk and closing the hard disk.
Step S103, copying the file data into a read buffer area when the hard disk is in an activated state, and triggering the read buffer area cleaning operation in real time according to preset conditions.
And if the hard disk is mounted, judging that the hard disk is in an activated state.
The preset condition is that the available residual capacity of the read buffer is smaller than a first threshold value.
Specifically, in this step, the first threshold is set according to specific conditions. Such as an unmanned astronomical telescope operating in the south Pole, the first threshold is determined to be 20% of the total read buffer capacity, depending on the observation strategy of the telescope.
As shown in fig. 3, the read buffer scrubbing operation includes the following steps:
step S1031, finding out all file data with the last access time being greater than the third preset time, sorting the file data according to the reverse order of the last access time, and deleting the file data one by one;
specifically, in this step, the third preset time is set according to a specific situation. For example, an unmanned astronomical telescope operating in the south Pole determines the third preset time to be 7 days according to the observation strategy of the telescope.
Step S1032, judging whether the available residual capacity of the read buffer is larger than a second threshold value; if not, go to step S1033, if yes, go to step S1034;
specifically, in this step, the second threshold is set according to specific conditions. Such as an unmanned astronomical telescope operating in the south Pole, the second threshold is determined to be 70% of the total read buffer capacity, depending on the observation strategy of the telescope.
Step S1033, sorting all file data in the read buffer area in a reverse order according to the total access times in a fourth preset time range traced back from the previous time, deleting the file data one by one until the available residual capacity of the read buffer area is larger than a second threshold value or all the file data are deleted completely, and finishing the cleaning;
specifically, in this step, the fourth preset time is set according to specific conditions. For example, an unmanned astronomical telescope operating in the south Pole determines the fourth preset time as the last access time of the file data with the top file data in the read buffer according to the observation strategy of the telescope.
Step S1034, the cleaning is ended.
Step S104, recording the read access time after the copying is finished.
Step S105, the interval between the current time and the last read access time is determined.
And step S106, when the interval is greater than the first preset time, triggering the reading closing operation.
Specifically, in this step, the first preset time is set according to a specific situation. For example an unmanned astronomical telescope operating in the south Pole, said first predetermined time is determined to be 30 minutes according to the observation strategy of the telescope.
As shown in fig. 4, the read shutdown operation includes the following steps:
step S1061, deleting the read access mark of the hard disk;
step S1062, judging whether the hard disk is marked with a write access mark; if not, go to step S1063; if yes, go to step S1064;
step S1063, generating a hard disk closing instruction to the PDU;
step S1064, keeping the current hard disk open state unchanged.
Compared with the prior art, the data reading method dynamically manages the hard disk running state (including hard disk opening, hard disk mounting and hard disk closing) and the hard disk marking state (including hard disk reading access marking, hard disk writing access marking and hard disk damage marking) in the reading process of the file data, so that the hard disk is opened and closed in real time, and the energy is saved to the maximum extent on the basis of minimizing influence on data reading experience.
In another embodiment, as shown in fig. 2, there is provided a data reading method including the steps of:
in step S201, a file data reading request is acquired.
Step S202, responding to the request for reading the file data in step S201 and judging the running state of the hard disk in which the file data is positioned; if the hard disk where the file data is located is opened, go to step S203, and if the hard disk is not opened, go to step S209.
Step S203, continuing to determine whether the hard disk is mounted in a state where the hard disk of the file data is opened, if so, determining that the hard disk is in an activated state, and going to step S204, and if not, going to step S211.
Step S204, copying the file data into a read buffer area, recording a read access mark of the file data, copying the file data generated in a second preset time before and after the file data into the read buffer area, and marking the hard disk with the read access mark.
Specifically, in this step, the second preset time is set according to a specific situation. For example, an unmanned astronomical telescope operating in the south Pole, and the second preset time is determined to be 6 minutes according to the observation strategy of the telescope
Specifically, in this step, step S215 is triggered in real time according to a preset condition during the process of copying the file data into the read buffer.
The preset condition is that the available residual capacity of the read buffer is smaller than a first threshold value;
in step S205, after the copy-in is completed, the read access time is recorded, and then the process goes to step S206.
Step S206, judging whether the interval between the current time and the last read access time is greater than a first preset time, if so, turning to step S207; if not, go to step S208.
Step S207, a read close operation is triggered.
And step S208, keeping the opening state of the current hard disk unchanged.
Step S209 is to open the hard disk.
Step S210, judging whether the hard disk is successfully started, if so, turning to step S211; if not, go to step S213.
In step S211, the hard disk is mounted.
Step S212, judging whether the hard disk is mounted successfully, if so, turning to step S204; if not, go to step S213.
Step S213, updating the hard disk array list, marking the hard disk and file data damage marks, sending a hard disk closing instruction to the PDU, and returning file data damage to the user.
In step S214, in the process of copying the file data into the read buffer, it is determined in real time whether the available remaining capacity of the read buffer is smaller than the first threshold, and if so, the process goes to step S215.
Step S215, a read buffer cleanup operation is triggered.
The application provides a two-stage hybrid cache replacement strategy to maximize the data hit rate and reduce the number of hard disk data reading times and energy consumption. Specifically, when the file data is copied into the read buffer, the file data generated before and after the file data within a second preset time is also copied into the read buffer, and the read buffer cleaning operation is triggered in real time according to the available residual capacity of the read buffer.
In one embodiment, as shown in fig. 5, there is provided a data storage method comprising the steps of:
step S501, obtaining data information of a current file to be stored.
The file data information comprises a file name and a file address.
Step S502, acquiring the number of the writable hard disk.
In this step, the step of obtaining the writable hard disk number includes:
finding out a hard disk with a health state and enough capacity from the hard disk array list;
judging the running state of the hard disk; and
and if the hard disk is in the activated state, returning the disk number of the hard disk.
And step S503, submitting a file data storage request according to the file address and the hard disk number.
Step S504, writing the requested file data into the hard disk.
And step S505, after the writing is finished, updating the file information database and the hard disk array list.
In this step, the file information database comprises the number of the hard disk where the file is located, the MD5 validation code, the size of the file and the marking state of the file data; the hard disk array list comprises a hard disk number, the running state of the hard disk, the available residual capacity of the hard disk and the hard disk marking state. The file data mark state comprises a file data write mark, a file data read access mark and a file data damage mark. The hard disk mark state comprises a hard disk write-in mark, a hard disk read-access mark, a hard disk damage mark and a hard disk full mark; the running state of the hard disk comprises hard disk opening, hard disk mounting and hard disk closing;
step S506, judging whether the available residual capacity of the hard disk is smaller than a third threshold value, if so, turning to step S507; if not, go to step S508.
Specifically, in this step, the third threshold is set according to specific conditions. Such as an unmanned astronomical telescope operating in the south Pole, the third threshold is determined to be the greater of both 5% of the total capacity of the read buffer and the maximum file data size in the single file data, depending on the observation strategy of the telescope.
In step S507, a write close operation is triggered.
As shown in fig. 6, the write close operation includes the following steps:
step S5071, delete the write access flag of the hard disk;
step 5072, determining whether the hard disk is marked with a read access mark, if yes, going to step 5073; if not, go to step S5074;
step S5073, sending a hard disk close command to the PDU;
in step S5074, the current hard disk open state is kept unchanged.
Step S508), the current hard disk opening state is kept unchanged.
It should be understood that although the various steps in the flow charts of fig. 1-6 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in fig. 1-6 may include multiple sub-steps or multiple stages that are not necessarily performed at the same time, but may be performed at different times, and the order of performance of the sub-steps or stages is not necessarily sequential, but may be performed in turn or alternating with other steps or at least some of the sub-steps or stages of other steps.
In one embodiment, there is provided a data reading apparatus comprising a memory and a processor, the memory storing a computer program, the processor implementing the following steps when executing the computer program:
receiving a request for reading file data;
responding to the request for reading the file data and judging the running state of the hard disk in which the file data is positioned; copying the file data into a read buffer area when the hard disk is in an activated state, and triggering read buffer area cleaning operation in real time according to preset conditions in the copying process; recording the read access time after the copying is finished; judging the interval between the current time and the last read access time;
and triggering the reading closing operation when the interval is greater than the first preset time.
In one embodiment, a reading apparatus is provided, the internal structure of which may be as shown in fig. 7. The reading device comprises a processor, a memory, a network interface, a display screen and an input device which are connected through a system bus. Wherein the processor of the reading device is used for providing calculation and control capability, the memory of the reading device is stored with an operating system and a computer program, the network interface of the reading device is used for communicating with an external terminal through network connection, and the computer program is executed by the processor to realize a data reading method.
It will be appreciated by those skilled in the art that the configuration shown in fig. 7 is a block diagram of only a portion of the configuration associated with the present application and does not constitute a limitation on the reading apparatus to which the present application is applied, and a particular reading apparatus may include more or less components than those shown in the drawings, or combine certain components, or have a different arrangement of components.
In one embodiment, there is provided a data storage device comprising a memory and a processor, the memory storing a computer program which when executed by the processor performs the steps of:
acquiring data information of a current file to be stored in a file storage queue, wherein the data information of the file comprises a file name and a file address;
acquiring a writable hard disk number;
submitting a file data storage request according to the file address and the hard disk number;
writing the file data requested by the file data storage request into the hard disk;
after the file is successfully written, updating a file information database and a hard disk array list, and marking a write access mark on the hard disk;
judging the available residual capacity of the hard disk;
and when the available residual capacity of the hard disk is smaller than a third threshold value, marking that the hard disk is full, and triggering write-off operation.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and can include the processes of the embodiments of the methods described above when the computer program is executed.
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features. When technical features in different embodiments are represented in the same drawing, it can be seen that the drawing also discloses a combination of the embodiments concerned.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims (6)

1. A data reading and storing method is characterized by comprising the following steps:
receiving a request for reading file data;
responding to the request for reading the file data and judging the running state of the hard disk in which the file data is positioned;
copying the file data into a read buffer area when the hard disk is in an activated state, and triggering read buffer area cleaning operation in real time according to preset conditions in the copying process;
recording the read access time after the copying is finished;
judging the interval between the current time and the last read access time;
when the interval is larger than first preset time, triggering a read closing operation;
the hard disk marking state comprises a hard disk writing mark, a hard disk reading access mark, a hard disk damage mark and a hard disk full mark; the operation state of the hard disk comprises hard disk opening, hard disk mounting and hard disk closing, and when the operation state of the hard disk where the file data is located is judged, the method comprises the following steps: judging whether the hard disk is started, mounted or closed; and
if the hard disk is mounted, judging that the hard disk is in an activated state, copying the file data into a read buffer area, recording a read access mark of the file data, copying the file data generated within a second preset time before and after the file data into the read buffer area, and marking the hard disk with the read access mark;
the data storage method comprises the following steps:
acquiring data information of a current file to be stored in a file storage queue, wherein the data information of the file comprises a file name and a file address;
finding out a hard disk with a health state and enough capacity from the hard disk array list, judging the running state of the hard disk, and returning a hard disk number if the hard disk is in an activated state;
submitting a file data storage request according to the file address and the hard disk number;
writing the file data requested by the file data storage request into the hard disk;
after the file is successfully written, updating a file information database and a hard disk array list, and marking a write access mark on the hard disk;
judging the available residual capacity of the hard disk;
and when the available residual capacity of the hard disk is smaller than a third threshold value, marking that the hard disk is full, and triggering write-off operation.
2. The data reading and storing method according to claim 1, wherein the preset condition is that the available remaining capacity of the read buffer is less than a first threshold; and
and in the process of copying the file data into the read buffer, when the available residual capacity of the read buffer is smaller than a first threshold value, meeting a preset condition and triggering the read buffer cleaning operation.
3. The data reading and storing method according to claim 1, wherein the read buffer scrubbing operation comprises the steps of:
finding out all file data with the last access time being greater than the third preset time, sorting the file data in a reverse order according to the last access time, deleting the file data one by one until the available residual capacity of the read buffer area is greater than a second threshold value, and finishing cleaning; and
if all the file data with the last access time larger than the third preset time are deleted, the available residual capacity of the read buffer is still smaller than the second threshold, all the file data in the read buffer are sorted in a reverse order according to the total access times in a range backtracking from the previous time to a fourth preset time, and are deleted one by one until the available residual capacity of the read buffer is larger than the second threshold or all the file data are deleted, and the clearing is finished.
4. The data reading and storing method according to claim 1, wherein the read shutdown operation comprises the steps of:
deleting a read access mark of the hard disk;
judging whether the hard disk is marked with a write access mark or not; and
and if not, sending a hard disk closing instruction to the program controllable power supply distribution unit.
5. The data reading and storing method according to claim 1, wherein the write-off operation comprises the steps of:
deleting the write access mark of the hard disk;
judging whether the hard disk is marked with a read access mark or not; and
and if not, sending a hard disk closing instruction to the program controllable power supply distribution unit.
6. A data reading and storing device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the data reading and storing method according to any one of claims 1 to 5 when executing the computer program.
CN201911105040.1A 2019-11-13 2019-11-13 Data reading and storing method and data reading and storing device Active CN111176557B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911105040.1A CN111176557B (en) 2019-11-13 2019-11-13 Data reading and storing method and data reading and storing device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911105040.1A CN111176557B (en) 2019-11-13 2019-11-13 Data reading and storing method and data reading and storing device

Publications (2)

Publication Number Publication Date
CN111176557A CN111176557A (en) 2020-05-19
CN111176557B true CN111176557B (en) 2022-08-02

Family

ID=70657129

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911105040.1A Active CN111176557B (en) 2019-11-13 2019-11-13 Data reading and storing method and data reading and storing device

Country Status (1)

Country Link
CN (1) CN111176557B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115981548A (en) * 2021-10-14 2023-04-18 华为技术有限公司 Flow control method and device

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1694556A (en) * 2004-04-30 2005-11-09 微软公司 Method for maintaining wireless network response time while saving wireless adapter power
CN102087586A (en) * 2009-12-02 2011-06-08 成都市华为赛门铁克科技有限公司 Data processing method and device
CN109766462A (en) * 2018-12-27 2019-05-17 山东信通电子股份有限公司 Image file read method, apparatus and system in transmission line of electricity monitoring system

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10572407B2 (en) * 2017-08-11 2020-02-25 Western Digital Technologies, Inc. Hybrid data storage array

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1694556A (en) * 2004-04-30 2005-11-09 微软公司 Method for maintaining wireless network response time while saving wireless adapter power
CN102087586A (en) * 2009-12-02 2011-06-08 成都市华为赛门铁克科技有限公司 Data processing method and device
CN109766462A (en) * 2018-12-27 2019-05-17 山东信通电子股份有限公司 Image file read method, apparatus and system in transmission line of electricity monitoring system

Also Published As

Publication number Publication date
CN111176557A (en) 2020-05-19

Similar Documents

Publication Publication Date Title
CN108268219B (en) Method and device for processing IO (input/output) request
US9177028B2 (en) Deduplicating storage with enhanced frequent-block detection
US7590722B2 (en) Apparatus and methods for managing data used by a mobile device
CN105302478B (en) A kind of date storage method and electronic equipment
EP3229142A1 (en) Read cache management method and device based on solid state drive
US20160011800A1 (en) Techniques for reducing memory write operations using coalescing memory buffers and difference information
KR100577384B1 (en) Method for page replacement using information on page
US20110167239A1 (en) Methods and apparatuses for usage based allocation block size tuning
CN104808952B (en) data cache method and device
CN107168647A (en) FLASH data read-write methods and system
EP3252609A1 (en) Cache data determination method and device
JP2014078262A (en) Techniques for multi-memory device lifetime management
CN107786623A (en) Message asynchronous processing method and device
CN110837480A (en) Processing method and device of cache data, computer storage medium and electronic equipment
CN110968524B (en) Data storage control method, device, storage medium and electronic device
CN106528026B (en) Text display method and device
CN108664577B (en) File management method and system based on FLASH idle area
CN110534151A (en) Realize method, apparatus, computer equipment and the storage medium wiped before write-in
CN111176557B (en) Data reading and storing method and data reading and storing device
CN109783019B (en) Intelligent data storage management method and device
US5584015A (en) Buffer memory management method, recording medium, and computer system incorporating same
US20120047330A1 (en) I/o efficiency of persistent caches in a storage system
CN108829345B (en) Data processing method of log file and terminal equipment
CN113672166A (en) Data processing method and device, electronic equipment and storage medium
CN111913913A (en) Access request processing method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant