CN112035068A - Data writing method and device, electronic equipment and storage medium - Google Patents

Data writing method and device, electronic equipment and storage medium Download PDF

Info

Publication number
CN112035068A
CN112035068A CN202010938141.3A CN202010938141A CN112035068A CN 112035068 A CN112035068 A CN 112035068A CN 202010938141 A CN202010938141 A CN 202010938141A CN 112035068 A CN112035068 A CN 112035068A
Authority
CN
China
Prior art keywords
management unit
storage
data
logic management
target
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010938141.3A
Other languages
Chinese (zh)
Inventor
刘泽许
李明强
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangzhou Tupu Network Technology Co ltd
Original Assignee
Guangzhou Tupu Network Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangzhou Tupu Network Technology Co ltd filed Critical Guangzhou Tupu Network Technology Co ltd
Priority to CN202010938141.3A priority Critical patent/CN112035068A/en
Publication of CN112035068A publication Critical patent/CN112035068A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0608—Saving storage space on storage systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0629—Configuration or reconfiguration of storage systems
    • G06F3/0631—Configuration or reconfiguration of storage systems by allocating resources to storage systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638—Organizing or formatting or addressing of data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

The application provides a data writing method, a data writing device, an electronic device and a storage medium, wherein the method comprises the following steps: acquiring a data writing request, wherein the data writing request carries a storage requirement; judging whether a first target logic management unit matched with a storage requirement exists in an expiration list in a database, wherein at least one occupied logic management unit which is expired is stored in the expiration list; and if so, writing the data to be written into the storage space corresponding to the first target logic management unit.

Description

Data writing method and device, electronic equipment and storage medium
Technical Field
The present application relates to the field of storage technologies in communication networks, and in particular, to a data writing method and apparatus, an electronic device, and a storage medium.
Background
In the conventional method for writing data in the distributed storage system, a data writing request is generally used to allocate a free storage space for at least a part of the data, and then a free logic management unit is applied to the storage device server, so that the data is written into the free storage space corresponding to the free logic management unit obtained by the application.
However, the above data writing method has the defect of insufficient storage resources in the face of increasingly huge data storage requirements.
Disclosure of Invention
An object of the embodiments of the present application is to provide a data writing method, an apparatus, an electronic device, and a storage medium, so as to solve a problem that storage resources are insufficient in a manner that data writing directly applies for an idle logical unit to a storage device server and then writes the data into an idle storage space corresponding to an idle logical management unit that is obtained by the application, in a case of meeting an increasingly large data storage requirement.
In a first aspect, an embodiment of the present invention provides a data writing method, where the method includes: acquiring a data writing request, wherein the data writing request carries a storage requirement; judging whether a first target logic management unit matched with the storage requirement exists in an expiration list in a database, wherein at least one occupied and expired logic management unit is stored in the expiration list; and if so, writing the data to be written into the storage space corresponding to the first target logic management unit.
In the designed data writing method, the first target logic management unit matched with the storage requirement of the data to be written is searched from at least one occupied and expired logic management unit in the expiration list stored in the database when the data is written, and the data to be written is written into the storage space corresponding to the searched first target logic management unit, so that the occupied and expired logic management unit can be repeatedly utilized, the problem that the storage resource is insufficient under the condition of meeting increasingly huge data storage requirements in a mode that the current data writing directly applies for an idle logic unit from a storage device server and then writes the data into the idle storage space corresponding to the idle logic management unit obtained by application is solved, and further mass storage can be carried out under the environment of limited storage resources, saving storage resources and cost.
In an optional implementation manner of the first aspect, the determining whether a first target logical management unit matching the storage requirement exists in an expiration list in a database includes: judging whether a logic management unit with the validity period matched with the storage duration exists in an expiration list in a database; and if so, determining the matched logic management unit as the first target logic management unit.
In an optional implementation manner of the first aspect, the determining whether a first target logic management unit matching the storage requirement exists in an expiration list in a database includes: judging whether at least one logic management unit with the validity period matched with the storage duration is found in the expiration list; if yes, obtaining the memory space of each searched logic management unit, and judging whether a logic management unit with the memory space larger than the data size exists or not; and if so, selecting the logic management unit with the smallest difference between the storage amount and the data amount from the logic management units with the storage amount larger than the data amount, and determining the logic management unit as the first target logic management unit.
In an optional implementation manner of the first aspect, after the determining whether there is a logical management unit having an amount of storage greater than the data amount of the write data, the method further includes: if not, judging whether the total storage capacity of all the searched logic management units is larger than the data volume of the written data or not; and if so, selecting a plurality of logic management units from all logic management units as the first target logic management unit, wherein the storage capacity of the plurality of logic management units is larger than the data volume of the written data.
In an optional implementation manner of the first aspect, before determining whether the first target logical management unit matching the storage requirement exists in the expiration list in the database, the method further includes: acquiring a current date, an occupied date and a valid period of each occupied logic management unit; judging whether occupied logic management units with the time length of subtracting the occupied date from the current date being greater than the effective period exist, if so, storing the occupied logic management units with the time length of subtracting the occupied date from the current date being greater than the effective period in the expiration list.
In an optional implementation manner of the first aspect, after the writing the data to be written into the storage space corresponding to the target logical management unit, the method further includes: and determining a path of the data to be written according to the storage cluster and the storage space of the first target logic management unit and marking the path of the written data in the first target logic management unit.
In an optional implementation manner of the first aspect, after determining whether there is a first target logical management unit matching the storage requirement in an expiration list in the database, the method further includes: if not, judging whether a second target logic management unit matched with the storage requirement exists in a plurality of logic management units in a free list corresponding to each storage space stored in the database; and if so, writing the data to be written into a storage space corresponding to the second target logic management unit.
In an optional implementation manner of the first aspect, after the writing the data to be written into the storage space corresponding to the first target logical management unit, the method further includes: and deleting the first target logic management unit from the expiration list.
In a second aspect, an embodiment of the present invention provides a data writing apparatus, where the apparatus includes: the data writing device comprises an acquisition module, a storage module and a storage module, wherein the acquisition module is used for acquiring a data writing request which carries a storage requirement; the judging module is used for judging whether a first target logic management unit matched with the storage requirement exists in an expiration list in a database, and at least one occupied logic management unit which is expired is stored in the expiration list; and the writing module is used for writing the data to be written into the storage space corresponding to the first target logic management unit after judging that the first target logic management unit exists.
In the data writing device designed above, the first target logic management unit matching with the storage requirement of the data to be written is searched from at least one occupied and expired logic management unit in the expiration list stored in the database when the data is written, and then the data to be written is written into the storage space corresponding to the searched first target logic management unit, so that the occupied and expired logic management unit can be reused, thereby solving the problem that the storage resource is insufficient in the case of meeting increasingly huge data storage requirements in the current way that data writing is directly applied to a storage device server for an idle logic unit and then the data is written into the idle storage space corresponding to the idle logic management unit obtained by application, and further enabling massive storage to be performed in the limited storage resource environment, saving storage resources and cost.
In an optional implementation manner of the second aspect, the storage requirement includes a storage duration of data to be written, and the determining module is specifically configured to determine whether a logic management unit whose validity period matches the storage duration exists in an expiration list in a database; and if so, determining the matched logic management unit as the first target logic management unit.
In an optional implementation manner of the second aspect, the storage requirement includes a storage duration and a data amount of data to be written, and the determining module is further specifically configured to determine whether at least one logic management unit with a validity period matching the storage duration is found in the expiration list; if yes, obtaining the memory space of each searched logic management unit, and judging whether a logic management unit with the memory space larger than the data size exists or not; and if so, selecting the logic management unit with the smallest difference between the storage amount and the data amount from the logic management units with the storage amount larger than the data amount, and determining the logic management unit as the first target logic management unit.
In an optional implementation manner of the second aspect, the obtaining module is further configured to obtain a current date, an occupation date of each occupied logic management unit, and a validity period; the judging module is also used for judging whether an occupied logic management unit with the time length of the current date minus the occupied date being greater than the effective period exists; and the storage module is used for storing the occupied logic management units with the duration of subtracting the occupied date from the current date larger than the effective period in the expiration list.
In an optional implementation manner of the second aspect, the apparatus further includes a determining module, configured to determine a path of the data to be written according to the storage cluster and the storage space to which the first target logical management unit belongs, and identify the path of the write data in the first target logical management unit.
In an optional implementation manner of the second aspect, the determining module is further configured to determine whether a second target logical management unit matching the storage requirement exists in the plurality of logical management units in the free list corresponding to each storage space stored in the database; the writing module is further configured to write the data to be written into a storage space corresponding to the second target logical management unit.
In an optional implementation manner of the second aspect, the apparatus further includes a deletion module configured to delete the first target logical management unit from the expiration list.
In a third aspect, an embodiment provides an electronic device, including a memory and a processor, where the memory stores a computer program, and the processor executes the computer program to perform the method in the first aspect or any optional implementation manner of the first aspect.
In a fourth aspect, embodiments provide a non-transitory readable storage medium having stored thereon a computer program which, when executed by a processor, performs the method of the first aspect, any optional implementation manner of the first aspect.
In a fifth aspect, embodiments provide a computer program product, which when run on a computer, causes the computer to execute the method of the first aspect or any optional implementation manner of the first aspect.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments of the present application will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and that those skilled in the art can also obtain other related drawings based on the drawings without inventive efforts.
Fig. 1 is a schematic structural diagram of a distributed storage system according to an embodiment of the present application;
FIG. 2 is a first flowchart of a data writing method according to an embodiment of the present application;
FIG. 3 is a second flowchart of a data writing method according to an embodiment of the present application;
FIG. 4 is a third flowchart of a data writing method according to an embodiment of the present application;
fig. 5 is a fourth flowchart of a data writing method according to an embodiment of the present application;
fig. 6 is a fifth flowchart of a data writing method according to an embodiment of the present application;
fig. 7 is a sixth flowchart of a data writing method according to an embodiment of the present application;
fig. 8 is a seventh flowchart of a data writing method according to an embodiment of the present application;
FIG. 9 is a schematic structural diagram of a data writing apparatus according to an embodiment of the present application;
fig. 10 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Icon: 10-distributed storage cluster; 20-a database; 30-calling end; 200-an obtaining module; 201-a judgment module; 202-a write module; 203-a storage module; 204-a determination module; 205-delete module; 3-an electronic device; 301-a processor; 302-a memory; 303-communication bus.
Detailed Description
The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application.
First embodiment
The embodiment of the application provides a data writing method, which is used for writing data into a storage space of a distributed storage cluster, and is applied to a calling end or a client end of a distributed storage system, where the distributed system may include a distributed storage cluster 10, a database 20, and a calling end 30, as shown in fig. 1, the calling end 30 may receive a user access request, such as a data writing request, a data reading request, and the like, may configure configuration information of the distributed storage cluster 10 into the database 20 in advance and configure a storage space bucket for which the cluster applies for registration into the database, so that the calling end 30 may directly perform message transfer with the database 20, and implement data writing and the like.
As shown in fig. 2, it specifically describes the method for writing data by using the above-mentioned distributed storage system, and the method is applied to the calling end 30, and specifically includes the following steps:
step S100: and acquiring a data writing request, wherein the data writing request carries a storage requirement.
Step S102: judging whether a first target logic management unit matched with the storage requirement exists in an expired list in the database, if so, turning to the step S104; if not, go to step S1050.
Step S104: and writing the data to be written into the storage space corresponding to the first target logic management unit.
In step S100, a calling end obtains a data writing request initiated by a user, where the data writing request carries a storage requirement of data to be written, and the storage requirement includes a storage duration of the data to be written, a data amount of the data to be written, and the like, and after obtaining the data writing request, step S102 may be executed.
Before explaining step S102, the logical management unit is explained first: the logical management unit, which is hereinafter referred to as "chunk" for short, may include a cluster to which the logical management unit belongs, the chunk to which the logical management unit belongs, a validity period, and a storage start sequence number and length corresponding to the storage start sequence number and length, that is, a storage amount, where the validity period represents a period during which data may be stored in a storage space managed by the chunk, and the chunk corresponds to a medium through which data to be written may be written into the storage space to which the chunk belongs.
On the basis of the above, in step S102, the calling end searches, in an expiration list stored in the database, whether the first target logical management unit matching the storage requirement exists, where the first target logical management unit matching the storage requirement, that is, some storage characteristics (such as a storage validity period and a storage space size) of the storage space managed by the chunk are consistent with the storage requirement of the data to be written, for example, the storage duration needs to be 30 days, then a chunk with a validity period of 30 days needs to be found in the expiration list, the data size is a certain length, and then the storage space needs to have a corresponding certain length. In addition, at least one occupied and occupied expired logical management unit chunk is stored in the expiration list, and chunks in the expiration list are all free, and the database can recycle the occupied and occupied expired logical management unit chunk and then store the same in the list in a unified manner to form the expiration list.
If the first target logical management unit chunk matching the storage requirement is found from the expiration list in step S102, step S104 is executed to write the data to be written into the storage space corresponding to the first target logical management unit chunk. In addition, it should be noted here that the first target logical management unit chunk is occupied before but occupied is expired, and then the data to be written in this time can be overwritten with the expired data when the data is written in this time, so as to save storage resources.
In the designed data writing method, the first target logic management unit with the validity period matched with the storage duration of the data to be written is searched from at least one occupied and expired logic management unit in the expiration list stored in the database when the data is written, and the data to be written is written into the storage space corresponding to the searched first target logic management unit, so that the occupied and expired logic management unit can be reused, and when the data to be written is written into the first target logic management unit, the data occupied and expired in the first target logic management unit can be covered by the data to be written, thereby solving the problem that the storage resources are insufficient under the condition of facing increasingly huge data storage requirements in the mode that the current data writing directly applies for a free logic unit from a storage device server and then writes the data into the free storage space corresponding to the free logic management unit obtained by application, and further, mass storage can be performed under the environment of limited storage resources, and the storage resources and the cost are saved.
In an optional implementation manner of this embodiment, after writing the data to be written into the storage space corresponding to the first target logical management unit through step S104, as shown in fig. 3, the method may further include the following steps:
step S106: and determining a path of data to be written according to the storage cluster and the storage space of the first target logic management unit and marking the path of the written data in the first target logic management unit.
In the above step, after the data to be written is written into the storage space corresponding to the first target logical management unit, a path of the data to be written may be determined according to the storage cluster and the storage space to which the first target management unit belongs, and then the path is identified in the first target logical management unit, so that the user may access the written data through the path when accessing.
In an optional implementation manner of this embodiment, after the step S104 is executed to write the data to be written into the storage space corresponding to the first target logical management unit, the first target logical management unit may be deleted from the expiration list.
In an optional implementation manner of this embodiment, it has been described above that the storage requirement may include a storage duration, and on this basis, the step S102 determines whether the first target logic management unit matching the storage requirement exists in the expiration list in the database, as shown in fig. 4, specifically, the following steps may be performed:
step S1020: and judging whether a logic management unit with the validity period matched with the storage duration exists in the expiration list in the database, and if so, turning to the step S1021.
Step S1021: and determining the matched logic management unit as the first target logic management unit.
In the above step, if the data to be written only has the requirement of the storage duration, the logic management unit with the validity period matching the storage duration is searched in the expiration list, and if the logic management unit is searched, the searched logic management unit is determined as the first target logic management unit, and then step S104 is executed.
In an optional implementation manner of this embodiment, when the storage requirement includes not only the storage duration but also the data size, as shown in fig. 5, step S102 may specifically include the following steps:
step S1022: and judging whether at least one logic management unit with the validity period matched with the storage duration is found in the expiration list, and if so, going to the step S1023.
Step S1023: and acquiring the storage capacity of each searched logic management unit.
Step S1024: and judging whether a logic management unit with the storage capacity larger than the data volume exists or not, if so, turning to a step S1025, and if not, turning to a step S1026.
Step S1025: and selecting the logic management unit with the smallest difference between the storage amount and the data amount from the logic management units with the storage amount larger than the data amount to determine as the first target logic management unit.
In the above steps, first, it is determined whether at least one logic management unit with a validity period matching the storage time length is found in the expiration list, if so, it indicates that there is a logic management unit with a sufficient validity period for the storage time length in the expiration list, step S1023 may be performed to obtain the storage amount of each found logic management unit, then step S1024 is performed to determine whether there is a logic management unit with a storage amount greater than the data amount, that is, if there is a logic management unit with a storage amount greater than the data amount, then writing of the data to be written may be implemented by using one logic management unit with a storage amount greater than the data amount, and after step S1024 determines that there is a logic management unit with a storage amount that is the smallest difference between the storage amount and the data amount, step S1025 may be performed to select the logic management unit with a storage amount greater than the data amount from the logic management units, in this step, if only one logic management unit with the storage capacity larger than the data volume exists, the logic management unit is directly determined as a first target logic management unit; if a plurality of logic management units with the storage capacity larger than the data volume exist, selecting the logic management unit with the storage capacity which is the smallest difference with the data volume from the plurality of logic management units and determining the logic management unit as a first target logic management unit.
In the embodiment designed above, after at least one logic management unit with the validity period matched with the storage duration is found in the expiration list, the storage amount of the matched logic management unit is obtained, and then the logic management unit with the minimum difference between the storage amount and the data amount is determined by analyzing the data amount and the storage amount and is further determined as the first target logic management unit, so that the determined first target logic management unit can select the logic management unit with the minimum difference between the storage amount and the data amount of the data to be written while meeting the storage requirement, and further, the storage resource is saved.
In an alternative implementation manner of this embodiment, it has been described above that if it is determined in step S1024 that there is no logic management unit with a storage amount greater than the data amount, the step goes to step S1026, where, as shown in fig. 6, the step of step S1026 specifically includes:
step S1026: and judging whether the total storage capacity of all the searched logic management units is larger than the data volume of the written data, if so, turning to the step S1027.
Step S1027: and selecting a plurality of logic management units from all the logic management units as a first target logic management unit, wherein the storage capacity of the plurality of logic management units is larger than the data capacity of the written data.
On the basis of determining that there is no logic management unit with a storage capacity greater than the data volume in the logic management units with the validity period and the storage duration matched, executing step S1026 to determine whether the total storage capacity of all the searched logic management units is greater than the data volume of the write-in data, if so, the number of the logic management units searched by the specification is multiple, but the storage capacity of each single logic management unit in the multiple is less than the data volume of the write-in data, on this basis, executing step S1027 to select a plurality of logic management units from all the logic management units as first target logic management units, specifically, the selection mode may be that the storage capacities are added according to the sequence of the logic management units in the expiration list, and further, when the added storage capacity is greater than the data volume, the plurality of added logic management units may be determined as the first target logic management unit, or the storage amounts may be added in order from the maximum storage amount to the minimum storage amount, and when the added storage amount is greater than the data amount, the added logical management units may be determined as the first target logical management unit.
In an alternative embodiment of this embodiment, it has been described above that the database stores the expiration list, and before that, a specific way for the database to reclaim the occupied and expired logical management units in the expiration list is as shown in fig. 7, which includes the following steps:
step S90: the current date, the occupied date of each occupied logical management unit, and the valid period are acquired.
Step S91: it is determined whether there is an occupied logical management unit whose length of time obtained by subtracting the occupied date from the current date is greater than the validity period, and if so, the process goes to step S92.
Step S92: and storing the occupied logic management unit of which the duration of the current date minus the occupied date is greater than the effective period in an expiration list.
In the above steps, a timing task may be designed, for example, acquiring the current date, the occupied date of each occupied logic management unit and the valid period every day, and then executing step S91 to determine whether there is an occupied logic management unit whose duration of the current date minus the occupied date is greater than the valid period, if so, it indicates that the logic management unit is occupied and expired, and then it is stored in the expiration list.
Specifically, the occupied logic management unit may be stored according to a certain rule, where the rule is stored in a corresponding occupation table according to the occupied time, for example, the rule is stored in the occupation table 2020-06-23 when the rule is occupied on 23/06/2020/07. And then setting a timing task to obtain all occupation tables, comparing the effective period of each logic management unit in the occupation tables according to the dates of the occupation tables, if the number of days of subtracting the dates of the occupation tables from the current date is greater than the effective period, considering that the logic management units of which the number of days of subtracting the dates of the occupation tables from the current date is greater than the effective period are occupied to be expired, and recovering the occupied expired logic management units, wherein the specific rule of recovery is to delete the logic management units from the occupation tables and then add the logic management units to the expired list.
In an alternative implementation manner of this embodiment, it has been described before that, after determining that the first target logical unit matching the storage requirement does not exist in the expiration list in the database in step S102, step S1050 is performed, where, as shown in fig. 8, step S1050 specifically includes the following steps:
step S1050: and judging whether a second target logic management unit matched with the storage requirement exists in a plurality of logic management units in the free list corresponding to each storage space stored in the database, and if so, turning to the step S1051.
Step S1051: and writing the data to be written into the storage space corresponding to the second target logic management unit.
In the above step, when the first target logic management unit matching the storage requirement is not found in the overdue list, then whether a second target logic management unit matching the storage requirement exists is found in a free list corresponding to each storage space stored in the database, and if so, the data to be written is written into the storage space corresponding to the second target logic management unit, wherein the manner of finding in the free list is similar to that of finding in the overdue list, and is not described herein again.
Specifically, after the second target logic management unit is found in the free list, the storage cluster corresponding to the second target logic management unit may be found, and then the second target logic management unit is applied to the corresponding storage cluster to be allocated, at this time, the storage space corresponding to the second target logic management unit also needs to update corresponding information, such as updating the allocated storage amount.
Second embodiment
Fig. 9 shows a schematic structural block diagram of a data writing device provided in the present application, and it should be understood that the device corresponds to the method embodiments executed in fig. 1 to 8, and can execute the steps involved in the method executed by the server in the first embodiment, and the specific functions of the device can be referred to the description above, and the detailed description is appropriately omitted here to avoid repetition. The device includes at least one software function that can be stored in memory in the form of software or firmware (firmware) or solidified in the Operating System (OS) of the device. Specifically, the apparatus includes: an obtaining module 200, configured to obtain a write data request, where the write data request carries a storage requirement; the judging module 201 is configured to judge whether a first target logic management unit matching the storage requirement exists in an expiration list in the database, where at least one occupied and expired logic management unit is stored in the expiration list; the writing module 202 is configured to write the data to be written into the storage space corresponding to the first target logical management unit after determining that the first target logical management unit exists.
In the data writing device designed above, the first target logic management unit matching with the storage requirement of the data to be written is searched from at least one occupied and expired logic management unit in the expiration list stored in the database when the data is written, and then the data to be written is written into the storage space corresponding to the searched first target logic management unit, so that the occupied and expired logic management unit can be reused, thereby solving the problem that the storage resource is insufficient in the case of meeting increasingly huge data storage requirements in the current way that data writing is directly applied to a storage device server for an idle logic unit and then the data is written into the idle storage space corresponding to the idle logic management unit obtained by application, and further enabling massive storage to be performed in the limited storage resource environment, saving storage resources and cost.
In an optional implementation manner of this embodiment, the storage requirement includes a storage duration of data to be written, and the determining module 201 is specifically configured to determine whether a logic management unit whose validity period matches the storage duration exists in an expiration list in the database; and if so, determining the matched logic management unit as a first target logic management unit.
In an optional implementation manner of this embodiment, the storage requirement includes a storage duration and a data amount of the data to be written, and the determining module 201 is further specifically configured to determine whether at least one logic management unit whose validity period matches the storage duration is found in the expiration list; if yes, the searched memory space of each logic management unit is obtained, and whether the logic management unit with the memory space larger than the data size exists is judged; and if so, selecting the logic management unit with the smallest difference between the storage amount and the data amount from the logic management units with the storage amount larger than the data amount to determine as the first target logic management unit.
In an optional implementation manner of this embodiment, the obtaining module 200 is further configured to obtain a current date, an occupied date of each occupied logic management unit, and a valid period; the judging module 201 is further configured to judge whether there is an occupied logic management unit whose duration of the current date minus the occupied date is greater than the validity period; the storage module 203 is configured to store the occupied logic management unit with the length of the current date minus the occupied date greater than the validity period in the expiration list.
In an optional implementation manner of this embodiment, the apparatus further includes a determining module 204, configured to determine a path to be written with data according to the storage cluster and the storage space to which the first target logical management unit belongs, and identify the path to be written with data in the first target logical management unit.
In an optional implementation manner of this embodiment, the determining module 201 is further configured to determine whether a second target logical management unit matching the storage requirement exists in the plurality of logical management units in the free list corresponding to each storage space stored in the database; the writing module 202 is further configured to write the data to be written into a storage space corresponding to the second target logical management unit.
In an optional implementation manner of this embodiment, the apparatus further includes a deleting module 205, configured to delete the first target logical management unit from the expiration list.
Third embodiment
As shown in fig. 10, the present application provides an electronic device 3 including: a processor 301 and a memory 302, the processor 301 and the memory 302 being interconnected and communicating with each other via a communication bus 303 and/or other form of connection mechanism (not shown), the memory 302 storing a computer program executable by the processor 301, the processor 301 executing the computer program when the computing device is running to perform the method of the first embodiment, any alternative implementation of the first embodiment, such as steps S100 to S104: acquiring a data writing request, wherein the data writing request carries a storage requirement; judging whether a first target logic management unit matched with the storage requirement exists in an expiration list in the database; and if so, writing the data to be written into the storage space corresponding to the first target logic management unit.
The present application provides a storage medium having a computer program stored thereon, where the computer program is executed by a processor to perform the method of the first embodiment or any alternative implementation manner of the first embodiment.
The storage medium may be implemented by any type of volatile or nonvolatile storage device or combination thereof, such as a Static Random Access Memory (SRAM), an Electrically Erasable Programmable Read-Only Memory (EEPROM), an Erasable Programmable Read-Only Memory (EPROM), a Programmable Read-Only Memory (PROM), a Read-Only Memory (ROM), a magnetic Memory, a flash Memory, a magnetic disk, or an optical disk.
The present application provides a computer program product which, when run on a computer, causes the computer to perform the method of the first embodiment, any of its alternative implementations.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and there may be other divisions when actually implemented, and for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some communication interfaces, and may be in an electrical, mechanical or other form.
In addition, units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
Furthermore, the functional modules in the embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
It should be noted that the functions, if implemented in the form of software functional modules and sold or used as independent products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application or portions thereof that substantially contribute to the prior art may be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
In this document, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions.
The above description is only an example of the present application and is not intended to limit the scope of the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims (10)

1. A method of writing data, the method comprising:
acquiring a data writing request, wherein the data writing request carries a storage requirement;
judging whether a first target logic management unit matched with the storage requirement exists in an expiration list in a database, wherein at least one occupied and expired logic management unit is stored in the expiration list;
and if so, writing the data to be written into the storage space corresponding to the first target logic management unit.
2. The method according to claim 1, wherein the storage requirement includes a storage duration of data to be written, and the determining whether the first target logical management unit matching the storage requirement exists in an expiration list in the database comprises:
judging whether a logic management unit with the validity period matched with the storage duration exists in an expiration list in a database;
and if so, determining the matched logic management unit as the first target logic management unit.
3. The method according to claim 1, wherein the storage requirement includes a storage duration and a data size of data to be written, and the determining whether the first target logical management unit matching the storage requirement exists in an expiration list in the database comprises:
judging whether at least one logic management unit with the validity period matched with the storage duration is found in the expiration list;
if yes, obtaining the memory space of each searched logic management unit, and judging whether a logic management unit with the memory space larger than the data size exists or not;
and if so, selecting the logic management unit with the smallest difference between the storage amount and the data amount from the logic management units with the storage amount larger than the data amount, and determining the logic management unit as the first target logic management unit.
4. The method of claim 3, wherein after the determining whether there is a logical management unit having an amount of storage greater than the amount of data of the write data, the method further comprises:
if not, judging whether the total storage capacity of all the searched logic management units is larger than the data volume of the written data or not;
and if so, selecting a plurality of logic management units from all logic management units as the first target logic management unit, wherein the storage capacity of the plurality of logic management units is larger than the data volume of the written data.
5. The method of claim 1, wherein before determining whether the first target logical management unit matching the storage requirement exists in the expiration list in the database, the method further comprises:
acquiring a current date, an occupied date and a valid period of each occupied logic management unit;
judging whether occupied logic management units with the time length of subtracting the occupied date from the current date being greater than the effective period exist, if so, storing the occupied logic management units with the time length of subtracting the occupied date from the current date being greater than the effective period in the expiration list.
6. The method according to claim 1, wherein after the writing the data to be written into the storage space corresponding to the target logical management unit, the method further comprises:
and determining a path of the data to be written according to the storage cluster and the storage space of the first target logic management unit and marking the path of the written data in the first target logic management unit.
7. The method of claim 1, wherein after determining whether a first target logical management unit matching the storage requirement exists in an expiration list in the database, the method further comprises:
if not, judging whether a second target logic management unit matched with the storage requirement exists in a plurality of logic management units in a free list corresponding to each storage space stored in the database;
and if so, writing the data to be written into a storage space corresponding to the second target logic management unit.
8. A data writing apparatus, characterized in that the apparatus comprises:
the data writing device comprises an acquisition module, a storage module and a storage module, wherein the acquisition module is used for acquiring a data writing request which carries a storage requirement;
the judging module is used for judging whether a first target logic management unit matched with the storage requirement exists in an expiration list in a database, and at least one occupied logic management unit which is expired is stored in the expiration list;
and the writing module is used for writing the data to be written into the storage space corresponding to the first target logic management unit after judging that the first target logic management unit exists.
9. An electronic device comprising a memory and a processor, the memory storing a computer program, wherein the processor implements the method of any one of claims 1 to 7 when executing the computer program.
10. A storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method of any of claims 1 to 7.
CN202010938141.3A 2020-09-08 2020-09-08 Data writing method and device, electronic equipment and storage medium Pending CN112035068A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010938141.3A CN112035068A (en) 2020-09-08 2020-09-08 Data writing method and device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010938141.3A CN112035068A (en) 2020-09-08 2020-09-08 Data writing method and device, electronic equipment and storage medium

Publications (1)

Publication Number Publication Date
CN112035068A true CN112035068A (en) 2020-12-04

Family

ID=73584310

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010938141.3A Pending CN112035068A (en) 2020-09-08 2020-09-08 Data writing method and device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN112035068A (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060236053A1 (en) * 2005-04-13 2006-10-19 Kenta Shiga Memory device system, storage device, and log recording method
US20170060918A1 (en) * 2015-08-31 2017-03-02 Commvault Systems, Inc. Automated intelligent provisioning of data storage resources in response to user requests in a data storage management system
CN109032505A (en) * 2018-06-26 2018-12-18 深圳忆联信息系统有限公司 Data read-write method, device, computer equipment and storage medium with timeliness
CN110737389A (en) * 2018-07-19 2020-01-31 杭州海康威视系统技术有限公司 Method and device for storing data

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060236053A1 (en) * 2005-04-13 2006-10-19 Kenta Shiga Memory device system, storage device, and log recording method
US20170060918A1 (en) * 2015-08-31 2017-03-02 Commvault Systems, Inc. Automated intelligent provisioning of data storage resources in response to user requests in a data storage management system
CN109032505A (en) * 2018-06-26 2018-12-18 深圳忆联信息系统有限公司 Data read-write method, device, computer equipment and storage medium with timeliness
CN110737389A (en) * 2018-07-19 2020-01-31 杭州海康威视系统技术有限公司 Method and device for storing data

Similar Documents

Publication Publication Date Title
US11474972B2 (en) Metadata query method and apparatus
CN110018989B (en) Snapshot comparison method and device
CN107786638B (en) Data processing method, device and system
CN106446044B (en) Storage space recovery method and device
US11100047B2 (en) Method, device and computer program product for deleting snapshots
CN110659259A (en) Database migration method, server and computer storage medium
CN107169126B (en) Log processing method and related equipment
CN112416972A (en) Real-time data stream processing method, device, equipment and readable storage medium
CN106708865B (en) Method and device for accessing window data in stream processing system
CN112463058B (en) Fragmented data sorting method and device and storage node
CN111475100B (en) Method, apparatus and computer readable medium for managing a storage system
CN109542841B (en) Method for creating data snapshot in cluster and terminal equipment
CN112035068A (en) Data writing method and device, electronic equipment and storage medium
CN107422991B (en) Storage strategy management system
CN112269665B (en) Memory processing method and device, electronic equipment and storage medium
CN115840731A (en) File processing method, computing device and computer storage medium
US10678453B2 (en) Method and device for checking false sharing in data block deletion using a mapping pointer and weight bits
CN107181715B (en) Service checking method and device
CN111399759B (en) Method for reading data and writing data and object file system
CN113254273A (en) Method, system, device and medium for real-time recovery of principal metadata
CN114675776A (en) Resource storage method and device, storage medium and electronic equipment
CN110019031B (en) File creation method and file management device
CN106959888B (en) Task processing method and device in cloud storage system
CN111209304A (en) Data processing method, device and system
CN108959517B (en) File management method and device and electronic equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20201204

RJ01 Rejection of invention patent application after publication