CN111831208B - Information processing method and device, terminal equipment and storage medium - Google Patents
Information processing method and device, terminal equipment and storage medium Download PDFInfo
- Publication number
- CN111831208B CN111831208B CN201910305339.5A CN201910305339A CN111831208B CN 111831208 B CN111831208 B CN 111831208B CN 201910305339 A CN201910305339 A CN 201910305339A CN 111831208 B CN111831208 B CN 111831208B
- Authority
- CN
- China
- Prior art keywords
- file
- identifier
- metadata
- content
- path
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0604—Improving or facilitating administration, e.g. storage management
- G06F3/0607—Improving or facilitating administration, e.g. storage management by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0629—Configuration or reconfiguration of storage systems
- G06F3/0635—Configuration or reconfiguration of storage systems by changing the path, e.g. traffic rerouting, path reconfiguration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/0643—Management of files
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the invention discloses an information processing method, an information processing device, terminal equipment and a storage medium, which are applied to the technical field of communication, wherein the information processing method comprises the following steps: acquiring a file path to be written in a target file; generating a first file identifier according to the file path; the file content of the target file and the first file identification are stored in an associated mode; matching a second file identifier with the first file identifier based on a reading request carrying the second file identifier; and if the matching is consistent, reading the file content stored in association with the first file identifier. The information processing method does not need to obtain the file content of the target file through reading and writing of the metadata content, effectively solves the problem of high cost caused by reading metadata information when reading massive files, and greatly improves the storage performance of the files.
Description
Technical Field
The present invention relates to the field of communications technologies, and in particular, to an information processing method and apparatus, a terminal device, and a storage medium.
Background
Entering the information and data society, the amount of various information data generated along with the rapid development of applications in various industries such as mobile communication, network video, electronic commerce, scientific experiments and the like is increasing: TB level, PB level and even EB level mass files are difficult to meet the requirement of storage service development in the traditional storage.
Generally, when a client writes a file, it determines which buckets store the file data, and then the buckets store the file data in a local file system in a file manner, and store metadata information of the file. When the client reads the file, the file data can be obtained through the metadata information. The files are stored in the local file system through the storage barrel, so that random read-write operation of a large number of small files of the local system is necessarily involved when the large number of small files are stored; and the corresponding metadata information reading and writing are also involved with the reading and writing of each small file.
Disclosure of Invention
In view of this, embodiments of the present invention provide an information processing method, a server, a terminal device, and a storage medium.
The technical scheme of the invention is realized as follows:
an information processing method, the method comprising:
acquiring a file path to be written in a target file;
generating a first file identifier according to the file path;
associating and storing the file content of the target file and the first file identifier;
matching a second file identifier with the first file identifier based on a reading request carrying the second file identifier;
and if the matching is consistent, reading the file content stored in association with the first file identifier.
In the above scheme, the method further comprises:
acquiring a file name prefix of the target file;
the generating a first file identifier according to the file path includes:
and generating a first file identifier based on the file name prefix and the file path.
In the foregoing solution, the associating and storing the file content of the target file and the first file identifier includes:
coding the file content of the target file to obtain a coded first key value;
storing the first key value and the first file identification in an associated manner;
the reading of the file content stored in association with the first file identifier comprises:
reading the first key value stored in association with the first file identifier;
and performing reverse coding on the first key value to obtain the file content.
In the foregoing solution, the method further includes:
determining a metadata path based on a directory path of the target file;
generating a first metadata identifier according to the metadata path;
the metadata content of the target file and the first metadata identification are stored in an associated mode;
matching a second metadata identifier with the first metadata identifier based on a read request carrying the second metadata identifier;
and if the matching is consistent, reading the metadata content associated with the first metadata identifier.
In the above scheme, the method further comprises:
establishing a corresponding relation between the first metadata identifier and the first file identifier;
querying the first metadata identifier through the first file identifier based on the corresponding relation; or querying the first file identification through the first metadata identification.
In the foregoing solution, the matching, based on the read request carrying the second file identifier, the second file identifier with the first file identifier includes:
and if the received read request is a read request carrying a second file identifier, calling a file data IO interface to perform matching operation of the second file identifier and the first file identifier.
In the above solution, before the associating and storing the file content of the target file and the first file identifier, the method further includes:
determining whether a third file identifier matching the first file identifier exists;
and if so, deleting the third file identifier and the file content stored in association with the third file identifier.
An embodiment of the present invention further provides an information processing apparatus, where the apparatus includes:
the device comprises an acquisition unit, a storage unit and a processing unit, wherein the acquisition unit is used for acquiring a file path to be written in a target file;
the generating unit is used for generating a first file identifier according to the file path;
the writing unit is used for storing the file content of the target file and the first file identifier;
the reading unit is used for matching a second file identifier with the first file identifier based on a reading request carrying the second file identifier; and if the matching is consistent, reading the file content stored in association with the first file identifier.
In the above scheme, the obtaining unit is further configured to obtain a filename prefix of the target file;
the generating unit is further configured to generate a first file identifier based on the file name prefix and the file path.
In the above scheme, the write-in unit is further configured to encode the file content of the target file to obtain an encoded first key value; storing the first key value and the first file identification in an associated manner;
the reading unit is further configured to read the first key value stored in association with the first file identifier; and performing reverse coding on the first key value to obtain the file content.
In the foregoing solution, the obtaining unit is further configured to determine a metadata path based on a directory path of the target file;
the generating unit is further configured to generate a first metadata identifier according to the metadata path;
the writing unit is further used for associating and storing the metadata content of the target file and the first metadata identifier;
the reading unit is further configured to match a second metadata identifier with the first metadata identifier based on a read request carrying the second metadata identifier; and if the matching is consistent, reading the metadata content associated with the first metadata identifier.
In the foregoing solution, the writing unit is further configured to establish a corresponding relationship between the first metadata identifier and the first file identifier;
the reading unit is further configured to query the first metadata identifier through the first file identifier based on the corresponding relationship; or querying the first file identification through the first metadata identification.
In the foregoing scheme, the reading unit is further configured to, if the received read request is a read request carrying a second file identifier, invoke a file data IO interface to perform matching operation between the second file identifier and the first file identifier.
In the foregoing solution, the writing unit is further configured to determine whether a third file identifier matching the first file identifier exists before the file content of the target file and the first file identifier are stored in the association manner; and if so, deleting the third file identifier and the file content stored in association with the third file identifier.
The embodiment of the invention also provides terminal equipment, which comprises a processor and a memory for storing the computer program capable of running on the processor; the processor is used for realizing the information processing method applied to the terminal equipment when running a computer program.
The embodiment of the invention also provides a storage medium of a computer, and the storage medium of the computer is provided with computer executable instructions, and the computer executable instructions are executed by a processor to realize the information processing method of any embodiment of the invention.
The information processing method provided by the embodiment of the invention comprises the steps of acquiring a file path to be written in a target file; generating a first file identifier according to the file path; associating and storing the file content of the target file and the first file identifier; matching a second file identifier with the first file identifier based on a read request carrying the second file identifier; if the matching is consistent, reading the file content stored in association with the first file identifier; therefore, the embodiment of the invention can directly read the file content of the target file through the first file identifier of the target file without reading the metadata content through the metadata identifier and reading the file content of the target file based on the metadata content, thereby effectively solving the problem of high cost caused by reading metadata information when massive files are read and written, and greatly improving the storage performance of the files. In addition, the embodiment of the invention can provide more storage modes based on the file identification and the file content of the associated storage target file, and improves the flexibility and the expansibility of the storage system.
Drawings
Fig. 1 is a schematic flowchart of an information processing method according to an embodiment of the present invention;
fig. 2 is a schematic flowchart of an information processing method according to an embodiment of the present invention;
fig. 3 is a schematic flowchart of an information processing method according to an embodiment of the present invention;
fig. 4 is a schematic flowchart of an information processing method according to an embodiment of the present invention;
fig. 5 is a schematic flowchart of a mapping method according to an embodiment of the present invention;
FIG. 6 is a flowchart illustrating an information processing apparatus according to an embodiment of the present invention;
fig. 7 is a schematic diagram of a hardware structure of a terminal device according to an embodiment of the present invention.
Detailed Description
The present invention will be described in further detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the invention.
In the following description, suffixes such as "module", "component", or "unit" used to denote elements are used only for the convenience of description of the present application, and have no specific meaning by themselves. Thus, "module", "component" or "unit" may be used mixedly.
As shown in fig. 1, an embodiment of the present invention provides an information processing method, where the method includes:
102, generating a first file identifier according to the file path;
103, associating and storing the file content of the target file and the first file identifier;
104, matching a second file identifier with the first file identifier based on a read request carrying the second file identifier;
and 105, if the matching is consistent, reading the file content stored in association with the first file identifier.
Here, the file includes a word file, an excel file, an RAR file, a txt file, a PDF file, an exe file, and the like. Here, the file includes: the first file and/or the second file.
Here, the target file is a file to be written. In an actual application scenario, if the information processing method is applied to a certain node of the distributed storage system, the target file may be a file distributed to the node by the cloud server. It will be appreciated that the node is a terminal device.
Here, the file path refers to a location where a file is stored. For example, the file path is C: \\ yidong \2019.
Here, the first file identifier is an identifier that uniquely identifies the first file; the second file identification is an identification which uniquely identifies the second file.
Here, the matching of the second file identifier and the first file identifier may be that the second file identifier is the same as the first file identifier.
Therefore, the embodiment of the invention can directly read the file content of the target file through the first file identifier of the target file, does not need to read the metadata content through the metadata identifier, and reads the file content of the target file based on the metadata content, thereby effectively solving the problem of high cost caused by reading metadata information when massive files are read and written, and greatly improving the storage performance of the files. In addition, the embodiment of the invention can provide more storage modes based on the file identification and the file content of the associated storage target file, and improves the flexibility and the expansibility of the storage system.
In some embodiments, the generating a first file identifier according to the file path includes: acquiring a file name prefix of the target file; and generating a first file identifier based on the file name prefix and the file path.
In some embodiments, the generating a first file identifier according to the file path includes: generating a raw _ key value by the file path through a hash (hash) algorithm; acquiring a file name prefix of the target file; generating a first key (key) value based on the filename prefix and the raw _ key value.
It is understood that, here, the hash algorithm is to convert an input of an arbitrary length into an output of a fixed length through a hash algorithm, and the output is a hash value. And generating raw _ key values of the file paths into a compression mapping.
Here, the file name prefix is constituted by a part of a file name, and for example, if a file name of a certain file is "abc _123", the file name prefix of the file is "abc". The first key value generated based on the filename prefix and the raw _ key value is abc _ key.
Here, the file name prefix may also be a preset character, for example, "data"; and a first key value generated based on the file name prefix data and the raw _ key value is data _ key.
Thus, in the embodiment of the present invention, a first key value may be constructed by using the filename prefix and the raw _ key value, where the first key value is a first file identifier; as such, the first file identification may uniquely identify the file content of the target file.
In some embodiments, said associating storing the file content of the target file and the first file identification comprises: coding the file content of the target file to obtain a coded first key value; storing the first key value and the first file identification in an associated manner; the reading of the file content stored in association with the first file identifier comprises: reading the first key value stored in association with the first file identifier; and performing reverse coding on the first key value to obtain the file content. Therefore, by encoding the file content, the encoded first key value obtained by storing is easier to store compared with the file content.
In some embodiments, the associating stores the first key value and the first file identification, including: establishing a first key value pair of the first key value and the first file identifier; storing the first key-value pair in the key-value storage medium. Therefore, the key-value storage medium can be stored in the form of key-value pairs in the embodiment of the invention, and the storage based on the key-value storage can make the read-write operation of the file simpler and the storage types more.
In some embodiments, the encoding the file content of the target file to obtain the encoded first key value includes: binary coding is carried out on the file content of the target file, and a coded first key value is obtained; the inversely encoding the first key value to obtain the file content includes: and carrying out binary reverse coding on the first key value to obtain the file content. It is understood that the encoding manner may be various, for example, based on hexadecimal encoding, or based on octal encoding, etc.; accordingly, the inverse coding mode can be various, for example, it can also be based on the hexadecimal inverse coding, or based on the octal inverse coding, etc.
In some application scenarios, a user may desire to know some basic attribute information of the target file in addition to the file content of the target file, and thus may need to read the metadata content of the target file.
To address at least the above issues, in some embodiments, as shown in fig. 2, the method further comprises:
Here, the directory path is a path of a directory where the target file is located. The directory path may include a file path and a file name. For example, the object file is represented at "c: \ yidong \2019", the file name of the target file is" abc ", and the metadata path is" c: \\ yidong \2019 \\ abc ".
Here, the metadata path is a location where the metafile content is stored. In some embodiments, the metadata path may also be the same as the file path.
Here, the metadata content is content that characterizes the file attribute. The metadata content comprises various attributes related to files and various attributes related to directories. Such as quota, file information, sub-directory information, etc. Wherein the metadata content comprises first metadata content and/or second metadata content; the first metadata content characterizes content of a first file attribute; the second metadata content characterizes the content of the second file attribute.
Here, the first metadata identification is an identification that uniquely identifies the first metadata; the second metadata identifier is an identifier uniquely identifying the second metadata.
Here, that the second metadata identifier matches the first file identifier may be that the second metadata identifier is the same as the first metadata identifier.
In some embodiments, said generating a first metadata identification from said metadata path comprises: acquiring a metadata prefix; and generating a first file identifier based on the metadata prefix and the metadata path.
In some embodiments, said generating a first metadata identifier from said metadata path comprises: generating a raw _ key value by the metadata path through a hash algorithm; acquiring a metadata prefix of the target file; generating a second key value based on the metadata prefix and the raw _ key value.
Here, the metadata prefix may be a preset character, for example, "metadata"; a second key value generated based on the metadata prefix and the raw _ key value is metadata _ key.
Thus, in the embodiment of the present invention, a second key value may be constructed by using the metadata prefix and the raw _ key value, where the second key value is a second metadata identifier; as such, the second metadata identification may uniquely identify the metadata content of the target file.
In some embodiments, said associating stores metadata content of said target file and said first metadata identification, including: encoding the metadata content of the target file to obtain an encoded second key value; the second key value and the first metadata identification are stored in an associated mode; the reading the metadata content stored in association with the first metadata identification comprises: reading the second key value stored in association with the first metadata identity; and performing reverse coding on the second key value to obtain the metadata content. In this way, in the embodiment of the present invention, by encoding the metadata content, the encoded second key value is obtained and is easier to store compared with the metadata content.
In some embodiments, said associating stores said second key value and said first metadata identification, including: establishing a second key value pair of the second key value and the first metadata identifier; storing the second key-value pair in the key-value storage medium. Thus, the reading and writing operation of the metadata content can be simpler.
In some embodiments, the encoding the metadata content of the target file to obtain an encoded second key value includes: binary coding is carried out on the metadata content of the target file, and a coded second key value is obtained; the inversely encoding the second key value to obtain the metadata content includes: and carrying out binary reverse coding on the second key value to obtain the metadata content. It is understood that the encoding manner may be various, for example, based on hexadecimal encoding, or based on octal encoding, etc.; accordingly, the inverse coding mode can be various, for example, it can be based on hexadecimal inverse coding, or based on octal inverse coding, etc.
In some embodiments, the method further comprises:
establishing a corresponding relation between the first metadata identifier and the first file identifier;
inquiring the first metadata identification through the first file identification based on the corresponding relation; or querying the first file identification through the first metadata identification.
Therefore, in the embodiment of the invention, when a file is read, the first file identifier can be inquired through the first metadata identifier, so that the file content of the file can be read based on the first file identifier; the first metadata identifier may also be queried by the first file identifier, so that the metadata content read to the file is identified based on the first metadata.
In some embodiments, the matching the second file identifier with the first file identifier based on the read request carrying the second file identifier includes:
and if the received read request is a read request carrying a second file identifier, calling a file data IO interface to perform matching operation of the second file identifier and the first file identifier.
In other embodiments, the matching the second metadata identifier with the first metadata identifier based on the read request carrying the second metadata identifier includes:
and if the received read request is a read request carrying a second metadata identifier, calling a metadata IO interface to perform matching operation of the second metadata identifier and the first metadata identifier.
Therefore, in the embodiment of the invention, which interface is called to perform matching operation according to the parameter type carried in the request, so that corresponding calling data is found based on the interface, and the corresponding file content or metadata content is easier to query, thereby improving the file reading efficiency.
In an actual application scenario, when a terminal device receives a read request and/or a write request, the type of the read request and/or the write request may be determined first; if the received read request and/or write request is determined to be a first type of request, calling the file data IO interface to perform read operation and/or write operation; if the received read request and/or write request is determined to be a second type of request, calling the metadata IO interface to perform read operation and/or write operation; wherein the first type of request is a file data type of request and the second type of request is a metadata type of request.
In application scenarios of some distributed storage systems, the information processing method is applied to a terminal device, and the terminal device is a node in a distributed cluster; in the cluster, the server can uniformly distribute files to each node; each node acquires a corresponding target file, and stores the file content and the file identification of the target file in a local storage medium in an associated manner; if one of the nodes fails, the other nodes cannot share data to that node.
To address at least the above issues, in some embodiments the method is applied to a first node, the method further comprising:
storing the file content of the target file and the first file identifier to a second node in an associated manner; the first node and the second node are nodes within a cluster.
Therefore, the embodiment of the invention can realize data sharing, so that when the first node fails, the file content of the target file can still be read through the second node.
In some application scenarios, before a certain time, the file identifier and the file content of the target file are stored in association; editing the target file at a certain moment, wherein the file content of the target file is changed; after a certain time, if the target file needs to be read, the correct file content cannot be read.
To address at least the above concerns, in some embodiments, prior to the associating storing the file content of the target file and the first file identification, the method further comprises:
determining whether a third file identifier matching the first file identifier exists;
and if so, deleting the third file identifier and the file content stored in association with the third file identifier.
Here, the first file identifier and the third file identifier are matched such that the first file identifier and the third file identifier are the same.
Therefore, in the embodiment of the present invention, by determining whether a third file identifier identical to the first file identifier written at that time exists, and if so, deleting the third file identifier and the file content stored in association with the third file identifier, on one hand, the latest stored file content can be read all the time by subsequent file reading operations; and the other side can ensure that the storage medium only stores the file content of one file corresponding to one file identifier all the time, thereby improving the accuracy and efficiency of file reading operation.
As shown in fig. 3, an embodiment of the present invention further provides an information processing method, where the method includes the following steps.
Step 301: receiving a write request of a first file; for first file data, a first file key value is obtained by adding a first file prefix to a first file path of the first file through a hash algorithm; and for the first metadata, adding a first metadata prefix to a first metadata path associated with the first file through a hash algorithm to obtain a first metadata key value.
Here, the key includes: key1, key2, key3.
Step 302: coding the first file content to obtain a first file value data; and encoding the first metadata content to obtain a first metadata value data.
Here, the data includes: data1, data2, data3.
Step 303: establishing a key-value pair of the key value to the value.
Step 304: and writing the key-value pair into a key-value storage medium.
Step 305: receiving a read request of a second file; for second file data, a second file key value is obtained by adding a second file prefix to a second file path of the second file through a hash algorithm; for second metadata, a second metadata key value is obtained by adding a second metadata prefix to a second metadata path of the second metadata through a hash algorithm;
step 306: if the second file key value is determined to be matched with the first file key value, reading the first file value as a second file value of the second file based on the key-value pair; if the second metadata key value is determined to be matched with the first metadata key value, reading the first metadata value as a second metadata value of the second file based on the key-value pair;
step 307: inversely encoding the value of the first file to obtain the file content of the second file; and inversely coding the first metadata value to obtain the metadata content of the second file.
In the embodiment of the invention, the storage can be carried out based on the key-value key value pair, and the mode of storing the key-value key value pair at the bottom layer is modified, so that the small file can be processed more quickly and efficiently, the IO loss caused by frequent addressing accompanied with random writing of the small file in the traditional storage process is avoided, and the storage efficiency is greatly improved. In addition, as the key-value key value pair is used for storing, more types of storage can be realized, and a single storage mode is avoided, so that the storage capacity is improved, and the expansibility of the storage system is enhanced.
As shown in fig. 4, an embodiment of the present invention further provides an information processing method, which is applied to an information processing apparatus; the device comprises: an IO interface layer 41, an IO mapping layer 42, and a key-value storage medium 43; the IO interface layer 41, the IO mapping layer 42 and the key-value storage medium 43 are connected in sequence.
In some embodiments, the IO interface layer 41 includes: a file data IO interface and/or a metadata IO interface; the IO mapping layer 42 includes: file data IO mapping and/or metadata IO mapping;
the file data IO interface, the file data IO mapping and the key-value storage medium are sequentially connected; acquiring a file path to be written in a target file by using the file data IO interface; generating a first file identifier according to the file path by using the file data IO mapping; storing the file content of the target file and the first file identification by using the key-value association; matching the second file identifier with the first file identifier by using the file IO interface based on a read request carrying the second file identifier; and if the matching is consistent, reading the file content stored in association with the first file identifier.
The metadata IO interface, the metadata IO mapping and the key-value storage medium are sequentially connected; determining a metadata path based on a directory path of the target file by using the metadata IO interface; generating a first metadata identifier according to the metadata path by using the metadata IO mapping; utilizing the key-value storage medium to store the metadata content of the target file and the first metadata identification in an associated manner; matching the second metadata identifier with the first metadata identifier by using the metadata IO interface based on a read request carrying the second metadata identifier; and if the matching is consistent, reading the metadata content associated with the first metadata identification.
As shown in fig. 5, an embodiment of the present invention further provides a mapping method, where the mapping method includes:
step 501: and the IO mapping layer calculates raw _ key according to the file path through a hash algorithm.
Here, the file data prefix is the file name prefix of the above-described embodiment.
Here, the data _ key value formed according to step 502a can ensure the uniqueness of the file data key value mapped by the file within the entire storage volume or other storage medium.
Here, the metadata _ key value formed according to step 502b may guarantee the uniqueness of the metadata key value mapped by the file throughout the storage volume or other storage medium.
Step 503a: and encoding the file data into binary value data1.
Step 503b: encoding the meta data into binary value data2.
Step 504a: and carrying out binary reverse coding on the value data1 to obtain the file data.
Step 504b: and carrying out binary reverse coding on the value data2 to obtain the file data meta data.
It is to be noted here that: the following description of the information processing apparatus items is similar to the above description of the information processing method items, and the description of the advantageous effects of the same method is omitted for brevity. For technical details not disclosed in the embodiments of the information processing apparatus of the present invention, please refer to the description of the embodiments of the information processing method of the present invention.
As shown in fig. 6, an embodiment of the present invention further provides an information processing apparatus, including: an acquisition unit 61, a generation unit 62, a writing unit 63, and a reading unit 64; wherein the content of the first and second substances,
an obtaining unit 61, configured to obtain a file path to be written in a target file;
a generating unit 62, configured to generate a first file identifier according to the file path;
a writing unit 63, configured to store the file content of the target file and the first file identifier;
a reading unit 64, configured to match a second file identifier with the first file identifier based on a read request carrying the second file identifier; and if the matching is consistent, reading the file content stored in association with the first file identifier.
In some embodiments, the obtaining unit 61 is further configured to obtain a file name prefix of the target file;
the generating unit 62 is further configured to generate a first file identifier based on the file name prefix and the file path.
In some embodiments, the writing unit 63 is further configured to encode the file content of the target file to obtain an encoded first key value; storing the first key value and the first file identification in an associated manner;
the reading unit 64 is further configured to read the first key value stored in association with the first file identifier; and performing reverse coding on the first key value to obtain the file content.
In some embodiments, the obtaining unit 61 is further configured to determine a metadata path based on the directory path of the target file;
the generating unit 62 is further configured to generate a first metadata identifier according to the metadata path;
the writing unit 63 is further configured to store the metadata content of the target file in association with the first metadata identifier;
the reading unit 64 is further configured to match a second metadata identifier with the first metadata identifier based on a read request carrying the second metadata identifier; and if the matching is consistent, reading the metadata content associated with the first metadata identifier.
In some embodiments, the writing unit 63 is further configured to establish a corresponding relationship between the first metadata identifier and the first file identifier;
the reading unit 64 is further configured to query the first metadata identifier through the first file identifier based on the corresponding relationship; or querying the first file identification through the first metadata identification.
In some embodiments, the reading unit 64 is further configured to, if the received read request is a read request carrying a second file identifier, call a file data IO interface to perform matching operation between the second file identifier and the first file identifier.
In some embodiments, the writing unit 63 is further configured to determine whether a third file identifier matching the first file identifier exists before the file content of the target file and the first file identifier are stored in association; and if so, deleting the third file identifier and the file content stored in association with the third file identifier.
As shown in fig. 7, an embodiment of the present invention further provides a terminal device, where the terminal device includes a processor 71 and a memory 72 for storing a computer program capable of running on the processor 71; the processor 71 is configured to implement an information processing method applied to the terminal device when running a computer program.
In some embodiments, memory in embodiments of the present invention may be either volatile memory or nonvolatile memory, or may include both volatile and nonvolatile memory. The non-volatile Memory may be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable PROM (EEPROM), or a flash Memory. Volatile Memory can be Random Access Memory (RAM), which acts as external cache Memory. By way of illustration and not limitation, many forms of RAM are available, such as Static random access memory (Static RAM, SRAM), dynamic Random Access Memory (DRAM), synchronous Dynamic random access memory (Synchronous DRAM, SDRAM), double Data Rate Synchronous Dynamic random access memory (ddr Data Rate SDRAM, ddr SDRAM), enhanced Synchronous SDRAM (ESDRAM), synchlink DRAM (SLDRAM), and Direct Rambus RAM (DRRAM). The memory of the systems and methods described herein is intended to comprise, without being limited to, these and any other suitable types of memory.
And the processor may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware in a processor or instructions in the form of software. The Processor may be a general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, or discrete hardware components. The various methods, steps and logic blocks disclosed in the embodiments of the present invention may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present invention may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The storage medium is located in a memory, and a processor reads information in the memory and completes the steps of the method in combination with hardware of the processor.
In some embodiments, the embodiments described herein may be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For a hardware implementation, the Processing units may be implemented within one or more Application Specific Integrated Circuits (ASICs), digital Signal Processors (DSPs), digital Signal Processing Devices (DSPDs), programmable Logic Devices (PLDs), field Programmable Gate Arrays (FPGAs), general purpose processors, controllers, micro-controllers, microprocessors, other electronic units designed to perform the functions described herein, or a combination thereof.
For a software implementation, the techniques described herein may be implemented with modules (e.g., procedures, functions, and so on) that perform the functions described herein. The software codes may be stored in a memory and executed by a processor. The memory may be implemented within the processor or external to the processor.
Still another embodiment of the present invention provides a computer storage medium storing an executable program that, when executed by a processor, can implement the steps of an information processing method applied to the server or terminal device. For example, as one or more of the methods shown in fig. 1-4.
In some embodiments, the computer storage medium may include: a U-disk, a removable hard disk, a Read Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
It should be noted that: the technical schemes described in the embodiments of the present invention can be combined arbitrarily without conflict.
The above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention, and all the changes or substitutions should be covered within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the appended claims.
Claims (7)
1. An information processing method, characterized in that the method comprises:
acquiring a file path to be written in a target file;
generating a first file identifier according to the file path;
coding the file content of the target file to obtain a coded first key value; storing the first key value and the first file identification in an associated manner;
matching a second file identifier with the first file identifier based on a read request carrying the second file identifier;
if the matching is consistent, reading the first key value stored in association with the first file identifier; performing reverse coding on the first key value to obtain the file content;
the method further comprises the following steps:
determining a metadata path based on a directory path of the target file;
generating a first metadata identifier according to the metadata path;
the metadata content of the target file and the first metadata identification are stored in an associated mode;
matching a second metadata identifier with the first metadata identifier based on a read request carrying the second metadata identifier;
if the matching is consistent, reading the metadata content associated with the first metadata identifier;
establishing a corresponding relation between the first metadata identifier and the first file identifier;
querying the first metadata identifier through the first file identifier based on the corresponding relation; or querying the first file identification through the first metadata identification.
2. The method of claim 1, further comprising:
acquiring a file name prefix of the target file;
the generating a first file identifier according to the file path includes:
and generating a first file identifier based on the file name prefix and the file path.
3. The method of claim 1,
the matching the second file identifier with the first file identifier based on the read request carrying the second file identifier includes:
and if the received read request is a read request carrying a second file identifier, calling a file data IO interface to perform matching operation of the second file identifier and the first file identifier.
4. The method of claim 1,
before the associating stores the first key and the first file identification, the method further includes:
determining whether a third file identifier matching the first file identifier exists;
and if so, deleting the third file identifier and the file content stored in association with the third file identifier.
5. An information processing apparatus characterized in that the apparatus comprises:
the device comprises an acquisition unit, a storage unit and a processing unit, wherein the acquisition unit is used for acquiring a file path to be written in a target file;
the generating unit is used for generating a first file identifier according to the file path;
the writing unit is used for coding the file content of the target file to obtain a coded first key value; storing the first key value and the first file identification in an associated manner;
the reading unit is used for matching a second file identifier with the first file identifier based on a reading request carrying the second file identifier; if the matching is consistent, reading the first key value stored in association with the first file identifier; performing reverse coding on the first key value to obtain the file content;
the obtaining unit is further configured to determine a metadata path based on a directory path of the target file;
the generating unit is further configured to generate a first metadata identifier according to the metadata path;
the writing unit is further used for storing the metadata content of the target file and the first metadata identifier in an associated manner;
the reading unit is further configured to match a second metadata identifier with the first metadata identifier based on a read request carrying the second metadata identifier; if the matching is consistent, reading the metadata content associated with the first metadata identifier;
the writing unit is further configured to establish a corresponding relationship between the first metadata identifier and the first file identifier;
the reading unit is further configured to query the first metadata identifier through the first file identifier based on the corresponding relationship; or querying the first file identification through the first metadata identification.
6. A terminal device, characterized in that the terminal device comprises a processor and a memory for storing a computer program operable on the processor; wherein the processor is configured to implement the information processing method according to any one of claims 1 to 4 when running the computer program.
7. A computer storage medium having computer-executable instructions stored therein, the computer-executable instructions being executed by a processor to implement the information processing method according to any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910305339.5A CN111831208B (en) | 2019-04-16 | 2019-04-16 | Information processing method and device, terminal equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910305339.5A CN111831208B (en) | 2019-04-16 | 2019-04-16 | Information processing method and device, terminal equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111831208A CN111831208A (en) | 2020-10-27 |
| CN111831208B true CN111831208B (en) | 2023-04-14 |
Family
ID=72915187
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910305339.5A Active CN111831208B (en) | 2019-04-16 | 2019-04-16 | Information processing method and device, terminal equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111831208B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112434015B (en) * | 2020-12-08 | 2022-08-19 | 新华三大数据技术有限公司 | Data storage method and device, electronic equipment and medium |
| CN112527802B (en) * | 2020-12-22 | 2023-11-10 | 北京百度网讯科技有限公司 | Soft link method and device based on key value database |
| CN112988062B (en) * | 2021-01-28 | 2023-02-14 | 腾讯科技（深圳）有限公司 | Metadata reading limiting method and device, electronic equipment and medium |
| CN112925755A (en) * | 2021-02-18 | 2021-06-08 | 安徽中科美络信息技术有限公司 | Intelligent storage method and device for ultra-long path of file system |
| CN113282551B (en) * | 2021-07-22 | 2022-05-31 | 阿里云计算有限公司 | Data processing method, system and equipment |
| CN114579061B (en) * | 2022-04-28 | 2022-07-29 | 苏州浪潮智能科技有限公司 | Data storage method, device, equipment and medium |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106484821A (en) * | 2016-09-27 | 2017-03-08 | 浪潮软件集团有限公司 | Hybrid cloud storage method under cloud computing architecture |
| CN109614373A (en) * | 2018-11-21 | 2019-04-12 | 安徽云融信息技术有限公司 | A kind of storage of small documents storage architecture and read method |
-
2019
- 2019-04-16 CN CN201910305339.5A patent/CN111831208B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN111831208A (en) | 2020-10-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111831208B (en) | Information processing method and device, terminal equipment and storage medium | |
| WO2018121430A1 (en) | File storage and indexing method, apparatus, media, device and method for reading files | |
| WO2019075978A1 (en) | Data transmission method and apparatus, computer device, and storage medium | |
| CN105159985A (en) | Data query device and method based on redis cluster | |
| CN107153644B (en) | Data synchronization method and device | |
| CN111324665B (en) | Log playback method and device | |
| CN108614837B (en) | File storage and retrieval method and device | |
| CN111273863B (en) | Cache management | |
| CN104484638A (en) | Method for processing code chart not bound with related information | |
| CN110572451B (en) | Data processing method, device and storage medium | |
| WO2021237467A1 (en) | File uploading method, file downloading method and file management apparatus | |
| CN112632375B (en) | Session information processing method, server and storage medium | |
| US20160275199A1 (en) | Data processing device, data processing method, and non-transitory computer readable medium | |
| CN111190899B (en) | Buried data processing method, buried data processing device, server and storage medium | |
| CN116578746A (en) | Object de-duplication method and device | |
| CN108133026B (en) | Multi-data processing method, system and storage medium | |
| US20190026299A1 (en) | Metadata separated container format | |
| CN114238240B (en) | Distributed multi-cluster data storage method and device and storage medium | |
| CN111367861A (en) | File caching method, system, device and medium | |
| CN109857719B (en) | Distributed file processing method, device, computer equipment and storage medium | |
| CN117271224B (en) | Data repeated storage processing method and device of storage system, storage medium and electronic equipment | |
| CN112860755B (en) | Service identifier generation method, device, computer equipment and medium | |
| CN115905120B (en) | Archive file management method, archive file management device, archive file management computer device and archive file management storage medium | |
| US20230385810A1 (en) | Cfilter cacheing for crypto currency wallet | |
| CN115686380B (en) | File storage method, device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |