CN114722021A - Metadata management method and electronic equipment - Google Patents

Metadata management method and electronic equipment Download PDF

Info

Publication number
CN114722021A
CN114722021A CN202210259434.8A CN202210259434A CN114722021A CN 114722021 A CN114722021 A CN 114722021A CN 202210259434 A CN202210259434 A CN 202210259434A CN 114722021 A CN114722021 A CN 114722021A
Authority
CN
China
Prior art keywords
electronic device
file
local file
directory
electronic equipment
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210259434.8A
Other languages
Chinese (zh)
Inventor
李志鹏
娄松
冒晶晶
韦龙平
张智伟
易见
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Priority claimed from CN202011377326.8A external-priority patent/CN114328377A/en
Publication of CN114722021A publication Critical patent/CN114722021A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/18File system types
    • G06F16/182Distributed file systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/11File system administration, e.g. details of archiving or snapshots
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/13File access structures, e.g. distributed indices
    • G06F16/137Hash-based
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/16File or folder operations, e.g. details of user interfaces specifically adapted to file systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/16File or folder operations, e.g. details of user interfaces specifically adapted to file systems
    • G06F16/162Delete operations
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/17Details of further file system functions
    • G06F16/172Caching, prefetching or hoarding of files
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0602Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0604Improving or facilitating administration, e.g. storage management
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0628Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638Organizing or formatting or addressing of data
    • G06F3/0643Management of files

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Human Computer Interaction (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

Provided are a metadata management method and an electronic device. In the method, a first electronic device includes a first local file and a second local file in a first directory in a first local file system, and the second local file includes metadata information of the first local file; when the second electronic equipment receives a request for accessing the first directory, the second electronic equipment sends a request for acquiring a second local file by the distributed file system to the first electronic equipment; the first electronic equipment sends a second local file to the second electronic equipment; the second electronic device caches the second local file in a first directory in the second local file system, and displays the first local file in the first directory in the second local file system. By implementing the technical scheme provided by the application, the stability of using the distributed file system is improved.

Description

Metadata management method and electronic equipment
Technical Field
The present application relates to the field of terminal and communication technologies, and in particular, to a metadata management method and an electronic device.
Background
With the vigorous development of network communication technology, the interconnection of everything has quietly entered the public life. The fundamental stone of everything interconnection is that data among different devices can be accessed to each other, and a distributed file system is just a technology for providing consistent view angles of data of different devices.
Existing distributed file systems typically include a metadata server and a data server. When a user accesses file data, the metadata server needs to be accessed first to obtain basic information and data index information of the file, and then the data server needs to be accessed to obtain the file data.
However, in the implementation process of the distributed file system, the metadata server is used as a central node of the distributed file system, and it is necessary to ensure that the metadata server provides a metadata query update service on line all the time. The services of the distributed file system are not available when the metadata server is offline. However, the current distributed file system may need to be composed of a plurality of electronic devices that can be offline at any time, and if the existing distributed file system is adopted, after the electronic device serving as the central node is offline, the distributed file system can not manage metadata information in the system any more, so that the distributed file system can not be used stably.
Disclosure of Invention
The embodiment of the application provides a metadata management method and electronic equipment, which are used for realizing metadata management of a distributed file system under the condition of no central node and improving the stability of using the distributed file system.
In a first aspect, the present application provides a metadata management method, including: the method comprises the steps that a first electronic device comprises a first local file and a second local file in a first directory in a first local file system of the first electronic device, and the second local file comprises metadata information of the first local file; when second electronic equipment receives a request for accessing a first directory in a second local file system of the second electronic equipment, the second electronic equipment sends the request for obtaining a second local file by a distributed file system to the first electronic equipment; the first electronic equipment sends the second local file to the second electronic equipment; the second electronic device caches the second local file in a first directory in a second local file system, and displays the first local file in the first directory in the second local file system according to the second local file.
In the above embodiment, the first directory of the first local file system includes the second local file, and the second local file may include metadata information of other files in the first directory. When the first directory in the second local file system needs to be accessed, the second electronic device may obtain the second local file and cache the second local file in the first directory in the second local file system, so that the first local file is displayed in the first directory in the second local file system. Therefore, the metadata information of each file in each directory in the local file system is directly stored in the directory without being stored in a central node of the distributed file system. On one hand, the local file containing the metadata information can be conveniently sent to other electronic equipment according to the requirement, a distributed file system without a central node is formed, the unavailability of the whole distributed file system due to the offline of the central node is avoided, and the stability of using the distributed file system is improved. On the other hand, the metadata information is managed in the local file system and transmitted and shared in the distributed file system according to the requirement, so that the files in the local file system can be conveniently used in the distributed file system.
In a second aspect, the present application provides a metadata management method, including: the method comprises the steps that a first electronic device comprises a first local file and a second local file in a first directory in a first local file system of the first electronic device, wherein the second local file comprises metadata information of the first local file; when the first electronic device receives a request sent by the second electronic device that the distributed file system acquires the second local file, the first electronic device sends the second local file to the second electronic device, so that the first local file is displayed in a first directory in the second local file system of the second electronic device.
In the above embodiment, the metadata information of each file in each directory in the local file system is directly stored in the directory, and is not required to be stored in the central node of the distributed file system, and is sent to other electronic devices according to the requirement, so that the files in the directory are displayed in the corresponding directories by the other electronic devices. On one hand, the local file containing the metadata information can be conveniently sent to other electronic equipment according to the requirement, a distributed file system without a central node is formed, the unavailability of the whole distributed file system due to the offline of the central node is avoided, and the stability of using the distributed file system is improved. On the other hand, the metadata information is managed in the local file system and is transmitted and shared in the distributed file system according to the requirement, so that the files in the local file system can be conveniently used in the distributed file system.
In some embodiments in combination with some embodiments of the second aspect, the metadata information comprises at least one of a file name, a file type, a file size, a modification time, and an access right.
In combination with some embodiments of the second aspect, in some embodiments, the method further comprises: when the first electronic equipment adds a third local file in the first directory in the first local file system, the first electronic equipment adds metadata information of the third local file in the second local file.
In the above embodiment, as the number of files in a directory increases, the metadata information of the corresponding file is added to the second local file containing the metadata information of the file in the directory, and the comprehensiveness of the metadata information of the file recorded in the second local file is maintained.
In combination with some embodiments of the second aspect, in some embodiments, the method further comprises: when the first electronic equipment deletes the first local file, the first electronic equipment deletes the metadata information of the first local file in the second local file.
In the above embodiment, when a local file in a directory is deleted, the metadata information of the corresponding file is deleted in a second local file containing the metadata information of the file in the directory, and the metadata information of the file recorded in the second local file is kept up to date.
In combination with some embodiments of the second aspect, in some embodiments, the method further comprises: when the first electronic equipment modifies the first local file, the first electronic equipment updates the metadata information of the first local file in the second local file according to the modification of the first local file.
In the above embodiment, when a local file in a directory is modified, metadata information of a corresponding file is modified in a second local file containing metadata information of files in the directory, and metadata information of files recorded in the second local file is kept up to date.
In some embodiments in combination with some embodiments of the second aspect, in some embodiments, the first directory is one of the second directories; the method further comprises the following steps: the first electronic equipment receives a request, sent by the second electronic equipment, for acquiring a fourth local file in a second directory in the first local file system by using the distributed file system; the fourth local file comprises metadata information of a first directory in the first local file system; the first electronic device sends the fourth local file to the second electronic device; the first electronic device receives a request, sent by the second electronic device, for acquiring the second local file in the first directory in the first local file system by using the distributed file system; the first electronic equipment sends the second local file to the second electronic equipment; the second local file comprises metadata information of a fifth local file in a first directory in the first local file system; the first electronic device receives access to the fifth local file by the second electronic device.
In the above embodiment, when the second electronic device needs to access a file under the multilayer directory, the metadata files under the directory in the other electronic devices may be sequentially obtained from the uppermost directory of the path of the file, and it is determined whether the next-level directory of the path of the file exists or not until the file is found.
In some embodiments in combination with some embodiments of the second aspect, in some embodiments, the first directory is one of the second directories; the method further comprises the following steps: the first electronic equipment receives a path of a fifth local file sent by the second electronic equipment; the fifth local file is a file in the first directory in the first local file system; when the first electronic device determines that the fifth local file exists in the first directory in the first local file system, the first electronic device sends the second local file to the second electronic device; the second local file comprises metadata information of the fifth local file; and the first electronic equipment receives the access of the second electronic equipment to the fifth local file.
In the above embodiment, when the second electronic device needs to access a file in the multi-layer directory, the path of the file may be directly sent to other electronic devices, for example, the first electronic device. When the first electronic device determines that the file exists in the first local file system under the path, a second local file containing metadata information of the file can be returned to the second electronic device, so that the second electronic device can access the file.
In combination with some embodiments of the second aspect, in some embodiments, the method further comprises: the first electronic equipment receives an updated first local file sent by the second electronic equipment; the first electronic equipment updates the second local file according to the updated first local file; the updated second local file comprises the metadata information of the updated first local file.
In the above embodiment, the first electronic device may receive the updated local file sent by the other electronic device, and update the metadata information in the metadata file in the corresponding directory according to the updated local file.
With reference to some embodiments of the second aspect, in some embodiments, the first electronic device includes a first read record, and the first read record includes a record that the second electronic device requests the first electronic device to obtain a local file including metadata information.
In combination with some embodiments of the second aspect, in some embodiments, the method further comprises: when the first electronic device sends the second local file to the second electronic device and the first electronic device updates the second local file, the first electronic device sends the updated second local file to the second electronic device.
In the above embodiment, the first electronic device may send the second local file containing the metadata information of the file in the first directory to the second electronic device that reads the second local file again after updating, so that the second local file in the second electronic device is also kept up to date.
In a third aspect, the present application provides a metadata management method, including: when second electronic equipment receives a request for accessing a first directory in a second local file system of the second electronic equipment, the second electronic equipment sends a request for acquiring a second local file by a distributed file system to the first electronic equipment; the second local file comprises metadata information of a first local file in a first directory in a first local file system of the first electronic device; the second electronic equipment receives the second local file sent by the first electronic equipment and caches the second local file in a first directory in a second local file system; and the second electronic equipment displays the first local file in a first directory in the second local file system according to the second local file.
In the above embodiment, the metadata information of each file in each directory in the local file system is directly stored in the directory, and does not need to be stored in the central node of the distributed file system. When a certain directory is accessed, the metadata files in the corresponding directories in other electronic devices can be obtained according to requirements, so that the files in the corresponding directories in the local file systems of other electronic devices are displayed in the directory of the local file system of the electronic device. On one hand, local files containing metadata information in other electronic equipment can be conveniently and rapidly acquired according to requirements, a distributed file system without a center node is formed, the unavailability of the whole distributed file system due to the offline of the center node is avoided, and the stability of using the distributed file system is improved. On the other hand, the metadata information is managed in the local file system and is transmitted and shared in the distributed file system according to the requirement, so that the files in the local file system can be conveniently used in the distributed file system.
In combination with some embodiments of the third aspect, in some embodiments, the metadata information includes at least one of a file name, a file type, a file size, a modification time, and an access right.
In some embodiments in combination with some embodiments of the third aspect, the first directory is one of the second directories; the method further comprises the following steps: when the second electronic device receives a request for accessing a fifth local file in a first directory in a second directory in the distributed file system, the second electronic device sends a request for obtaining a fourth local file in the second directory in the first local file system to the first electronic device; the fourth local file comprises metadata information of the first directory; when the second electronic device determines that the received fourth local file contains the metadata information of the first directory, the second electronic device sends a request of a distributed file system for acquiring a second local file in the first directory in the first local file system to the first electronic device; the second electronic equipment accesses the fifth local file according to the received second local file; the second local file includes metadata information of the fifth local file.
In some embodiments in combination with some embodiments of the third aspect, the first directory is one of the second directories; the method further comprises the following steps: when the second electronic device receives a request for accessing a fifth local file in a first directory in a second directory in the distributed file system, the second electronic device sends a path of the fifth local file to the first electronic device; the fifth local file is a file in the first directory in the first local file system; and when the second electronic equipment receives the second local file sent by the first electronic equipment, the second electronic equipment accesses the fifth local file according to the second local file.
In combination with some embodiments of the third aspect, in some embodiments, the method further comprises: the second electronic device caches the first local file in a first directory in the second local file system; and the second electronic equipment updates the first local file and sends the updated first local file to the first electronic equipment.
In combination with some embodiments of the third aspect, in some embodiments, the method further comprises: after the second electronic device receives the second local file sent by the first electronic device, the second electronic device receives the updated second local file sent by the first electronic device; and the second electronic equipment displays the updated file in the first directory in the first local file system in the first directory in the second local file system according to the updated second local file.
In combination with some embodiments of the third aspect, in some embodiments, the method further comprises: and after the distributed file system stops mounting the second local file system, the second electronic equipment deletes the second local file cached in the first directory in the second local file system.
In the above embodiment, after the distributed file system stops mounting the second local file system, the second electronic device may delete the metadata files of the other electronic devices cached in the directories, thereby ensuring that the cached metadata files of the other electronic devices are the latest.
In a fourth aspect, an embodiment of the present application provides a first electronic device, where the first electronic device includes: one or more processors and memory; the memory coupled with the one or more processors, the memory for storing computer program code, the computer program code including computer instructions, the one or more processors invoking the computer instructions to cause the first electronic device to perform: the method comprises the steps that a first local file and a second local file are included in a first directory in a first local file system of a first electronic device, and metadata information of the first local file is included in the second local file; when a request sent by second electronic equipment for acquiring the second local file by the distributed file system is received, the second local file is sent to the second electronic equipment, so that the first local file is displayed in a first directory in the second local file system of the second electronic equipment.
In some embodiments in combination with some embodiments of the fourth aspect, the metadata information includes at least one of a file name, a file type, a file size, a modification time, and an access right.
In some embodiments combined with some embodiments of the fourth aspect, in some embodiments, the one or more processors are further configured to invoke the computer instructions to cause the first electronic device to perform: and when a third local file is added in the first directory in the first local file system, adding metadata information of the third local file in the second local file.
In some embodiments combined with the fourth aspect, in some embodiments, the one or more processors are further configured to invoke the computer instructions to cause the first electronic device to perform: and when the first local file is deleted, deleting the metadata information of the first local file in the second local file.
In some embodiments combined with the fourth aspect, in some embodiments, the one or more processors are further configured to invoke the computer instructions to cause the first electronic device to perform: when the first local file is modified, the metadata information of the first local file is updated in the second local file according to the modification of the first local file.
In some embodiments combined with the fourth aspect, in some embodiments, the one or more processors are further configured to invoke the computer instructions to cause the first electronic device to perform: receiving a request sent by the second electronic device that the distributed file system acquires a fourth local file in a second directory in the first local file system; the fourth local file comprises metadata information of the first directory in the first local file system; sending the fourth local file to the second electronic device; receiving a request of a distributed file system sent by the second electronic device for acquiring the second local file in the first directory in the first local file system; sending the second local file to the second electronic device; the second local file comprises metadata information of a fifth local file in a first directory in the first local file system; and receiving the access of the second electronic equipment to the fifth local file.
In some embodiments combined with the fourth aspect, in some embodiments, the one or more processors are further configured to invoke the computer instructions to cause the first electronic device to perform: receiving a path of a fifth local file sent by the second electronic equipment; the fifth local file is a file in the first directory in the first local file system; when the fifth local file exists in the first directory in the first local file system, sending the second local file to the second electronic device; the second local file comprises metadata information of the fifth local file; and receiving the access of the second electronic equipment to the fifth local file.
In some embodiments combined with the fourth aspect, in some embodiments, the one or more processors are further configured to invoke the computer instructions to cause the first electronic device to perform: receiving an updated first local file sent by the second electronic equipment; updating the second local file according to the updated first local file; the updated second local file comprises the metadata information of the updated first local file.
With reference to some embodiments of the fourth aspect, in some embodiments, the first electronic device includes a first read record, and the first read record includes a record that the second electronic device requests the first electronic device to obtain a local file including metadata information.
In some embodiments combined with the fourth aspect, in some embodiments, the one or more processors are further configured to invoke the computer instructions to cause the first electronic device to perform: and when the second local file is sent to the second electronic equipment and the second local file is updated, sending the updated second local file to the second electronic equipment.
In a fifth aspect, the present application provides a chip system, which is applied to a first electronic device, and the chip system includes one or more processors, and the processors are configured to invoke computer instructions to cause the first electronic device to execute a method as described in any possible implementation manner of the second aspect and the second aspect.
In a sixth aspect, the present application provides a computer program product containing instructions, which, when run on a first electronic device, causes the first electronic device to perform the method described in any one of the second aspect and the possible implementation manner of the second aspect.
In a seventh aspect, an embodiment of the present application provides a computer-readable storage medium, which includes instructions that, when executed on a first electronic device, cause the first electronic device to perform a method as described in the second aspect and any possible implementation manner of the second aspect.
It is to be understood that the electronic device provided in the fourth aspect, the chip system provided in the fifth aspect, the computer program product provided in the sixth aspect, and the computer storage medium provided in the seventh aspect are all configured to execute the metadata management method provided in any possible implementation manner of the second aspect and the second aspect of the embodiments of the present application. Therefore, the beneficial effects achieved by the method can refer to the beneficial effects in the corresponding method, and are not described herein again.
In an eighth aspect, an embodiment of the present application provides a second electronic device, where the second electronic device includes: one or more processors and memory; the memory coupled with the one or more processors, the memory for storing computer program code, the computer program code including computer instructions, the one or more processors invoking the computer instructions to cause the second electronic device to perform: when a request for accessing a first directory in a second local file system of second electronic equipment is received, sending a request for acquiring a second local file by a distributed file system to the first electronic equipment; the second local file comprises metadata information of the first local file in a first directory in a first local file system of the first electronic equipment; receiving the second local file sent by the first electronic device, and caching the second local file in a first directory in the second local file system; and displaying the first local file in a first directory in the second local file system according to the second local file.
In some embodiments in combination with some embodiments of the eighth aspect, the metadata information includes at least one of a file name, a file type, a file size, a modification time, and an access right.
In some embodiments combined with some embodiments of the eighth aspect, in some embodiments, the one or more processors are further configured to invoke the computer instructions to cause the second electronic device to perform: when a request for accessing a fifth local file in a first directory in a second directory in a distributed file system is received, sending a request for obtaining a fourth local file in the second directory in the first local file system from the distributed file system to the first electronic device; the fourth local file comprises metadata information of the first directory; when the received fourth local file is determined to contain the metadata information of the first directory, sending a request of a distributed file system for acquiring a second local file in the first directory in the first local file system to the first electronic device; accessing the fifth local file according to the received second local file; the second local file includes metadata information of the fifth local file.
In some embodiments combined with some embodiments of the eighth aspect, in some embodiments, the one or more processors are further configured to invoke the computer instructions to cause the second electronic device to perform: when a request for accessing a fifth local file in a first directory in a second directory in the distributed file system is received, sending a path of the fifth local file to the first electronic device; the fifth local file is a file in the first directory in the first local file system; and when the second local file sent by the first electronic equipment is received, accessing the fifth local file according to the second local file.
In some embodiments combined with some embodiments of the eighth aspect, in some embodiments, the one or more processors are further configured to invoke the computer instructions to cause the second electronic device to perform: caching the first local file in a first directory in the second local file system; and updating the first local file, and sending the updated first local file to the first electronic equipment.
In some embodiments combined with some embodiments of the eighth aspect, in some embodiments, the one or more processors are further configured to invoke the computer instructions to cause the second electronic device to perform: after receiving a second local file sent by the first electronic device, receiving an updated second local file sent by the first electronic device; and displaying the updated file in the first directory in the first local file system in the first directory in the second local file system according to the updated second local file.
In some embodiments combined with some embodiments of the eighth aspect, in some embodiments, the one or more processors are further configured to invoke the computer instructions to cause the second electronic device to perform: and deleting the second local file cached in the first directory in the second local file system after the distributed file system stops mounting the second local file system.
In a ninth aspect, the present application provides a chip system, which is applied to a second electronic device, and the chip system includes one or more processors, and the processors are configured to invoke computer instructions to cause the second electronic device to perform a method as described in any possible implementation manner of the third aspect and the third aspect.
In a tenth aspect, embodiments of the present application provide a computer program product including instructions, which, when run on a second electronic device, cause the second electronic device to perform the method described in any one of the possible implementations of the third aspect and the third aspect.
In an eleventh aspect, the present application provides a computer-readable storage medium, which includes instructions that, when executed on a second electronic device, cause the second electronic device to perform a method as described in any one of the possible implementation manners of the third aspect and the third aspect.
It is to be understood that the electronic device provided by the eighth aspect, the chip system provided by the ninth aspect, the computer program product provided by the tenth aspect, and the computer storage medium provided by the eleventh aspect are all configured to execute the metadata management method provided by any one of the third aspect and the possible implementation manner of the third aspect of the embodiment of the present application. Therefore, the beneficial effects achieved by the method can refer to the beneficial effects in the corresponding method, and the details are not repeated here.
The embodiment of the application also provides a method for identifying homonymous files and electronic equipment, which are used for rapidly determining whether homonymous files exist in a path of each electronic equipment in a distributed file system without a central node.
In a twelfth aspect, the present application provides a method for identifying a file with the same name, including: the method comprises the steps that a first electronic device responds to a request for displaying files under a first directory in a distributed file system, and whether the files with the same name do not exist under the first directory in the first electronic device and a first directory in a second electronic device is determined according to a bloom filter bit array of the first electronic device and a bloom filter bit array of the second electronic device; when the file with the same name does not exist, the first electronic device displays file information under the first directory in the distributed file system, wherein the file information under the first directory comprises file information of a local file under the first directory in the first electronic device and file information of a local file under the first directory in the second electronic device.
In the above embodiment, the possibility that the files with the same name exist in the same directory in the multiple electronic devices in the distributed file system is integrally judged according to the bit group of the bloom filter of each electronic device, and then the subsequent processing is performed according to the result of the integral judgment. Since there are not many cases where the same-name files exist in the same directory in each electronic device, in most cases, the result of identifying that there is no same-name file between two directories can be obtained directly and quickly by the overall judgment. Even if the files with the same name possibly exist in some directories which are judged integrally, the final recognition result cannot be directly obtained, and the range of carrying out the file recognition with the same name can be obviously reduced. The processing time length when the homonymous files are identified is greatly reduced, and the resources consumed when the homonymous files are identified are reduced.
In combination with some embodiments of the twelfth aspect, in some embodiments, the method further comprises: when determining that the files with the same name possibly exist under a first directory in the first electronic equipment and a first directory in the second electronic equipment, the first electronic equipment determines whether the files with the same name exist under the first directory in the first electronic equipment and the first directory in the second electronic equipment or not; and when the files with the same name exist, renaming the files with the same name until the files with the same name do not exist under the first directory in the first electronic equipment and the first directory in the second electronic equipment.
Optionally, renaming the file with the same name may be: when the file with the same name is displayed, the equipment identification of the equipment for storing the file is added to the file name of the file with the same name.
Alternatively, the device identifier may be a device name, a MAC address, and the like, which is not limited herein.
In the above embodiment, when it is determined that the files with the same name may exist in the two directories, the first electronic device may find the files with the same name in the two directories and rename the files until the files with the same name do not exist in the two directories and then display the files. As only the directories with the same name possibly exist need to be compared, the processing time for identifying the files with the same name is greatly reduced, and the resources consumed during identifying the files with the same name are reduced.
With reference to some embodiments of the twelfth aspect, in some embodiments, the determining, by the first electronic device, whether the file with the same name exists in the first directory of the first electronic device and the first directory of the second electronic device includes: the first electronic equipment compares the file name of each local file in the first directory of the first electronic equipment with the file name of each file in the first directory of the second electronic equipment respectively, and determines whether the file with the same name exists.
In the above embodiment, when it is determined that the files with the same name may exist in the two directories and it is required to determine whether the files with the same name exist therein, the first electronic device may compare the files in the two directories pairwise to find the files with the same name existing therein, or determine that the files with the same name do not exist in the two directories.
With reference to some embodiments of the twelfth aspect, in some embodiments, the determining, by the first electronic device, whether a file with the same name exists in a first directory in the first electronic device and in a first directory in the second electronic device specifically includes: the first electronic equipment respectively determines whether each file in a first directory in the second electronic equipment possibly has a file with the same name in the first directory in the first electronic equipment according to the bloom filter bit array of the first electronic equipment; when determining that a file under the first directory in the second electronic equipment possibly has a file with the same name under the first directory in the first electronic equipment, comparing the file with the file name of each file under the first directory in the first electronic equipment respectively, and determining whether the file with the same name exists.
In the above embodiment, when it is determined that the files with the same name may exist in the two directories and it is required to determine whether the files with the same name exist therein, the first electronic device may respectively determine whether each file in the first directory in the second electronic device may possibly exist in the first directory of the first electronic device according to the bloom filter bit array of the first electronic device. The file name of the file is compared to the file name of each file under the first directory of the first electronic device only if it is possible. Otherwise, it may be directly determined that the file does not have a file with the same name in the first directory of the first electronic device, and the file name does not need to be compared with the file name of each file in the first directory of the first electronic device. Since the files without the same name are most occupied, when the files in the first directory of each electronic device are more, the time for determining the same name files in the files can be greatly reduced.
In combination with some embodiments of the twelfth aspect, in some embodiments, the file information includes: file names and/or file icons.
Optionally, in some embodiments, the file information may further include actual data of the file.
In combination with some embodiments of the twelfth aspect, in some embodiments, the method further comprises: the first electronic equipment calculates a bloom filter bit array of the first electronic equipment according to the file name of the local file in the first directory of the first electronic equipment; the first electronic device obtains a bloom filter bit array of a second electronic device; the bloom filter bit array of the second electronic device is calculated according to the file name of the local file in the first directory of the second electronic device.
It is understood that the first electronic device and the second electronic device both include bloom filters, and the parameters of the bloom filters in the two electronic devices are the same, that is, the two electronic devices include the same K hash functions and N bit numbers of bit arrays of the bloom filters. The specific values of K and N may be set according to practical situations, and are not limited herein.
In the above embodiment, the first electronic device may first obtain the bloom filter bit array of the first electronic device and the bloom filter bit array of the second electronic device. The bloom filter bit array of the first electronic device is obtained by the first electronic device through calculation according to the file name of the local file in the first directory of the first electronic device.
The bloom filter bit array of the second electronic device is obtained by calculation according to the file name of the local file in the first directory of the second electronic device, but the process of the first electronic device obtaining the bloom filter bit array of the second electronic device may have a plurality of different ways:
optionally, in some embodiments, the obtaining, by the first electronic device, the bloom filter bit array of the second electronic device specifically includes: the first electronic equipment sends a request for acquiring a bloom filter bit array of the second electronic equipment to the second electronic equipment, wherein the request carries an identifier of a first directory; and the first electronic equipment receives the bloom filter bit array of the second electronic equipment sent by the second electronic equipment.
In the above embodiment, the bloom filter bit array of the second electronic device is calculated by the second electronic device itself according to the file name of the local file in the first directory in the second electronic device. After the bloom filter bit array of the second electronic device is obtained through calculation by the second electronic device, the bloom filter bit array of the second electronic device may be sent to the first electronic device based on the request sent by the first electronic device.
Optionally, in some embodiments, the obtaining, by the first electronic device, the bloom filter bit array of the second electronic device specifically includes: the first electronic equipment acquires the file name of a local file in a first directory in the second electronic equipment; and the first electronic equipment calculates the bit array of the bloom filter of the second electronic equipment according to the file name of the local file in the first directory of the second electronic equipment.
In the above embodiment, the first electronic device may first obtain a file name of a local file in the first directory in the second electronic device, and then the first electronic device calculates the bloom filter bit array of the second electronic device.
In practical applications, the first electronic device may obtain the file name of the local file in the first directory in the second electronic device in a plurality of different manners:
optionally, in some embodiments, the obtaining, by the first electronic device, a file name of a local file in a first directory in the second electronic device specifically includes: the first electronic equipment sends a request for acquiring the file name of the local file in the first directory in the second electronic equipment to the second electronic equipment; and the first electronic equipment receives the file name of the local file in the first directory in the second electronic equipment, which is sent by the second electronic equipment.
In the above embodiment, the first electronic device may obtain the file name of the local file in the first directory in the second electronic device by sending a request to the second electronic device.
Optionally, in some embodiments, the obtaining, by the first electronic device, a file name of a local file in a first directory in the second electronic device specifically includes: the first electronic device obtains a file name of the local file in the first directory in the second electronic device according to the stored metadata information of the local file in the first directory in the second electronic device.
In the above embodiment, since the first directory of the first electronic device and the first directory of the second electronic device are both mounted in the distributed file system, the first electronic device may store therein metadata information of a local file under the first directory of the second electronic device, where the metadata information may include a file name. The first electronic device may obtain the file name of the local file in the first directory of the second electronic device according to the stored metadata information of the local file in the second directory of the second electronic device, and does not need to temporarily request the second electronic device to obtain the file name of the local file in the first directory of the second electronic device.
With reference to some embodiments of the twelfth aspect, in some embodiments, the calculating, by the first electronic device, a bloom filter bit array of the first electronic device according to a file name of a local file in a first directory in the first electronic device specifically includes: the first electronic equipment calculates the hash value of the file name of each local file in a first directory in the first electronic equipment by using K hash functions in a bloom filter of the first electronic equipment; mapping the hash value of the file name of each local file to a bit array of N bits contained in a bloom filter of the first electronic equipment to obtain the bit array of the bloom filter of the first electronic equipment; wherein K is a positive integer and N is a positive integer greater than K.
With reference to some embodiments of the twelfth aspect, in some embodiments, the bloom filter bit array of the second electronic device is obtained by calculating a hash value of a file name of each local file in the first directory of the second electronic device according to the K hash functions in the bloom filter of the second electronic device, and mapping the hash value of the file name of each local file into a bit array of N bits included in the bloom filter of the second electronic device.
With reference to some embodiments of the twelfth aspect, in some embodiments, the determining, according to the bloom filter bit array of the first electronic device and the bloom filter bit array of the second electronic device, whether there may be a file with the same name in the first directory of the first electronic device and in the first directory of the second electronic device specifically includes: the first electronic equipment determines whether the number of bits, of which the same positions are simultaneously 1, in the bloom filter bit array of the first electronic equipment and the bloom filter bit array of the second electronic equipment is smaller than K, wherein K is the number of hash functions in the bloom filter of the first electronic equipment and the bloom filter of the second electronic equipment; when the file is determined to be smaller than the K, the first electronic device determines that the same-name file does not exist under a first directory in the first electronic device and under a first directory in the second electronic device; when the number K is determined to be not less than the number K, the first electronic device determines that files with the same name possibly exist under a first directory in the first electronic device and under a first directory in the second electronic device.
In the above embodiment, by integrally comparing the bloom filter bit array of the first electronic device with the bloom filter bit array of the second electronic device, it can be determined whether there is a file with the same name possibly existing under the first directory in the first electronic device and under the first directory in the second electronic device. When a plurality of directories need to be compared and judged, the partial directories and other partial directories can be directly determined to have no files with the same name, and the files in the directories do not need to be compared pairwise, so that the processing time for identifying the files with the same name is greatly shortened, and the resources consumed in identifying the files with the same name are reduced.
In combination with some embodiments of the twelfth aspect, in some embodiments, the method further comprises: the first electronic equipment acquires a bloom filter bit array of third electronic equipment; the bloom filter bit array of the third electronic device is calculated according to the file name of the local file in the first directory in the third electronic device.
In the above embodiment, the first directory of the distributed file system may also be populated with first directories of other electronic devices, such as a third electronic device. The first electronic device may also obtain a bloom filter bit array for the third electronic device.
In combination with some embodiments of the twelfth aspect, in some embodiments, the method further comprises: the first electronic equipment determines whether files with the same name possibly exist under a first directory in the first electronic equipment and under a first directory in the third electronic equipment according to the bloom filter bit array of the first electronic equipment and the bloom filter bit array of the third electronic equipment; the first electronic equipment determines whether files with the same name possibly exist under a first directory in the second electronic equipment and under a first directory in the third electronic equipment according to the bloom filter bit array of the second electronic equipment and the bloom filter bit array of the third electronic equipment; when it is determined that the file with the same name does not exist, the first electronic device displays file information in the first directory in the distributed file system, and the method specifically includes: when it is determined that the same-name file does not exist under a first directory in the first electronic device and a first directory in the second electronic device, the same-name file does not exist under the first directory in the first electronic device and a first directory in the third electronic device, and the same-name file does not exist under the first directory in the second electronic device and the first directory in the third electronic device, the first electronic device displays file information under the first directory in a distributed file system; the file information of the local file in the first directory in the first electronic device, the file information of the local file in the first directory in the second electronic device, and the file information of the local file in the first directory in the third electronic device are included.
In the above embodiment, the first electronic device may compare the bloom filter bit array of the first electronic device, the bloom filter bit array of the second electronic device, and the bloom filter bit array of the third electronic device two by two, respectively, and when it is determined that no file with the same name exists in the first directory of each electronic device, then display the file information of the distributed file system in the first directory.
In combination with some embodiments of the twelfth aspect, in some embodiments, the method further comprises: when determining that the same-name file does not exist in the first directory of the first electronic device and the first directory of the second electronic device, the first electronic device performs logic or operation on the bloom filter bit array of the first electronic device and the bloom filter bit array of the second electronic device to obtain a logic or bloom filter bit array; the first electronic device determines whether a file with the same name possibly exists under a first directory of at least one of the first electronic device and the second electronic device and under a first directory of the third electronic device according to the logical or bloom filter bit array and the bloom filter bit array of the third electronic device; when it is determined that the file with the same name does not exist, the first electronic device displays file information in the first directory in the distributed file system, and the method specifically includes: when determining that the same-name file does not exist under a first directory in the first electronic device and a first directory in the second electronic device, and determining that no same-name file exists under the first directory in the first electronic device and the second electronic device and the first directory in the third electronic device, the first electronic device displays file information under the first directory in a distributed file system; the file information of the local file in the first directory in the first electronic device, the file information of the local file in the first directory in the second electronic device, and the file information of the local file in the first directory in the third electronic device are included.
In the above embodiment, the first electronic device may perform a logic or operation on the bloom filter bit array of the electronic device that determines that the file with the same name does not exist, to obtain a logic or bloom filter bit array, and then compare the whole with the bloom filter bit array of the third electronic device. The processing time of the identification of the files with the same name is further reduced.
With reference to some embodiments of the twelfth aspect, in some embodiments, the determining, by the first electronic device, whether a file with a same name may exist in the first directory of at least one of the first electronic device and the second electronic device and in the first directory of the third electronic device according to the logical or bloom filter bit array and the bloom filter bit array of the third electronic device includes: the first electronic device determining whether a number of bits in the logical or bloom filter bit array and a bloom filter bit array of the third electronic device that are co-located and are 1 is less than K, the K being a number of hash functions in a bloom filter of the first electronic device, a bloom filter of the second electronic device, and a bloom filter of the third electronic device; when the file is determined to be smaller than the K, the first electronic equipment determines that no file with the same name exists under the first directory in the first electronic equipment and the second electronic equipment and the first directory of the third electronic equipment; when the number K is determined to be not less than the number K, the first electronic device determines that the files with the same name possibly exist under the first directory of at least one of the first electronic device and the second electronic device and the first directory of the third electronic device, and determines and renames the files with the same name until none of the files with the same name exists under the first directory of the first electronic device and the second electronic device and the first directory of the third electronic device.
In a thirteenth aspect, an embodiment of the present application provides a first electronic device, including: one or more processors and memory; the memory coupled with the one or more processors, the memory for storing computer program code, the computer program code including computer instructions, the one or more processors invoking the computer instructions to cause the first electronic device to perform: in response to a request for displaying a file under a first directory in a distributed file system, determining whether a file with the same name does not exist under the first directory in the first electronic device and under the first directory in the second electronic device according to a bloom filter bit array of the first electronic device and a bloom filter bit array of the second electronic device; when the file with the same name does not exist, the first electronic device displays file information under the first directory in the distributed file system, wherein the file information under the first directory comprises file information of a local file under the first directory in the first electronic device and file information of a local file under the first directory in the second electronic device.
In the above embodiment, the possibility that the files with the same name exist in the same directory in the multiple electronic devices in the distributed file system is wholly judged according to the bit set of the bloom filter of each electronic device, and then the subsequent processing is performed according to the result of the whole judgment. Since the electronic devices have files with the same name in the same directory, the identification result of the files without the same name between the two directories can be obtained directly and quickly through the integral judgment in most cases. Even if the files with the same name possibly exist in some directories which are judged integrally, the final recognition result cannot be directly obtained, and the range of carrying out the file recognition with the same name can be obviously reduced. The processing time length when the homonymous files are identified is greatly reduced, and the resources consumed when the homonymous files are identified are reduced.
In some embodiments combined with some embodiments of the thirteenth aspect, in some embodiments, the one or more processors are further configured to invoke the computer instructions to cause the first electronic device to perform: when determining that the files with the same name possibly exist under a first directory in the first electronic equipment and a first directory in the second electronic equipment, determining whether the files with the same name exist under the first directory in the first electronic equipment and the first directory in the second electronic equipment or not; and when the files with the same name exist, renaming the files with the same name until the files with the same name do not exist under the first directory in the first electronic equipment and the first directory in the second electronic equipment.
With reference to some embodiments of the thirteenth aspect, in some embodiments, the one or more processors are specifically configured to invoke the computer instructions to cause the first electronic device to perform: and comparing the file name of each local file in the first directory of the first electronic equipment with the file name of each file in the first directory of the second electronic equipment respectively to determine whether the file with the same name exists.
With reference to some embodiments of the thirteenth aspect, in some embodiments, the one or more processors are specifically configured to invoke the computer instructions to cause the first electronic device to perform: respectively determining whether each file in a first directory in the second electronic equipment possibly has a file with the same name in the first directory in the first electronic equipment according to the bloom filter bit array of the first electronic equipment; when determining that a file under the first directory in the second electronic equipment possibly has a file with the same name under the first directory in the first electronic equipment, comparing the file with the file name of each file under the first directory in the first electronic equipment respectively, and determining whether the file with the same name exists.
In combination with some embodiments of the thirteenth aspect, in some embodiments, the file information includes: a file name and/or a file icon.
In some embodiments combined with some embodiments of the thirteenth aspect, in some embodiments, the one or more processors are further configured to invoke the computer instructions to cause the first electronic device to perform: calculating a bloom filter bit array of the first electronic equipment according to the file name of the local file in the first directory of the first electronic equipment; acquiring a bloom filter bit array of the second electronic device; the bloom filter bit array of the second electronic device is calculated according to the file name of the local file in the first directory of the second electronic device.
With reference to some embodiments of the thirteenth aspect, in some embodiments, the one or more processors are specifically configured to invoke the computer instructions to cause the first electronic device to perform: sending a request for acquiring a bloom filter bit array of the second electronic equipment to the second electronic equipment, wherein the request carries an identifier of the first directory; and receiving the bloom filter bit array of the second electronic device sent by the second electronic device.
With reference to some embodiments of the thirteenth aspect, in some embodiments, the one or more processors are specifically configured to invoke the computer instructions to cause the first electronic device to perform: acquiring the file name of a local file in a first directory in the second electronic equipment; and calculating to obtain the bit array of the bloom filter of the second electronic equipment according to the file name of the local file in the first directory of the second electronic equipment.
With reference to some embodiments of the thirteenth aspect, in some embodiments, the one or more processors are specifically configured to invoke the computer instructions to cause the first electronic device to perform: sending a request for acquiring the file name of the local file in the first directory in the second electronic equipment to the second electronic equipment; and receiving the file name of the local file in the first directory in the second electronic equipment, which is sent by the second electronic equipment.
With reference to some embodiments of the thirteenth aspect, in some embodiments, the one or more processors are specifically configured to invoke the computer instructions to cause the first electronic device to perform: and acquiring the file name of the local file in the first directory in the second electronic equipment according to the stored metadata information of the local file in the first directory in the second electronic equipment.
With reference to some embodiments of the thirteenth aspect, in some embodiments, the one or more processors are specifically configured to invoke the computer instructions to cause the first electronic device to perform: calculating a hash value of a file name of each local file in a first directory in the first electronic equipment by using K hash functions in a bloom filter of the first electronic equipment; mapping the hash value of the file name of each local file to a bit array of N bits contained in a bloom filter of the first electronic equipment to obtain the bit array of the bloom filter of the first electronic equipment; wherein K is a positive integer and N is a positive integer greater than K.
With reference to some embodiments of the thirteenth aspect, in some embodiments, the bloom filter bit array of the second electronic device is obtained by calculating a hash value of a file name of each local file in the first directory of the second electronic device according to the K hash functions in the bloom filter of the second electronic device, and mapping the hash value of the file name of each local file to a bit array of N bits included in the bloom filter of the second electronic device.
With reference to some embodiments of the thirteenth aspect, in some embodiments, the one or more processors are specifically configured to invoke the computer instructions to cause the first electronic device to perform: determining whether the number of bits of which the same positions in the bloom filter bit array of the first electronic device and the bloom filter bit array of the second electronic device are simultaneously 1 is less than K, wherein K is the number of hash functions in the bloom filter of the first electronic device and the bloom filter of the second electronic device; when the file is determined to be smaller than the K, determining that the same-name file does not exist under a first directory in the first electronic equipment and a first directory in the second electronic equipment; and when the K is determined to be not less than the K, determining that the files with the same name can exist under the first directory in the first electronic equipment and the first directory in the second electronic equipment.
In some embodiments, in combination with some embodiments of the thirteenth aspect, the one or more processors are further configured to invoke the computer instructions to cause the first electronic device to perform: acquiring a bloom filter bit array of a third electronic device; the bloom filter bit array of the third electronic device is calculated according to the file name of the local file in the first directory in the third electronic device.
In some embodiments, in combination with some embodiments of the thirteenth aspect, the one or more processors are further configured to invoke the computer instructions to cause the first electronic device to perform: determining whether files with the same name possibly exist under a first directory in the first electronic equipment and under a first directory in the third electronic equipment according to the bloom filter bit array of the first electronic equipment and the bloom filter bit array of the third electronic equipment; determining whether files with the same name possibly exist under a first directory in the second electronic equipment and under a first directory in the third electronic equipment according to the bloom filter bit array of the second electronic equipment and the bloom filter bit array of the third electronic equipment; the one or more processors are specifically configured to invoke the computer instructions to cause the first electronic device to perform: when it is determined that the same-name file does not exist under a first directory in the first electronic device and a first directory in the second electronic device, the same-name file does not exist under the first directory in the first electronic device and a first directory in the third electronic device, and the same-name file does not exist under the first directory in the second electronic device and the first directory in the third electronic device, file information under the first directory in the distributed file system is displayed; the file information of the local file in the first directory in the first electronic device, the file information of the local file in the first directory in the second electronic device, and the file information of the local file in the first directory in the third electronic device are included.
In some embodiments, in combination with some embodiments of the thirteenth aspect, the one or more processors are further configured to invoke the computer instructions to cause the first electronic device to perform: when determining that the files with the same name do not exist under the first directory in the first electronic device and the first directory in the second electronic device, performing logic or operation on the bloom filter bit array of the first electronic device and the bloom filter bit array of the second electronic device to obtain a logic or bloom filter bit array; determining whether a file with the same name possibly exists in a first directory of at least one of the first electronic device and the second electronic device and a first directory of the third electronic device according to the logic or bloom filter bit array and the bloom filter bit array of the third electronic device; the one or more processors are specifically configured to invoke the computer instructions to cause the first electronic device to perform: when determining that the same-name file does not exist under a first directory in the first electronic device and a first directory in the second electronic device and determining that no same-name file exists under the first directory in the first electronic device and the second electronic device and the first directory in the third electronic device, displaying file information under the first directory in the distributed file system; the file information of the local file in the first directory in the first electronic device, the file information of the local file in the first directory in the second electronic device, and the file information of the local file in the first directory in the third electronic device are included.
With reference to some embodiments of the thirteenth aspect, in some embodiments, the one or more processors are specifically configured to invoke the computer instructions to cause the first electronic device to perform: determining whether the number of bits in the logical or bloom filter bit array and the bloom filter bit array of the third electronic device that are at the same time 1 and at the same time are less than K, where K is the number of hash functions in the bloom filter of the first electronic device, the bloom filter of the second electronic device, and the bloom filter of the third electronic device; when the file is determined to be smaller than the K, determining that no file with the same name exists under the first directory in the first electronic device and the second electronic device and the first directory in the third electronic device; and when the K is determined to be not less than the K, determining that the files with the same name possibly exist under the first directory of at least one of the first electronic equipment and the second electronic equipment and the first directory of the third electronic equipment, and determining and renaming the files with the same name until none of the files with the same name exists under the first directories of the first electronic equipment and the second electronic equipment and the first directory of the third electronic equipment.
In a fourteenth aspect, the present application provides a chip system, which is applied to a first electronic device, where the chip system includes one or more processors, and the processors are configured to invoke computer instructions to cause the first electronic device to execute a method as described in any possible implementation manner of the twelfth aspect and the twelfth aspect.
In a fifteenth aspect, the present application provides a computer program product containing instructions, which when run on a first electronic device, causes the first electronic device to perform the method as described in any one of the possible implementations of the twelfth aspect and the twelfth aspect.
Sixteenth, the present application provides a computer-readable storage medium, which includes instructions that, when executed on a first electronic device, cause the first electronic device to perform a method as described in any one of the twelfth and the twelfth possible implementation manners.
It is to be understood that the first electronic device provided in the thirteenth aspect, the chip system provided in the fourteenth aspect, the computer program product provided in the fifteenth aspect, and the computer storage medium provided in the sixteenth aspect are all used to execute the method for identifying files under the same name provided in the embodiments of the present application. Therefore, the beneficial effects achieved by the method can refer to the beneficial effects in the corresponding method, and the details are not repeated here.
The embodiment of the application also provides a data pre-reading method and electronic equipment, which are used for improving the pre-reading performance of the electronic equipment in the distributed file system.
In a seventeenth aspect, the present application provides a data pre-reading method, including: the method comprises the steps that a first electronic device obtains a file page of a first numerical value in a first file from a second electronic device, and a pre-reading flag bit is set in the Nth file page in the file page; the N is determined by the file type of the first file, and/or the application type of the first application, and/or the device type of the second electronic device; the first application is an application in the first electronic equipment, which accesses a first file on the second electronic equipment; when the first application accesses the file page with the pre-reading flag bit, the first electronic device obtains the file page with the second numerical value in the first file from the second electronic device; the second value is determined by a network delay and a network bandwidth between the first electronic device and the second electronic device.
In the embodiment of the application, the first electronic device may adaptively set the pre-reading flag bits with different numbers and positions according to the size of the pre-reading window determined by the network environment adaptability and the file type of the first file to be accessed, and/or the application type of the first application accessing the first file and/or the device type of the second electronic device, so that the first electronic device may receive new data read from the second electronic device more likely before accessing the read data, and does not need to store excessive pre-reading data, and while the pre-reading performance of the electronic device in the distributed file system is greatly improved, less network and storage resources are consumed.
With reference to some embodiments of the seventeenth aspect, in some embodiments, before the step of the first electronic device obtaining the file page of the second numerical value in the first file from the second electronic device, the method further includes: the first electronic device monitors a network round trip delay with the second electronic device; the first electronic device determines a network bandwidth with the second electronic device; and the first electronic equipment determines the size of the current pre-reading window to be the second value according to the network round-trip delay and the network bandwidth between the first electronic equipment and the second electronic equipment.
In the above embodiment, the first electronic device may dynamically determine the size of the current pre-reading window according to the network round-trip delay and the network bandwidth between the first electronic device and the second electronic device, so that the pre-reading performance of the electronic device in the distributed file system is improved.
With reference to some embodiments of the seventeenth aspect, in some embodiments, the determining, by the first electronic device, the size of the pre-read window to be the second value according to a network round trip delay and a network bandwidth between the first electronic device and the second electronic device specifically includes: the first electronic device divides a value obtained by multiplying the network round trip delay by the network bandwidth by a memory page size as the second numerical value.
In the above embodiment, the size of the pre-read window is related to the size of the memory page, so that the calculated size of the pre-read window better meets the actual requirement.
With reference to some embodiments of the seventeenth aspect, in some embodiments, the determining, by the first electronic device, a network bandwidth with the second electronic device specifically includes: the first electronic device estimates the network bandwidth between the first electronic device and the second electronic device according to the network signal strength received by the first electronic device and/or the network frequency band used by the first electronic device.
In the above embodiment, when the network bandwidth with the second electronic device cannot be directly acquired, the network bandwidth with the second electronic device may be estimated, so that the size of the pre-reading window is calculated in real time, and adaptability of the process of calculating the size of the pre-reading window to different network environments is improved.
With reference to some embodiments of the seventeenth aspect, in some embodiments, the monitoring, by the first electronic device, a network round trip delay with the second electronic device specifically includes: the first electronic device obtains a round trip delay RTT parameter value of a TCP link between the first electronic device and the second electronic device in real time to serve as a network round trip delay between the first electronic device and the second electronic device.
In the above embodiment, the RTT parameter value of the TCP link is used as the network round trip delay with the second electronic device, so that the efficiency of obtaining the network round trip delay is improved.
With reference to some embodiments of the seventeenth aspect, in some embodiments, the first electronic device obtains, from the second electronic device, a file page of the first numerical value in the first file, and sets a pre-read flag in an nth file page of the file page, specifically including: the first electronic equipment acquires a file page of a first numerical value in a first file from second electronic equipment; the first electronic equipment determines whether the file type of the first file is a video file; when the file type of the first file is determined to be a video file, the first electronic device sets a pre-reading flag bit in a plurality of file pages in the file pages with the first numerical value.
In the above embodiment, when the file type of the first file is a video file, the pre-reading flag bits are set in a plurality of file pages in the pre-read file pages, so that even if a user skips over a video, the pre-reading flag bits set in the file pages are not skipped, so that the first electronic device is more fluent when accessing video files in other electronic devices.
With reference to some embodiments of the seventeenth aspect, in some embodiments, the determining, by the first electronic device, whether the file type of the first file is a video file specifically includes: the first electronic equipment determines whether the extension of the first file is the extension of the video file; when the extension name of the first file is determined to be the extension name of the video file, the first electronic equipment determines that the file type of the first file is the video file; when it is determined that the extension of the first file is not the extension of the video file, the first electronic device determines that the file type of the first file is not the video file.
In the embodiment, the file type of the file is determined through the extension of the file, so that the efficiency of determining the file type of the file by the electronic equipment is improved.
In combination with some embodiments of the seventeenth aspect, in some embodiments, the method further comprises: when the file type of the first file is determined not to be a video file, the first electronic equipment determines whether the application type of the first application is a foreground application; when the application type of the first application is determined not to be the foreground application, the first electronic device sets a pre-reading flag bit in the N1 th document page in the document pages with the first numerical value; the position of the N1 th document page in the first value document page is the first position, and the N1 is greater than 1/2 of the first value.
In the above embodiment, when the file type of the first file is not a video file and the first application accessing the first file is not a foreground application, the pre-reading flag bit may be set in the file page of the second half of the read file page of the first value, and the next pre-reading may be triggered later, so as to reduce the occupation of unnecessary applications on network resources.
With reference to some embodiments of the seventeenth aspect, in some embodiments, the determining, by the first electronic device, whether the application type of the first application is a foreground application specifically includes: the first electronic equipment determines whether the process of the first application contains a foreground application identifier or not; when determining that the process of the first application contains the foreground application identifier, the first electronic device determines that the application type of the first application is a foreground application; when the foreground application identification is determined not to be contained in the process of the first application, the first electronic device determines that the application type of the first application is not the foreground application.
In the embodiment, the application type of the first application can be quickly determined by whether the process of the first application contains the foreground application identifier, so that the efficiency of determining the application type of the first application is improved.
In combination with some embodiments of the seventeenth aspect, in some embodiments, the method further comprises: when the application type of the first application is determined to be a foreground application, the first electronic equipment determines whether the equipment type of the second electronic equipment is first capacity equipment; the first capacity equipment is electronic equipment with capacity exceeding a preset capacity value; when the device type of the first electronic device is determined to be a first capability device, the first electronic device sets a pre-reading flag bit in the Nth 2 th file page in the file pages of the first numerical value; the position of the Nth 2-th document page in the first-value document page is a second position, and the second position is before the first position; when the device type of the first electronic device is determined not to be the first capability device, the first electronic device sets a pre-reading flag bit in the N3 th file page in the file pages of the first numerical value; the position of the N3 th document page in the first value document page is a third position, the third position is before the second position, and the N3 is less than 1/2 of the first value.
In the above embodiment, when the application type of the first application accessing the first file is the foreground application and the device type of the first electronic device is not the first capability device, the pre-read flag bit may be set in the first half of the file page of the read first value, so that the next data pre-read can be triggered earlier, and the timeliness of the foreground application for data access is ensured.
In some embodiments in combination with some embodiments of the seventeenth aspect, the preset capability value includes a frequency of the CPU being a first preset frequency, and/or a memory size being a first capacity.
In combination with some embodiments of the seventeenth aspect, in some embodiments, the method further comprises: the first electronic device determines whether the difference value between the network round trip delay when the size of the pre-reading window is calculated to be the second value and the current network round trip delay reaches a preset change threshold value; and when the preset change threshold is determined to be reached, the first electronic equipment determines the size of the current pre-reading window to be a third numerical value according to the current network round-trip delay and the current network bandwidth, wherein the third numerical value is different from the second numerical value.
In the above embodiment, if the network round-trip delay is greatly changed, the first electronic device may update the size of the pre-reading window in real time, so that the first electronic device can smoothly access the first file even if the network environment is changed drastically.
With reference to some embodiments of the seventeenth aspect, in some embodiments, after the step of determining, by the first electronic device, that the current pre-read window size is the third value according to the current network round trip delay and the current network bandwidth, the method further includes: and when the first application accesses a file page with a pre-reading flag bit in the file page of the second numerical value, the first electronic equipment acquires the file page of a third numerical value in the first file from the second electronic equipment.
In an eighteenth aspect, an embodiment of the present application provides a first electronic device, including: one or more processors and memory; the memory coupled with the one or more processors, the memory for storing computer program code, the computer program code including computer instructions, the one or more processors invoking the computer instructions to cause the first electronic device to perform: acquiring a file page of a first numerical value in a first file from second electronic equipment, and setting a pre-reading flag bit in an Nth file page in the file page; the N is determined by the file type of the first file, and/or the application type of the first application, and/or the device type of the second electronic device; the first application is an application in the first electronic equipment, which accesses a first file on the second electronic equipment; when the first application accesses the file page with the pre-reading flag bit, acquiring the file page with a second numerical value in the first file from the second electronic equipment; the second value is determined by a network delay and a network bandwidth between the first electronic device and the second electronic device.
In some embodiments, in combination with some embodiments of the eighteenth aspect, the one or more processors are further configured to invoke the computer instructions to cause the first electronic device to perform: monitoring a network round trip delay with the second electronic device; determining a network bandwidth with the second electronic device; and determining the size of the current pre-reading window as the second value according to the network round trip delay and the network bandwidth between the second electronic equipment and the current pre-reading window.
With reference to some embodiments of the eighteenth aspect, in some embodiments, the one or more processors are specifically configured to invoke the computer instructions to cause the first electronic device to perform: the product of the network round trip delay and the network bandwidth is divided by a memory page size to obtain a value, and the value is used as the second numerical value.
With reference to some embodiments of the eighteenth aspect, in some embodiments, the one or more processors are specifically configured to invoke the computer instructions to cause the first electronic device to perform: and estimating the network bandwidth between the first electronic device and the second electronic device according to the network signal strength received by the first electronic device and/or the network frequency band used by the first electronic device.
With reference to some embodiments of the eighteenth aspect, in some embodiments, the one or more processors are specifically configured to invoke the computer instructions to cause the first electronic device to perform: and acquiring a round trip delay RTT parameter value of a TCP link between the second electronic equipment in real time to serve as the network round trip delay between the second electronic equipment and the TCP link.
With reference to some embodiments of the eighteenth aspect, in some embodiments, the one or more processors are specifically configured to invoke the computer instructions to cause the first electronic device to perform: acquiring a file page of a first numerical value in a first file from second electronic equipment; determining whether the file type of the first file is a video file; and when the file type of the first file is determined to be the video file, setting a pre-reading flag bit in a plurality of file pages in the file pages with the first value.
With reference to some embodiments of the eighteenth aspect, in some embodiments, the one or more processors are specifically configured to invoke the computer instructions to cause the first electronic device to perform: determining whether the extension of the first file is the extension of the video file; when the extension name of the first file is determined to be the extension name of the video file, determining the file type of the first file to be the video file; when it is determined that the extension of the first file is not the extension of the video file, it is determined that the file type of the first file is not the video file.
In some embodiments, in combination with some embodiments of the eighteenth aspect, the one or more processors are further configured to invoke the computer instructions to cause the first electronic device to perform: when the file type of the first file is determined not to be the video file, determining whether the application type of the first application is a foreground application; when the application type of the first application is determined not to be the foreground application, setting a pre-reading flag bit in the N1 th file page in the file pages of the first numerical value; the position of the N1 th document page in the first value document page is the first position, and the N1 is greater than 1/2 of the first value.
With reference to some embodiments of the eighteenth aspect, in some embodiments, the one or more processors are specifically configured to invoke the computer instructions to cause the first electronic device to perform: determining whether the process of the first application contains a foreground application identifier; when the foreground application identifier is contained in the process of the first application, determining that the application type of the first application is foreground application; when the foreground application identification is determined not to be contained in the process of the first application, determining that the application type of the first application is not the foreground application.
In some embodiments, in combination with some embodiments of the eighteenth aspect, the one or more processors are further configured to invoke the computer instructions to cause the first electronic device to perform: when the application type of the first application is determined to be a foreground application, determining whether the device type of the second electronic device is a first capability device; the first capacity equipment is electronic equipment with capacity exceeding a preset capacity value; when the device type of the first electronic device is determined to be a first capability device, setting a pre-reading flag bit in the N2 th file page in the file pages of the first numerical value; the position of the Nth 2-th document page in the first-value document page is a second position, and the second position is before the first position; when the device type of the first electronic device is determined not to be the first capability device, setting a pre-reading flag bit in the N3 th file page in the file pages with the first value; the position of the N3 th document page in the first value document page is a third position, the third position is before the second position, and the N3 is less than 1/2 of the first value.
In some embodiments, in combination with some embodiments of the eighteenth aspect, the preset capability value includes that the frequency of the CPU is a first preset frequency, and/or the memory size is a first capacity.
In some embodiments, in combination with some embodiments of the eighteenth aspect, the one or more processors are further configured to invoke the computer instructions to cause the first electronic device to perform: determining whether the difference value between the network round-trip delay when the size of the pre-reading window is calculated to be the second value and the current network round-trip delay reaches a preset change threshold value or not; and when the preset change threshold is reached, determining the size of the current pre-reading window to be a third numerical value according to the current network round-trip delay and the current network bandwidth, wherein the third numerical value is different from the second numerical value.
In some embodiments, in combination with some embodiments of the eighteenth aspect, the one or more processors are further configured to invoke the computer instructions to cause the first electronic device to perform: and when the first application accesses a file page with a pre-reading flag bit in the file page with the second numerical value, acquiring the file page with a third numerical value in the first file from the second electronic equipment.
In a nineteenth aspect, the present application provides a chip system, where the chip system is applied to a first electronic device, and the chip system includes one or more processors, and the processors are configured to invoke computer instructions to cause the first electronic device to execute a method as described in any one of the seventeenth aspect and the seventeenth aspect.
In a twentieth aspect, embodiments of the present application provide a computer program product including instructions that, when run on a first electronic device, cause the first electronic device to perform the method as described in any one of the seventeenth and seventeenth aspects.
In a twenty-first aspect, an embodiment of the present application provides a computer-readable storage medium, which includes instructions that, when executed on a first electronic device, cause the first electronic device to perform a method as described in any one of the seventeenth and seventeenth aspects.
It is to be understood that the first electronic device provided by the eighteenth aspect, the chip system provided by the nineteenth aspect, the computer program product provided by the twentieth aspect, and the computer storage medium provided by the twentieth aspect are all used to execute the data pre-reading method provided by the embodiments of the present application. Therefore, the beneficial effects achieved by the method can refer to the beneficial effects in the corresponding method, and the details are not repeated here.
The embodiment of the application also provides a method for keeping the data consistency and electronic equipment, which are used for improving the data access speed and reducing the consumption of network resources when the data consistency is kept in the distributed file system.
In a twenty-second aspect, the present application provides a method for maintaining data consistency, comprising: the method comprises the steps that first electronic equipment receives a request which is sent by second electronic equipment and used for modifying first data of a first file in the first electronic equipment into second data; the first data is data with offset X in data with N memory pages of the first file, wherein N and X are positive integers, and X is not greater than N; the first electronic equipment modifies the first data in the radix tree of the first file in the local memory into the second data; the first electronic device modifies the first data in a first file stored in a local non-volatile memory to the second data.
It will be appreciated that the first file is stored in a local non-volatile memory of the first electronic device. When the second electronic device reads the first file stored in the first electronic device through the distributed file system, the first electronic device caches the first file in a local memory of the second electronic device to generate a radix tree of the first file. After reading the first file, the second electronic device also generates a radix tree of the first file in its own local memory. When the second electronic device modifies the first data in the radix tree of the first file in the local memory of the second electronic device into the second data, a request for modifying the first data of the first file in the first electronic device into the second data is sent to the first electronic device, so that the data consistency in the distributed file system is maintained. After receiving the modification request, the first electronic device does not need to delete or invalidate all radix trees of the first file in the local memory. Only the first data of the memory page size is modified into the second data, and then the first data in the first file stored in the local nonvolatile memory is modified into the second data correspondingly. The radix tree of the modified first file is prevented from being reloaded by each electronic device, so that the data access speed is greatly improved and the consumption of network resources is reduced when the data consistency is kept in the distributed file system.
With reference to some embodiments of the twenty-second aspect, in some embodiments, after the step of modifying, by the first electronic device, the first data in the radix tree of the first file in the local memory into the second data, the method further includes: the first electronic device determines that the third electronic device has also read the first data; the first electronic device sends a notice for marking that the first data are invalid to the third electronic device, so that the third electronic device clears the cache of the first data on the radix tree of the first file in the local memory.
In the above embodiment, after the first electronic device modifies the first data into the second data, it is further determined whether other electronic devices in the distributed file system read the first data. When it is determined that the first data has been read by the third electronic device through the distributed file system, a notification marking that the first data is invalid may be sent to the third electronic device, so that the third electronic device clears the cache of the first data in the radix tree of the first file in the local memory. Therefore, the third electronic device only needs to invalidate one memory page on the radix tree of the first file, does not need to invalidate the whole radix tree of the first file, and does not need to reload the modified radix tree of the whole first file, thereby greatly reducing the consumption of network resources.
With reference to some embodiments of the twenty-second aspect, in some embodiments, the determining, by the first electronic device, that the third electronic device has read the first data further includes: the first electronic equipment determines that the third electronic equipment has read the first data according to the local data reading record; the local data reading record is used for recording the corresponding relation of the other electronic equipment for reading the data of the file in the first electronic equipment.
In the above embodiment, the first electronic device may determine, according to the local data reading record, the electronic device that has read the first data in the distributed file system, so that accuracy of the determination process is improved.
In some embodiments, in combination with some embodiments of the twenty-second aspect, after the step of the first electronic device sending a notification to the third electronic device marking that the first data is invalid, the method further comprises: and the first electronic equipment deletes the reading record of the third electronic equipment on the first data.
In the above embodiment, after the first electronic device sends the notification that the first data is marked to be invalid to the third electronic device, the read record of the third electronic device for the first data may be deleted, so that the real-time performance of the read record of the data of the file in the first electronic device by the other recorded electronic devices is ensured.
In combination with some embodiments of the twenty-second aspect, in some embodiments, the method further comprises: the first electronic equipment modifies third data in a radix tree of a second file in the local memory into fourth data; the second file is a file in the second electronic device; the third data is data with an offset of Y in the data with the size of M memory pages of the second file, where M and Y are positive integers, and Y is not greater than M; after the preset delay time, the first electronic device sends a request for modifying third data of a second file in the second electronic device into fourth data to the second electronic device.
In the above embodiment, after the first electronic device modifies the third data of the second file in the second electronic device read from the distributed file system to be the fourth data, a modification request may be sent to the second electronic device to request that the third data in the second electronic device be modified to be the fourth data, so as to maintain data consistency in the distributed file system. And the third data is only required to be modified into the fourth data, and the whole second file is not required to be deleted and then reloaded, so that the efficiency of keeping the data consistency in the distributed file system is improved, and the consumption of network resources is reduced.
In combination with some embodiments of the twenty-second aspect, in some embodiments, the method further comprises: the first electronic equipment receives a notice which is sent by the second electronic equipment and marks that the fifth data are invalid; the fifth data is data in a third file in the second electronic equipment; and the first electronic equipment clears the cache of the fifth data on the radix tree of the third file in the local memory.
In the above embodiment, the first electronic device reads data in the third file from the second electronic device through the distributed file system, and generates a radix tree of the third file in the local memory. When a notification that fifth data marking the size of the memory page in the third file is invalid, which is sent by the second electronic device, is received, the cache of the fifth data in the radix tree of the third file in the local memory can be directly cleared, that is, only one piece of data marking the size of the memory page needs to be cleared, and the whole third file does not need to be cleared, so that the efficiency of maintaining data consistency in the distributed file system is improved, and the consumption of network resources is reduced.
With reference to some embodiments of the twenty-second aspect, in some embodiments, the clearing, by the first electronic device, the cache of the fifth data in the radix tree of the third file in the local memory specifically includes: and the first electronic equipment marks the memory page where the fifth data on the radix tree of the third file in the local memory is located as dirty.
In the above embodiment, when the first electronic device clears the cache of the fifth data in the local memory, it is only necessary to mark the memory page where the fifth data is located as dirty to indicate that the data in the memory page is invalid, so that the efficiency of clearing the cache is improved.
In some embodiments, in combination with some embodiments of the twenty-second aspect, before the step of the first electronic device receiving a request sent by the second electronic device to modify the first data of the first file in the first electronic device into the second data, the method further includes: the first electronic equipment receives a first request of reading first data of a first file, which is sent by the second electronic equipment; the first electronic equipment reads the first data from the local nonvolatile memory to a radix tree of a first file in a local memory for caching; the first electronic equipment returns a first response to the second electronic equipment, and the first response comprises the first data.
In the above embodiment, the first electronic device can receive a request from another electronic device in the distributed file system to read data of the memory page size in the file stored in the local nonvolatile memory. When a request is received, the data of the file stored in the local nonvolatile memory may be cached in the local memory, a radix tree of the file is generated, and then a response carrying the data requested to be accessed is returned to the other electronic device. The granularity of file access is refined, and the access efficiency of data of files in the distributed file system is improved.
With reference to some embodiments of the twenty-second aspect, in some embodiments, after the step of the first electronic device reading the first data from the local non-volatile memory to the radix tree of the first file in the local memory for caching, the method further comprises: the first electronic equipment records the corresponding relation between the second electronic equipment and the first data in a local data reading record; the local data reading record is used for recording the corresponding relation of the other electronic equipment for reading the data of the file in the first electronic equipment.
In some embodiments, in combination with some embodiments of the twenty-second aspect, the first request includes a command number of the read instruction, a device identification of the second electronic device, a path of the first file, and an offset of the first data in the first file; the first response also includes the same command number of the read instruction as in the first request, and is used to indicate that the first response is a response to the first request.
In combination with some embodiments of the twenty-second aspect, in some embodiments, the method further comprises: the first electronic equipment receives a request for accessing sixth data of a fourth file in the second electronic equipment; the sixth data is data with an offset of K from data with H memory pages of the fourth file, where H and K are positive integers, and K is not greater than H; the first electronic device searches whether the sixth data is cached in a radix tree of a fourth file in a local memory; when determining that the sixth data is not cached, the first electronic device sends a second request for reading the sixth data of a fourth file in the second electronic device to the second electronic device; when determining that the sixth data is cached, the first electronic device reads the sixth data from the radix tree of the fourth file in the local memory.
In the above embodiment, when the first electronic device receives a request for accessing data in a file of another electronic device through the distributed file system, it may first search whether there is data requested by the cache in the radix tree of the file in the local memory. If there is a cache, the requested data may be returned directly from the cached radix tree for that file. And when the cache does not exist, a data reading request is sent to the target electronic equipment through the distributed file system, so that the efficiency of accessing the data in the distributed file system is improved.
In a twenty-third aspect, an embodiment of the present application provides a first electronic device, where the first electronic device includes: one or more processors and memory; the memory comprises a local memory and a local nonvolatile memory; the memory coupled with the one or more processors, the memory for storing computer program code, the computer program code including computer instructions, the one or more processors invoking the computer instructions to cause the first electronic device to perform: receiving a request which is sent by second electronic equipment and modifies first data of a first file in the first electronic equipment into second data; the first data is data with offset X in data with N memory pages of the first file, wherein N and X are positive integers, and X is not greater than N; modifying the first data in the radix tree of the first file in the local memory into the second data; and modifying the first data in the first file stored in the local nonvolatile memory into the second data.
In the above embodiment, the first file is stored in the local non-volatile memory of the first electronic device. When the second electronic device reads the first file stored in the first electronic device through the distributed file system, the first electronic device caches the first file in a local memory of the second electronic device to generate a radix tree of the first file. After the second electronic device reads the first file, the radix tree of the first file is also generated in a local memory of the second electronic device. When the second electronic device modifies the first data in the radix tree of the first file in the local memory of the second electronic device into the second data, a request for modifying the first data of the first file in the first electronic device into the second data is sent to the first electronic device, so that the data consistency in the distributed file system is maintained. After receiving the modification request, the first electronic device does not need to delete or invalidate all radix trees of the first file in the local memory. Only the first data of the memory page size is modified into the second data, and then the first data in the first file stored in the local nonvolatile memory is modified into the second data correspondingly. The radix tree of the modified first file is prevented from being reloaded by each electronic device, so that the data access speed is greatly improved and the consumption of network resources is reduced when the data consistency is kept in the distributed file system.
In some embodiments, in combination with some embodiments of the twenty-third aspect, the one or more processors are further configured to invoke the computer instructions to cause the first electronic device to perform: determining that the third electronic device has also read the first data; and sending a notice for marking invalidity of the first data to the third electronic equipment, so that the third electronic equipment clears the cache of the first data on the radix tree of the first file in the local memory.
With reference to some embodiments of the twenty-third aspect, in some embodiments, the one or more processors are specifically configured to invoke the computer instructions to cause the first electronic device to perform: determining that the third electronic device has read the first data according to the local data reading record; the local data reading record is used for recording the corresponding relation of the other electronic equipment for reading the data of the file in the first electronic equipment.
In some embodiments combined with some embodiments of the twenty-third aspect, in some embodiments, the one or more processors are further configured to invoke the computer instructions to cause the first electronic device to perform: and deleting the reading record of the third electronic equipment on the first data.
In some embodiments combined with some embodiments of the twenty-third aspect, in some embodiments, the one or more processors are further configured to invoke the computer instructions to cause the first electronic device to perform: modifying third data in a radix tree of a second file in the local memory into fourth data; the second file is a file in the second electronic device; the third data is data with an offset of Y in the data with the size of M memory pages of the second file, where M and Y are positive integers, and Y is not greater than M; and after the preset delay time, sending a request for modifying third data of a second file in the second electronic equipment into fourth data to the second electronic equipment.
In some embodiments, in combination with some embodiments of the twenty-third aspect, the one or more processors are further configured to invoke the computer instructions to cause the first electronic device to perform: receiving a notice which is sent by the second electronic equipment and marks that the fifth data is invalid; the fifth data is data in a third file in the second electronic equipment; and clearing the cache of the fifth data on the radix tree of the third file in the local memory.
With reference to some embodiments of the twenty-third aspect, in some embodiments, the one or more processors are specifically configured to invoke the computer instructions to cause the first electronic device to perform: and marking the memory page where the fifth data on the radix tree of the third file in the local memory is located as dirty.
In some embodiments combined with some embodiments of the twenty-third aspect, in some embodiments, the one or more processors are further configured to invoke the computer instructions to cause the first electronic device to perform: receiving a first request for reading first data of a first file sent by the second electronic equipment; reading the first data from the local nonvolatile memory to a radix tree of a first file in a local memory for caching; and returning a first response to the second electronic equipment, wherein the first response comprises the first data.
In some embodiments, in combination with some embodiments of the twenty-third aspect, the one or more processors are further configured to invoke the computer instructions to cause the first electronic device to perform: recording the corresponding relation between the second electronic equipment and the first data in a local data reading record; the local data reading record is used for recording the corresponding relation of the other electronic equipment for reading the data of the file in the first electronic equipment.
With reference to some embodiments of the twenty-third aspect, in some embodiments, the first request includes a command number of the read instruction, a device identification of the second electronic device, a path of the first file, and an offset of the first data in the first file; the first response also contains the same command number of the read instruction as in the first request, which is used to indicate that the first response is a response to the first request.
In some embodiments combined with some embodiments of the twenty-third aspect, in some embodiments, the one or more processors are further configured to invoke the computer instructions to cause the first electronic device to perform: receiving a request for accessing sixth data of a fourth file in the second electronic device; the sixth data is data with an offset of K from the data with the size of H memory pages of the fourth file, where H and K are positive integers, and K is not greater than H; searching whether the sixth data is cached in a radix tree of a fourth file or not in a local memory; when determining that the sixth data is not cached, sending a second request for reading the sixth data of a fourth file in the second electronic equipment to the second electronic equipment; and when the sixth data is determined to be cached, reading the sixth data from the radix tree of the fourth file in the local memory.
In a twenty-fourth aspect, the present application provides a chip system, where the chip system is applied to a first electronic device, and the chip system includes one or more processors, where the processors are configured to invoke computer instructions to cause the first electronic device to execute a method as described in any one of the twenty-second and twenty-second aspects.
In a twenty-fifth aspect, embodiments of the present application provide a computer program product containing instructions, which, when run on a first electronic device, cause the first electronic device to perform a method as described in any one of the twenty-second and twenty-second aspects.
In a twenty-sixth aspect, an embodiment of the present application provides a computer-readable storage medium, which includes instructions that, when executed on a first electronic device, cause the first electronic device to perform a method as described in any one of the twenty-second and twenty-second possible implementation manners.
It is to be understood that the first electronic device provided by the twenty-third aspect, the chip system provided by the twenty-fourth aspect, the computer program product provided by the twenty-fifth aspect, and the computer storage medium provided by the twenty-sixth aspect are all configured to perform the method for maintaining data consistency provided by the embodiments of the present application. Therefore, the beneficial effects achieved by the method can refer to the beneficial effects in the corresponding method, and are not described herein again.
Drawings
FIG. 1 is an exemplary diagram of metadata information of a file in an embodiment of the present application;
FIG. 2 is an exemplary diagram illustrating the distribution of metadata files in a file system in an embodiment of the present application;
FIG. 3 is an exemplary diagram illustrating metadata information obtained from a metadata file in an embodiment of the present application;
FIG. 4 is an exemplary diagram illustrating a file modification resulting in a metadata file update in an embodiment of the present application;
FIG. 5 is an exemplary diagram illustrating a file deletion causing a metadata file update in an embodiment of the present application;
FIG. 6 is an exemplary diagram illustrating a file addition causing a metadata file update in an embodiment of the present application;
FIG. 7 is a schematic illustration of an exemplary set of user interfaces in an embodiment of the present application;
fig. 8 is a schematic diagram of an exemplary state in a memory of an electronic device in an embodiment of the present application;
FIG. 9 is an exemplary diagram of a directory tree after metadata files are obtained from other electronic devices in an embodiment of the application;
FIG. 10 is a diagram of an exemplary scenario for a distributed file system in the prior art;
FIG. 11 is a diagram of an exemplary signaling interaction for modifying a file in a distributed file system in the prior art;
FIG. 12 is a schematic diagram of an exemplary scenario of a centerless distributed file system in an embodiment of the present application;
Fig. 13 is an exemplary structural diagram of an electronic device 100 provided in an embodiment of the present application;
fig. 14 is a block diagram of an exemplary software structure of the electronic device 100 according to an embodiment of the present application;
FIG. 15 is a signaling interaction diagram of a metadata management method in an embodiment of the present application;
FIG. 16 is another set of exemplary user interface diagrams in an embodiment of the present application;
FIG. 17 is a schematic signaling diagram illustrating another exemplary metadata management method according to an embodiment of the present disclosure;
FIG. 18 is a schematic view of another set of exemplary user interfaces in an embodiment of the present application;
FIG. 19 is a schematic signaling diagram illustrating another exemplary metadata management method according to an embodiment of the present disclosure;
FIG. 20 is another set of exemplary user interface diagrams in an embodiment of the present application;
FIG. 21 is a schematic signaling diagram illustrating another exemplary metadata management method according to an embodiment of the present disclosure;
FIG. 22 is another set of exemplary user interface diagrams in an embodiment of the present application;
FIG. 23 is a schematic signaling diagram illustrating another exemplary metadata management method according to an embodiment of the present disclosure;
FIG. 24 is another set of exemplary user interface diagrams in an embodiment of the present application;
FIG. 25 is a schematic signaling diagram illustrating another exemplary metadata management method according to an embodiment of the present disclosure;
FIG. 26 is a schematic illustration of another set of exemplary user interfaces in an embodiment of the present application;
FIG. 27 is a schematic illustration of a set of user interfaces in an embodiment of the present application;
FIG. 28 is a diagram illustrating a set of states of an array of bloom filter bits in an embodiment of the present application;
FIG. 29 is a schematic diagram of an exemplary scenario in which bloom filter bit arrays of two electronic devices are compared in an embodiment of the present application;
FIG. 30 is a schematic view of another set of user interfaces in an embodiment of the present application;
fig. 31 is a schematic block diagram of another exemplary software structure of the electronic device 100 provided in the embodiment of the present application;
FIG. 32 is a signaling interaction diagram illustrating a method for identifying homonymous files in an embodiment of the present application;
FIG. 33 is a flowchart illustrating a process of determining whether a file with the same name may exist in the embodiment of the present application;
FIG. 34 is a diagram illustrating an exemplary scenario for determining whether a file with the same name may exist in an embodiment of the present application;
FIG. 35 is a schematic diagram of another exemplary scenario for determining whether a file with the same name may exist in an embodiment of the present application;
FIG. 36 is another flowchart illustrating the determination of whether a file with the same name may exist in the embodiment of the present application;
FIG. 37 is an exemplary diagram of the logical OR operation of a bloom filter bit array in an embodiment of the present application;
FIG. 38 is a schematic diagram of an exemplary scenario in which logic or operations are used to determine whether a file with the same name may exist in embodiments of the present application;
FIG. 39 is a pseudo code diagram in an embodiment of the present application;
FIG. 40 is a diagram illustrating the relationship between application programs, virtual memory, and physical memory in an embodiment of the present application;
FIG. 41 is an exemplary diagram illustrating memory paging management in an embodiment of the present application;
FIG. 42 is a schematic diagram of an exemplary scenario for data access;
FIG. 43 is a schematic diagram of an exemplary scenario for performing data pre-reading;
FIG. 44 is a diagram illustrating an exemplary scenario of pre-reading flag bits in the embodiment of the present application;
FIG. 45 is an exemplary diagram of setting different pre-read flag bits according to different device types in the embodiment of the present application;
FIG. 46 is a diagram of an exemplary signaling interaction for data pre-reading in the prior art;
FIG. 47 is a state diagram of an exemplary state of the art memory page with data pre-read;
FIG. 48 is a diagram illustrating an exemplary scenario of data pre-reading in an embodiment of the present application;
fig. 49 is a schematic block diagram of another exemplary software structure of the electronic device 100 provided in the embodiment of the present application;
FIG. 50 is a schematic signaling diagram illustrating a pre-read window size determination phase in an embodiment of the present application;
FIG. 51 is a flowchart illustrating the pre-read flag bit determination stage in the embodiment of the present application;
FIG. 52 is an exemplary illustration of various positions of the pre-read flag bits in an embodiment of the present application;
FIG. 53 is a schematic signaling interaction diagram of the pre-read phase in the embodiment of the present application;
FIG. 54 is an exemplary diagram of a radix tree of document F in memory in an embodiment of the present application;
FIG. 55 is a diagram of an exemplary scenario for a prior art method of maintaining data consistency;
FIG. 56 is a schematic diagram of an exemplary scenario of a method for maintaining data consistency in an embodiment of the present application;
fig. 57 is a block diagram of an exemplary software structure of the electronic device 100 according to an embodiment of the present application;
FIG. 58 is a signaling interaction diagram illustrating a method for maintaining data consistency in an embodiment of the present application;
fig. 59 is a schematic structural diagram of an exemplary electronic device 200 provided in an embodiment of the present application;
fig. 60 is a schematic block diagram of another exemplary software structure of the electronic device of the embodiment.
Detailed Description
The terminology used in the following embodiments of the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the present application. As used in the specification of the present application and the appended claims, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the listed items.
In the following, the terms "first", "second" are used for descriptive purposes only and are not to be construed as implying or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of embodiments of the application, unless stated otherwise, "plurality" means two or more.
The embodiment of the application provides a metadata management method, a homonymous file identification method, a data pre-reading method, a method for keeping data consistency and electronic equipment.
The metadata management method in the embodiment of the application can realize metadata management of the distributed file system under the condition of no central node, and improves the stability of the electronic equipment using the distributed file system.
In the distributed file system without the central node, after the electronic device obtains the metadata file in a path in another electronic device by using the metadata management method, the file in the path in another electronic device may be displayed in the path according to the metadata file, for example, the path in the electronic device a may be: and displaying paths in the electronic equipment B under the root directory/directory A: files in root directory/directory a.
In order to make the file name of each file displayed in one path unique, it is necessary for each electronic device in the distributed file system to identify whether a file with the same name exists in the same path. When the file with the same name does not exist in the identification and determination, the file under the same path in other electronic equipment can be directly displayed, and when the file with the same name does exist in the identification and determination, the file with the same name needs to be renamed. The method for identifying the files with the same name in the embodiment of the application can rapidly determine whether the files with the same name exist in one path of each electronic device in a distributed file system without a central node, and reduces the processing time for identifying the files with the same name.
After the files in other electronic devices are displayed in the electronic device, the files in the other electronic devices can be read, and a data pre-reading technology can be adopted to accelerate the speed of data reading between different electronic devices. In a distributed file system without a central node, the network environment between electronic devices is often in dynamic change, and the performance of each electronic device is different. When one electronic device performs data pre-reading on another electronic device, the pre-reading effect is often poor due to changes in network environment or differences in device performance. The data pre-reading method provided by the embodiment of the application can improve the pre-reading performance of the electronic equipment in the distributed file system.
An electronic device may modify a file after reading the file in another electronic device, and in order to obtain the latest file when the other electronic device reads the file, it is necessary to maintain data consistency in the distributed file system. The method for maintaining data consistency provided by the embodiment of the application can improve the data access speed and reduce the consumption of network resources when maintaining the data consistency in the distributed file system.
The following describes the above methods in detail with reference to specific scenarios:
the metadata management method comprises the following steps:
for ease of understanding, the related terms and concepts related to the metadata management method according to the embodiments of the present application will be described below.
(1) Metadata information;
metadata (Metadata) is defined as: data describing data is descriptive information for data and information resources.
Metadata is data that describes other data, or structural data that provides information about a resource. The purpose of its use lies in: identifying a resource; evaluating the resources; tracking changes of the resource in the using process; the method realizes simple and efficient management of a large amount of networked data; the method realizes effective discovery, search, integrated organization, effective management of used resources and the like of the information resources.
In the embodiment of the present application, metadata information of a file includes metadata describing the file. The metadata describing the file contained in the metadata information of the file may be different according to the file type.
In general, the metadata of a file contained in the metadata information of the file may include: file name, file type, file size, modification time, access rights, etc.
Different metadata may be included in the metadata information for different types of files. For some files with specific formats, more specific metadata can be contained in the metadata information of the file.
Fig. 1 is an exemplary diagram of metadata information of a file in an embodiment of the present application. As shown in fig. 1 (a), for a file of one text document type: document 1.txt, which metadata information may include: file name: document 1.txt, document type: text documents, and metadata for file size, modification time, and the like. As shown in fig. 1 (b), for one picture file: the metadata information of the picture 2, txt may include metadata generally included in metadata information of other types of files, such as a file name, a document type, a file size, and modification time, and may include file-specific metadata of the picture type, such as shooting time, resolution, bit depth, width, and height. As shown in fig. 1 (c), for one audio file: song 3.mp3, the metadata information may include metadata specific to the audio type file, such as album, genre, duration, bit rate, in addition to metadata typically included in metadata information for other types of files, such as file name, document type, file size, modification time, etc.
(2) A metadata file;
2.1, distribution structure of metadata files:
in the embodiment of the present application, a metadata file is maintained in each directory in a file system mounted by a distributed file system in an electronic device, and metadata information of all other files in the directory is stored in the metadata file.
Alternatively, the metadata file is a file in a database format, such as a file in a format for storing directory entry data in a F2FS (flash friendly file system), a file in a table format, and the like, and is not limited herein. According to the identification information of one other file in the directory where the metadata file is located, such as a file name, a unique identification number, and the like, the metadata information of the other file can be obtained from the metadata file.
In this embodiment, the file name of the metadata file may carry an identifier of the electronic device to distinguish metadata files from different electronic devices. Optionally, the file name of the metadata file may carry an identifier of a directory to distinguish metadata files in different directories, or the file name of the metadata file does not carry an identifier of a directory, but the metadata file may distinguish different directories through a path of the metadata file, which is not limited herein.
Alternatively, the metadata file may not be visible to the user.
Fig. 2 is an exemplary diagram illustrating distribution of metadata files in a file system in an embodiment of the present application. Fig. 2 (a) shows an exemplary user interface 210 of an electronic device. The user interface 210 is a user interface of a root directory of a file system mounted by the distributed file system in the electronic device 1. A directory a and a file 1 are stored in the root directory of the file system, and a metadata file 1 is stored in the root directory, which stores metadata information of all other files in the root directory. In response to an operation of the user to open the directory a, the electronic device 1 may display a user interface 220 as shown in (b) of fig. 2. The user interface 220 is a user interface of the directory a under the root directory in the electronic device 1. The file system has a directory a in which files 2 and 3 are stored, and a metadata file 2 in which metadata information of all other files in the directory a is stored.
In the drawings in the embodiments of the present application, the short dashed line indicates that the metadata file may be invisible or visible to the user, and is not limited herein. Such as the metadata file 1 shown in (a) in fig. 2 and the metadata file 2 shown in (b) in fig. 2. In other embodiments, the representation in this manner may also be adopted, and will not be described in detail later.
As shown in fig. 2 (c), it is a tree diagram of the file system mounted by the distributed file system in the electronic device 1 shown in fig. 2 (a) and (b). As can be seen, directory a, file 1 and metadata file 1 are stored under the root directory, and metadata information of directory a and file 1 is stored in the metadata file 1. The directory a stores files 2 and 3 and a metadata file 2, and the metadata file 2 stores metadata information of the files 2 and 3.
It is understood that in the file system, a lower directory (e.g., directory a) of an upper directory (e.g., root directory) is also treated as a file, and corresponding metadata information is also stored in a metadata file (e.g., metadata file 1) under the upper directory.
Alternatively, there may be a plurality of file systems in an electronic device, and only the file system mounted by the distributed file system may maintain a metadata file storing metadata information of all other files in each directory. And is not limited herein.
2.2, use and update of metadata files:
the use and updating of metadata files is illustratively described below in conjunction with the exemplary metadata file distribution structure shown in FIG. 2.
Fig. 3 is an exemplary schematic diagram illustrating obtaining metadata information from a metadata file in the embodiment of the present application. Since the metadata information of all other files in the root directory is stored in the metadata file 1, the metadata information of the file 1 is included. Therefore, according to the identification of the file 1, for example, the file name of the file 1, the metadata information of the file 1 can be searched from the metadata file 1.
It can be understood that, although there may be no path of the file in the metadata information of the file, since the file in which the metadata information is recorded in the metadata file and the metadata file are in the same directory, after obtaining the metadata information of the file, the electronic device may locate the file by retrieving the file in the directory where the metadata file is located according to the metadata information of the file.
Fig. 4 is an exemplary diagram illustrating a file modification causing a metadata file update in the embodiment of the present application. Fig. 4 (a) shows a user interface 410 of the electronic device 1. The user interface 410 is the user interface after the user opens file 1 under the root directory of the electronic device 1. If the user modifies the contents of File 1 and clicks on the save control in this user interface 410 at 2020/02/0202: 02 as shown in FIG. 4 (b), the modification to File 1 is saved. The electronic device 1 will also update the metadata information of file 1 saved in the metadata file 1 accordingly. For example, as shown in (c) of fig. 4, if the metadata information of file 1 is queried again from the metadata file 1 by using the identifier of file 1, it is found that the metadata information of file 1 has been updated, and the updating content may include: the modification time has been updated to 2020/02/0202: 02. According to different contents contained in the metadata information of the file 1, other content updates, such as file size update, character number update, etc., may also be contained, and are not limited herein.
Fig. 5 is an exemplary schematic diagram illustrating a file deletion causing a metadata file update in the embodiment of the present application. Fig. 5 (a) shows an exemplary user interface 210 of the electronic device. The user interface 210 is a user interface of a root directory of a file system mounted by the distributed file system in the electronic device 1. A directory a and a file 1 are stored in the root directory of the file system, and a metadata file 1 is stored in the root directory, which stores metadata information of all other files in the root directory. If the user operation deletes file 1 under the root directory, the user interface 210 may be updated as shown in (b) of fig. 5. The electronic device 1 updates the metadata file 1 in the root directory accordingly, and deletes the metadata information of the file 1 from the metadata file 1. As shown in fig. 5 (c), if the metadata information of file 1 is searched from the metadata file 1 using the identifier of file 1, the metadata information of file 1 cannot be searched.
Fig. 6 is an exemplary diagram illustrating a file addition causing a metadata file update in the embodiment of the present application. Fig. 6 (a) shows an exemplary user interface 210 of an electronic device. The user interface 210 is a user interface of a root directory of a file system mounted by the distributed file system in the electronic device 1. A directory a and a file 1 are stored in the root directory of the file system, and a metadata file 1 is stored in the root directory, which stores metadata information of all other files in the root directory. If a user action adds a new file 4 under the root directory, the user interface 210 may be updated as shown in fig. 6 (b). The electronic device 1 will update the metadata file 1 in the root directory accordingly, and add the metadata information of the file 4 to the metadata file 1. As shown in (c) of fig. 6, at this time, the metadata information of the file 4 can be searched from the metadata file 1 using the identification of the file 4.
2.3, synchronization effect of the metadata file:
in this embodiment, one electronic device may obtain metadata files in the same directory in other electronic devices in the distributed file system, and store the metadata files in the directory in the electronic device. Then, files recorded in the metadata files acquired from other electronic devices may be displayed in the directory in the electronic device.
It will be appreciated that since the purpose of a distributed file system is to make the data views of different electronic devices consistent, file systems mounted by the same distributed file system in each electronic device generally have the same or similar directory structure. Even if the directory structures are different, the same directory structure can be obtained after the metadata files in the directory of other electronic equipment are obtained and the files recorded in the obtained metadata files are displayed in the electronic equipment.
It is to be understood that the file recorded in the metadata file indicates a file in which metadata information is recorded in the metadata file. For example, if the metadata information of file 1 is recorded in the metadata file 1, it can be understood that file 1 is recorded in the metadata file 1.
Optionally, the file recorded in the metadata file acquired from the other electronic device is displayed in the directory, and specifically, the file may be a link file displaying the file recorded in the metadata file. The file may be an actual file showing files recorded in the metadata file, and is not limited herein. If the link file is displayed, the link file is generated according to the metadata information of the file recorded in the metadata file, and does not contain the actual data of the file, unlike the actual file in which the actual data of the file is stored in the electronic device. The display effect of the link file may be the same as or different from that of the real file, and is not limited herein. When the electronic device receives a request for accessing the link file, the request can be converted into a request for accessing a real file in other electronic devices corresponding to the link file. If the real file is displayed, after the electronic device acquires the metadata file from other electronic devices, the electronic device may acquire and display the corresponding real file from other electronic devices according to the metadata information of the file recorded in the metadata file.
It can be understood that, in the process of displaying a file according to metadata information in a metadata file acquired from another electronic device, an index node (inode) structure of the file to be displayed may be generated in a memory of the electronic device. According to the difference of metadata files where metadata information of one file is obtained, when the file is displayed in the electronic equipment, the file stores an electronic equipment identifier of a metadata file source in an inode structure body corresponding to a memory. Therefore, when the electronic device receives a request for accessing the file, the electronic device can determine that the file actually originates from the file in the electronic device according to the identifier of the electronic device in the inode structure body corresponding to the file in the memory.
In the embodiment of the present application, the lower directory is a directory type file, and the metadata information of the lower directory is also recorded in the metadata file in the same upper directory as the lower directory. However, in the determination of whether or not the files of the directory type are the same, if the paths of the files of the directory type are the same and the file names in the metadata information are the same, it is determined that the files are files of the same directory type. When a plurality of metadata files exist in one directory, the files of the same directory type recorded in the plurality of metadata files are not repeatedly displayed. For other types of files, this processing method may be used or not used, and is not limited herein.
FIG. 7 shows an exemplary set of user interface diagrams for embodiments of the present application after metadata files are obtained from other electronic devices.
Fig. 7 (a) shows a user interface 710. The user interface 710 is a user interface of a root directory of a file system mounted by the distributed file system in the electronic device 1. Directory a, file 1 and metadata file 1-1 are stored under the root directory of the file system, and metadata information of directory a and file 1 is recorded in the metadata file 1-1. The file name in the metadata file 1-1 includes the identifier of the electronic device 1: 1.
fig. 7 (b) shows a user interface 720. The user interface 720 is a user interface of a root directory of a file system mounted by the distributed file system in the electronic device 2. The root directory of the file system stores a directory a, a file 5, a file 6 and a metadata file 2-1, the metadata file 2-1 records metadata information of the directory a, the file 1, the file 5 and the file 6, and the file name of the metadata file includes an identifier of the electronic device 2: 2.
as shown in (c) of fig. 7, the electronic device 1 may acquire the metadata file 2-1 under the root directory of the electronic device 2 and save under the root directory of the electronic device 1.
As shown in (d) of fig. 7, the electronic device 1 may display the file recorded in the metadata file 2-1 according to the metadata information of the file recorded in the metadata file 2-1. Since the file name of the directory a in the electronic device 2 is the same as the file name of the directory a in the root directory of the electronic device 1 in the metadata information of the directory a recorded in the metadata file 2-1, the electronic device 1 confirms that the directory a recorded in the metadata file 2-1 and the directory a in the root directory of the electronic device 1 are the same directory file and does not display repeatedly. The electronic device 1 displays the other files under the root directory of the electronic device 2 recorded in the metadata file 2-1 under the root directory: file 5 and file 6.
Referring to fig. 7 (d), fig. 8 is a schematic diagram illustrating an exemplary state in a memory when an electronic device displays files in other electronic devices according to a metadata file in the embodiment of the present application. In the process of displaying the file 5 and the file 6 in the root directory according to the metadata information recorded in the metadata file 2-1, the electronic device 1 generates an inode structure of the file 5 and an inode structure of the file 6 in the memory of the electronic device 1, respectively. Since the files 5 and 6 are generated from the metadata file 2-1 and the metadata file 2-1 is derived from the electronic device 2, the identities of the electronic device 2 are stored in the inode structure of the file 5 and the inode structure of the file 6, respectively. When the electronic device 1 receives an access request for the file 5, the electronic device 1 can determine that the file 5 is a file in the electronic device 2 in the distributed file system according to the identifier of the electronic device 2 in the inode structure of the file 5 in the kernel. So that the index path of the file 5 can be determined from the metadata information of the file 5, and the file 5 in the electronic device 2 can be accessed according to the index path.
With reference to the schematic user interface diagram shown in fig. 7, fig. 9 is an exemplary schematic diagram of a directory tree after metadata files are obtained from other electronic devices in the embodiment of the present application.
As shown in fig. 9 (a), the directory tree is a directory tree of a file system mounted by a distributed file system in the electronic device 1. Wherein, the root directory contains directory a, file 1 and metadata file 1-1. The file name of the metadata file 1-1 includes the identifier of the electronic device 1: 1. the metadata file 1-1 stores metadata information of the directory a and the file 1. The directory a includes a file 7 and metadata files 1-a, and metadata information of the file 7 is stored in the metadata files 1-a. The file name of the metadata file 1-a includes the identifier of the electronic device 1: 1.
as shown in fig. 9 (b), the directory tree is a directory tree of the file system mounted by the distributed file system in the electronic device 2. Wherein, the root directory contains directory a, file 5, file 6 and metadata file 2-1. The file name of the metadata file 2-1 includes the identifier of the electronic device 2: 2. the metadata file 2-1 stores metadata information of the directory a, the file 5, and the file 6. The directory a includes a file 8 and a metadata file 2-a, and metadata information of the file 7 is stored in the metadata file 2-a. The file name of the metadata file 2-a includes the identifier of the electronic device 2: 2.
As shown in fig. 9 (c), the metadata file 2-1 of the electronic device 2 is pulled for the file system mounted by the distributed file system in the electronic device 1, and is stored in the directory tree after being stored in the root directory. The root directory includes a directory a, a file 1, a file 5, a file 6, a metadata file 1-1, and a metadata file 2-1.
However, the files 5 and 6 may not include actual data, but the files 5 and 6 have the identifier of the electronic device 2 in the corresponding inode structure in the memory, so that a request for accessing the files 5 or 6 in the root directory is converted into a request for accessing the files 5 or 6 in the root directory of the electronic device 2.
The file name of the metadata file 1-1 includes the identifier of the electronic device 1: 1, and the metadata information of the directory a and the file 1 is stored in the metadata file 1-1. The directory a includes a file 7 and metadata files 1-a, and metadata information of the file 7 is stored in the metadata files 1-a. The file name of the metadata file 1-a includes the identifier of the electronic device 1: 1.
the file name of the metadata file 2-1 includes the identifier of the electronic device 2: 2. the metadata file 2-1 stores metadata information of the directory a, the file 5, and the file 6.
Fig. 10 is a diagram illustrating an exemplary scenario of a distributed file system in the prior art. In the prior art, a distributed file system generally comprises a metadata server and a plurality of data servers, wherein the metadata server is used as a central node and stores metadata information, and the data servers manage and store actual data of files in the distributed file system. When a user accesses file data, the metadata server serving as a central node needs to be accessed to acquire basic information and data index information of a file, and then the data server is accessed to acquire the file data according to the basic information and the data index information. As shown in fig. 10, device a, device B, and device C all need to access files in the data server through a metadata server as a central node in the distributed file system.
Specifically, referring to the scenario diagram shown in fig. 10, fig. 11 is an exemplary signaling interaction diagram for modifying a file in a distributed file system in the prior art. If the device a needs to access and modify the file a in the distributed file system, the device a needs to query the metadata server for the metadata information of the file a. The metadata server returns the metadata information for file a to device a. And the device A accesses and modifies the file A from the data server according to the address of the file A in the metadata information of the file A. The data server may then save the latest information for file a and update the metadata information for file a in the metadata server.
Therefore, in the distributed file system in the prior art, the metadata server is used as a central node of the distributed file system, and it is required to ensure that the metadata query and update service is provided online at any time. When the metadata server goes offline, the services of the distributed file system are unavailable.
The metadata management method provided by the embodiment of the application realizes metadata management of the distributed file system in the centerless distributed file system formed by a plurality of electronic devices, the distributed file system cannot be unavailable due to offline of a certain electronic device, and the stability of using the distributed file system is improved.
Fig. 12 is a schematic diagram of an exemplary scenario of a centerless distributed file system in an embodiment of the present application. The electronic devices such as the device A, the device B, the device C, the device D, the device E and the like form a centerless distributed file system. Each electronic device in the centerless distributed file system can directly manage the metadata information of files in other electronic devices, so that the files in other electronic devices can be directly displayed or accessed without passing through a central node. For example, file a is stored in device a, file B is stored in device B, and file C is stored in device C, but all online devices in the distributed file system may directly display or access these files. When any electronic device in the centerless distributed file system is offline, other electronic devices are not influenced to continue to normally use the service of the centerless distributed file system.
An exemplary electronic device 100 provided by embodiments of the present application is first described below.
Fig. 13 is a schematic structural diagram of an electronic device 100 according to an embodiment of the present application.
The following describes an embodiment specifically by taking the electronic device 100 as an example. It should be understood that electronic device 100 may have more or fewer components than shown, may combine two or more components, or may have a different configuration of components. The various components shown in the figures may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and/or application specific integrated circuits.
The electronic device 100 may include: the mobile terminal includes a processor 110, an external memory interface 120, an internal memory 121, a Universal Serial Bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, a Subscriber Identity Module (SIM) card interface 195, and the like. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, and the like.
It is to be understood that the illustrated structure of the embodiment of the present application does not specifically limit the electronic device 100. In other embodiments of the present application, electronic device 100 may include more or fewer components than shown, or some components may be combined, some components may be split, or a different arrangement of components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.
Processor 110 may include one or more processing units, such as: the processor 110 may include an Application Processor (AP), a modem processor, a Graphics Processing Unit (GPU), an Image Signal Processor (ISP), a controller, a memory, a video codec, a Digital Signal Processor (DSP), a baseband processor, and/or a neural-Network Processing Unit (NPU), etc. The different processing units may be separate devices or may be integrated into one or more processors.
The controller may be, among other things, a neural center and a command center of the electronic device 100. The controller can generate an operation control signal according to the instruction operation code and the timing signal to complete the control of instruction fetching and instruction execution.
A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may hold instructions or data that have just been used or recycled by the processor 110. If the processor 110 needs to reuse the instruction or data, it can be called directly from the memory. Avoiding repeated accesses reduces the latency of the processor 110, thereby increasing the efficiency of the system.
In some embodiments, processor 110 may include one or more interfaces. The interface may include an integrated circuit (I2C) interface, an integrated circuit built-in audio (I2S) interface, a Pulse Code Modulation (PCM) interface, a universal asynchronous receiver/transmitter (UART) interface, a Mobile Industry Processor Interface (MIPI), a general-purpose input/output (GPIO) interface, a Subscriber Identity Module (SIM) interface, and/or a Universal Serial Bus (USB) interface, etc.
The I2C interface is a bi-directional synchronous serial bus that includes a serial data line (SDA) and a Serial Clock Line (SCL). In some embodiments, processor 110 may include multiple sets of I2C buses. The processor 110 may be coupled to the touch sensor 180K, charger, flash, camera 193, etc. through different I2C bus interfaces, respectively. For example: the processor 110 may be coupled to the touch sensor 180K via an I2C interface, such that the processor 110 and the touch sensor 180K communicate via an I2C bus interface to implement the touch functionality of the electronic device 100.
The I2S interface may be used for audio communication. In some embodiments, processor 110 may include multiple sets of I2S buses. The processor 110 may be coupled to the audio module 170 via an I2S bus to enable communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 may communicate audio signals to the wireless communication module 160 via the I2S interface, enabling answering of calls via a bluetooth headset.
The PCM interface may also be used for audio communication, sampling, quantizing and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 may be coupled by a PCM bus interface. In some embodiments, the audio module 170 may also transmit audio signals to the wireless communication module 160 through the PCM interface, so as to implement a function of answering a call through a bluetooth headset. Both the I2S interface and the PCM interface may be used for audio communication.
The UART interface is a universal serial data bus used for asynchronous communications. The bus may be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, a UART interface is generally used to connect the processor 110 with the wireless communication module 160. For example: the processor 110 communicates with a bluetooth module in the wireless communication module 160 through a UART interface to implement a bluetooth function. In some embodiments, the audio module 170 may transmit the audio signal to the wireless communication module 160 through a UART interface, so as to realize the function of playing music through a bluetooth headset.
MIPI interfaces may be used to connect processor 110 with peripheral devices such as display screen 194, camera 193, and the like. The MIPI interface includes a Camera Serial Interface (CSI), a Display Serial Interface (DSI), and the like. In some embodiments, processor 110 and camera 193 communicate through a CSI interface to implement the capture functionality of electronic device 100. The processor 110 and the display screen 194 communicate through the DSI interface to implement the display function of the electronic device 100.
The GPIO interface may be configured by software. The GPIO interface may be configured as a control signal and may also be configured as a data signal. In some embodiments, a GPIO interface may be used to connect the processor 110 with the camera 193, the display 194, the wireless communication module 160, the audio module 170, the sensor module 180, and the like. The GPIO interface may also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, and the like.
The SIM interface may be used to communicate with the SIM card interface 195, enabling functions of transferring data to or reading data from the SIM card.
The USB interface 130 is an interface conforming to the USB standard specification, and may be a Mini USB interface, a Micro USB interface, a USB Type C interface, or the like. The USB interface 130 may be used to connect a charger to charge the electronic device 100, and may also be used to transmit data between the electronic device 100 and a peripheral device. And the method can also be used for connecting a headset and playing audio through the headset. The interface may also be used to connect other electronic devices, such as AR devices and the like.
It should be understood that the connection relationship between the modules illustrated in the embodiment of the present application is only an exemplary illustration, and does not limit the structure of the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection manners or a combination of multiple interface connection manners in the above embodiments.
The charging management module 140 is configured to receive charging input from a charger. The charger may be a wireless charger or a wired charger.
The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and/or the charge management module 140 and provides power to the processor 110, the internal memory 121, the external memory, the display 194, the camera 193, the wireless communication module 160, and the like.
The wireless communication function of the electronic device 100 may be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, a modem processor, a baseband processor, and the like.
The antennas 1 and 2 are used for transmitting and receiving electromagnetic wave signals. Each antenna in the electronic device 100 may be used to cover a single or multiple communication bands. Different antennas can also be multiplexed to improve the utilization of the antennas. For example: the antenna 1 may be multiplexed as a diversity antenna of a wireless local area network. In other embodiments, the antenna may be used in conjunction with a tuning switch.
The mobile communication module 150 may provide a solution including 2G/3G/4G/5G wireless communication applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a Low Noise Amplifier (LNA), and the like. The mobile communication module 150 may receive the electromagnetic wave from the antenna 1, filter, amplify, etc. the received electromagnetic wave, and transmit the electromagnetic wave to the modem processor for demodulation. The mobile communication module 150 may also amplify the signal modulated by the modem processor, and convert the signal into electromagnetic wave through the antenna 1 to radiate the electromagnetic wave. In some embodiments, at least some of the functional modules of the mobile communication module 150 may be disposed in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 may be disposed in the same device as at least some of the modules of the processor 110.
The modem processor may include a modulator and a demodulator. The modulator is used for modulating a low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used for demodulating the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then passes the demodulated low frequency baseband signal to a baseband processor for processing. The low frequency baseband signal is processed by the baseband processor and then transferred to the application processor. The application processor outputs a sound signal through an audio device (not limited to the speaker 170A, the receiver 170B, etc.) or displays an image or video through the display screen 194. In some embodiments, the modem processor may be a stand-alone device. In other embodiments, the modem processor may be provided in the same device as the mobile communication module 150 or other functional modules, independent of the processor 110.
The wireless communication module 160 may provide a solution for wireless communication applied to the electronic device 100, including Wireless Local Area Networks (WLANs) (e.g., wireless fidelity (Wi-Fi) networks), bluetooth (bluetooth, BT), Global Navigation Satellite System (GNSS), Frequency Modulation (FM), Near Field Communication (NFC), Infrared (IR), and the like. The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, performs frequency modulation and filtering processing on electromagnetic wave signals, and transmits the processed signals to the processor 110. The wireless communication module 160 may also receive a signal to be transmitted from the processor 110, perform frequency modulation and amplification on the signal, and convert the signal into electromagnetic waves through the antenna 2 to radiate the electromagnetic waves.
In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150 and antenna 2 is coupled to wireless communication module 160 so that electronic device 100 can communicate with networks and other devices through wireless communication techniques. The wireless communication technology may include global system for mobile communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (long term evolution, LTE), BT, GNSS, WLAN, NFC, FM, and/or IR technologies, etc. The GNSS may include a Global Positioning System (GPS), a global navigation satellite system (GLONASS), a beidou satellite navigation system (BDS), a quasi-zenith satellite system (QZSS), and/or a Satellite Based Augmentation System (SBAS).
The electronic device 100 implements display functions via the GPU, the display screen 194, and the application processor. The GPU is a microprocessor for image processing, and is connected to the display screen 194 and an application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or alter display information.
The display screen 194 is used to display images, video, and the like. The display screen 194 includes a display panel. The display panel may adopt a Liquid Crystal Display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (active-matrix organic light-emitting diode, AMOLED), a flexible light-emitting diode (FLED), a miniature, a Micro-oeld, a quantum dot light-emitting diode (QLED), and the like. In some embodiments, the electronic device 100 may include 1 or N display screens 194, with N being a positive integer greater than 1.
The electronic device 100 may implement a shooting function through the ISP, the camera 193, the video codec, the GPU, the display 194, the application processor, and the like.
The ISP is used to process the data fed back by the camera 193. For example, when a photo is taken, the shutter is opened, light is transmitted to the camera photosensitive element through the lens, the optical signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing and converting into an image visible to naked eyes. The ISP can also carry out algorithm optimization on the noise, brightness and skin color of the image. The ISP can also optimize parameters such as exposure, color temperature and the like of a shooting scene. In some embodiments, the ISP may be provided in camera 193.
The camera 193 is used to capture still images or video. The object generates an optical image through the lens and projects the optical image to the photosensitive element. The photosensitive element may be a Charge Coupled Device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The light sensing element converts the optical signal into an electrical signal, which is then passed to the ISP where it is converted into a digital image signal. And the ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into image signal in standard RGB, YUV and other formats. In some embodiments, the electronic device 100 may include 1 or N cameras 193, N being a positive integer greater than 1.
The digital signal processor is used for processing digital signals, and can process other digital signals besides digital image signals. For example, when the electronic device 100 selects a frequency bin, the digital signal processor is used to perform fourier transform or the like on the frequency bin energy.
Video codecs are used to compress or decompress digital video. The electronic device 100 may support one or more video codecs. In this way, the electronic device 100 may play or record video in a variety of encoding formats, such as: moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, and the like.
The NPU is a neural-network (NN) computing processor that processes input information quickly by using a biological neural network structure, for example, by using a transfer mode between neurons of a human brain, and can also learn by itself continuously. Applications such as intelligent recognition of the electronic device 100 can be realized through the NPU, for example: image recognition, face recognition, speech recognition, text understanding, and the like.
The internal memory 121 may include one or more Random Access Memories (RAMs) and one or more non-volatile memories (NVMs).
The random access memory may include static random-access memory (SRAM), dynamic random-access memory (DRAM), synchronous dynamic random-access memory (SDRAM), double data rate synchronous dynamic random-access memory (DDR SDRAM), such as fifth generation DDR SDRAM generally referred to as DDR5 SDRAM, and the like;
the nonvolatile memory may include a magnetic disk storage device, a flash memory (flash memory).
The FLASH memory may include NOR FLASH, NAND FLASH, 3D NAND FLASH, etc. according to the operation principle, may include single-level cells (SLC), multi-level cells (MLC), three-level cells (TLC), four-level cells (QLC), etc. according to the level order of the memory cells, and may include universal FLASH memory (UFS), embedded multimedia memory cards (eMMC), etc. according to the storage specification.
The random access memory may be read and written directly by the processor 110, may be used to store executable programs (e.g., machine instructions) of an operating system or other programs in operation, and may also be used to store data of users and applications, etc.
The nonvolatile memory may also store executable programs, data of users and application programs, and the like, and may be loaded into the random access memory in advance for the processor 110 to directly read and write.
The external memory interface 120 may be used to connect an external nonvolatile memory to extend the storage capability of the electronic device 100. The external non-volatile memory communicates with the processor 110 through the external memory interface 120 to implement data storage functions. For example, files such as music, video, etc. are saved in an external nonvolatile memory.
The electronic device 100 may implement audio functions via the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone interface 170D, and the application processor. Such as music playing, recording, etc.
The audio module 170 is used to convert digital audio information into an analog audio signal output and also to convert an analog audio input into a digital audio signal. The audio module 170 may also be used to encode and decode audio signals. In some embodiments, the audio module 170 may be disposed in the processor 110, or some functional modules of the audio module 170 may be disposed in the processor 110.
The speaker 170A, also called a "horn", is used to convert the audio electrical signal into a sound signal. The electronic apparatus 100 can listen to music through the speaker 170A or listen to a hands-free call.
The receiver 170B, also called "earpiece", is used to convert the electrical audio signal into a sound signal. When the electronic apparatus 100 receives a call or voice information, it is possible to receive voice by placing the receiver 170B close to the human ear.
The microphone 170C, also referred to as a "microphone," is used to convert sound signals into electrical signals. When making a call or sending voice information, the user can input a voice signal into the microphone 170C by uttering a voice signal by the mouth of the user near the microphone 170C. The electronic device 100 may be provided with at least one microphone 170C. In other embodiments, the electronic device 100 may be provided with two microphones 170C to achieve a noise reduction function in addition to collecting sound signals. In other embodiments, the electronic device 100 may further include three, four or more microphones 170C to collect sound signals, reduce noise, identify sound sources, perform directional recording, and so on.
The earphone interface 170D is used to connect a wired earphone. The headset interface 170D may be the USB interface 130, or may be a 3.5mm open mobile electronic device platform (OMTP) standard interface, a cellular telecommunications industry association (cellular telecommunications industry association of the USA, CTIA) standard interface.
The pressure sensor 180A is used for sensing a pressure signal, and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A may be disposed on the display screen 194. The pressure sensor 180A can be of a variety of types, such as a resistive pressure sensor, an inductive pressure sensor, a capacitive pressure sensor, and the like. The capacitive pressure sensor may be a sensor comprising at least two parallel plates having an electrically conductive material. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the strength of the pressure from the change in capacitance. When a touch operation is applied to the display screen 194, the electronic apparatus 100 detects the intensity of the touch operation according to the pressure sensor 180A. The electronic apparatus 100 may also calculate the touched position from the detection signal of the pressure sensor 180A. In some embodiments, the touch operations that are applied to the same touch position but different touch operation intensities may correspond to different operation instructions. For example: and when the touch operation with the touch operation intensity smaller than the first pressure threshold value acts on the short message application icon, executing an instruction for viewing the short message. And when the touch operation with the touch operation intensity larger than or equal to the first pressure threshold value acts on the short message application icon, executing an instruction of newly building the short message.
The gyro sensor 180B may be used to determine the motion attitude of the electronic device 100. In some embodiments, the angular velocity of electronic device 100 about three axes (i.e., the x, y, and z axes) may be determined by gyroscope sensor 180B. The gyro sensor 180B may be used for photographing anti-shake. For example, when the shutter is pressed, the gyro sensor 180B detects a shake angle of the electronic device 100, calculates a distance to be compensated for by the lens module according to the shake angle, and allows the lens to counteract the shake of the electronic device 100 through a reverse movement, thereby achieving anti-shake. The gyroscope sensor 180B may also be used for navigation, somatosensory gaming scenes.
The air pressure sensor 180C is used to measure air pressure. In some embodiments, electronic device 100 calculates altitude, aiding in positioning and navigation, from barometric pressure values measured by barometric pressure sensor 180C.
The magnetic sensor 180D includes a hall sensor. The electronic device 100 may detect the opening and closing of the flip holster using the magnetic sensor 180D. In some embodiments, when the electronic device 100 is a flip phone, the electronic device 100 may detect the opening and closing of the flip according to the magnetic sensor 180D. And then according to the opening and closing state of the leather sheath or the opening and closing state of the flip cover, the automatic unlocking of the flip cover is set.
The acceleration sensor 180E may detect the magnitude of acceleration of the electronic device 100 in various directions (typically three axes). The magnitude and direction of gravity can be detected when the electronic device 100 is stationary. The method can also be used for recognizing the posture of the electronic equipment, and is applied to horizontal and vertical screen switching, pedometers and other applications.
A distance sensor 180F for measuring a distance. The electronic device 100 may measure the distance by infrared or laser. In some embodiments, taking a picture of a scene, electronic device 100 may utilize range sensor 180F to range for fast focus.
The proximity light sensor 180G may include, for example, a Light Emitting Diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The electronic device 100 emits infrared light to the outside through the light emitting diode. The electronic device 100 detects infrared reflected light from nearby objects using a photodiode. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 100. When insufficient reflected light is detected, the electronic device 100 may determine that there are no objects near the electronic device 100. The electronic device 100 can utilize the proximity light sensor 180G to detect that the user holds the electronic device 100 close to the ear for talking, so as to automatically turn off the screen to achieve the purpose of saving power. The proximity light sensor 180G may also be used in a holster mode, a pocket mode automatically unlocks and locks the screen.
The ambient light sensor 180L is used to sense ambient light brightness. Electronic device 100 may adaptively adjust the brightness of display screen 194 based on the perceived ambient light level. The ambient light sensor 180L may also be used to automatically adjust the white balance when taking a picture. The ambient light sensor 180L may also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in a pocket to prevent accidental touches.
The fingerprint sensor 180H is used to collect a fingerprint. The electronic device 100 can utilize the collected fingerprint characteristics to unlock the fingerprint, access the application lock, photograph the fingerprint, answer an incoming call with the fingerprint, and so on.
The temperature sensor 180J is used to detect temperature. In some embodiments, electronic device 100 implements a temperature processing strategy using the temperature detected by temperature sensor 180J. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold, the electronic device 100 performs a reduction in performance of a processor located near the temperature sensor 180J, so as to reduce power consumption and implement thermal protection. In other embodiments, the electronic device 100 heats the battery 142 when the temperature is below another threshold to avoid the low temperature causing the electronic device 100 to shut down abnormally. In other embodiments, when the temperature is lower than a further threshold, the electronic device 100 performs boosting on the output voltage of the battery 142 to avoid abnormal shutdown due to low temperature.
The touch sensor 180K is also referred to as a "touch panel". The touch sensor 180K may be disposed on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, which is also called a "touch screen". The touch sensor 180K is used to detect a touch operation applied thereto or nearby. The touch sensor can communicate the detected touch operation to the application processor to determine the touch event type. Visual output associated with the touch operation may be provided through the display screen 194. In other embodiments, the touch sensor 180K may be disposed on a surface of the electronic device 100, different from the position of the display screen 194.
The keys 190 include a power-on key, a volume key, and the like. The keys 190 may be mechanical keys. Or may be touch keys. The electronic apparatus 100 may receive a key input, and generate a key signal input related to user setting and function control of the electronic apparatus 100.
The motor 191 may generate a vibration cue. The motor 191 may be used for incoming call vibration cues, as well as for touch vibration feedback. For example, touch operations applied to different applications (e.g., photographing, audio playing, etc.) may correspond to different vibration feedback effects. The motor 191 may also respond to different vibration feedback effects for touch operations applied to different areas of the display screen 194. Different application scenes (such as time reminding, receiving information, alarm clock, game and the like) can also correspond to different vibration feedback effects. The touch vibration feedback effect may also support customization.
Indicator 192 may be an indicator light that may be used to indicate a state of charge, a change in charge, or a message, missed call, notification, etc.
The SIM card interface 195 is used to connect a SIM card. The SIM card can be brought into and out of contact with the electronic apparatus 100 by being inserted into the SIM card interface 195 or being pulled out of the SIM card interface 195. The electronic device 100 may support 1 or N SIM card interfaces, N being a positive integer greater than 1. The SIM card interface 195 may support a Nano SIM card, a Micro SIM card, a SIM card, etc. The same SIM card interface 195 can be inserted with multiple cards at the same time. The types of the plurality of cards may be the same or different. The SIM card interface 195 may also be compatible with different types of SIM cards. The SIM card interface 195 may also be compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to implement functions such as communication and data communication.
In the embodiment of the present application, the file stored in the internal memory 121 may be displayed in a file system mounted by a distributed file system. The electronic device 100 can acquire metadata files in other devices constituting the distributed file system through the mobile communication module 150 or the wireless communication module 160 and save them in the internal memory 121. The processor 110 may cause the electronic device 100 to perform the metadata management method in the embodiment of the present application by calling the computer instructions stored in the internal memory 121.
Fig. 14 is a block diagram of a software configuration of the electronic device 100 according to the embodiment of the present application.
The layered architecture divides the software into several layers, each layer having a clear role and division of labor. The layers communicate with each other through a software interface. In some embodiments, the system is divided into four layers, an application layer, an application framework layer, a Runtime (Runtime) and system library, and a kernel layer, from top to bottom.
The application layer may include a series of application packages.
As shown in fig. 14, the application package may include camera, gallery, calendar, phone call, map, navigation, WLAN, bluetooth, music, video, short message, etc. applications (also referred to as applications).
In this embodiment, the application layer may further include a metadata management module, and the metadata management module may be configured to execute the metadata management method in this embodiment.
Optionally, in this embodiment of the application, the application layer may further include an online determination module, where the online determination module may be configured to determine, in real time, an online status of other electronic devices in the distributed file system. The online determination module may provide a reading interface to the metadata management module in the embodiments of the present application. The metadata management module can determine the online state of other electronic equipment in the distributed file system in real time through the reading interface of the online determination module.
The online determining module may determine online states of other electronic devices in the distributed file system in many ways, for example, periodically sends heartbeat detection signals to the other electronic devices, determines that the other electronic devices are online when response signals of the heartbeat detection signals are detected, and determines that the other electronic devices are offline when the response signals of the heartbeat detection signals are not detected for a plurality of times; for another example, after receiving the online notification sent by the other electronic device, it is determined that the other electronic device is online, and after receiving the offline notification sent by the other electronic device, it is determined that the other electronic device is offline, and the like.
The application framework layer provides an Application Programming Interface (API) and a programming framework for the application program of the application layer. The application framework layer includes a number of predefined functions.
As shown in FIG. 14, the application framework layers may include a window manager, content provider, view system, phone manager, resource manager, notification manager, and the like.
The window manager is used for managing window programs. The window manager can obtain the size of the display screen, judge whether a status bar exists, lock the screen, intercept the screen and the like.
Content providers are used to store and retrieve data and make it accessible to applications. The data may include video, images, audio, calls made and answered, browsing history and bookmarks, phone books, etc.
The view system includes visual controls such as controls to display text, controls to display pictures, and the like. The view system may be used to build applications. The display interface may be composed of one or more views. For example, the display interface including the short message notification icon may include a view for displaying text and a view for displaying pictures.
The phone manager is used to provide communication functions of the electronic device 100. Such as management of call status (including on, off, etc.).
The resource manager provides various resources for the application, such as localized strings, icons, pictures, layout files, video files, and the like.
The notification manager enables the application to display notification information in the status bar, can be used to convey notification-type messages, can disappear automatically after a short dwell, and does not require user interaction. Such as a notification manager used to inform download completion, message alerts, etc. The notification manager may also be a notification that appears in the form of a chart or scroll bar text at the top status bar of the system, such as a notification of a background running application, or a notification that appears on the screen in the form of a dialog interface. For example, prompting text information in the status bar, sounding a prompt tone, vibrating the electronic device, flashing an indicator light, etc.
The Runtime (Runtime) includes a core library and a virtual machine. Runtime is responsible for scheduling and management of the system.
The core library comprises two parts: one part is a function which needs to be called by a programming language (e.g. java language), and the other part is a core library of the system.
The application layer and the application framework layer run in a virtual machine. The virtual machine executes programming files (e.g., java files) of the application layer and the application framework layer as binary files. The virtual machine is used for performing the functions of object life cycle management, stack management, thread management, safety and exception management, garbage collection and the like.
The system library may include a plurality of functional modules. For example: surface managers (surface managers), Media Libraries (Media Libraries), three-dimensional graphics processing Libraries (e.g., OpenGL ES), two-dimensional graphics engines (e.g., SGL), and the like.
The surface manager is used to manage the display subsystem and provides a fusion of two-Dimensional (2-Dimensional, 2D) and three-Dimensional (3-Dimensional, 3D) layers for multiple applications.
The media library supports a variety of commonly used audio, video format playback and recording, and still image files, among others. The media library may support a variety of audio-video encoding formats, such as MPEG4, h.264, MP3, AAC, AMR, JPG, PNG, and the like.
The three-dimensional graphic processing library is used for realizing 3D graphic drawing, image rendering, synthesis, layer processing and the like.
The 2D graphics engine is a drawing engine for 2D drawing.
The kernel layer is a layer between hardware and software. The kernel layer at least comprises a display driver, a camera driver, an audio driver, a sensor driver and a virtual card driver.
The following describes the method in the embodiment of the present application in detail with reference to the software and hardware structure of the above exemplary electronic device 100:
the metadata management method in the embodiment of the present application is described below with reference to different scenarios by taking an example in which the electronic device a, the electronic device B, and the electronic device C form a centerless distributed file system. It will be appreciated that the centerless distributed file system may be comprised of more or fewer electronic devices and is not limited thereto. In the following description, for simplicity of description, the centerless distributed file system may be simply referred to as a distributed file system.
In the embodiment of the present application, the device group forming the distributed file system may be electronic devices under the same network account, may be electronic devices under the same local area network, may be electronic devices with the same specific identifier, may also be a combination of these situations, and the like, and may be specifically set according to an actual situation, which is not limited herein.
The metadata management method in the embodiment of the present application is described below by combining signaling interaction diagrams between electronic devices in different scenarios according to (1) a method execution basis, (2) accessing or modifying files in the electronic device, (3) caching metadata files of other electronic devices, (4) accessing files in other electronic devices, (5) modifying cached files in other electronic devices, and (6) metadata file cleaning after and before the electronic devices are offline.
It is to be understood that the metadata management method in the embodiment of the present application is described below in steps performed by the electronic device a, the electronic device B, and the electronic device C in a specific scenario, and thus different electronic devices may perform different steps. All electronic devices in the distributed file system may have the capability to perform any of the steps described below and are not limited thereto.
(1) The method is implemented as follows:
fig. 15 is a schematic diagram of signaling interaction of a metadata management method in this embodiment.
S1501, in the file system mounted in the electronic device by the distributed file system, maintaining a metadata file under each directory, where the metadata file stores metadata information of all other files under the directory.
For the specific distribution structure of the metadata file, refer to (2) description in the metadata file 2.1 in the above introduction of terms and description in the distribution structure of the metadata file, which are not described herein again.
In a centerless distributed file system composed of an electronic device a, an electronic device B and an electronic device C, the electronic device a, the electronic device B and the electronic device C may respectively maintain a metadata file under each directory in the file system mounted by the centerless distributed file system in their own electronic devices, and the metadata file stores metadata information of all other files under the directory.
For the description of the metadata information, reference may be made to the description in (1) the metadata information in the above introduction of terms, which is not described herein again.
Illustratively, as shown in fig. 16, a schematic diagram of a set of user interfaces in the embodiment of the present application is shown. The electronic device a, the electronic device B, and the electronic device C constitute a distributed file system.
As shown in fig. 16 (a), the user interface 1610 is a directory a under the root directory in the file system mounted by the distributed file system in the electronic device a. The directory a of the electronic device a stores a file a and maintains a metadata file a. The metadata file a stores metadata information of the file a.
As shown in fig. 16 (B), the user interface 1620 is a directory a under the root directory in the file system mounted by the distributed file system in the electronic device B. The directory a of the electronic device B stores a file B and maintains a metadata file B. The metadata file B stores metadata information of the file B.
As shown in (C) of fig. 16, the user interface 1630 is for the directory a under the root directory in the file system mounted by the distributed file system in the electronic device C. The directory a of the electronic device C stores a file C1 and a file C2, and maintains a metadata file C. The metadata file C stores metadata information of the file C1 and the file C2.
It is understood that each of the other directories of the electronic device a, the electronic device B, and the electronic device C also maintains a metadata file for storing metadata information of all other files in the directory.
(2) Accessing or modifying files in the electronic device:
fig. 17 is a schematic signaling interaction diagram of a metadata management method in this embodiment.
The following describes a metadata management method in an embodiment of the present application, taking as an example a process in which an electronic device a in a distributed file system accesses or modifies a file a in a directory a of the file system mounted by the distributed file system in the electronic device a.
It is understood that the electronic devices in the distributed file system have performed the above step S1501 before performing the steps shown in fig. 17. Therefore, the electronic device a maintains a metadata file a under the directory a, and the metadata file a stores therein metadata information of all other files under the directory a of the electronic device a.
S1701, the electronic equipment A accesses a file A under the directory A;
the electronic device A can directly access the file A under the directory A in the electronic device without reading the metadata information of the file A.
S1702, the electronic device A modifies and stores the file A under the directory A;
when the electronic device A accesses the file A, the file A can be directly modified and stored.
S1703, the electronic device A updates the metadata information of the file A in the metadata file A under the directory A;
after the electronic device a modifies and saves the file a, the electronic device a updates the metadata information of the file a in the metadata file a in the directory a where the file a is located. For example, the description of the metadata file update caused by the file modification shown in fig. 4 in the above-mentioned terminology introduction (2) metadata file may be referred to, and details are not repeated here.
S1704, the electronic device A determines whether an online electronic device requesting to acquire the metadata file A exists in the distributed file system;
After the electronic device a updates the metadata file a, the electronic device a may determine whether there is an online electronic device in the distributed file system that requests to obtain the metadata file a.
Optionally, the electronic device a may also determine whether it has requested to acquire the metadata file a by sending a query request to other electronic devices in the distributed file system.
Optionally, each electronic device in the distributed file system may store a read record, and the read record may record an identifier of each other electronic device in the distributed file system, whether each electronic device is currently online, and a corresponding relationship of which metadata files in the electronic device are requested by each electronic device. The identifier of the electronic device is used to uniquely identify one electronic device in the distributed file system, and may be any one of or a combination of a device number, a device name, an International Mobile Equipment Identity (IMEI), and the like of the electronic device, which is not limited herein.
It is understood that the electronic device a may also confirm whether there is an electronic device in the distributed file system that requests to obtain the metadata file a and is online in other ways, which is not limited herein.
When the electronic device a determines that there is an electronic device that requests to acquire the metadata file a and is online in the distributed file system, the electronic device a may perform step S1705;
when the electronic device a determines that there is an off-line electronic device that requests to acquire the metadata file a in the distributed file system, the electronic device a may perform step S1706;
when the electronic device a determines that there is no electronic device in the distributed file system that requests acquisition of the metadata file a, the electronic device a may perform step S1707.
Taking the electronic device a determining whether there is an electronic device that requests to acquire the metadata file a and is online through the stored read record as an example, the following describes step S1704 specifically and exemplarily:
as shown in table 1 below, this is an exemplary example of reading records in the electronic device a in the embodiment of the present application:
electronic equipment identification On-line status Metadata file requested to be read
Electronic equipment B On-line
Electronic equipment C On-line Metadata file A
TABLE 1
It is understood that the read record may be in other forms besides a table form, such as an array, a database, etc., and is not limited herein.
The electronic device a may determine whether there is an online electronic device that requests to obtain the metadata file a in the distributed file system by querying the read record:
For example, if the read record in the electronic device a is shown in table 1, the electronic device a may determine that there is an electronic device C that requests to acquire the metadata file a and is online, and the electronic device a may perform step S1705;
as shown in table 2 below, is another illustrative example of reading records in the electronic device a in the embodiment of the present application:
electronic device identification On-line status Metadata file requested to be read
Electronic equipment B Out of line Metadata file A
Electronic equipment C On-line
TABLE 2
For example, if the read record in the electronic device a is as shown in table 2, the electronic device a may determine that there is an electronic device B that requests to acquire the metadata file a and is not online, and the electronic device a may execute step S1706;
as shown in table 3 below, is another illustrative example of the read record of the electronic device a in the embodiment of the present application:
electronic equipment identification On-line status Metadata file requested to be read
Electronic equipment B Out of line
Electronic equipment C On-line
TABLE 3
For example, if the read record in the electronic device a is as shown in table 3, the electronic device a may determine that there is no electronic device that requests to acquire the metadata file a, and the electronic device a may perform step S1707.
Optionally, in other embodiments, if one electronic device in the device group does not request to acquire the metadata file in the electronic device a, an entry corresponding to the electronic device may not exist in the read record of the electronic device a.
For example, the electronic device a may store the read records shown in table 1 in another way. As follows
Table 4 shows another exemplary example of reading records of the electronic device a in the embodiment of the present application.
Electronic equipment identification On-line status Metadata file requested to be read
Electronic equipment C On-line Metadata file A
TABLE 4
Correspondingly, the read record shown in table 2 or table 3 may not have an entry corresponding to the electronic device that does not read the metadata file in the electronic device a, and is not described here again.
S1705, the electronic device A pushes the metadata file A to the electronic device which requests to acquire the metadata file A and is online in the distributed file system;
after the electronic device a updates the metadata file a, when the electronic device a determines that an electronic device that requests to acquire the metadata file a and is online exists in the distributed file system, the electronic device a may push the metadata file a to the electronic device that requests to acquire the metadata file a and is online, so that the electronic device updates the acquired metadata file a.
For example, if the electronic device a determines, through the read record shown in table 1 above, that there is an electronic device C that requests to acquire the metadata file a and is online, the electronic device a may push the metadata file a to the electronic device C.
S1706, the electronic device A deletes the record of the electronic device which requests to acquire the metadata file A but is not on-line in the read record to read the metadata file A;
after the electronic device a updates the metadata file a, when the electronic device a determines that an electronic device that requests to acquire the metadata file a and is not online exists in the distributed file system, the electronic device a may delete a record of the electronic device that acquires the metadata file a and is not online in the read record to read the metadata file a.
For example, if the electronic device a determines, through the read record shown in table 2 above, that there is an electronic device B that requests to acquire the metadata file a but is not online, the electronic device a may delete the record of the read record of the metadata file a read by the electronic device B.
It is understood that the electronic device a may delete the record of the electronic device B reading the metadata file a in the read record in various ways:
illustratively, as shown in table 5 below, an exemplary example of the read record shown in table 2 that is deleted by the electronic device a after the record of the electronic device B that reads the metadata file a is deleted is shown in this application embodiment:
Electronic device identification On-line status Metadata file requested to be read
Electronic equipment B Out of line
Electronic equipment C On-line
TABLE 5
For example, in some embodiments, if the other electronic devices in the read record of the electronic device a do not have the metadata file requested to be read, the electronic device a may directly delete the entire entry of the electronic device in the read record. And is not limited herein.
S1707, the electronic device a waits to execute the new instruction.
After the electronic device a updates the metadata file a, when the electronic device a determines that there is no electronic device in the distributed file system that requests to acquire the metadata file a, the electronic device a may not perform any operation and wait for a new instruction to be executed.
For example, if the electronic device a determines that there is no electronic device that requests to acquire the metadata file a by reading the record shown in table 3 above, the electronic device a may not perform any operation and wait for a new instruction to be executed.
Optionally, if the electronic device a directly deletes the whole entry in the read record of the electronic device that does not request to read the metadata file, the electronic device may determine that the electronic device that requests to acquire the metadata file a does not exist in the distributed file system when it is determined that there is no data in the read record. And may then perform no operation waiting for the new instruction to be executed.
Illustratively, in conjunction with the user interface shown in fig. 16, as shown in fig. 18, a set of schematic user interfaces in the embodiments of the present application is shown.
As shown in fig. 18 (a), in the user interface 1610 of the electronic apparatus a, the user modifies and saves the file a in the directory a of the electronic apparatus a, and the file a is updated to the file a 1. The metadata file a updates the metadata information of the file a recorded therein, which is updated to the metadata file a 1.
After the electronic device a updates the metadata file a to the metadata file a1, the electronic device a determines that the electronic device C has requested to acquire the metadata file a of the electronic device a before by reading the record. As shown in (b) of fig. 18, the electronic device C has acquired the metadata file a, which is cached under the directory a of the electronic device C. And displays the file a recorded in the metadata file a. Then, electronic device A pushes metadata file A1 to electronic device C, indicating that metadata file A has been updated for metadata file A1;
the file displayed according to the metadata file may be a real file or a link file, and is not limited herein.
In the drawings in the embodiments of the present application, a long dashed line indicates that a file displayed according to a metadata file may be a real file or a link file, and is not limited herein. For example, file a shown in (b) in fig. 18. In other embodiments, the representation in this manner may also be adopted, and will not be described in detail later.
As shown in fig. 18 (C), after receiving the metadata file a1, the electronic device C updates the metadata file a cached under the directory a to a metadata file a1, and correspondingly updates the file a recorded in the displayed metadata file a1 to an updated file a 1.
(3) Caching metadata files of other electronic devices:
fig. 19 is a schematic diagram of another signaling interaction of the metadata management method in the embodiment of the present application.
The following describes a metadata management method in this embodiment by taking as an example a process in which an electronic device a in a distributed file system accesses a directory a of a file system mounted by the distributed file system in the electronic device a.
It is understood that the electronic devices in the distributed file system have performed the above step S1501 before performing the steps shown in fig. 19. Therefore, the electronic device a maintains a metadata file a under the directory a, and the metadata file a stores metadata information of all other files under the directory a of the electronic device a. The electronic device B maintains a metadata file B under the directory a of the electronic device B, and the metadata file B stores metadata information of all other files under the directory a of the electronic device B. The electronic device C maintains a metadata file C under the directory a of the electronic device C, and the metadata file C stores metadata information of all other files under the directory a of the electronic device C.
S1901, the electronic device A receives a request for accessing the directory A in the electronic device A;
electronic device a may receive a request to access directory a in a file system mounted by a distributed file system in electronic device a.
Optionally, the request for accessing the directory a may be a request for opening the directory a, or may be a request for accessing a file in the directory a, which is not limited herein.
It is understood that the above-mentioned accessing of the file a under the directory a in the step S1701 is also a request for accessing the directory a, and therefore, after the step S1701 is executed, the electronic device a may also execute the steps S1902 to S1904 while executing the steps S1702 to S1706, which is not limited herein.
S1902, the electronic device A determines whether a metadata file valid under the directory A of all other online electronic devices in the distributed file system is cached locally;
after the electronic device a receives a request for accessing the directory a in the electronic device a, the electronic device a may determine whether the directory a of the file system mounted by the distributed file system in the electronic device caches metadata files valid under the directories a of all other online electronic devices.
Specifically, the determination process can be divided into 3 steps:
1. The electronic equipment A determines the current online electronic equipment in the distributed file system;
optionally, the electronic device a may determine, by the presence determination module, a current presence electronic device in the distributed file system.
For example, electronic device a may determine that electronic device B and electronic device C in the distributed file system are both online.
2. The electronic equipment A determines whether metadata files in the directory A of the online electronic equipment are cached in the directory A;
because the file name of the metadata file maintained by each electronic device carries the identifier of the electronic device, the electronic device a can determine whether the metadata file in the directory a of the online electronic device is cached in the directory a.
For example, if the metadata file maintained in the directory a of the electronic device B is the metadata file B, the metadata file maintained in the directory a of the electronic device C is the metadata file C. Electronic device a may determine whether the metadata file B and the metadata file C are cached under directory a of electronic device a. If the electronic device a determines that the metadata file B is not cached in the directory a of the electronic device a, the electronic device a may determine that the metadata file valid in the directory a of the electronic device B is not cached. If the electronic device a determines that the metadata file C is not cached in the directory a of the electronic device a, the electronic device a may determine that the metadata file valid in the directory a of the electronic device C is not cached.
3. The electronic equipment A determines whether the cached metadata file is valid;
the cached metadata file may have a validity period. When the cache duration of the cached metadata file does not exceed the preset valid duration, the electronic device a may determine that the metadata file is valid. When the cache duration of the cached metadata file exceeds the preset valid duration, the electronic device a may determine that the metadata file is invalid.
Illustratively, if the predetermined effective time period is 5 minutes. If the electronic device a determines that the metadata file B and the metadata file C are cached in the directory a, the caching duration of the metadata file B does not exceed 5 minutes, but the caching duration of the metadata file C exceeds 5 minutes. Electronic device a may determine that the metadata files valid under directory a of electronic device B are cached, but not cached.
When the electronic device a determines that the metadata file valid under the directory a of all other online electronic devices in the distributed file system is not cached under the directory a of the electronic device a, the electronic device a may perform step S1903;
when the electronic device a determines that the metadata files valid under the directory a of all other online electronic devices in the distributed file system have been cached under the directory a of the electronic device a, the electronic device a may perform step S1904.
S1903, the electronic device A obtains metadata files which are not effectively cached in the directory A of all other online electronic devices in the distributed file system;
when the electronic device a determines that the metadata files that are valid under the directories a of all other online electronic devices in the distributed file system are not cached under the directories a of the electronic device a, the electronic device a may obtain the metadata files that are not valid cached under the directories a of all other online electronic devices in the distributed file system.
For example, if the electronic device a determines that the metadata file valid under the directory a of the electronic device a is not cached under the directory a of the electronic device B, the electronic device a may obtain the metadata file B that is not cached effectively under the directory a of the electronic device B.
Specifically, the electronic device a may perform the following steps to cache the metadata file B:
s19031, electronic device a sends to electronic device B a request to obtain metadata file B of directory a of electronic device B;
when electronic device a determines that the metadata file valid under directory a of electronic device B is not cached under directory a of electronic device a, electronic device a may send a request to electronic device B to obtain metadata file B under directory a of electronic device B.
S19032, the electronic device B records the correspondence between the identifier of the electronic device a and the read metadata file B in the read record of the electronic device B;
after receiving the request sent by the electronic device a to acquire the metadata file B in the directory a of the electronic device B, the electronic device B may record the correspondence between the identifier of the electronic device a and the read metadata file B in the read record of the electronic device B.
As shown in table 6 below, this is an exemplary example of the read record of the electronic device B in the embodiment of the present application:
Figure BDA0003549537550000451
Figure BDA0003549537550000461
TABLE 6
S19033, electronic device B pushes metadata file B to electronic device A;
after the electronic device B updates the read record, the metadata file B under the directory a of the electronic device B requested by the electronic device a may be pushed to the electronic device a.
S19034, the electronic device a caches the metadata file B in the directory a of the electronic device a.
After receiving the metadata file B pushed by the electronic device B under the directory a of the electronic device B, the electronic device a may cache the metadata file B under the directory a of the electronic device a.
For example, if the electronic device a determines that the metadata file valid under the directory a of the electronic device a is not cached under the directory a of the electronic device C, the electronic device a may obtain the metadata file C that is not cached effectively under the directory a of the electronic device C.
Specifically, the electronic device a may perform the following steps to cache the metadata file C:
s19035, the electronic device a sends to the electronic device C a request to acquire the metadata file C of the directory a of the electronic device C;
s19036, the electronic device C records the correspondence between the identifier of the electronic device a and the read metadata file C in the read record of the electronic device C;
s19037, electronic device C pushes metadata file C to electronic device A;
s19038, the electronic device a caches the metadata file C in the directory a of the electronic device a.
Steps S19035 to S19038 are similar to steps S19031 to S19034, and reference may be made to the description of steps S19031 to S19034, which is not repeated here.
S1904, the electronic device a displays files recorded in the metadata file under the directory a of all online electronic devices in the distributed file system.
After determining that the metadata files valid under the directories a of all other online electronic devices in the distributed file system have been cached under the directories a of the electronic device a, or after obtaining the metadata files that are not cached effectively under the directories a of all other online electronic devices in the distributed file system, the electronic device a may display the files recorded in the metadata files under the directories a of all online electronic devices in the distributed file system.
Illustratively, a metadata file B under directory a of electronic device B and a metadata file C under directory a of electronic device C have been cached under directory a of electronic device a. Since the metadata information of all other files in the directory a of the electronic device B is recorded in the metadata file B, the metadata information of all other files in the directory a of the electronic device C is recorded in the metadata file C. Therefore, in addition to displaying the files existing in the directory a of the electronic device, the directory a of the electronic device a may display the files in the directory a of the electronic device B and the files in the directory a of the electronic device C according to the metadata file B and the metadata file C.
For the process of displaying the corresponding file by the electronic device a according to the metadata file B and the metadata file C, reference may be made to the above term to introduce (2) 2.3 in the metadata file and the description in the synchronization effect of the metadata file, which is not described herein again.
Illustratively, in conjunction with the user interface shown in fig. 16, as shown in fig. 20, a schematic diagram of a user interface in the embodiment of the present application is shown. After receiving a request for accessing the directory a or the files in the directory a, the electronic device a may cache the metadata file B in the directory a of the electronic device B in the distributed file system, and the metadata file C in the directory a of the electronic device a.
In some embodiments, the file B recorded in the metadata file B, and the file C1 and the file C2 recorded in the metadata file C may be displayed in the user interface 1610 displaying the directory a of the electronic device a. It will be appreciated that the files displayed in accordance with the metadata file may be link files that do not contain actual data.
In some embodiments, after the metadata file B or the metadata file C is cached under the directory a of the electronic device a, the files recorded in the metadata file B or the metadata file C may not be displayed. The corresponding file may be displayed after receiving an access request for a file recorded in the metadata file B or the metadata file C.
(4) Accessing files in other electronic devices:
the following describes a metadata management method in the embodiment of the present application, taking a process in which an electronic device a in a distributed file system accesses a file B in a directory a of the electronic device B as an example.
It is understood that the electronic devices in the distributed file system have performed the above step S1501 before performing the steps shown in fig. 21. Therefore, the electronic device a maintains a metadata file a under the directory a, and the metadata file a stores metadata information of all other files under the directory a of the electronic device a. The electronic device B maintains a metadata file B under the directory a of the electronic device B, and the metadata file B stores metadata information of all other files under the directory a of the electronic device B.
Alternatively, before executing the steps shown in fig. 21, the electronic device a may already execute steps S1901 to S1904, or may not execute steps S1901 to S1904, which is not limited herein.
Fig. 21 is a schematic signaling interaction diagram of another metadata management method in the embodiment of the present application.
S2101, the electronic device A receives a request for accessing a file B under a directory A of the electronic device B;
for example, the electronic device a may receive an access request from an application program in the electronic device a, where the access request may specify that an access path is a path of a file B under a directory a of the electronic device B, so as to access the file B under the directory a of the electronic device B.
For example, a metadata file B under the directory a of the electronic device a may be cached, and all files under the directory a of the electronic device B are displayed. The electronic device a may receive a request to access a file B under the directory a of the electronic device B in response to a user operation to click on the file B under the directory a of the electronic device B displayed under the directory a of the electronic device a.
It is understood that the electronic device a may also receive the request for accessing the file B under the directory a of the electronic device B by other means, which is not limited herein.
S2102, the electronic equipment A determines whether a metadata file B under the directory A of the electronic equipment A is cached effectively;
after receiving a request for accessing a file B in a directory A of an electronic device B, the electronic device A can determine whether a valid metadata file B in the directory A of the electronic device B is cached in the directory A of the electronic device A;
specifically, this determination process can be divided into 3 steps:
1. the electronic device A determines whether the electronic device B is online in the distributed file system;
optionally, the electronic device a may determine, by the presence determination module, a current presence electronic device in the distributed file system.
When electronic device a determines that electronic device B is online, subsequent steps of the determination process may continue to be performed.
When the electronic device A determines that the electronic device B is not online, prompt information can be fed back, and the prompt information can be used for indicating that the electronic device B is not online and the file B cannot be accessed.
2. The method comprises the steps that the electronic equipment A determines whether a metadata file under the directory A of the electronic equipment B is cached under the directory A of the electronic equipment A or not;
when electronic device a determines that electronic device B is online, electronic device a may determine whether metadata file B under directory a of electronic device B is cached under directory a of electronic device a.
When the electronic device a determines that the metadata file B under the directory a of the electronic device B is not cached in the directory a of the electronic device a, the electronic device a may perform step S2103;
when the electronic device a determines that the metadata file B under the directory a of the electronic device B is cached in the directory a of the electronic device a, the electronic device a may continue to perform the subsequent steps of the determination process.
3. The electronic equipment A determines whether the cached metadata file B is valid;
the cached metadata file may have a validity period. When the cache duration of the cached metadata file does not exceed the preset valid duration, the electronic device a may determine that the metadata file is valid. When the cache duration of the cached metadata file exceeds the preset valid duration, the electronic device a may determine that the metadata file is invalid.
Illustratively, if the preset effective time is 5 minutes. If the electronic device a determines that the metadata file B is cached in the directory a, and the caching duration of the metadata file B does not exceed 5 minutes, the electronic device a may determine that the cached metadata file B is valid. If the electronic device determines that the metadata file B is cached in the directory a and the caching duration of the metadata file B exceeds 5 minutes, the electronic device a may determine that the cached metadata file B is invalid.
When the electronic device a determines that the cached metadata file B is valid, the electronic device a may determine that the metadata file B under the directory a of the electronic device a that is valid is cached under the directory a of the electronic device B, and may perform step S2107.
When the electronic device a determines that the cached metadata file B is invalid, the electronic device a may determine that the metadata file B under the directory a of the electronic device a is not cached as valid, and may perform step S2103.
S2103, the electronic equipment A sends a request for acquiring the metadata file B under the directory A of the electronic equipment B to the electronic equipment B;
s2104, the electronic device B records the corresponding relation between the identifier of the electronic device A and the read metadata file B in the read record of the electronic device B;
s2105, the electronic equipment B pushes the metadata file B to the electronic equipment A;
s2106, caching the metadata file B by the electronic equipment A under the directory A of the electronic equipment A;
steps S2103 to S2106 are similar to steps S19031 to S19034, and reference may be made to the description in steps S19031 to S19034, which is not described herein again.
S2107, the electronic device A requests to acquire a file B in a directory A in the electronic device B according to the index path of the file B in the metadata file B;
After determining that the effective metadata file B is cached in the electronic device a, the electronic device a may determine an index path of the file B according to metadata information recorded in the metadata file B. And requesting the electronic equipment B to acquire the file B in the directory A in the electronic equipment B according to the index path.
S2108, the electronic equipment B sends the file B to the electronic equipment A;
the electronic device B may send the file B to the electronic device a after receiving the request for obtaining the file B sent by the electronic device a.
S2109, caching the file B by the electronic equipment A under the directory A of the electronic equipment A.
It is understood that after the electronic device a receives the file B sent by the electronic device, the file B may be cached in the directory a of the electronic device a.
Optionally, the electronic device may not perform the steps S2107 to S2109, and when the file recorded in the metadata file B is displayed after the metadata file B is cached in the directory a, the electronic device may obtain the real data of all files recorded in the metadata file B from the electronic device B. All files recorded in the metadata B are cached under the directory a. In this case, the file B in the directory a of the electronic device a may be cached in the directory a of the electronic device B.
Illustratively, as shown in fig. 22, a set of schematic user interfaces in the embodiment of the present application is shown.
As shown in (a) in fig. 22, the metadata file B under the directory a of the electronic device B and the metadata file C under the directory a of the electronic device C are cached under the directory a of the electronic device a, and the file B recorded in the metadata file B and the file C1 and the file C2 recorded in the metadata file C are displayed. However, these files in the other devices shown are link files without actual data, generated based on the metadata information in the metadata file.
After the electronic device a receives the request for accessing the file B in the directory a of the electronic device B, the electronic device a may cache the actual data of the file B in the directory a of the electronic device a according to the steps of S2101 to S2109 described above. As shown in (B) of fig. 22, the electronic device a caches the file B under the directory a, and the file B under the directory a of the electronic device a has actual data therein, and can be directly accessed or modified, and is no longer a link file without actual data.
(5) Modifying the cached files in the other electronic devices:
the following describes a metadata management method in this embodiment by taking, as an example, a process in which an electronic device a in a distributed file system modifies a file B in a directory a of an electronic device B cached in the directory a of the electronic device a.
It is understood that the electronic devices in the distributed file system have performed the above step S1501 before performing the steps shown in fig. 23. Therefore, the electronic device a maintains a metadata file a under the directory a, and the metadata file a stores metadata information of all other files under the directory a of the electronic device a. The electronic device B maintains a metadata file B under the directory a of the electronic device B, and the metadata file B stores metadata information of all other files under the directory a of the electronic device B.
Alternatively, before executing the steps shown in fig. 23, the electronic device a may have already executed steps S2101 to S2104, or may not have executed steps S2101 to S2104, which is not limited herein.
It is understood that electronic device a has cached file B under directory a of electronic device a before performing the steps shown in fig. 23.
Fig. 23 is a schematic signaling interaction diagram of a metadata management method in this embodiment.
S2301, the electronic device A modifies and stores a file B cached under the directory A in the electronic device B under the directory A in the electronic device A;
S2302, the electronic device A sends the updated file B to the electronic device B;
s2303, updating the file B under the directory A of the electronic equipment B by the electronic equipment B;
s2304, updating metadata information of the file B in the metadata file B in the directory A of the electronic device B by the electronic device B;
for the description of the file modification causing the metadata file update, reference may be made to the description shown in fig. 4 in the above description (2) of the usage and update of the 2.2 metadata file in the metadata file, and details are not repeated here.
S2305, the electronic device B determines whether an online electronic device which requests the metadata file B exists in the distributed file system;
s2306, the electronic device B pushes the metadata file B to the electronic device which requests to acquire the metadata file B and is online in the distributed file system;
s2307, the electronic device B deletes the record of the electronic device which requests to acquire the metadata file B but is not on line in the read record to read the metadata file B;
s2308, the electronic device B waits for execution of a new instruction;
steps S2305 to S2307 are similar to steps S1704 to S1707, and reference may be made to the description of steps S1704 to S1707, which will not be described herein again.
S2309, the electronic device A updates the metadata file B cached under the directory A in the electronic device A.
When the electronic device B determines that the electronic device a requests the metadata file B of the electronic device B in the distributed file system, the electronic device B transmits the updated metadata file B to the electronic device a. After receiving the updated metadata file B, the electronic device a updates the metadata file B cached under the directory a in the electronic device a.
Illustratively, in conjunction with the user interface shown in fig. 22, as shown in fig. 24, another group of schematic user interfaces according to the embodiments of the present application is shown.
As shown in fig. 24 (a), the user may modify and save a file B cached under the directory a of the electronic device a, and the file B may be updated to a file B1. Electronic device A may send the updated file B1 to electronic device B indicating that file B has been updated to file B1.
As shown in (B) of fig. 24, after the electronic device B receives the file B1, it is possible to update the file B under the directory a of the electronic device B to the file B1 and update the metadata information of the file B recorded in the metadata file B under the directory a. The metadata file B under the directory a of the electronic device B is updated to the metadata file B1. The electronic device B determines that the electronic device a has previously requested to acquire the metadata file B of the electronic device B by reading the record. The updated metadata file B1 is transmitted to the electronic device A, indicating that the metadata file B is updated to the metadata file B1.
As shown in (c) of fig. 24, after the electronic device a receives the metadata file B1, the metadata file B is updated to a metadata file B1 under the directory a of the electronic device a.
(6) Clearing up metadata files after the electronic equipment is offline and before the electronic equipment is online:
it is understood that, in order to ensure that the metadata files of other electronic devices cached by the electronic device in the distributed file system are valid, the electronic device may clean the cached metadata files of other electronic devices after going offline or before going online. And acquiring the latest metadata file again when needed.
The following describes a metadata management method in the embodiment of the present application, taking a process of cleaning up a metadata file after an electronic device a in a distributed file system goes offline or before the electronic device a goes online as an example.
Fig. 25 is a schematic signaling interaction diagram of a metadata management method in the embodiment of the present application.
Stage one: clearing the offline metadata file:
s2501, the electronic device A disconnects the network connection with the distributed file system;
it is understood that there may be many situations when the electronic device a disconnects from the network of the distributed file system, for example, the electronic device a quits the network account for establishing the distributed file system, or the electronic device a disconnects from the local area network for establishing the distributed file system, or the electronic device a disconnects from all the networks, and the like, and the present disclosure is not limited thereto.
S2502, the electronic device A clears metadata files of other electronic devices cached in each directory in the file system;
after the electronic device A is disconnected from the network of the distributed file system, metadata files of other electronic devices cached under each directory in the file system mounted by the distributed file system can be cleared.
Optionally, the electronic device a may also clean files of other electronic devices cached in each directory, which is not limited herein.
And a second stage: clearing up metadata files before online:
s2503, the electronic device A receives a request for accessing the distributed file system;
it is understood that the electronic device a may receive the request for accessing the distributed file system in various ways, for example, a network account for establishing the distributed file system for login, or a local area network connection for establishing the distributed file system for access, or receive invitation to join information sent by the electronic device in the distributed file system, which is not limited herein.
S2504, the electronic device A cleans up metadata files of other electronic devices cached under various directories in the file system;
after receiving the request for accessing the distributed file system, the electronic device may first clear the metadata files of other electronic devices cached in each directory in the file system that will be mounted by the distributed file system.
Optionally, the electronic device a may also clean files of other electronic devices cached in each directory, which is not limited herein.
S2505, the electronic device A accesses the distributed file system.
After the metadata files of other electronic devices cached in each directory are cleared, the electronic device a may access the distributed file system. The required metadata files may then be cached in the distributed file system to corresponding directories according to access requirements.
It is understood that the settings may be different according to actual needs, factory settings or user settings. In some embodiments, steps S2501 to S2502 of stage one may or may not be executed, and are not limited herein. In some embodiments, steps S2503 to S2505 of stage two may or may not be executed, and are not limited herein.
Illustratively, fig. 26 is a schematic view of another set of user interfaces in the embodiments of the present application.
As shown in (a) of fig. 26, a metadata file B under the directory a of the electronic device B and a metadata file C under the directory a of the electronic device C in the distributed file system may be cached under the directory a of the electronic device a. The file B in the electronic device B under the directory A, and the file C1 and the file C2 displayed according to the metadata file C can be cached under the directory A of the electronic device A.
As shown in fig. 26 (B), after the electronic device a gets offline, and/or before the electronic device a gets online, the electronic device a may clear the cached metadata file B in the electronic device B and the cached metadata file C in the electronic device C according to the above steps S2501 to S2505. Meanwhile, the files B in the electronic device B, and the files C1 and C2 displayed according to the metadata file C can be cleaned up. And only keeping the files in the electronic equipment of the electronic equipment A under the directory A of the electronic equipment A.
As another embodiment of the present application, a process of the electronic device a accessing a file with a path of "/a/B/file.txt" in the electronic device B is taken as an example to describe a process of the electronic device a accessing a file in a multi-layer directory in a distributed file system in the embodiment of the present application:
suppose that the first local file system in the electronic device a has just been mounted by the distributed file system, and the metadata files in the local file systems of the other electronic devices mounted by the distributed file system have not been cached in the first local file system. The access procedure may have the following two alternatives:
scheme 1:
and the electronic device A determines that the "/A/B/file.txt" exists in the local file system, and directly returns the local file file.txt.
Otherwise, electronic device a may request metadata files under path "/" from other electronic devices in the distributed file system, e.g., request metadata files under path "/" from electronic device B.
Analyzing the metadata files, and analyzing which metadata files of the electronic devices contain the metadata information of the directory a, it can be determined that the local file systems of the electronic devices contain the directory a. And then requesting metadata files under the path "/A/" from the electronic equipment containing the directory A (if none of the metadata files exist, a path error is returned; if any metadata file exists under the path "/A/" but the directory A does not exist in the local file system of the electronic equipment A, the directory A is created in the local file system of the electronic equipment A).
Analyzing the metadata files, analyzing which metadata files of the electronic equipment comprise the metadata information of the directory B, and requesting the metadata files under the path "/A/B/" from the electronic equipment comprising the directory B.
And analyzing the metadata files, and analyzing the metadata information of the file "file.
If there is metadata information of a file "file.txt" in a plurality of metadata files, the name file on one of the electronic devices (for example, the name file on the electronic device with the smallest device number) may be returned, or the name files on all the electronic devices may be returned, which is not limited herein.
During the process of accessing the file "/a/B/file.txt", the electronic device a may cache the accessed metadata files of other electronic devices to the local file system of the electronic device a, and when accessing the file under the path "/", "/a/B/", the next time, the corresponding metadata file cached locally may be accessed to accelerate the access process.
Scheme 2:
and the electronic device A determines that the "/A/B/file.txt" exists in the local file system, and directly returns the local file file.txt.
Otherwise, the electronic device A sends a file path "/A/B/file.txt" to other electronic devices, and the electronic device containing "file.txt" in the path "/A/B/" returns a metadata file under the path "/A/B/". The electronic device a accesses the file "file.
If there is metadata information of a file "file.txt" in a plurality of metadata files, the name file on one of the electronic devices (for example, the name file on the electronic device with the smallest device number) may be returned, or the name files on all the electronic devices may be returned, which is not limited herein.
In combination with the above embodiments, in some embodiments of the present application, the electronic device a may be referred to as a second electronic device, and the electronic device B may be referred to as a first electronic device.
In some embodiments of the present application, the file system mounted by the distributed file system in the electronic device B may be referred to as a first local file system, and the file system mounted by the distributed file system in the electronic device a may be referred to as a second local file system.
In some embodiments of the present application, a file B in a directory a in an electronic device B may be referred to as a first local file; the metadata file B in the directory a in the electronic device B may be referred to as a second local file.
Through the metadata management method, the file in another electronic device can be displayed in one electronic device in the distributed file system without the central node. In the distributed file system, it is assumed that a file X exists under a certain path (e.g., path P) of an electronic device a, and a file X of the same name exists under the same path P of another electronic device B. When all files under the path P in the distributed file system are displayed under the path P in the electronic device a, the electronic device a needs to identify the files under the path P of all electronic devices in the distributed file system with the same name and modify the file names of the files with the same name, so that the file names of all files under the path P of the distributed file system are unique.
For an existing distributed file system with a central node, the central node maintains metadata information for all files in the distributed file system. When electronic equipment is accessed into the distributed file system with the central node, the accessed electronic equipment synchronizes metadata information of files under each path to the central node, and the central node checks whether the files have the same name with the files under each path in the distributed file system, so that the files with the same name do not appear under the same path.
For a distributed file system without a central node, any device in the distributed file system is likely to be offline at any time, and the central node is not used for managing global information of the distributed file system, so that a better homonymous file identification method is not available at present.
The method for identifying the files with the same name provided by the embodiment of the application can rapidly determine whether the files with the same name exist in one path of each electronic device in a distributed file system without a central node.
Secondly, a method for identifying the same-name file:
the following first introduces related terms and concepts related to the method for identifying the file with the same name in the embodiment of the present application.
(1) Distributed file system with central node:
a distributed file system with a central node usually consists of a metadata server as a central node, which stores metadata information, and a plurality of data servers, which manage the actual data of files stored in the distributed file system. When a user accesses file data, the metadata server serving as a central node needs to be accessed to acquire basic information and data index information of a file, and then the data server is accessed to acquire the file data according to the basic information and the data index information.
FIG. 10 is a diagram of an exemplary scenario for a distributed file system with a central node. The device a, the device B, and the device C constituting the distributed file system having the center node all need to access the files in the data server through the metadata server as the center node. The device a, the device B, and the device C may each serve as a data server, and the metadata server serving as the central node may be one of the electronic devices or may be an independent server. When the metadata server as the central node goes offline, the services of the distributed file system with the central node are unavailable, and each electronic device can no longer read the files in other electronic devices.
(2) Distributed file system without central node:
the method for identifying the file with the same name in the embodiment of the application can be suitable for a distributed file system without a central node. In a distributed file system without a central node, each electronic device itself manages metadata information of files in other electronic devices. The files in other electronic equipment can be displayed or accessed directly according to the metadata information of the files in other electronic equipment managed by the user without passing through a central node.
Fig. 12 is a schematic diagram of an exemplary scenario of a distributed file system without a central node in an embodiment of the present application. The device a, the device B, the device C, the device D, and the device E constituting the distributed file system without the center node manage metadata information of files in other electronic devices, respectively, and can directly display or access files stored in other electronic devices. For example, file a is stored in electronic device a, file B is stored in electronic device B, and file C is stored in electronic device C. The electronic device a may store metadata information of the file B and the file C, so that the file B or the file C may be directly displayed or accessed. Similarly, the electronic device D may also store metadata information of the file a, the file B, and the file C, so that the file a, the file B, or the file C may be directly displayed or accessed.
(3) File display in a distributed file system:
the distributed file system provides a logical file system structure for resources distributed at any position on each electronic device, so that access to shared files distributed on each electronic device is easier.
When displaying files in a directory (or called a path) in a file system mounted by the distributed file system on a certain electronic device in the distributed file system, the file stored in the directory in the electronic device may be displayed, and files in the directory in other electronic devices in the distributed file system may also be displayed.
Illustratively, fig. 27 is a set of schematic user interfaces in an embodiment of the present application.
Fig. 27 (a) shows the user interface 310 in the electronic apparatus a. The user interface 310 is a user interface for displaying the files in the root directory of the file system mounted by the distributed file system in the electronic device a. As can be seen, the root directory of the electronic device a stores the file 1 and the directory a.
Fig. 27 (B) shows the user interface 320 in the electronic device B. The user interface 320 is a user interface for displaying the files in the root directory of the file system mounted by the distributed file system in the electronic device B. As can be seen, the root directory of the electronic device B stores the file 2, the file 3 and the directory a.
Fig. 27 (c) shows the user interface 310 in the electronic device a after the electronic device a and the electronic device B constitute the distributed file system. At this time, the user interface 310 may display the file 1 and the directory a stored under the root directory in the electronic device a. Files 2 and 3 stored in the electronic device B under the root directory may be displayed. Since directory a represents the same path, the display is not repeated.
The files 2 and 3 shown by long dashed lines in the figure are files in other electronic devices, and may be link files or real files. The link file does not contain actual data of the file, and is a file generated by the electronic device a according to the metadata information of the files 2 and 3. In other drawings of the present application, the representation can also be performed in this way, and the description is omitted.
(4) Bloom Filter (Bloom Filter):
a bloom filter is an algorithm that uses a multi-hash function to determine whether an element is in a set. It is a probabilistic data structure that determines that something must not exist or may exist, and that returns a probabilistic result. The bloom filter comprises a bit array of N bits and K hash functions.
FIG. 28 is a state diagram of a bloom filter bit array. The working process of the bloom filter is described in an exemplary manner in connection with fig. 28:
illustratively, (a) in fig. 28 is a schematic structural diagram of a bloom filter bit array in the embodiment of the present application. In fig. 28, the bloom filter includes a bit array of 8 bits and 3 hash functions as an example. The value of each bit is initialized to 0 at this time.
If it is desired to map a value into a bloom filter bit array, then it is necessary to generate multiple hash values using multiple different hash functions and for each generated hash value point to one bit in the bloom filter bit array.
For example, for the value "filename", Hash values 1, 4, 7 may be generated using three different Hash functions (e.g., Hash1, Hash2, and Hash3), respectively. The state of the bloom filter bit array shown in (a) in fig. 28 may be changed to the state of the bloom filter bit array shown in (b) in fig. 28: the values of the 1 st, 4 th and 7 th bits are set to 1.
For another example, for the value "shouji", if three hash functions return hash values of 3, 4, and 8. The state of the bloom filter bit array shown in (b) in fig. 28 may be changed to the state of the bloom filter bit array shown in (c) in fig. 28: the values of the 3 rd, 4 th and 8 th bits are set to 1. It will be appreciated that the 4 th bit is overwritten because the hash function of both values returns this position, and the value of the 4 th bit is still 1.
At this time, the bloom filter bit array shown in (c) in fig. 28 records information of two values "filename" and "shouji".
If the value of "diannao" is inquired whether the hash value exists at this time, the hash values returned by the three hash functions are three values of 1, 5 and 8. We can find that the value at bit 5 is 0. Indicating that none of the recorded values maps to this bit and therefore it is very certain that the value "diannao" does not exist.
If the user wants to query whether the value of "filename" exists, the hash values returned by the three hash functions are 1, 4 and 7. We can find that the values on these three bits are all 1. Thus, it may be determined that the value "filename" may exist.
Since as the records increase in value more and more bits are set to 1 more and more. Thus, even if a certain value is not stored, the three bits returned by the hash function may all be set to 1 by other values. The results returned by the bloom filter are probabilistic.
It can be seen that a bloom filter with too few bits in the bit array quickly has all bits 1, and then any query value returns a possible value, which does not serve the purpose of filtering. The longer the bloom filter bit array, the lower its false alarm rate.
The larger the number of hash functions, the faster the bit position of the bloom filter bit array is set to 1, and the less efficient the bloom filter is. However, if the number of hash functions is too small, the false alarm rate increases.
Therefore, the number of bits in the appropriate bit array and the number of hash functions need to be set according to actual requirements.
In the embodiment of the present application, a bloom filter may be used to determine whether files with the same name exist in the same directory of each electronic device:
for example, if it is required to determine whether files with the same name exist in the directories a of the electronic device a and the electronic device B. Then the same parameter bloom filters (i.e., the same respective hash functions used and the same number of bits in the bit array) may be arranged on both electronic device a and electronic device B. Assume that the number of bits of the bit array is N and the number of hash functions is K.
The electronic equipment A can take all file names under the own directory A as key values and map the key values to a bloom filter bit array BA through K hash functions; the electronic device B may map all file names under its own directory a to the bloom filter bit array BB through K hash functions as key values. In this case, the bloom filter bit array of each electronic device may be determined as a whole. Counting the number of bits at the same position of BA and BB which are 1 at the same time:
Fig. 29 is a schematic diagram of an exemplary scenario in which bloom filter bit arrays of two electronic devices are compared in this embodiment. If a file with the same name exists in the directory a of the electronic device a and the directory a of the electronic device B, the file with the same name inevitably obtains the same K hash values based on the same K hash functions, and the hash values are mapped to the BA and the BB respectively. It will make it necessary that at least K identical bits in BA and BB are 1 at the same time. Thus:
if the number of bits of the same positions of BA and BB which are simultaneously 1 is smaller than the number K of the hash functions, it indicates that the files with the same name cannot exist under the directories A of the electronic equipment A and the electronic equipment B;
if the number of bits of the same positions of BA and BB, which are simultaneously 1, is greater than or equal to the number K of the hash function, it indicates that the files with the same name may exist in the directories a of the electronic device a and the electronic device B.
As can be seen from the file display in the distributed file system shown in fig. 27, one electronic device in the distributed file system can display all files in other electronic devices in a certain directory under the same directory. Files with the same name may exist in the same directory of the electronic equipment, and the files with the same name are generally not allowed to exist in the same directory when displayed in a file system. Therefore, in order to display different file names to the user, the electronic devices need to perform the same-name file check and perform the renaming process on the same-name files. It will be appreciated that renaming refers to modifying the file name displayed to the user, and may not modify the actual name of the file store.
The following description will exemplarily take an example that files with the same name exist in the directories a of the electronic device a and the electronic device B, and after the electronic device a and the electronic device B form a distributed file system, the files in the directory a of the electronic device B need to be displayed in the directory a of the electronic device a.
Illustratively, fig. 30 is another set of user interface diagrams in the embodiments of the present application.
Fig. 30 (a) shows a user interface 610 in the electronic apparatus a. The user interface 610 is a user interface for displaying files in directory a under a root directory in a file system mounted by a distributed file system in the electronic device a. As can be seen, the directory a of the electronic device a stores files with file names file 1, file 2, and file 3.
Fig. 30 (B) shows the user interface 620 in the electronic device B. The user interface 620 is a user interface for displaying files in directory a under the root directory in the file system mounted by the distributed file system in the electronic device B. As can be seen, the directory a of the electronic device B stores files with file names file 1, file 4, and file 5.
Fig. 30 (c) shows the user interface 610 in the electronic device a after the electronic device a and the electronic device B constitute the distributed file system. In this case, the user interface 610 needs to display not only the existing files with file names of file 1, file 2, and file 3, but also files with file names of file 1, file 4, and file 5 in the directory a in the electronic device B. The electronic device A can recognize that a file with the same name exists in the directory A of the electronic device A and the directory A of the electronic device B through a file with the same name recognition method: document 1. Thus, electronic device A may rename the file 1 under the directory A in electronic device B to files 1-B.
In the prior art, as shown in fig. 10, in a distributed file system having a central node, a metadata server as the central node identifies files with the same name in each directory, thereby ensuring that the files with the same name do not exist in the same directory. However, in the distributed file system without the central node, since there is no central node, the same-name file identification cannot be performed in this manner.
In another identification mode of files with the same name, if all files in a certain directory in a distributed file system without a central node need to be displayed, the file names of all files in the directory in all electronic equipment can be compared pairwise to judge whether the file names are the same. Assume that the distributed file system consists of 3 electronic devices: electronic device a, electronic device B, and electronic device C. The 3 electronic devices each have 5 files under directory a of the distributed file system. If the electronic device a needs to display all files in the directory a, it needs to determine whether the electronic device B and the electronic device C have files with the same name in the directory a. The electronic device a needs to compare the 5 files in the directory a of the electronic device a with the file names of the 5 files in the directory a of the electronic device B one by one, and obtain the result after 25 times of comparison. Then, the 5 files under the directory a of the electronic device a are compared with the file names of the 5 files under the directory a of the electronic device C one by one, and the result is obtained by comparing the two files for 25 times. Finally, comparing the 5 files under the directory A of the electronic device B with the file names of the 5 files under the directory A of the electronic device C one by one, and comparing for 25 times to obtain a result. The results were obtained in a total of 75 comparisons.
In practical application, it is displayed that all files in a certain path of the distributed file system are operated frequently, more electronic devices exist in the distributed file system, and a great number of files exist in directories of each electronic device.
In the method for identifying the file with the same name in the embodiment of the application, a bloom filter can be adopted to integrally judge the possibility that the file with the same name exists in the same directory of a plurality of electronic devices in the distributed file system without the central node, so that a result that the file with the same name does not exist and the file with the same name possibly exists is obtained, and subsequent processing is performed according to the integral judgment result. Since the electronic devices are not so many that the files of the same name exist in the same directory, the result of identifying the files of the same name can be obtained directly by the overall judgment in most cases, and the range of identifying the files of the same name can be remarkably narrowed even if the final result is not obtained. The processing time length when the homonymous files are identified is greatly reduced, and the resources consumed when the homonymous files are identified are reduced.
In conjunction with the description of the hardware structure diagram of the electronic device 100 shown in fig. 13, in some embodiments of the present application, the electronic device 100 may form a distributed file system without a central node with other electronic devices through the mobile communication module 150 or the wireless communication module 160. The processor 110 may call the computer instructions stored in the internal memory 121 to cause the electronic device 100 to execute the file identification method of the same name in the embodiment of the present application.
In conjunction with the description of the software structure diagram of the electronic device 100 shown in fig. 14, as shown in fig. 31, in some embodiments of the present application, the electronic device 100 may further include a homonym file identification module at the application layer.
The homonymous file identification module is used for executing the homonymous file identification method in the embodiment of the application.
It is understood that the file identification module with the same name can be located at other levels in the hierarchical framework, and is not limited herein.
In the following, by taking the example that the electronic device a in the distributed file system needs to display all files under the directory a in combination with the software and hardware structures of the exemplary electronic device 100, a method for identifying files with the same name in the embodiment of the present application is specifically described:
fig. 32 is a schematic signaling interaction diagram of a method for identifying a file with the same name in the embodiment of the present application.
In the embodiment of the present application, for convenience of description, online electronic devices other than the electronic device a in the distributed file system are all referred to as an electronic device X.
It is understood that each electronic device in the distributed file system includes a bloom filter, and parameters of the bloom filter are the same. Specifically, the digits of the digit group of the bloom filter in each electronic device are all first preset values N, and the numbers of the hash functions used are all second preset values K. The K hash functions are different from each other, but each electronic device uses the same K hash functions.
S3201, the electronic device A receives a request for displaying all files in a directory A in the distributed file system;
it is understood that the request received by the electronic device a to display all files in the directory a in the distributed file system may be a request of a user to open the directory a on the electronic device a, or may also be a request of an application program in the electronic device a to access the directory a or files in the directory a, which is not limited herein.
S3202, the electronic equipment A calculates a bloom filter bit array of the electronic equipment A according to the file names of all local files in the directory A;
after the electronic device a receives a request for displaying all files in the directory a in the distributed file system, the bloom filter bit array of the electronic device a may be calculated according to the filenames of all local files in the directory a.
Specifically, the electronic device a may use file names of all files in the directory a as key values, evaluate the key values by using K hash functions in a bloom filter of the electronic device a, and then map the key values to N bit arrays of the bloom filter to obtain a bit array of the bloom filter of the electronic device a. At this time, the bloom filter bit array of the electronic device a records information about the file names of all local files in the directory a of the electronic device a.
The process of using the hash function to evaluate and map the file name as the key value onto the bit array can refer to the above description of (4) the working process of the bloom filter, and is not described herein again.
S3203, the electronic device A sends a request for acquiring file names of all local files under the directory A in the electronic device X and a bloom filter bit array of the electronic device X to the electronic device X;
after the electronic device a receives the request for displaying all files in the directory a in the distributed file system, a request for obtaining the file names of all local files in the directory a in the electronic device X and the bloom filter bit array of the electronic device X may be sent to the electronic device X.
It is understood that the electronic device X is an online electronic device other than the electronic device a in the distributed file system. The electronic device X may be one electronic device, or may be a plurality of electronic devices, for example, two, three, four or more electronic devices, which are not limited herein.
If the electronic device X is a plurality of electronic devices, the electronic device a may send a request to the electronic devices X to obtain the filenames of all local files under the directory a in the electronic device X and the bloom filter bit array of the electronic device X, or broadcast a request to obtain the filenames of all local files under the directory a in the electronic device X and the bloom filter bit array of the electronic device X in the distributed file system, which is not limited herein.
S3204, the electronic device X calculates a bloom filter bit array of the electronic device X according to the file names of all local files in the directory A in the electronic device X;
after receiving the request sent by the electronic device a to acquire the filenames of all local files in the directory a of the electronic device X and the bloom filter bit array of the electronic device X, the electronic device X may calculate the bloom filter bit array of the electronic device X according to the filenames of all local files in the directory a of the electronic device X.
Specifically, the electronic device X may use file names of all files in the directory a as key values, evaluate the key values by using K hash functions in a bloom filter of the electronic device X, and then map the key values to N bit arrays of the bloom filter, thereby obtaining a bloom filter bit array of the electronic device X. At this time, the bloom filter bit array of the electronic device X records information about the file names of all local files in the directory a of the electronic device X.
The process of using the hash function to evaluate and map the file name as the key value onto the bit array can refer to the above description of (4) the description of the working process of the bloom filter in the bloom filter, and is not described herein again.
S3205, the electronic device A receives the file names of all local files in the directory A in the electronic device X returned by the electronic device X and the calculated bloom filter bit array of the electronic device X;
After the bloom filter bit array of the electronic device X is calculated by the electronic device X, the calculated bloom filter bit array of the electronic device X and file names of all local files in the directory a may be sent to the electronic device a.
Steps S3203 to S3205 described above are one way for the electronic device a to obtain the file names of all local files in the directory a in the electronic device X and the bloom filter bit array of the electronic device X. In practical applications, the electronic device a may also obtain the file names of all local files in the directory a in the electronic device X and the bloom filter bit array of the electronic device X in other manners:
optionally, because the electronic device a and the electronic device X form a distributed file system, metadata information of all local files in the electronic device X may be stored in the electronic device a, and the electronic device a may obtain filenames of all local files in the directory a in the electronic device X according to the stored metadata information of all local files in the directory a in the electronic device X.
In some embodiments, in step S3203, electronic device X may only need to send a request to electronic device X to obtain the bloom filter bitmap of electronic device X; in step S3205, electronic device X may only need to return the calculated bloom filter bit array for electronic device X.
In some embodiments, the electronic device a may not perform steps S3203 to S3205, and after obtaining the filenames of all local files in the directory a in the electronic device X according to the stored metadata information of all local files in the directory a in the electronic device X, the electronic device a may directly calculate the bloom filter bit array of the electronic device X.
S3206, the electronic equipment A judges whether the files with the same name possibly exist in the directory A in the electronic equipment according to the bloom filter bit array of the electronic equipment;
after the electronic device a acquires the bloom filter bit arrays of other online electronic devices in the distributed file system, whether the files with the same name possibly exist in the directory a in each electronic device can be judged according to the bloom filter bit arrays of each electronic device;
judging whether the files with the same name possibly exist in the directory A of each electronic device, specifically: determining whether files with the same name possibly exist under the directory A in every two pieces of electronic equipment;
when it is determined that the files with the same name may exist under the directory a in the two electronic devices, the electronic device a may perform step S3207;
optionally, as long as the electronic device a determines that there are files with the same name in the directory a in the two electronic devices, step S3207 may be executed;
Optionally, after the electronic device a may determine that the same-name files may exist in the directory a of the two electronic devices in the preset number of groups at most, step S3207 is executed again;
optionally, the electronic device a may perform step S3207 after comparing the bloom filter bit arrays of all the electronic devices.
When it is determined that the files of the same name are unlikely to exist under the directory a in each electronic device, the electronic device a may perform step S3208.
If there are only two electronic devices, e.g., electronic device a and electronic device B, online in the distributed file system. The process of determining whether the file with the same name may exist in the directory a of the electronic device a and the directory a of the electronic device B by the electronic device a according to the bloom filter bit array of each electronic device refers to the description of fig. 5, and details are not repeated here.
If there are more than two online electronic devices in the distributed file system, the electronic device a may determine, according to the bloom filter bit array of each electronic device, whether there are files with the same name in the directory a of each electronic device in multiple ways, which is described below:
optionally, the electronic device a may perform pairwise comparison and determination on the obtained bloom filter bit array:
fig. 33 is a flowchart illustrating a process of determining whether a file with the same name may exist in the embodiment of the present application.
S3301, the electronic device A respectively judges whether the number of bits, which are at the same time 1 in the same position, in the bloom filter bit array of each electronic device is smaller than K two by two;
after the electronic equipment A acquires the bloom filter bit arrays of other online electronic equipment in the distributed file system, two-by-two judgment can be made whether the bit number of the bloom filter bit arrays of the electronic equipment, which has the same position and is 1 at the same time, is less than K;
when it is determined that the number of bits of which the same positions in the bloom filter bit arrays are simultaneously 1 is less than K, the electronic device a may perform step S3302;
when it is determined that the number of bits of which the same position in the bloom filter bit arrays is simultaneously 1 is not less than K, the electronic device a may perform step S3303.
S3302, the electronic device A determines that files with the same name cannot exist in the directories A of the two electronic devices;
when it is determined that the number of bits of which the same positions in the bloom filter bit arrays are simultaneously 1 is smaller than K, the electronic device a may determine that the files of the same name may not exist under the directories a of the two electronic devices.
S3303, the electronic device A determines that files with the same name possibly exist in the directories A of the two electronic devices.
When it is determined that the number of bits of which the same position in the bloom filter bit arrays is 1 at the same time is not less than K, the electronic device a may determine that the files of the same name may exist in the directories a of the two electronic devices.
Fig. 34 is a schematic diagram of an exemplary scenario for determining whether a file with the same name may exist in the embodiment of the present application.
The electronic device a acquires two other online electronic devices in the distributed file system: bloom filter bit arrays for electronic device B and electronic device C. For convenience of description, the bloom filter bit array of the electronic device a is denoted as BA, the bloom filter bit array of the electronic device B is denoted as BB, and the bloom filter bit array of the electronic device C is denoted as BC.
Then, the electronic device a will respectively compare the BA, BB, and BC pairwise, so as to determine whether the files with the same name may exist in the directories a of the electronic device a, the electronic device B, and the electronic device C, specifically:
the electronic equipment A compares BA and BB, and when the digits of the same positions of the two bloom filter digit groups, which are simultaneously 1, are less than K, the electronic equipment A can determine that the files with the same name cannot exist in the directories A of the electronic equipment A and the electronic equipment B; when the number of bits of which the same position in the two bloom filter bit arrays is 1 at the same time is not less than K, the electronic device a may determine that the files of the same name may exist under the directories a of the electronic device a and the electronic device B.
The electronic equipment A compares the BA and the BC, and when the digits of the same positions of the two bloom filter digit groups, which are simultaneously 1, are less than K, the electronic equipment A can determine that the files with the same name cannot exist in the directories A of the electronic equipment A and the electronic equipment C; when the number of bits of which the same position in the two bloom filter bit arrays is 1 at the same time is not less than K, the electronic device a may determine that the files of the same name may exist under the directories a of the electronic device a and the electronic device C.
The electronic device A compares BB with BC, and when the digits of the same positions of the two bloom filter digit groups, which are simultaneously 1, are smaller than K, the electronic device A can determine that the files with the same name cannot exist in the directories A of the electronic device B and the electronic device C; when the number of bits of which the same position in the two bloom filter bit arrays is 1 at the same time is not less than K, the electronic device a may determine that the files of the same name may exist under the directories a of the electronic device B and the electronic device C.
Fig. 35 is a schematic diagram of an exemplary scenario for determining whether a file with the same name may exist in the embodiment of the present application.
The electronic device a acquires three other online electronic devices in the distributed file system: bloom filter bit arrays for electronic device B, electronic device C, and electronic device D. For convenience of description, the bloom filter bit set of the electronic device a is denoted by BA, the bloom filter bit set of the electronic device B is denoted by BB, the bloom filter bit set of the electronic device C is denoted by BC, and the bloom filter bit set of the electronic device D is denoted by BD.
Then, the electronic device a will respectively compare the BA, BB, BC, and BD pairwise, so as to determine whether the same-name files may exist in the directories a of the electronic device a, the electronic device B, the electronic device C, and the electronic device D, specifically:
The electronic equipment A compares BA and BB, and when the digits of the same positions in the two bloom filter digit arrays, which are simultaneously 1, are less than K, the electronic equipment A can determine that the files with the same name cannot exist in the directories A of the electronic equipment A and the electronic equipment B; when the number of bits in the two bloom filter bit arrays, the same position of which is 1 at the same time, is not less than K, the electronic device a may determine that the file with the same name may exist under the directories a of the electronic device a and the electronic device B.
The electronic equipment A compares the BA and the BC, and when the digits of the same positions of the two bloom filter digit groups, which are simultaneously 1, are less than K, the electronic equipment A can determine that the files with the same name cannot exist in the directories A of the electronic equipment A and the electronic equipment C; when the number of bits of which the same position in the two bloom filter bit arrays is 1 at the same time is not less than K, the electronic device a may determine that the files of the same name may exist under the directories a of the electronic device a and the electronic device C.
The electronic equipment A compares the BA and the BD, and when the digits of which the same positions are 1 in the two bloom filter digit groups are smaller than K, the electronic equipment A can determine that the same-name files cannot exist in the directories A of the electronic equipment A and the electronic equipment D; when the number of bits of which the same position in the two bloom filter bit arrays is 1 at the same time is not less than K, the electronic device a may determine that the files of the same name may exist under the directories a of the electronic device a and the electronic device D.
The electronic equipment A compares BB with BC, and when the digits of the same positions in the two bloom filter digit arrays, which are simultaneously 1, are smaller than K, the electronic equipment A can determine that the files with the same name cannot exist in the directories A of the electronic equipment B and the electronic equipment C; when the number of bits of which the same position in the two bloom filter bit arrays is 1 at the same time is not less than K, the electronic device a may determine that the files of the same name may exist under the directories a of the electronic device B and the electronic device C.
The electronic device A compares BB with BD, and when the digits of the same positions in the two bloom filter digit groups, which are 1 at the same time, are smaller than K, the electronic device A can determine that the files with the same name cannot exist in the directories A of the electronic device B and the electronic device D; when the number of bits of which the same position in the two bloom filter bit arrays is 1 at the same time is not less than K, the electronic device a may determine that the files of the same name may exist under the directories a of the electronic device B and the electronic device D.
The electronic device A compares BC and BD, and when the digits of the same positions in the two bloom filter digit groups, which are 1 at the same time, are smaller than K, the electronic device A can determine that the files with the same name cannot exist in the directories A of the electronic device C and the electronic device D; when the number of bits of which the same position in the two bloom filter bit arrays is 1 at the same time is not less than K, the electronic device a may determine that the files of the same name may exist under the directories a of the electronic device C and the electronic device D.
After comparing the bloom filter bit arrays of all electronic devices two by two, the electronic device may perform step S3207, rename the file with the same name, and then perform step S3208.
Preferably, the electronic device may further reduce the number of comparisons by performing a logical or operation on the bloom filter bit array of the electronic device that determines that the file with the same name is unlikely to exist, and comparing the bit array with other bloom filter bit arrays:
electronic device a may sort the obtained m bloom filter bit arrays, and correspond to m electronic devices (including electronic device a). The electronic device a compares the bloom filter bit arrays of the first electronic device and the second electronic device, and if it is determined that the files with the same name may exist in the directories a of the two electronic devices, step S3207 is executed to rename the files with the same name or determine that the files with the same name may not exist.
After renaming the files with the same name or after determining that the files with the same name cannot exist in the directories A of the two electronic devices, the electronic device A carries out logic or operation on the bit arrays of the bloom filters of the two electronic devices and records the bit arrays as logic or bloom filter bit arrays. And comparing the logic or bloom filter bit array with a bloom filter bit array of a third electronic device to determine whether the files with the same name exist in the directory A of the third electronic device and the first electronic device and the directory A of the third electronic device and the second electronic device. If it is determined that the same-name files may exist in some two electronic devices, step S3207 is executed to rename the same-name files or determine that the same-name files may not exist.
After renaming the files with the same name or after determining that the files with the same name cannot exist in the directory A of the three electronic devices, the electronic device A carries out logic or operation on the logic or bloom filter bit array and the bloom filter bit array of the third electronic device, and the logic or bloom filter bit array is updated. And comparing the logical or bloom filter bit array with a bloom filter bit array of a fourth electronic device, and repeating the steps to finally determine that no files with the same name exist in the m electronic devices, and then executing step S3208.
The following specifically describes the manner of determining whether the file with the same name may exist, by taking the example that the electronic device a acquires the bloom filter bit arrays of the electronic device B and the electronic device C:
fig. 36 is a schematic flowchart illustrating a process of determining whether a file with the same name may exist in the embodiment of the present application.
S3601, determining whether the number of bits, of which the same positions in the bloom filter bit array of the electronic device A and the same positions in the bloom filter bit array of the electronic device B are 1 at the same time, is less than K;
when determining that the value is smaller than K, executing step S3602;
when it is determined to be not less than K, step S3603 is performed.
S3602, determining that the same-name files cannot exist under the directories A of the electronic device A and the electronic device B;
When it is determined that the number of bits, which is 1 at the same time, in the bloom filter bit array of the electronic device a and the bloom filter bit array of the electronic device B is smaller than K, it may be determined that the files of the same name may not exist under the directories a of the electronic device a and the electronic device B. Step S3605 may be performed.
S3603, determining that files with the same name possibly exist in the directories A of the electronic device A and the electronic device B;
when it is determined that the number of bits of which the same position in the bloom filter bit array of the electronic device a and the same position in the bloom filter bit array of the electronic device B are simultaneously 1 is not less than K, it may be determined that files with the same name may exist under the directories a of the electronic device a and the electronic device B. Step S3604 may be performed.
S3604, executing step S3207;
when it is determined that the files with the same name may exist under the directories a of the electronic devices a and B, step S3207 may be performed to rename the files with the same name under the directories a of the electronic devices a and B or to determine that the files with the same name do not exist under the directories a of the electronic devices a and B.
After step S3604 is executed, step S3605 may be executed.
S3605, carrying out logic or operation on the bloom filter bit array of the electronic device A and the bloom filter bit array of the electronic device B to obtain a logic or bloom filter bit array;
After determining that the files with the same name cannot exist in the directories a of the electronic device a and the electronic device B in step S3602, or after renaming the files with the same name in the directories a of the electronic device a and the electronic device B in step S3604, or after eliminating the possibility that the files with the same name exist in the directories a of the electronic device a and the electronic device B in step S3604, the bit array of the bloom filter of the electronic device a and the bit array of the bloom filter of the electronic device B may be logically or-operated to obtain a logical or bloom filter bit array.
Performing logic or operation, specifically, in two bloom filter bit arrays with the same bit number, if any value at the same position is 1, setting the value at the position in the obtained logic or bloom filter bit array as 1; if the values of the same position are all 0, the value of the position in the obtained logical or bloom filter bit array is set to 0.
Illustratively, FIG. 37 is an exemplary diagram of a logical OR operation on a bloom filter bit array. The number of bits in both bloom filter bit array 1 and bloom filter bit array 2 is 8. In the bloom filter bit array 1, the values of the 3 rd, 4 th, 7 th and 8 th bits are 1; in bloom filter bit array 2, the values in bits 1, 4, and 6 are 1. The logical or operation is performed on the bloom filter bit array 1 and the bloom filter bit array 2, and the obtained bloom filter bit array 3 has 1 in the 1 st, 3 rd, 4 th, 6 th, 7 th and 8 th bits and 0 in the other bits.
S3606, determining whether the bit number of the logic or bloom filter bit array and the bit number of the bloom filter bit array of the electronic equipment C, wherein the same position of the logic or bloom filter bit array is 1 at the same time, is smaller than K;
after the logic or bloom filter bit array is obtained, whether the bit number of the logic or bloom filter bit array and the bit number of the bloom filter bit array of the electronic device C, which has the same position and is 1 at the same time, is less than K can be determined;
when it is determined to be less than K, step S3607 may be performed;
when it is determined to be not less than K, step S3608 may be performed.
S3607, determining that files with the same name cannot exist under the directories A of the electronic equipment A, the electronic equipment B and the electronic equipment C;
when it is determined that the number of bits of the logic or bloom filter bit array and the number of bits of the bloom filter bit array of the electronic device C which are at the same time 1 is less than K, it can be determined that the files with the same name cannot exist in the directories a of the electronic device a, the electronic device B and the electronic device C.
S3608, determining that files with the same name possibly exist in directories A of the electronic equipment C and the electronic equipment A and in directories A of the electronic equipment C and the electronic equipment B;
when it is determined that the logical or bloom filter bit array and the bit number of the bloom filter bit array of the electronic device C whose same position is 1 at the same time are not less than K, it may be determined that the files of the same name may exist under the directories a of the electronic device C and the electronic device a, and the electronic device C and the electronic device B. Steps S3609 and S3613 may be performed to determine whether the same-name file may exist under the directories a of the electronic device C and the electronic device a, and whether the same-name file may exist under the directories a of the electronic device C and the electronic device B, respectively.
S3609, determining whether the bit number of the bloom filter bit array of the electronic device C and the bit number of the bloom filter bit array of the electronic device A, which have the same position and are 1 at the same time, is less than K;
when it is determined to be less than K, step S3610 may be performed;
when it is determined to be not less than K, step S3611 may be performed.
S3610, determining that the files with the same name cannot exist in the directories A of the electronic equipment C and the electronic equipment A;
when it is determined that the number of bits, which is 1 at the same time, in the bloom filter bit array of the electronic device C and the bloom filter bit array of the electronic device a is smaller than K, it may be determined that the files of the same name may not exist under the directories a of the electronic device C and the electronic device a.
S3611, determining that files with the same name possibly exist in the directories A of the electronic equipment C and the electronic equipment A;
when it is determined that the number of bits of the bloom filter bit array of the electronic device C and the number of bits of the bloom filter bit array of the electronic device a that are at the same time 1 are not less than K, it may be determined that files of the same name may exist under the directories a of the electronic device C and the electronic device a. Step S3612 may be performed.
S3612, go to step S3207;
when it is determined that the files with the same name may exist under the directories a of the electronic device C and the electronic device a, step S3207 may be performed to rename the files with the same name under the directories a of the electronic device a and the electronic device C or to determine that the files with the same name do not exist under the directories a of the electronic device C and the electronic device a.
S3613, determining whether the number of bits, of which the same positions are 1 at the same time, in the bloom filter bit array of the electronic device C and the bloom filter bit array of the electronic device B is smaller than K;
when it is determined to be less than K, step S3614 may be performed;
when it is determined to be not less than K, step S3615 may be performed.
S3614, determining that the files with the same name cannot exist in the directories A of the electronic equipment C and the electronic equipment B;
when it is determined that the number of bits, which is 1 at the same time, in the bloom filter bit array of the electronic device C and the bloom filter bit array of the electronic device B is smaller than K, it may be determined that the files of the same name may not exist under the directories a of the electronic device C and the electronic device B.
S3615, determining that files with the same name possibly exist in the directories A of the electronic equipment C and the electronic equipment B;
when it is determined that the number of bits of which the same position in the bloom filter bit array of the electronic device C and the same position in the bloom filter bit array of the electronic device B are simultaneously 1 is not less than K, it may be determined that files of the same name may exist under the directories a of the electronic device C and the electronic device B. Step S3616 may be performed.
S3616, step S3207 is executed.
When it is determined that the files with the same name may exist under the directories a of the electronic devices C and B, step S3207 may be performed to rename the files with the same name under the directories a of the electronic devices a and B or to determine that the files with the same name do not exist under the directories a of the electronic devices C and B.
After any of the above steps S3610 and S3612 and any of S3614 and S3616 are executed, or after step S3607, it may be determined that the same-name file does not exist under the directory a of the electronic device a, the electronic device B, and the electronic device C at this time, and step S3208 may be executed.
If the electronic device a not only acquires the bloom filter bit arrays of the electronic device B and the electronic device C, but also acquires the bloom filter bit array of the electronic device D. Then the logical or bloom filter bit array needs to be compared with the bloom filter bit array of the electronic device C and corresponding operations are performed so that it can be determined that the same-name file does not exist under the directories a of the electronic device a, the electronic device B, the electronic device C and the electronic device D, and then step S3208 can be performed again. If the electronic device a acquires more bloom filter bit arrays of the electronic device, the operation can be performed by analogy, which is not described herein again.
For example, fig. 38 is a schematic diagram of an exemplary scenario in which logic or operations are used to determine whether a file with the same name may exist in the embodiment of the present application.
The electronic device a acquires two other online electronic devices in the distributed file system: bloom filter bit arrays for electronic device B and electronic device C. For convenience of description, the bloom filter bit array of the electronic device a is denoted as BA, the bloom filter bit array of the electronic device B is denoted as BB, and the bloom filter bit array of the electronic device C is denoted as BC.
The electronic device a may compare BA and BB first, and when the number of bits of which the same position is 1 in the two bloom filter bit arrays is not less than K, the electronic device a may determine that the same-name file may exist in the directories a of the electronic device a and the electronic device B; when the number of bits of which the same position in the two bloom filter bit arrays is 1 at the same time is smaller than K, the electronic device a may determine that the files of the same name may not exist under the directories a of the electronic device a and the electronic device B.
After determining that the files with the same name cannot exist in the directories a of the electronic device a and the electronic device B, the electronic device a may perform a logic or operation on BA and BB to obtain a logic or bloom filter bit array, which is denoted as BAB.
The electronic device A can compare BC and BAB, and when the digits of the same positions in the two bloom filter digit groups, which are 1 at the same time, are less than K, the electronic device A can determine that the files with the same name cannot exist in the directories A of the electronic device A and the electronic device C; when the number of bits of which the same position in the two bloom filter bit arrays is 1 at the same time is not less than K, the electronic device a may determine that the electronic device C and the electronic device a, or the electronic device C and the electronic device B may have the same-name file in the directory a.
The electronic device A can compare BC with BA, and when the digits of the same positions in the two bloom filter digit groups, which are 1 at the same time, are less than K, the electronic device A can determine that the files with the same name cannot exist in the directories A of the electronic device C and the electronic device A; when the number of bits of which the same position in the two bloom filter bit arrays is 1 at the same time is not less than K, the electronic device a may determine that the files of the same name may exist under the directories a of the electronic device C and the electronic device a.
Meanwhile, the electronic device A can compare BC and BB again, and when the digits of the same positions in the two bloom filter digit groups, which are 1 at the same time, are less than K, the electronic device A can determine that the files with the same name cannot exist in the directories A of the electronic device C and the electronic device B; when the number of bits of which the same position in the two bloom filter bit arrays is 1 at the same time is not less than K, the electronic device a may determine that the files of the same name may exist under the directories a of the electronic device C and the electronic device B.
For example, assume that the distributed file system is composed of m electronic devices, each electronic device utilizes a bloom filter with the same parameters, and the number of hash functions used by the bloom filter is k. When displaying all files in a path of the distributed file system, the file names of all files in the path of each electronic device are mapped and modified by the hash function of the bloom filter, and the bit array of the electronic device is modified by BF iA bit array representing the ith (1 ≦ i ≦ m) electronic device. By F (BF)i,BFj) Representing bit array BFiAnd BFjThe bits of the same position are simultaneously the total number of 1's. One way in which steps S3206 and S3207 determine whether files of the same name may exist under directory a in each electronic device may be as shown in pseudo code in fig. 39.
The pseudo code in FIG. 39 includes a Check function and a checkFileName function. The execution process of the checkFileName function is an implementation manner of the step S3206, and the execution process of the checkfunction is an implementation manner of the step S3207.
The CheckFileName function in fig. 39 uses the bit arrays of the electronic devices to count the total number of 1 at the same time for the same position of two different bit arrays, so as to achieve the purpose of quickly determining whether the file with the same name exists between the electronic devices. If the digits of the two digit groups at the same position, which are 1 at the same time, are less than K, the same-name file does not exist; otherwise, the file with the same name may exist, and the Check function in fig. 39 is continuously called to confirm whether the file with the same name exists by means of violent enumeration.
Specifically, the first line in the function body of the CheckFileName function represents: setting a logical or bloom filter bit array BF firsttmpEqual to BF1
The second row represents: the variable i is cycled from 2 to m.
The third row shows: in each cycle of variable i, if BFtmpAnd BFiIf the total number of the bits at the same position which are simultaneously 1 is more than or equal to K, executing a fourth line; otherwise, skipping the subsequent steps and carrying out the next circulation of the variable i.
The fourth row represents: in each loop of variable i, the variable j is looped from 1 to i-1.
The fifth row represents: in each cycle of variable j, if BFiAnd BFjIf the total number of bits of the same position which are simultaneously 1 is greater than or equal to K, executing a sixth row; otherwise, skipping the subsequent steps and carrying out the next circulation of the variable j.
The sixth row shows: the Check function is performed on i and j of the loop of the current round.
The seventh row shows: in each cycle of variable i, after the operation of the sixth line is completed, the BF is checkedtmpBF against this wheeliPerforming logical OR operation to update the result to BFtmpBF in the next cycle as variable itmp
Specifically, the first line in the function body of the Check function represents: the subsequent loop operation is performed for each file x in the electronic device i under the path.
The second row represents: the subsequent loop operation is performed for each file y in the electronic device j under the path.
The third row shows: if the filename of file x is the same as the filename of file y, then the fourth row is executed; otherwise, skipping the subsequent steps, and performing the next cycle, and comparing the file x in the electronic device i with the next file y in the electronic device j.
The fourth row represents: file x is renamed.
The fifth row represents: the loop for each file y in the path in electronic device j is skipped and the loop for the next file x in electronic device i is started.
S3207, the electronic device A determines the files with the same name in the directory A in the two electronic devices which possibly have the files with the same name, and renames the files with the same name;
when it is determined in step S3206 that the same-name files may exist in the directory a in the two electronic devices, the electronic device a may determine the same-name files in the directory a in the two electronic devices that the same-name files may exist, and rename the same-name files; or determining that the same-name file does not exist under the directory a in the two electronic devices where the same-name file may exist. Then, the result may be returned and step S3206 may be performed again.
Specifically, there are many ways for the electronic device a to determine the files with the same name in the directory a in the two electronic devices where the files with the same name may exist:
optionally, the electronic device a may directly compare the file names of each file in the directory a in the two electronic devices pairwise, and determine whether the file with the same name exists. When the same-name file exists, the same-name file is renamed. When the files with the same name are not found after the comparison is completed, it can be determined that the files with the same name do not exist in the directory a of the two electronic devices. For example, the method may be represented by a Check function in the pseudo code shown in fig. 39, and details are not repeated here.
Optionally, the electronic device a may determine, according to the bloom filter bit array of the first electronic device in the two electronic devices, whether a file under the directory a in the second electronic device may have a file with the same name under the directory a in the first electronic device; the file name of the file in directory a in the second electronic device is used to compare with the file name of each file in directory a of the first electronic device when it is determined that a file of the same name is likely to exist. When it is determined that the same-name file exists, the same-name file is renamed. After each file in the directory a in the second electronic device is compared, if the file with the same name is not found, it may be determined that the file with the same name does not exist in the directory a in the two electronic devices.
It is understood that there may be other ways to determine the same-name files under directory a in the two electronic devices where the same-name files may exist, and the method is not limited herein.
S3208, the electronic device a displays all files under directory a in the distributed file system.
After the electronic device a determines that the files under the directory a in each electronic device cannot exist under the same name, the electronic device a may display all the files under the directory a in the distributed file system.
It will be appreciated that displaying all files under directory a in the distributed file system may be displaying file information under directory a in the distributed file system. Specifically, the file information in directory a may include file information of local files in directory a of each electronic device in the distributed file system. For example, the file information of the local file under the directory a of the electronic device a, the file information of the local file under the directory a of the electronic device B, and the like are included.
Optionally, the file information may include at least a file name and a file icon, and may also include actual data of the file, and the like, which is not limited herein.
For example, all the files in the directory a in the distributed file system may be shown as described with reference to fig. 6, and are not described herein again.
It can be understood that the method for identifying files with the same name in the embodiment of the present application may be applied to a distributed file system without a central node, and may also be applied to a distributed file system with a central node, which is not limited herein.
In the embodiment of the application, the electronic device a can quickly perform overall judgment on the possibility that the files with the same name exist in the same directory in every two electronic devices by comparing the bloom filters of the electronic devices in the distributed file system without the central node, and obtain a result that the files with the same name cannot exist and the files with the same name may exist. And then according to the integral judgment result, carrying out subsequent processing on the electronic equipment possibly having the files with the same name, and determining the renaming of the files with the same name. Since the same-name files exist in the same directory of each electronic device not very much, the result of identifying the same-name files can be directly obtained through the overall judgment in most cases, and even if the final result is not obtained, most electronic devices which cannot have the same-name files can be quickly eliminated, so that the range of identifying the same-name files is remarkably reduced. The processing time length when the homonymous files are identified is greatly reduced, and the resources consumed when the homonymous files are identified are reduced.
By the method for identifying the homonym files, the homonym files can be quickly detected when the files in the other electronic equipment are displayed in one electronic equipment. In a distributed file system, one electronic device can not only display files in another electronic device, but also read files in another electronic device. In order to increase the speed of reading data among different electronic devices, a data pre-reading technology can be adopted.
In the current data pre-reading technology, after the requested data is read, data of a plurality of fixed memory pages is pre-read. And then setting a pre-reading flag bit on the memory page with the first fixed numerical value of the reciprocal. When the pre-read flag is accessed, the next pre-read can be triggered.
However, in a distributed file system, the network environment between electronic devices is often in dynamic change. And the performance of each electronic device is also different. When one electronic device performs data pre-reading on another electronic device, the pre-reading effect is often poor due to changes in network environment or differences in device performance.
The data pre-reading method provided by the embodiment of the application can improve the pre-reading performance of the electronic equipment in the distributed file system.
Thirdly, a data pre-reading method:
the following first introduces the related terms and concepts related to the data pre-reading method in the embodiments of the present application.
(1) A distributed file system:
in the embodiment of the application, a plurality of electronic devices form a distributed file system.
The distributed file system provides a logical file system structure for resources distributed at any position on each electronic device, so that access to shared files distributed on each electronic device is easier.
It can be understood that, in the embodiment of the present application, the distributed file system may be a centralized distributed file system, that is, one host electronic device manages metadata information of all files in the distributed file system; the file system may also be a decentralized distributed file system, i.e. all electronic devices may manage metadata information of all files in the distributed file system, and is not limited herein. The metadata information of a file contains metadata describing the file. For example, the metadata of a file contained in the metadata information of the file may include: file name, file type, file size, modification time, access rights, data index information, etc., without limitation.
Fig. 12 is an exemplary schematic diagram of a distributed file system formed by multiple electronic devices in this embodiment. For example, file a is a file in electronic device a, file B is a file in electronic device B, and file C is a file in electronic device C. After the device a, the device B, the device C, and the device D form a distributed file system, the device a can access the file B, and the device D can also access the file C. That is, any electronic device in the distributed file system can conveniently access files in other electronic devices in the distributed file system.
(2) Memory page:
the operating system manages the memory space by dividing it into pages of the same size, which are called memory pages.
The size of the memory page in the operating system of different electronic devices may be different according to the setting of the operating system, for example, 4KB, 8KB or 16 KB.
In an operating system of an electronic device, a memory page of a fixed size is used as a minimum allocation unit of a memory space, and data in the electronic device is managed in the form of the memory page. Therefore, the memory page may be a unit of the data size.
The following description relates to memory paging management:
2.1, memory management:
two important concepts in an operating system are the CPU and memory. The CPU is used to execute instructions. The memory is used to store computer executable program code, operating systems, programs, data, and the like. The memory may also be referred to as internal storage.
For memory, it is resource-limited, and if a process occupies a large or even larger memory than the physical memory, and multiple processes are executed simultaneously, it needs to involve memory management.
Ideally, the memory expectation of the user is large capacity, high speed and durability, but in reality, the memory is a memory architecture composed of a cache, a main memory and a disk. In the architecture, the cache has low capacity, high speed and high cost, the main memory has medium speed, medium capacity and medium cost, and the disk has high capacity, durability and low speed.
Memory management is to provide a uniform abstraction for "users" and to shield the differences between cache, main memory and disk, even to make them imperceptible. The user does not need to worry about whether the program is stored in a cache, a main memory or a disk, and only needs to operate and output the same result.
When the operating system executes programs of a plurality of processes simultaneously, the programs need to be ensured not to interfere with each other, that is, one process cannot access the memory space of another process. When a program reads and writes specific memory data, the specific memory data cannot be directly mapped to the physical memory, that is, the memory address issued by the program and the physical memory need to be independent.
The use of virtual memory may satisfy these memory management needs.
2.2, virtual memory:
virtual memory is an important milestone in the operating system development history. The use of the virtual memory avoids the direct interaction between the program and the physical memory, and the uniform abstraction is made for the cache, the main memory and the disk, so that the program can break through the size limitation of the physical memory, and the program is still limited by the size of the virtual memory.
After the virtual memory is used, the memory address seen by the program is the virtual memory address, and the reading and writing of the virtual memory address by the program are mapped into the actual physical memory, and the mapping is called translation. The translation is performed by a Memory Management Unit (MMU), the MMU receives a virtual address sent by the CPU, translates the virtual address into a physical address, and sends the physical address to the memory, and the memory performs corresponding access according to the physical address and then reads or writes related data.
Fig. 40 is a diagram illustrating a relationship between an application program, a virtual memory, and a physical memory according to an embodiment of the present application. Physical memory interfaces of physical memories such as a cache, a main memory and a hard disk in the electronic equipment can be abstracted into virtual memory uniformly, and virtual memory addresses are provided for application programs in the electronic equipment on the virtual machine interfaces. The user can map the virtual memory address read-write to the actual physical memory through the application program.
2.3, memory space paging management:
the core of paging management of memory space is to divide the virtual memory space and the physical memory space into pages with the same size, and use the pages as the minimum allocation unit of the memory space, and one page of a program can be stored in any one physical page.
Through paging management of the memory space, mapping of a continuous virtual space to a non-continuous physical memory can be realized. Therefore, continuous data segments in the process can be stored in the discontinuous physical memory, and the management of the memory space is more flexible.
Fig. 41 is an exemplary diagram illustrating memory paging management according to an embodiment of the present application. The paging mechanism may split data into memory page sizes, allocating contiguous addresses in the virtual address space. These memory pages that are consecutive in address in the virtual address space may be scattered over the unallocated pages in the physical address space.
In some embodiments of the present application, a memory page may also be referred to as a page for short, which is not described in detail later.
In some embodiments, the memory pages used to cache file data may be referred to as file pages, and are not limited herein.
(3) Pre-reading data:
in the embodiment of the application, when one electronic device in the distributed file system accesses a file in another electronic device, in order to improve the access efficiency, a data pre-reading technology may be adopted.
The following description takes an example in which the electronic device a accesses the file a in the electronic device B in the distributed file system as follows:
3.1, if data pre-reading is not carried out:
FIG. 42 is a diagram of an exemplary scenario for data access.
In the process that the electronic device a accesses the file a of the electronic device B, the electronic device a reads data of a next part of the file a from the electronic device B as needed. For example, suppose that the file A is a document which is a document whose data occupy 25 memory pages in the order of Y1 to Y25.
As shown in (a) of fig. 42, if the electronic device a has read a part of the data of the file a, the corresponding memory pages are Y1 to Y10. After the user turns the pages, the electronic device a displays the next part of data of the file a, and the corresponding memory pages are Y11 to Y15. The electronic device a may request access to Y11-Y15 from the electronic device B.
As shown in fig. 42 (b), the electronic apparatus a may cache and display Y11 to Y15. After the user turns the page, the electronic device a needs to display the next part of data of the file a, and the corresponding memory pages are Y16-Y20. The electronic device a needs to request access to Y16-Y20 from the electronic device B again in order to redisplay the next part of the file a.
This may cause the user to not continuously view the file a on the electronic device a, and after viewing one part of the data, the user may wait for a period of time before viewing another part of the data.
3.2, if data pre-reading is carried out:
FIG. 43 is a diagram of an exemplary scenario for performing data pre-reading.
In the process that the electronic device a accesses the file a of the electronic device B, the electronic device a reads the data of the next part of the file a from the electronic device B according to the current progress of reading the data of the file a. For example, suppose that the file A is a document which is a document whose data occupy 25 memory pages in the order of Y1 to Y25.
As shown in (a) of fig. 43, if the electronic device a has read a part of the data of the file a, the corresponding memory pages are Y1 to Y10. When the electronic device A reads Y8 in the process of reading the cached Y6-Y10 and displaying Y6-Y10, the electronic device A sends a request for pre-reading the memory pages of Y11-Y15 of the file A to the electronic device B.
As shown in fig. 43 (b), when the user finishes viewing the data of Y6 to Y10 displayed by the electronic device a and turns the page to view the next part of the data of the file a, the electronic device a finishes pre-reading Y11 to Y15, and can directly display Y11 to Y15. And the electronic device B does not need to be requested to access Y11-Y15 any more and can be displayed after waiting for the data returned by the electronic device B.
In addition, when the electronic device a reads Y13 while the electronic device a reads the cached Y11 to Y15 and displays Y11 to Y15, the electronic device a may send a request to the electronic device B to read in advance the memory pages of Y16 to Y20 of the file a.
Therefore, the fluency of the electronic equipment A when accessing the files in the electronic equipment B can be greatly improved by using the data pre-reading, and the data acquisition time is saved.
(4) Pre-read window size:
in the embodiment of the application, the size of the pre-reading window is the number of memory pages occupied by data acquired by the electronic device during each pre-reading.
For example, if N memory pages are read in advance each time, the read-in window size is N.
(5) Pre-reading a flag bit:
in this embodiment, the electronic device may set a pre-reading flag bit in the pre-read memory page, and when the memory page with the pre-reading flag bit is accessed, the electronic device is triggered to perform the next pre-reading.
With reference to fig. 43, fig. 44 is a schematic diagram of an exemplary scenario of pre-reading a flag bit in the embodiment of the present application. When the electronic device a caches the pre-read memory pages Y6 to Y10, the pre-read flag bit may be set in the memory page Y8. When the electronic device a accesses the memory page Y8 while displaying Y6 to Y10, the electronic device a is triggered to perform the next read-ahead operation because the memory page Y8 has the read-ahead flag bit. For example, the data of the Y11-Y15 parts of the file A are read in advance to the electronic device B according to the size of the read-ahead window 5.
It will be appreciated that there may be some space in a memory page to store other special fields in addition to data. For example, the value of a particular field may be modified so that the field becomes a pre-read flag, which is not limited herein.
(6) Network round trip delay:
6.1, definition of network round-trip delay:
the network round-trip delay, which is an important performance indicator in a computer network, represents the total time delay from the time when a sender sends data to the time when the sender receives an acknowledgement from a receiver (the receiver sends an acknowledgement immediately after receiving the data).
For example, if the electronic device a transmits data to the electronic device B and receives an acknowledgement from the electronic device B200 ms after the electronic device a transmits the data, it may be considered that the network delay between the electronic device a and the electronic device B is 200ms at this time.
6.2, fluctuation of network round-trip delay:
in the embodiment of the present application, when data access is performed between electronic devices in a distributed file system, because types of the electronic devices may be different, network environments in which the electronic devices are located may be different, or the network environments are in a change or the like, network round-trip delay fluctuation between the electronic devices is large, and may be in a continuously changing state.
For example, if the electronic device a accesses data in the electronic device B, the electronic device a and the electronic device B are both mobile terminals, and use a wireless mobile network. The network delay between the electronic device a and the electronic device B may be dynamically changed due to factors such as the number of base stations arranged around by the mobile network service provider used by the electronic device a and the electronic device B, the number of users using the wireless mobile network around, the distance from the base stations, and the like.
6.3, determination of network round trip delay:
in this embodiment, an electronic device may determine a network round trip delay (RTT) with another electronic device in a distributed file system by obtaining the RTT in a TCP link with the electronic device.
Because each communication between the sending end and the receiving end can obtain the latest RTT, the electronic equipment can monitor the network round-trip delay with another electronic equipment in real time.
(7) Network bandwidth:
network bandwidth refers to the amount of data that can be transmitted in a unit of time (e.g., 1 second).
For example, if 500 bits of data can be transmitted between the electronic device a and the electronic device B within 1 second, the network bandwidth between the electronic device a and the electronic device B may be considered to be 500 bits/s.
In the embodiment of the present application, the network bandwidth between the electronic device a and the electronic device B in the distributed file system may be determined in multiple ways: for example, a network transmission test may be performed between the electronic device a and the electronic device B, so as to determine a network bandwidth between the electronic device a and the electronic device B; for another example, the network bandwidth of the electronic device a may be estimated according to the signal strength of the electronic device a and/or the network frequency band used by the electronic device a. There may be many other ways to determine the network bandwidth between electronic device a and electronic device B, and is not limited herein.
(8) The device type:
in the embodiment of the application, the electronic devices can be classified into different device types according to the capacity of the electronic devices. The read-ahead flag bits may be set at different locations in the read-ahead memory page for different device types.
Optionally, the electronic device with the capability value exceeding the preset capability value may be referred to as a first capability device, such as a mobile phone, a computer, and the like; an electronic device that does not exceed the preset capability value is referred to as a second capability device, such as a watch, router, or the like. The preset capability value may include parameters such as cpu core number and frequency, and memory size, which are not limited herein.
For example, fig. 45 is an exemplary schematic diagram illustrating setting of different pre-read flag bits according to different device types in the embodiment of the present application. Take an example where the electronic device a reads in advance the data of the memory pages Y6 to Y10 in the file a containing the data of the memory pages Y1 to Y25 from another electronic device.
As shown in fig. 45 (a), if the electronic device a is data pre-read from the first capability device. Because the first capability device has stronger capability, the feedback processing can be performed faster each time the pre-reading request is received, and therefore, the electronic device a can compare the position setting of the pre-reading flag bit in the pre-read memory page later. For example, on Y9 of the memory pages Y6-Y10, and thus, it is also sufficient for the first capable device to transmit new read-ahead data at a later time.
As shown in fig. 45 (b), if the electronic device a is data pre-read from the second capability device. Since the second capability device has a weak capability, it takes a lot of time to perform the feedback process each time the read-ahead request is received. Therefore, the electronic device a needs to compare the setting of the position of the pre-read flag in the pre-read memory page to the front. For example, on Y7 of the memory pages Y6-Y10, thus leaving enough time for the second capable device to transmit new read-ahead data. Therefore, when the electronic device a finishes reading the data of the memory page Y10, it is ensured that the next part of data pre-read by the electronic device a sent by the second capability device has been received.
In the embodiment of the application, the electronic device stores the corresponding relationship between the electronic device which has established the communication connection in the distributed file system and the device type thereof. After the electronic device reads data in advance from another electronic device in the distributed file system, a pre-read flag may be set at a corresponding position in the memory page where the pre-read data is located according to the device type of the another electronic device in the stored correspondence.
It can be understood that, in this embodiment of the present application, taking dividing the device type into the first capability device and the second capability device as an example, in an actual application, the electronic device may be divided into more different device types according to the size of the capability of the electronic device, and more different relative positions of the pre-read flag bits are set for more different device types, which is not limited herein.
(9) Foreground application:
in this embodiment, the electronic device may determine whether the application requesting the pre-read data is a foreground application.
Specifically, in the electronic device, the process of the application initiates the request, and the process of the application may have an application type identifier, where the application type identifier is used to identify an application type of the application to which the process belongs. If the application type identifier is a foreground application identifier, it may be determined that the application to which the process belongs is currently a foreground application. If the application type identifier is not a foreground application identifier, it may be determined that the application to which the process belongs is not currently a foreground application.
(10) The file type:
in an embodiment of the application, the electronic device may determine a file type of the access file.
Specifically, when accessing a certain file, the electronic device needs to send an access request, where the access request may carry index information of the file to be accessed, and the index information may include an extension (or called a suffix) of the file to be accessed. The electronic device may determine the file type of the accessed file from the extension.
For example, if the extension of the accessed file is an extension of a video file such as rmvb, mp4, avi, etc., the electronic device may determine that the accessed file is a video file.
It is understood that the extension of the video file may be in various forms, such as flv, mov, wmv, etc., and is not limited thereto.
Fig. 46 is a schematic diagram of an exemplary signaling interaction for data pre-reading in the prior art. Referring to FIG. 46, FIG. 47 is a diagram illustrating an exemplary state of a memory page with data read ahead in the prior art. Assume that the default pre-reading window size preset in the electronic device a is 10, and the default position of the pre-reading flag bit is-5. After the electronic device a receives the request for accessing the file a in the electronic device B, the electronic device a may also pre-read the data of 10 memory pages of the file a after pre-reading the start position according to the default pre-reading window size 10 after reading the data that needs to be currently used. Then, in the pre-read memory pages, according to the default position-5, a pre-read flag bit is set in the 5 th last memory page. When the electronic device a reads the memory page with the pre-reading flag bit, the electronic device a triggers the next pre-reading of the file a. It will be appreciated that the next read ahead will also be processed according to the default read ahead window size 10, the default position-5 of the read ahead flag bit.
Therefore, in a scenario in which electronic devices access files to each other between distributed file systems, the network environment between the electronic devices is often in dynamic change, and the performance of each electronic device is different. For example, if the network environment between the current electronic device a and the current electronic device B is poor, the delay is high. It may occur that when the electronic device a finishes reading the data of the last pre-read 10 memory pages, the data of the next pre-read 10 memory pages is not received yet. The electronic device a needs to wait for more time to acquire the pre-read data, so that the electronic device a cannot continuously process the data, and the processing efficiency of the electronic device a is reduced. And the process that the user uses the electronic equipment A to view the file A is possibly discontinuous, and the user experience is reduced.
By adopting the data pre-reading method in the embodiment of the application, the electronic device a can determine a proper pre-reading window according to the network delay and the network bandwidth between the current electronic device a and the current electronic device B. And the number and the position of the pre-reading flag bit set in the pre-read memory page can be determined according to the type of the file to be accessed, the type of the application accessing the file and the device type of the electronic device B. Therefore, the determined size of the pre-reading window and the set pre-reading zone bit can better meet the current requirements, the influence of factors such as network environment change, the type of the electronic equipment B and the like on the pre-reading effect is reduced, the pre-reading performance of the electronic equipment in the distributed file system can be improved, and the data access speed between the electronic equipment is accelerated.
Illustratively, fig. 48 is a schematic diagram of an exemplary scenario of data pre-reading in the embodiment of the present application. If the electronic device a pre-reads the data of the file a in the electronic device B, the electronic device a pre-reads N memory pages, where the N memory pages have K pre-read flag bits, and the position of the last pre-read flag is-M. The size N of the pre-read window is determined according to the network round-trip delay and the network bandwidth between the electronic device a and the electronic device B, K is determined according to the file type of the file a, and M is determined according to the device type of the electronic device B. Rather than the default settings used.
Thus, according to different network environments, the pre-reading window size determined by the electronic device a is different: more memory pages can be read in advance when the network environment is poor, and less memory pages can be read in advance when the network environment is good. The power consumption of the electronic device A is reduced, and meanwhile the electronic device A can pre-read a proper number of memory pages each time.
According to the difference between the file type of the file a to be accessed, the application type of the file a to be accessed, and the device type of the electronic device B, the electronic device a may set pre-read flag bits with different numbers and positions in the pre-read memory page:
For a video type file, because a user may need to skip to view, several pre-reading flag bits may be additionally set in each pre-read memory page, or even a pre-reading flag bit may be set in each pre-read memory page, so that the user can trigger subsequent pre-reading flag bits to pre-read after skipping to view any position.
For other types of files, even if only one pre-read flag bit is set: if the foreground application accesses the remote file, in order to enable the foreground application to obtain better remote data read-write performance, the pre-reading flag bit can be set at a position close to the front, for example, the pre-reading flag bit is set on a first memory page in the pre-reading memory pages; for the background application to access the remote file, in order to reduce the network load of the electronic device, the pre-reading flag bit may be set at a later position, so that the pre-reading time is later. For example, the read-ahead flag is set at the position of the reciprocal 1/4 in the read-ahead memory page, and so on.
Even if the applications are foreground applications, pre-reading flag bits at different positions can be set in the pre-read memory page according to the device type of the remote electronic device B: because the stronger the capability of the electronic device B is, the faster the speed of the electronic device B feeding back data is, the pre-reading flag bit can be set at a relatively backward position, so that there is enough time to receive the pre-reading data fed back by the electronic device B. If the capability of the electronic device B is weak and the speed of the electronic device B feeding back data is slow, the pre-reading flag needs to be set at a relatively front position. Therefore, the electronic equipment A can receive the pre-read data returned by the electronic equipment B before the existing data is processed.
Therefore, according to the size of the pre-reading window determined by the network environment adaptability, the pre-reading flag bits with different numbers and positions are set according to the file type of the file A to be accessed, the application type of the file A to be accessed and the device type adaptability of the electronic device B, so that the electronic device A can receive new pre-reading data returned by the electronic device B before the existing pre-reading data is processed, excessive pre-reading data does not need to be stored, and less network and storage resources are consumed while the pre-reading performance of the electronic device in the distributed file system is greatly improved.
In conjunction with the description of the hardware structure diagram of the electronic device 100 shown in fig. 13, in some embodiments of the present application, the electronic device 100 may form a distributed file system with other electronic devices through the mobile communication module 150 or the wireless communication module 160. The processor 110 may call computer instructions stored in the internal memory 121 to cause the electronic device 100 to execute the data pre-reading method in the embodiment of the present application.
In conjunction with the description of the software structure diagram of the electronic device 100 shown in fig. 14, as shown in fig. 49, in some embodiments of the present application, the kernel layer may further include a data pre-reading module.
The data pre-reading module is configured to execute the data pre-reading method in the embodiment of the present application when the electronic device 100 accesses files of other electronic devices in the distributed file system.
It is understood that in some embodiments, the data read-ahead module may also be located at other levels of the hierarchical architecture, such as an application layer, an application framework layer, a system library, and the like, without limitation.
The data pre-reading method in the embodiment of the present application is specifically described below with reference to the software and hardware structure of the above exemplary electronic device 100. In the embodiment of the present application, the electronic device a or the electronic device B may adopt the software and hardware structure of the above exemplary electronic device 100, and is not limited herein.
The data pre-reading method in the embodiment of the application can be divided into three stages:
(1) pre-reading window size determination stage: and determining the size of the pre-reading window in real time according to the network delay and the network bandwidth.
(2) Pre-reading zone bit determining stage: determining the number and the position of pre-reading flag bits required to be set in a pre-read memory page according to the file type of the accessed file, the application type of the application of the accessed file and the equipment type of opposite-end electronic equipment;
(3) and (3) a pre-reading stage: pre-reading data and setting pre-reading zone bits from the electronic equipment at the opposite end according to (1) the size of the pre-reading window determined in real time in the pre-reading window size determining stage and (2) the number and the position of the pre-reading zone bits to be set determined in the pre-reading zone bit determining stage.
It will be appreciated that the above-described,
(1) the pre-reading window size determining stage can be continuously and repeatedly executed, and the pre-reading window size is updated in real time.
(2) The pre-reading flag bit determining stage may be executed only once when a new file is accessed to an electronic device, and the number and the positions of the pre-reading flag bits that need to be set when data pre-reading is performed when the file in the electronic device is accessed are determined, and the number and the positions of the pre-reading flag bits that are determined before may be directly used when the data of the file is pre-read in the pre-reading stage (3) subsequently. Or may be repeatedly executed for a plurality of times before or during the pre-reading of the file in the pre-reading stage (3), and the number and the position of the pre-reading flag bits to be set are updated in real time, which is not limited herein.
(3) The pre-reading stage triggers the execution every time the electronic device reads a memory page with a pre-reading flag bit.
The following describes in detail the three stages of data pre-reading in the embodiment of the present application, taking an example that an application a in an electronic device a in a distributed file system accesses a file F in an electronic device B:
(1) pre-reading window size determination stage:
fig. 50 is a schematic diagram of signaling interaction in the stage of determining the size of the read-ahead window in the embodiment of the present application.
S5001, accessing a file F in an electronic device B by an application A in the electronic device A;
in response to a user operation or an automatic task in the electronic device a, the application a in the electronic device a may access the file F in the electronic device B that is in the same distributed file system as the electronic device a.
When the electronic device a accesses the file F in the electronic device B, the metadata information of the file F may be first queried from the only metadata server in the distributed file system, and then the file F may be accessed to the electronic device B according to the metadata information of the file F; the metadata information of the file F cached in the electronic device may also be queried, and the file F is accessed to the electronic device B according to the metadata information of the file F, which is not limited herein.
In the process that the application A in the electronic equipment A accesses the file F in the electronic equipment B, partial data of the file F can be acquired from the electronic equipment B by a section according to the current requirement of the application A, and all data of the file F cannot be acquired directly. If the remote file is read to the local area in one time and cached in the local memory or disk, the network overhead and the storage resource occupation are much larger than the overhead of reading according to the requirement, and the network and the storage resource are greatly wasted.
It can be understood that when the application a in the electronic device a needs to access the file F in the electronic device B, a request for accessing the file F in the electronic device B is first sent to the electronic device a. After the electronic device a receives the request for accessing the file F in the electronic device B, step S5002 may be executed.
S5002, monitoring the network round-trip delay between the electronic equipment A and the electronic equipment B in real time;
after electronic device a receives the request to access file F in electronic device B, electronic device a can monitor the network round trip delay with electronic device B in real time.
It can be understood that, during the process that the application a in the electronic device a accesses the file F in the electronic device B, the electronic device a establishes a TCP link with the electronic device B.
Preferably, the electronic device a may acquire the RTT parameter value of the TCP link with the electronic device B in real time as the network round trip delay with the electronic device B.
Optionally, the electronic device a may periodically send a request to the electronic device B, and use a duration of receiving a response returned by the electronic device B as a network round-trip delay with the electronic device B.
The electronic device a may also monitor the network round-trip delay with the electronic device B in real time by other means, which is not limited herein.
S5003, determining a network bandwidth between the electronic device A and the electronic device B;
after electronic device a receives a request to access file F in electronic device B, electronic device a can determine the network bandwidth with electronic device B.
The manner in which electronic device a determines the network bandwidth with electronic device B may be many:
for example, the upper network bandwidth limit of the electronic device a and the upper network bandwidth limit of the electronic device B may be determined separately, and the smaller upper network bandwidth limit is used as the network bandwidth between the electronic device a and the electronic device B.
For another example, a network transmission test may be performed between the electronic device a and the electronic device B, so as to determine a network bandwidth between the electronic device a and the electronic device B;
as another example, the network bandwidth between the electronic device a and the electronic device B may be estimated according to the signal strength of the electronic device a and/or the network frequency band used by the electronic device a.
There may be many other ways for electronic device a to determine the network bandwidth with electronic device B, and is not limited herein.
It is understood that the network bandwidth determined by the electronic device a and the electronic device B may be an accurate value or a possible estimation value, and is not limited herein.
S5004, calculating the size of a pre-reading window by the electronic equipment A according to the network round-trip delay and the network bandwidth of the electronic equipment B;
after determining the network round trip delay and the network bandwidth with the electronic device B, the electronic device a may calculate the pre-read window size according to the network round trip delay and the network bandwidth.
Specifically, the following is a way to calculate the size of the read-ahead window according to the network round-trip delay and the network bandwidth:
the symbol S, T, L, N, D is first defined as follows:
s represents the maximum speed of reading data;
t represents the network bandwidth;
l represents the network round trip delay;
n represents the pre-read window size;
d represents the size of one memory page (typically, the size of one memory page is 4 KB).
Through modeling the performance S of the electronic equipment for reading the remote data, the network bandwidth T, the network round-trip delay L and the size N of the pre-reading window, the relationship between the size N of the pre-reading window and the performance S of the electronic equipment for reading the remote data can be obtained as follows:
Figure BDA0003549537550000751
according to the formula 1, it can be determined that when the size N of the pre-read window is set according to the following formula 2, the performance S of reading the remote data is large and the consumed network bandwidth and storage resources are small:
n ═ L × T/D (formula 2)
Optionally, an upper limit value may be set for the size N of the pre-read window to prevent the calculated value from being too large.
It is understood that the above formula 2 is only one way to calculate the size of the read-ahead window according to the network round trip delay and the network bandwidth, and in practical applications, a certain offset may be added or subtracted on the basis of the formula 2, or a certain coefficient may be multiplied or divided, and many other ways may be adopted to calculate the size of the read-ahead window according to the network round trip delay and the network bandwidth, which is not limited herein.
For example, if the electronic device a determines that the network round trip delay L with the electronic device B is 200ms, the network bandwidth T with the electronic device B is 800kB/s, and one memory page size D is 4kB, the pre-read window size N ═ 0.2s × 800 kB/4 kB ═ 40 can be calculated according to equation 2.
In the embodiment of the application, the size of the pre-reading window is calculated by using the network round trip delay as a parameter. Alternatively, in practical applications, the electronic device a may also use other types of network delay as parameters to calculate the size of the pre-reading window, for example, the network delay for sending data from the electronic device a to the electronic device B, or the network delay for sending data from the electronic device B to the electronic device a, which is not limited herein.
In steps S5001 to S5004, the electronic device a may calculate an initial size of the pre-read window according to the network round-trip delay and the network bandwidth of the electronic device B, and provide the initial size of the pre-read window to the pre-read stage (3). In practical application, the electronic device a can also continue to monitor the network round-trip delay with the electronic device B in real time, and update the size of the pre-reading window in real time and provide the updated size for the pre-reading stage (3).
S5005, when the change of the network round trip delay with the electronic device B is determined to reach the preset change threshold, updating the size of the pre-reading window according to the new network round trip delay.
After the electronic device a calculates the size of the pre-reading window according to the network round-trip delay and the network bandwidth with the electronic device B in step S5004, the electronic device a may further continue to execute step S5002, monitor the network round-trip delay with the electronic device B in real time, and determine whether the change of the network round-trip delay with the electronic device B relative to the network round-trip delay used in the calculation in step S5004 reaches a preset change threshold:
when it is determined that the change of the network round trip delay with the electronic device B reaches the preset change threshold, which indicates that the network environment with the electronic device B has large fluctuation, the electronic device a may update the size of the pre-reading window according to the new network round trip delay.
It is understood that the read-ahead window size is updated according to the new network round trip delay, and the new read-ahead window size can be calculated according to the network bandwidth and the new network round trip delay of the electronic device B using the same method in step S5004.
Illustratively, if the predetermined variation threshold is 200 ms. After the electronic device a calculates the size of the pre-reading window to be 40 according to the network round-trip delay of the electronic device B being 200ms, if the electronic device a monitors that the network round-trip delay of the electronic device B is updated to be 300ms, the electronic device a may determine that the change does not exceed the preset change threshold for 200ms, not update the size of the pre-reading window, and continue to monitor the network round-trip delay of the electronic device B. If the electronic device a monitors that the network round-trip delay with the electronic device B is updated to 400ms, the electronic device a determines that the change of the network round-trip delay with the electronic device B reaches a preset change threshold of 200ms, and the size of the pre-reading window needs to be updated. The electronic device a may recalculate the read-ahead window size N ═ 0.4s × 800kB/s)/4kB ═ 80 according to the equation 2 and according to the latest network round trip delay.
(2) Pre-reading zone bit determination stage:
it is understood that (2) the pre-reading flag bit determining stage is performed after the application a in the electronic device a accesses the file F in the electronic device B, and may be performed before the electronic device a pre-reads data from the electronic device B, after the electronic device a pre-reads data from the electronic device B, or during the process that the electronic device a pre-reads data from the electronic device B, which is not limited herein.
(2) The purpose of the pre-reading flag bit determining stage is to determine the number and the position of pre-reading flag bits to be set in a pre-read memory page according to the file type of a file accessed by the electronic device a, the application type of an application accessing the file, and the device type of the opposite-end electronic device. The following description will be made in detail with reference to an example in which an application a in an electronic device a accesses a file F in an electronic device B:
fig. 51 is a schematic flow chart of the pre-reading flag bit determining stage in the embodiment of the present application.
It is understood that the electronic device a has already performed the step S5001 before performing the following steps, and the description thereof is omitted here.
S5101, the electronic device A determines whether the file F is a video file;
after application a in electronic device a accesses file F in electronic device B, electronic device a may determine the file type of file F:
when determining that the file F is a video file, the electronic device a may perform step S5102;
when it is determined that the file F is not a video file, the electronic device a may perform step S5103.
It is understood that, in practical applications, the electronic device a may also perform different steps by determining that the file F is a file with other different formats, and determine that different numbers or positions of the pre-read flag bits need to be set in the pre-read memory pages, which is not limited herein.
For how to determine the file type of the file F, the description in the file type (10) can be introduced with reference to the above-mentioned terms, and will not be described herein again.
In some embodiments, step S5103 may be directly executed without the determination of the type of the accessed file in step S5101, which is not limited herein.
S5102, the electronic device A determines that a plurality of pre-reading flag bits need to be set;
when the electronic device a determines that the file F is a video file, it may be determined that a plurality of pre-reading flag bits need to be set.
The specific number of the plurality of pre-read flag bits to be set may be preset, and is not limited herein.
For example, when determining that the file F is a video file, the electronic device a may determine that a pre-read flag is to be set in each memory page to be pre-read.
For example, when the electronic device a determines that the file F is a video file, it may be determined that a pre-read flag bit needs to be set in every two memory pages in the pre-read memory pages.
It is understood that there are other ways to set the pre-read flag bits, which is not limited herein.
S5103, the electronic device A determines whether the application A is a foreground application;
the electronic device a may determine the application type of the application a accessing the file F:
When it is determined that the application a is not the foreground application, the electronic device a may perform step S5104;
when it is determined that the application a is the foreground application, the electronic device a may perform step S5105.
For how to determine the application type of the application a, reference may be made to the description in the aforementioned terminology introduction (9) foreground application, which is not described herein again.
It is understood that, in practical applications, the electronic device a may also perform different steps by using the application a as another different application type, and determine that different numbers or positions of the pre-read flag bits need to be set in the pre-read memory pages, which is not limited herein.
In some embodiments, step S5105 may be directly performed without determining the application type of the application of the accessed file in step S5103, which is not limited herein.
S5104, the electronic device A determines that a pre-reading zone bit needs to be set, and the position is a first position;
when the electronic device a determines that the file F is not a video file and the application a is not a foreground application, the electronic device a may determine that a pre-read flag bit needs to be set in a pre-read memory page, and the position is a first position. Since the application is not foreground, the requirement on the timeliness of data processing is low, and therefore, the first position can be a later position in the pre-read memory page.
The specific position of the first position may be set in advance, for example, the position of the reciprocal 1/4 in the pre-read memory page, and the like, and is not limited herein.
S5105, the electronic device A determines whether the electronic device B is a first capacity device;
when electronic device a determines that file F is not a video file and application a is a foreground application, electronic device a may determine the device type of electronic device B:
when the electronic device a determines that the electronic device B is the first capability device, the electronic device a may perform step S5106;
when the electronic device a determines that the electronic device B is not the first capability device, the electronic device a may perform step S5107.
It is understood that, in practical applications, the electronic device a may also perform different steps by using the file B as a different device type, and determine the set read-ahead flag bit at a different position in the read-ahead memory page, which is not limited herein.
For how to determine the device type of the electronic device B, the description in the description of the device type (8) may be referred to, and will not be described herein again.
In some embodiments, the determination of the device type of the opposite-end electronic device in steps S5105 to S5107 may not be provided, and is not limited herein.
S5106, the electronic device A determines that a pre-reading zone bit needs to be set, and the position is a second position;
when the electronic device a determines that the file F is not a video file, the application a is a foreground application, and the electronic device B is a first capability device, the electronic device a may determine that a pre-read flag bit is to be set in a pre-read memory page, and the position is a second position. Since the application a is a foreground application, the processing requirement on the data is large, and the processing speed is high, the second position is before the first position, so that the pre-reading of the next piece of data is triggered earlier.
The specific location of the second location may be set in advance, for example, the location of 1/2 in the pre-read memory page, and is not limited herein.
S5107, the electronic device determines that a pre-reading zone bit needs to be set, and the position is a third position;
when the electronic device a determines that the file F is not a video file, the application a is a foreground application, and the electronic device B is not a first capability device, the electronic device a may determine that a pre-read flag bit is to be set in a pre-read memory page, and the position is a third position. Since the electronic device B is not a first-capability device, which has a slower processing speed of data and requires an earlier pre-reading of data from the electronic device B, the third position precedes the second position so that the pre-reading of the next piece of data is triggered earlier.
The specific position of the third position may be preset, for example, the third position may be set as a start position in a pre-read memory page, and the like, which is not limited herein.
Illustratively, fig. 52 is an exemplary schematic diagram of different positions of the pre-read flag bit in the embodiment of the present application. According to the example that the reading sequence is from left to right when the pre-read memory pages are read, the first position of the pre-read flag bit is set to the rightmost side, and the memory page at the position is read last compared with the memory pages at the second position and the third position. The memory page in the second location is read before the memory page in the first location, and is read after the memory page in the third location. The memory page in the third location is read first compared to the memory pages in the first and second locations. It is to be understood that fig. 52 is an exemplary comparison diagram of different positions for setting the pre-read flag bit for three different situations, and does not indicate that the pre-read flag bit is set in all three positions of the pre-read memory page.
(3) And (3) a pre-reading stage:
it can be understood that, in the (3) pre-reading stage, the size of the pre-reading window can be determined in the (1) pre-reading window size determining stage, and in the (3) pre-reading stage, subsequent pre-reading can be performed according to the size of the pre-reading window updated in the (1) pre-reading window size determining stage.
(2) The pre-read flag bit determining stage may be completed before the pre-read stage (3), or may be completed during the execution of the pre-read stage (3), which is not limited herein.
Fig. 53 is a schematic signaling interaction diagram of the pre-reading stage in the embodiment of the present application.
S5301, pre-reading, by the electronic device a, data of a memory page of a first numerical value of the file F from the electronic device B, with the size of the determined pre-reading window being the first numerical value;
in step S5004, the electronic device a calculates the size of the read-ahead window according to the network round-trip delay and the network bandwidth of the electronic device B, and assumes that the calculated size of the read-ahead window is the first value. The electronic device a may pre-read the data of the first number of memory pages of the file F from the electronic device B according to the determined size of the pre-read window being the first number.
S5302, setting, by the electronic device a, a pre-read flag bit in the first numerical memory page according to the file type of the file F, and/or the application type of the application a, and/or the device type of the electronic device B;
for different file types, and/or application types, and/or device types, the number and the positions of the set pre-read flag bits are different, which may specifically refer to the description in the step (2) of determining the pre-read flag bits, and details are not described here.
After the electronic device a pre-reads data of a first numerical memory page of the file F, a pre-read flag bit may be set in the first numerical memory page according to a file type of the file F, and/or an application type of the application a, and/or an equipment type of the electronic device B.
Optionally, if the step of (2) determining the pre-read flag bit has been executed before the step S5302 is executed, the electronic device a may set the pre-read flag bit in the first number of memory pages according to the number and the position of the pre-read flag bits that need to be set and are determined in the step (2) determining the pre-read flag bit.
Optionally, if the step (2) of the pre-read flag determining stage is not executed before the step S5302 is executed, when the electronic device executes step S5302, the step (2) of the pre-read flag determining stage may be executed first: and determining the number and the positions of the pre-reading flag bits to be set according to the file type of the file F, the application type of the application A and/or the equipment type of the electronic equipment B. And then setting the pre-reading flag bits in the memory pages with the first numerical value according to the determined number and position of the pre-reading flag bits required to be set.
S5303, reading the memory page with the pre-reading flag bit by the electronic device A;
The electronic device a may read the pre-read memory page, and when the electronic device a reads the memory page with the pre-read flag bit, step S5304 may be triggered.
S5304, the electronic device a performs the next pre-reading.
When the electronic device a reads the memory page with the pre-reading flag bit, the next pre-reading may be triggered.
If the size of the current pre-reading window is not updated, the next pre-reading will pre-read the data of the first numerical memory page in the subsequent data of the file F from the electronic device B, and set a pre-reading flag bit in the first numerical memory page according to the file type of the file F, and/or the application type of the application a, and/or the device type of the electronic device B. And so on, the data of the file F is read out or the application A stops reading the data of the file F.
In steps S5301 to S5304, the electronic device a reads the memory page from the electronic device B in advance and sets the pre-read flag bit according to the determined size of the pre-read window and the number and the positions of the pre-read flag bits that need to be set. In practical applications, if the electronic device a executes step S5005 to update the size of the read-ahead window due to network fluctuation, the electronic device a may execute steps S5305 to S5308 to perform read-ahead according to the updated size of the read-ahead window.
S5305, updating the size of the pre-reading window to be a second value by the electronic device A;
if the electronic device a executes step S5005, the size of the pre-reading window is updated. The updated pre-read window size is assumed to be a second value, which is different from the first value.
S5306, reading the memory page with the pre-reading flag bit by the electronic device A;
after the electronic device a updates the size of the pre-read window to the second value, the electronic device a may continue to read the pre-read memory page, and when the electronic device a reads the memory page with the pre-read flag, step S5307 may be triggered.
S5307, pre-reading, by the electronic device a, data of a second number of memory pages of the file F from the electronic device B;
after the electronic device a updates the size of the pre-reading window to the second numerical value, and when the electronic device a reads the memory page with the pre-reading flag bit, the electronic device a may pre-read data of the second numerical value memory pages of the file F from the electronic device B.
And S5308, setting, by the electronic device A, a pre-reading flag bit in the second memory pages according to the file type of the file F, the application type of the application A, and/or the device type of the electronic device B.
Similar to step S5302, detailed description thereof is omitted.
In some embodiments, steps S5305-S5308 may not be performed, and are not limited herein.
In the embodiment of the present application, the electronic device a may determine the size of the pre-reading window in real time according to the network round-trip delay and the network bandwidth, so that the size of the pre-reading window is adapted to the current network situation, and can change with the change of the network environment, so that the electronic device a pre-reads a sufficient number of memory pages without occupying too much storage space. After the proper memory pages are pre-read, the electronic device a may set, according to different situations of a file type of the file to be accessed, an application type of the application accessing the file, and a device type of the opposite-end electronic device, memory pages in different numbers and locations in the pre-read memory pages. Therefore, the continuity of data reading is guaranteed, the data pre-read next time can be received before the pre-read memory page is read, and excessive storage space is not occupied. The pre-reading performance of the electronic equipment in the distributed file system is greatly improved, and meanwhile, less network resources and storage resources are consumed.
In the above embodiment, the second electronic device is an electronic device located in the same distributed file system as the first electronic device. In some embodiments, the second electronic device may not be an electronic device in the same distributed file system as the first electronic device, which is not limited herein.
The data pre-reading method in the embodiment of the present application is exemplarily described in stages.
As another embodiment of the present application, a method for pre-reading data in the embodiment of the present application is described below from the perspective of actual method execution:
y101, the first electronic equipment acquires a file page of a first numerical value in a first file from the second electronic equipment, and a pre-reading flag bit is set in an Nth file page in the file page of the first numerical value;
the first file is stored in a local file system or an external storage of the second electronic device. When a first application in the first electronic device accesses a first file in the second electronic device, the first electronic device may read data of the first file from a local file system or an external storage of the second electronic device. Due to the paging management mechanism of the memory space, the first electronic device may first read a part of data of the first file into the memory of the second electronic device in units of the size of the memory page, and cache the part of data as the file page of the first numerical value.
Optionally, the first value may be a preset read value, a value determined according to a network delay and a network bandwidth between the first electronic device and the second electronic device, or a value determined according to other parameters (for example, a file type of the first file, an application type of the first application, and the like), which is not limited herein.
After the first electronic device obtains the file page of the first numerical value in the first file from the second electronic device, the first electronic device may set a pre-reading flag bit in an nth file page of the first numerical value. Wherein N may be determined by a file type of the first file, and/or an application type of the first application, and/or a device type of the second electronic device.
In this embodiment, after the first electronic device reads a file page of a certain value in the first file from the second electronic device, the number and the position of the pre-read flag bits set in the file page may be determined according to the file type of the first file, and/or the application type of the first application, and/or the device type of the second electronic device.
The following describes different cases of setting different numbers and positions of pre-read flag bits, taking the setting of the pre-read flag bit in the nth file page of the file pages with the first value as an example:
in some embodiments, the first electronic device may determine whether a file type of the first file is a video file; when the file type of the first file is determined to be a video file, the first electronic device may set a pre-read flag bit in a plurality of file pages of the first value.
It is understood that the plurality of document pages is at least two document pages, and may be each document page, which is not limited herein.
There are various ways for the first electronic device to determine whether the file type of the first file is a video file:
preferably, the first electronic device may determine whether the extension of the first file is an extension of a video file; when the extension of the first file is determined to be the extension of the video file, the first electronic device may determine that the file type of the first file is the video file; when it is determined that the extension of the first file is not the extension of the video file, the first electronic device may determine that the file type of the first file is not the video file.
Optionally, the first electronic device may further determine a file type of the first file according to the metadata information of the first file.
The first electronic device may also have many other ways of determining whether the file type of the first file is a video file, and is not limited herein.
In some embodiments, when determining that the file type of the first file is not a video file, the first electronic device may also determine whether an application type of the first application is a foreground application; when the application type of the first application is determined not to be the foreground application, the first electronic device may set a pre-reading flag bit in an nth 1 file page of the file pages of the first numerical value; the position of the N1 th document page in the first value document page is the first position, wherein N1 is greater than 1/2 of the first value. I.e., the position of the nth 1 file page in the latter half of the first value file page.
There are various ways for the first electronic device to determine whether the application type of the first application is a foreground application:
preferably, the first electronic device may determine whether a foreground application identifier is included in the process of the first application; when determining that the process of the first application includes a foreground application identifier, the first electronic device may determine that the application type of the first application is a foreground application; when it is determined that the process of the first application does not include the foreground application identifier, the first electronic device may determine that the application type of the first application is not the foreground application.
Optionally, the first electronic device may further determine whether the application type of the first application is a foreground application according to data fed back by the GPU.
The first electronic device may also have many other ways of determining whether the application type of the first application is a foreground application, and is not limited herein.
In some embodiments, upon determining that the application type of the first application is a foreground application, the first electronic device may determine whether the device type of the second electronic device is a first capable device; the first capacity equipment is electronic equipment with capacity exceeding a preset capacity value; when the device type of the first electronic device is determined to be a first-capability device, the first electronic device may set a pre-read flag in an nth 2 file page of the file pages of the first numerical value; the position of the Nth 2-th document page in the first-value document page is a second position, and the second position is before the first position; when the device type of the first electronic device is determined not to be the first capability device, the first electronic device may set a pre-read flag in the nth 3 file pages in the file pages of the first numerical value; the position of the Nth 3 file page in the first value file page is a third position, the third position being before the second position, wherein N3 is less than 1/2 of the first value. I.e. the position of the first half of the N3 th document page in the first value document page.
It can be seen that, according to different situations of the file type of the first file, and/or the application type of the first application, and/or the device type of the second electronic device, the first electronic device may set the pre-reading flag bits in different numbers and positions of file pages in the file pages of the first value.
It is understood that, after the subsequent first electronic device reads a certain number of file pages (e.g., a second number of file pages, a third number of file pages, etc.) from the second electronic device, and sets the pre-read flag bits in these file pages, the number and the positions of the pre-read flag bits set in these file pages may also be determined according to the file type of the first file, and/or the application type of the first application, and/or the device type of the second electronic device. Specifically, reference may be made to a process of setting a pre-reading flag in an nth file page of the file pages of the first numerical value, which is not described herein again.
Y102, monitoring the network round trip delay between the first electronic device and the second electronic device;
y103, the first electronic equipment determines the network bandwidth with the second electronic equipment;
y104, the first electronic equipment determines that the size of the current pre-reading window is a second numerical value according to the network round-trip delay and the network bandwidth between the first electronic equipment and the second electronic equipment;
The execution process of steps Y102 to Y104 can refer to steps S5002 to S5004, which are not described herein again.
Y105, when the first application accesses a file page with a pre-reading flag bit, the first electronic equipment obtains a file page with a second numerical value in the first file from the second electronic equipment;
the first electronic device reads the file page of the first numerical value in the first file from the second electronic device, and after the flag bit is set, when the first application accesses the file page with the pre-reading flag bit, the next pre-reading of the first electronic device is triggered. The size of the pre-reading window for next pre-reading is the size of the current pre-reading window calculated in real time according to the network round-trip delay and the network bandwidth. According to step Y104, since the size of the current pre-reading window is the second value, the first electronic device may obtain a file page of the second value in the first file from the second electronic device.
It will be appreciated that the second value of the document page in the first document is the data following the first value of the document page in the first document.
Y106, setting a pre-reading flag bit in the Mth file page in the file pages of the second numerical value in the first file by the first electronic equipment;
the first electronic device may set a pre-reading flag in an mth file page of the file pages of the second value in the first file according to the file type of the first file, and/or the application type of the first application, and/or the device type of the second electronic device. The specific process may refer to the process of setting the pre-reading flag bit in the nth file page of the file of the first value in the first file by the first electronic device in step Y101, which is not described herein again.
Y107, the first electronic device determines whether the difference value between the network round-trip delay when the size of the pre-reading window is calculated to be a second value and the current network round-trip delay reaches a preset change threshold value;
y108, when the preset change threshold is determined to be reached, the first electronic device determines that the size of the current pre-reading window is a third numerical value according to the current network round-trip delay and the network bandwidth;
the step S5005 can be referred to in the execution process of steps Y107 to Y108, and is not described herein again.
Y109, when the first application accesses a file page with a pre-reading flag bit in a file page with a second numerical value, the first electronic equipment obtains the file page with a third numerical value in the first file from the second electronic equipment;
and Y110, setting a pre-reading flag bit in the J-th file page in the file page of the third numerical value in the first file by the first electronic equipment.
In steps Y109 to Y110, when the size of the current pre-reading window is updated to the third value and the first application accesses the file page with the pre-reading flag bit again, the next pre-reading of the first electronic device may be triggered. Because the size of the current pre-reading window is the third numerical value, the first electronic device can obtain the file page of the third numerical value in the first file from the second electronic device, and set the pre-reading flag bit. The detailed process is similar to steps Y105 to Y106, and is not described herein again.
By the data pre-reading method, the pre-reading performance of the electronic equipment when the electronic equipment reads data from other electronic equipment can be improved. In the distributed file system, if the electronic device a and the electronic device C read the data D on the disk of the electronic device B, the data D is cached in the memories of the electronic device a and the electronic device C. If the electronic device a updates the data D to data D', and at this time, the data in the memories of the electronic devices B and C are still D, the data accessed by the electronic devices B and C is not the latest data. Therefore, the electronic device B and the electronic device C also need to update the data D in the memory to the latest data D' to maintain data consistency.
Currently, to maintain data consistency, electronic devices in a distributed file system may use files as granularity to clean up caches in memory. When the data in the file is judged not to be latest, the data cache of the whole file in the memory is completely marked as invalid, and the cache cleaning mode can keep the data consistency in the distributed file system.
However, cleaning the cache in the memory with the file as the granularity will also clean most of the data in the file that is not updated. When the file needs to be read, all the data of the file needs to be loaded again. Not only is the speed of data access reduced in a distributed file system, but also excessive network resources are consumed.
The method for maintaining data consistency provided by the embodiment of the application can improve the data access speed and reduce the consumption of network resources when maintaining the data consistency in the distributed file system.
Fourthly, a method for keeping data consistency:
the following description will first discuss the related terms and concepts of the embodiments of the present application relating to a method for maintaining data consistency.
(1) Distributed file system:
the distributed file system provides a logical file system structure for resources distributed at any position on each electronic device, so that access to shared files distributed on each electronic device is easier.
Optionally, the distributed file system in the embodiment of the present application may be a distributed file system with a central node, and may also be a distributed file system without a central node, which is not limited herein.
Specifically, a distributed file system with a central node generally comprises a metadata server and a plurality of data servers, wherein the metadata server is used as the central node and stores metadata information, and the data servers manage and store actual data of files in the distributed file system. When a user accesses file data, the metadata server serving as a central node needs to be accessed to acquire basic information and data index information of a file, and then the data server is accessed to acquire the file data according to the basic information and the data index information.
Illustratively, FIG. 10 is a schematic diagram of an exemplary scenario of a distributed file system having a central node. The device a, the device B, and the device C constituting the distributed file system having the center node all need to access the files in the data server through the metadata server as the center node. The device a, the device B, and the device C may each serve as a data server, and the metadata server serving as the central node may be one of the electronic devices or may be an independent server.
Specifically, in a distributed file system without a central node, each electronic device manages metadata information of files in other electronic devices. The files in other electronic equipment can be displayed or accessed directly according to the metadata information of the files in other electronic equipment managed by the user, and the user does not need to pass through a central node.
Fig. 12 is a schematic diagram of an exemplary scenario of a distributed file system without a central node in this embodiment. The device a, the device B, the device C, and the device D constituting the distributed file system without the center node manage metadata information of files in other electronic devices, respectively, and can directly display or access files stored in other electronic devices. For example, file a is stored in device a, file B is stored in device B, and file C is stored in device C. The metadata information of the file B and the file C may be stored in the device a, so that the file B or the file C may be directly displayed or accessed. Similarly, the device D may also store metadata information of the file a, the file B, and the file C, so that the file a, the file B, or the file C may be directly displayed or accessed.
(2) Memory page (b):
the operating system manages the memory space by dividing it into pages of the same size, which are called memory pages.
The size of the memory page in the operating system of different electronic devices may be different according to the setting of the operating system, for example, 4KB, 8KB, 16KB, etc.
In an operating system of an electronic device, a memory page of a fixed size is used as a minimum allocation unit of a memory space, and data in the electronic device is managed in the form of the memory page. Therefore, the memory page may be used as a unit of the data size.
The following description relates to memory paging management:
2.1, memory management:
two important concepts in an operating system are the CPU and memory. The CPU is used to execute instructions. The memory is used to store computer executable program code, operating systems, programs, data, and the like. The memory may also be referred to as internal storage.
For memory, it is resource-limited, and if a process occupies a large or even larger memory than the physical memory, and multiple processes are executed simultaneously, it needs to involve memory management.
Ideally, the memory expectation of the user is large capacity, high speed and durability, but in reality, the memory is a memory architecture composed of a cache, a main memory and a disk. In the architecture, the cache has low capacity, high speed and high cost, the main memory has medium speed, medium capacity and medium cost, and the disk has high capacity, durability and low speed.
Memory management is to provide a uniform abstraction for "users" and to shield the differences between cache, main memory and disk, even to make them imperceptible. The user does not need to worry about whether the program is stored in a cache, a main memory or a disk, and only needs to operate and output the same result.
When the operating system executes programs of a plurality of processes simultaneously, the programs need to be ensured not to interfere with each other, that is, one process cannot access the memory space of another process. When a program reads and writes specific memory data, the specific memory data cannot be directly mapped to the physical memory, that is, the memory address issued by the program and the physical memory need to be independent.
The use of virtual memory may satisfy these memory management needs.
2.2, virtual memory:
virtual memory is an important milestone in the operating system development history. The use of the virtual memory avoids the direct interaction between the program and the physical memory, and the uniform abstraction is made for the cache, the main memory and the disk, so that the program can break through the size limitation of the physical memory, and the program is still limited by the size of the virtual memory.
After the virtual memory is used, the memory address seen by the program is the virtual memory address, and the reading and writing of the virtual memory address by the program are mapped into the actual physical memory, and the mapping is called translation. The translation is performed by a Memory Management Unit (MMU), the MMU receives a virtual address sent by the CPU, translates the virtual address into a physical address, and sends the physical address to the memory, and the memory performs corresponding access according to the physical address and then reads or writes related data.
Fig. 40 is a schematic diagram illustrating a relationship between an application program, a virtual memory, and a physical memory in an embodiment of the present application. Physical memory interfaces of physical memories such as a cache, a main memory and a hard disk in the electronic equipment can be abstracted into a virtual memory uniformly, and a virtual memory address is provided for an application program in the electronic equipment on the virtual machine interface. The user can map the virtual memory address read and write to the actual physical memory through the application program.
2.3, memory space paging management:
the core of paging management of memory space is to divide the virtual memory space and the physical memory space into pages with the same size, and use the pages as the minimum allocation unit of the memory space, and one page of a program can be stored in any one physical page.
Through paging management of the memory space, mapping of a continuous virtual space to an unnecessarily continuous physical memory can be realized. Therefore, continuous data segments in the process can be stored in the discontinuous physical memory, and the management of the memory space is more flexible.
FIG. 41 is a diagram illustrating memory paging management according to an embodiment of the present application. The paging mechanism may split data into memory page sizes, allocating contiguous addresses in the virtual address space. These memory pages that are consecutive in address in the virtual address space may be scattered over the unallocated pages in the physical address space.
In some embodiments of the present application, a memory page may also be referred to as a page for short, which is not described in detail later.
In some embodiments, the memory page used for caching the file may also be referred to as a file page, which is not limited herein.
(3) Radix tree:
the file data exists in the memory in the form of a Page (Page), for example, the size of each Page of the memory may be 4KB, and the Page of the memory is managed by a Page Cache (Page Cache) mechanism. The page cache uses a data structure of a Radix Tree (Radix Tree) to organize the pages of a certain file, namely, each file in the page cache has a Radix Tree belonging to the page cache.
Illustratively, FIG. 54 is a radix tree of file F in memory. The root (radix tree root) of the radix tree contains the height information (height) of the tree, each node (radix tree node) of the tree contains 64 pointers, and each pointer can be invalid or point to the node or page of the next level tree. In addition, each node of the tree contains the number (count) of valid pointers in the 64 pointers of that node. As shown in fig. 54, the tree root of the radix tree of the file F indicates that the height information of the radix tree is 2 and points to the node of the first level of the radix tree.
The node at the first level of the radix tree indicates that the node contains 2 valid pointers:
1. The valid pointer slots [0] at position 0 points to a node in the second level of the radix tree that also contains 2 valid pointers. The pointer slots [0] at the position 0 points to the memory page (page) with the index (index) of the file F being 0; the pointer slots [4] at location 4 points to the memory page of file F, index 4.
2. The valid pointer slots [2] at position 2 points to another node at the second level of the radix tree, which contains 1 valid pointer. Pointer slots [3] at location 3 points to the memory page of file F, indexed 131.
In this embodiment of the present application, if data in a memory page is updated, the page may be marked as dirty. When the number of dirty pages exceeds a certain proportion, the disk can be brushed back. Before the page is back-flushed, the data in the page and the data in the disk are inconsistent, and the latest data can be read because the system always accesses the data in the memory first.
FIG. 55 is a diagram illustrating an exemplary scenario of a method for maintaining data consistency according to the prior art. Electronic device a, electronic device B, and electronic device C constitute a distributed file system. The file F is stored in a disk of the electronic device B. The electronic device A, the electronic device B and the electronic device C access the file F, and the file F is read from a magnetic disk of the electronic device B to the memory of the electronic device B. If so, the electronic device a modifies the file F to a file F' in the memory. In order to maintain data consistency, the electronic device B and the electronic device C need to directly discard the file F in the memory, that is, all the radix trees of the file F in the memory are set to be invalid. And then after the electronic equipment A writes the modified file F ' back to the disk of the electronic equipment B, the electronic equipment B and the electronic equipment C read the modified file F ' from the disk of the electronic equipment B into the memory of the electronic equipment B, and the radix tree of the file F ' is reestablished.
Thus, the cache in the memory is cleaned by taking the file as granularity, and most of the data which is not updated in the file is also cleaned. When the file needs to be read, all the data of the file needs to be loaded again. Not only is the speed of data access reduced in a distributed file system, but also excessive network resources are consumed.
By adopting the method for keeping the data consistency in the embodiment of the application, the data of the size of the memory page is directly updated on the radix tree of the file in the memory, and the data (namely the whole radix tree) of the whole file cached in the memory does not need to be deleted and reloaded. Only the data of the corresponding memory pages need to be updated on the radix tree, so that the data access speed is greatly increased and the consumption of network resources is reduced while the data consistency is maintained in the distributed file system.
Fig. 56 is a schematic diagram of an exemplary scenario of a method for maintaining data consistency in an embodiment of the present application. The file F on the disk in the electronic device B is divided into pages of memory size, and may be regarded as data with the page sizes of F1-fd, which includes data fx. The electronic device a, the electronic device B, and the electronic device C all read the data of F1 to fx of the file F from the disk of the electronic device B and cache the data in their own memories. If so, the electronic device a modifies the file F in the memory, specifically, modifies the data fx on a certain memory page in the radix tree of the file F in the memory to fx'. In order to maintain data consistency, the electronic device a notifies the electronic device B of the modification, and after receiving the notification, the electronic device B does not need to discard the entire radix tree of the file F in the memory, and only needs to modify the data in the memory page in which the data fx is stored on the radix tree of the file F in the memory to fx'. The electronic device B determines that the electronic device C has also read the data fx, and in order to maintain data consistency, the electronic device B may notify the electronic device C that the data fx has failed. The electronic device C may discard the data fx of the memory page size on the radix tree of the file F in its memory, that is, the memory page storing the data fx is marked as dirty, and the whole radix tree of the file F does not need to be discarded. After a while, the electronic device B may write back the modified data fx' to the disk, and update the data fx in the file F stored in the disk.
It is understood that, in the embodiment of the present application, a magnetic disk is taken as an example, and in some embodiments, the electronic device may also use other nonvolatile memory to store files, which is not limited herein.
In conjunction with the description of the hardware structure diagram of the electronic device 100 shown in fig. 13, in some embodiments of the present application, the electronic device 100 may use the mobile communication module 150 or the wireless communication module 160 to form a distributed file system with other electronic devices. The processor 110 may invoke computer instructions stored in the internal memory 121 to cause the electronic device 100 to perform the method of maintaining data consistency in the embodiments of the present application.
In conjunction with the description of the software structure diagram of the electronic device 100 shown in fig. 14, as shown in fig. 57, in some embodiments of the present application, the application layer may further include a data consistency module.
The data consistency module can be used for executing the method for maintaining the data consistency in the embodiment of the application.
In some embodiments, the data consistency module may also be located at other levels in the hierarchical architecture, such as an application framework layer, a system library or kernel layer, and the like, which are not limited herein.
In the following, by taking the example that the electronic device a accesses or modifies the file F in the electronic device B in the distributed file system composed of the electronic device a, the electronic device B, and the electronic device C, the method for maintaining data consistency in the embodiment of the present application is specifically described with reference to the software and hardware structures of the above-mentioned exemplary electronic device 100:
fig. 58 is a schematic signaling interaction diagram of a method for maintaining data consistency in the embodiment of the present application.
S5801, the electronic device A receives a request for accessing data fx of a file F in the electronic device B;
the data fx is data of a memory page size offset by x in the file F. The file F is divided into data of N memory page sizes, and if the offset of the data of the first memory page size of the file F is set to 0, the offset of the data of the second memory page size of the file F is 1, the offset of the data of the third memory page size of the file F is 2, and so on, the offset of the data of the nth memory page size of the file F is N-1.
In some embodiments, if the offset of the data of the first memory page size of the file F is set to 1, the offset of the data of the second memory page size of the file F is 2, the offset of the data of the third memory page size of the file F is 3, and so on, the offset of the data of the nth memory page of the file F is N.
The specific offset may be set according to actual conditions, and is not limited herein.
In some embodiments of the present application, the data fx may also be referred to as first data.
S5802, the electronic device A searches whether the data fx is cached in the radix tree of the file F or not in the local memory;
it can be understood that when the electronic device a accesses a file, the cached file is stored in the memory in the form of a radix tree. The description of the radix tree can refer to the above-mentioned terminology to introduce a (3) radix tree, and will not be described herein.
After receiving a request for accessing the data fx in the file F in the electronic device B, the electronic device a may search in the local memory whether the data fx has been cached in the radix tree of the file F;
when it is determined that there is no cache data fx, the electronic apparatus a may perform step S5803.
When determining that the data fx has been cached, the electronic apparatus a may perform step S5807.
S5803, the electronic device A sends a request for reading data fx of the file F in the electronic device B to the electronic device B;
when electronic device a determines that data fx is not cached in the radix tree of file F in local memory, electronic device a may send a request to electronic device B to read data fx of file F in electronic device B.
The request may include: the command number of the read instruction, the device identification of electronic device a, the path to access the file (i.e., the path of file F), and the offset of the data to access in the file (i.e., offset x).
S5804, reading the data fx of the local file F from the disk by the electronic equipment B to a radix tree of the file F in the memory for caching;
after receiving a request sent by the electronic device a to read the data fx of the file F in the electronic device B, the electronic device B may read the data fx of the local file F from the disk to the radix tree of the file F in the memory for caching. Step S5805 is then performed.
S5805, recording the corresponding relation between the data fx and the equipment identification of the electronic equipment A by the electronic equipment B;
each electronic device in the distributed file system may store a local data reading record, and the local data reading record may be used to record a corresponding relationship between data of the file read by the other electronic devices in the distributed file system from the electronic device.
After the electronic device B receives the request sent by the electronic device a to read the data fx of the file F in the electronic device B, the electronic device B may record the correspondence between the data fx and the device identifier of the electronic device a in its local data reading record according to the request.
Optionally, the electronic device B may cache the local data reading record in the memory.
Illustratively, table 7 below is an exemplary illustration of the local data read record stored in the electronic device B:
device identification of electronic device Read data
Electronic equipment A f1、f2、f3、f4、fx
Electronic equipment C f1、f2、f3、f4、fx、f10、f20
TABLE 7
Optionally, the device identifier of the electronic device may be an identifier, such as a device number, a MAC address, an IP address, or a device name of the electronic device, that can uniquely identify one electronic device in the distributed file system, which is not limited herein.
It is understood that table 7 is only an example of the local data reading record, and in practical applications, the local data reading record may also be other data types or data structures, such as a matrix, an array, a database, etc., and is not limited herein.
S5806, the electronic device B returns a response, and the response includes the data fx;
after the electronic device B records the correspondence between the data fx and the device identifier of the electronic device a in its local data reading record, it may return a response to the electronic device a, where the response includes the data fx requested in the request.
Optionally, the response may include the same number as the command number of the read instruction in the request, to indicate that the response is a response to the request.
S5807, the electronic device A reads data fx from the radix tree of the file F;
when the electronic device a determines that the data fx is cached in the radix tree of the file F in the local memory, the electronic device a may directly read the data fx from the radix tree of the file F, and may not need to perform the above steps S5804 to S5806.
In steps S5806 to S5807, the electronic device a reads the data fx of the file F upon request. Then, in the following steps, the electronic device a may modify the data fx and maintain consistency of the data in the distributed file system.
S5808, modifying the data fx in the radix tree of the file F in the local memory into data fx' by the electronic device A;
after the electronic device a reads the data fx, in response to a user operation or in response to processing of an application program in the electronic device a, the electronic device a may modify the data fx in the radix tree of the file F in the local memory to be the data fx'.
In some embodiments of the present application, the data fx' may also be referred to as second data.
For example, file F is a document and electronic device A modifies some consecutive characters in the document.
S5809, delaying the preset time by the electronic device A;
after the electronic device a modifies the data fx in the radix tree of the file F in the local memory to be the data fx', a preset time duration may be delayed and then the step S5810 is performed.
Therefore, the subsequent data updating step can be prevented from being frequently triggered when the data in the memory is continuously modified.
S5810, the electronic equipment A sends a request for rewriting data fx of the file F in the electronic equipment B into data fx' to the electronic equipment B;
after the electronic device a modifies the data fx in the radix tree of the file F in the local memory to the data fx ', and when no other data modification is received after delaying a preset time, the electronic device a may send a request to the electronic device B to rewrite the data fx of the file F in the electronic device B to the data fx'.
It is understood that the request for rewriting the data fx of the file F in the electronic device B into the data fx' may include a device identifier of the electronic device a, a path of the file F, an offset of the data fx in the file F, and the like, which is not limited herein.
S5811, the electronic device B rewrites data fx in the radix tree of the file F in the local memory into data fx';
in response to a request for rewriting data fx of the file F to data fx ', the electronic device B may rewrite the data fx in the radix tree of the file F in the local memory to data fx'.
It can be understood that the electronic device B only needs to rewrite, as the data fx', the data on the memory page where the data fx is located in the radix tree of the file F in the local memory.
It can be understood that, in response to the request sent by the electronic device a to rewrite the data fx of the file F into the data fx ', the electronic device B may also modify the read record of the electronic device a reading the data fx from its local data read record into the read record of the electronic device a reading the data fx'.
For example, if the previous local data reading record in the electronic device B is shown in table 7 above, the electronic device B may update the local data reading record to be shown in table 8 below:
device identification of electronic device Read data
Electronic equipment A f1、f2、f3、f4、fx'
Electronic equipment C f1、f2、f3、f4、fx、f10、f20
TABLE 8
S5812, the electronic device B determines that the data fx is read by the electronic device C;
after the electronic device B responds to the request of the electronic device B and modifies the certain data in the memory, the electronic device B may determine, according to the correspondence between the data reads stored in the local data read record of the electronic device B, which electronic devices in the distributed file system have read the data, except the electronic device from which the data is modified.
When it is determined that the other electronic device has read the data, step S5813 may be performed;
when it is determined that no other electronic device has read the data, step S5815 may be directly performed.
For example, if the local data reading record of the electronic device B is shown in table 8 above. Electronic device B can determine that the data fx has also been read by electronic device C. Step S5813 may be performed.
S5813, the electronic device B sends a notice that the marking data fx are invalid to the electronic device C;
after the electronic device B rewrites the data fx in the memory to fx' and determines that the data fx has been read by the electronic device C, the electronic device B may send a notification to the electronic device C that the flag data fx is invalid.
It is to be understood that if the electronic device B determines that the data fx is also read by other electronic devices in the distributed file system, the electronic device B may also send a notification to the other electronic devices that the flag data fx is invalid.
S5814, the electronic device C clears the cache of the data fx on the radix tree of the file F in the local memory;
and the electronic equipment C responds to the received notice that the marking data fx is invalid and clears the cache of the data fx on the base tree of the file F in the local memory.
It can be understood that the electronic device C clears the cache of the data fx in the radix tree of the file F in the local memory, and may specifically mark the memory page where the data fx in the radix tree of the file F is located as dirty.
S5815, the electronic device B deletes the reading information of the electronic device C on the data fx;
after the electronic device B sends the notification that the flag data fx is invalid to the electronic device C, the electronic device B may delete the read information of the data fx by the electronic device C in its local data read record.
For example, if the previous local data reading record in the electronic device B is shown in table 8 above, the electronic device B may update the local data reading record to be shown in table 9 below:
device identification of electronic device Read data
Electronic equipment A f1、f2、f3、f4、fx'
Electronic equipment C f1、f2、f3、f4、f10、f20
TABLE 9
S5816, the electronic device B modifies fx of the file F stored in the disk to fx'.
After the electronic device B modifies the data fx of the file F in the memory to fx ', and after the preset write-back duration, the electronic device B may modify the data fx of the file F stored in the disk to fx'.
It is understood that if there are other modifications to the data in the file F in the memory within the preset write-back time, the data can also be written back to the file F on the disk together.
In this embodiment, when the electronic device a modifies the data fx of the file in the electronic device B in the memory, the electronic device B may directly update the data fx of the memory page size on the radix tree of the file F in the memory, and it is not necessary to delete and reload the data of the entire file F (i.e., the whole radix tree) cached in the memory. The data of the corresponding memory page only needs to be updated on the radix tree, and a notification that the flag data fx is invalid can also be directly sent to other electronic devices in the distributed file system, so that the received electronic device only needs to set the memory page where fx is located in the radix tree of the file F as invalid, and the data of the whole file F cached in the memory does not need to be deleted and reloaded. When the data consistency is kept in the distributed file system, the data access speed is greatly improved, and the consumption of network resources is reduced.
It is to be understood that the electronic device in the foregoing embodiments of the present application may be a mobile electronic device, and may also be other types of electronic devices, such as a television, a computer, an intelligent printer, and the like, which are not limited herein.
For example, fig. 59 is another schematic structural diagram of an exemplary electronic device 200 provided in an embodiment of the present application.
The electronic device 200 may include: an input device 5901, an output device 5902, a processor 5903, and a memory 5904 (where the number of processors 5903 in the electronic apparatus 200 may be one or more, one processor 5903 is exemplified in fig. 59). In some embodiments of the present application, the input device 5901, the output device 5902, the processor 5903, and the memory 5904 may be connected by a bus or other means, wherein the connection by a bus is illustrated in fig. 59.
In some embodiments of the present application, the processor 5903 may cause the electronic device 200 to execute the metadata management method in the embodiments of the present application by calling an operation instruction stored in the memory 5904. The specific process is similar to the above-mentioned metadata management method executed by the electronic device 100 in the embodiment of the present application, and is not described herein again;
in some embodiments of the present application, the processor 5903 may cause the electronic device 200 to execute the file identification method of the same name in the embodiments of the present application by calling the operation instructions stored in the memory 5904. The specific process is similar to the above-mentioned electronic device 100 executing the method for identifying files with the same name in the embodiment of the present application, and is not described herein again;
In some embodiments of the present application, the processor 5903 may cause the electronic device 200 to execute the data pre-reading method in the embodiment of the present application by calling an operation instruction stored in the memory 5904. The specific process is similar to the above-mentioned electronic device 100 executing the data pre-reading method in the embodiment of the present application, and is not described herein again;
in some embodiments of the present application, the processor 5903 may cause the electronic device 200 to perform the method for maintaining data consistency in the embodiments of the present application by calling the operation instructions stored in the memory 5904. The specific process is similar to the method for the electronic device 100 to perform the data consistency keeping in the embodiment of the present application, and is not described here again.
In the above embodiments, the electronic device a, the electronic device B, and the electronic device C may also be referred to as a first electronic device, a second electronic device, a third electronic device, or the like, and are not limited herein.
For example, in some embodiments, electronic device a may be a first electronic device, electronic device B may be a second electronic device, and electronic device C may be a third electronic device; in some embodiments, electronic device a may be a first electronic device, electronic device B may be a third electronic device, and electronic device C may be a second electronic device; in some embodiments, the electronic device a may be a first electronic device, the electronic device B may be a second electronic device, and the electronic device C may also be a second electronic device; in some embodiments, electronic device a may be a second electronic device, electronic device B may be a first electronic device, and electronic device C may be a third electronic device; in some embodiments, electronic device a may be a second electronic device, electronic device B may be a third electronic device, and electronic device C may be a first electronic device; in some embodiments, the electronic device a may be a second electronic device, the electronic device B may be a first electronic device, and the electronic device C may also be a second electronic device; in some embodiments, the electronic device a may be a second electronic device, the electronic device B may also be a second electronic device, and the electronic device C may be a first electronic device; in some embodiments, electronic device a may be a third electronic device, electronic device B may be a first electronic device, and electronic device C may be a second electronic device; in some embodiments, electronic device a may be a third electronic device, electronic device B may be a second electronic device, and electronic device C may be a first electronic device; there may be other correspondences between the electronic device a, the electronic device B, and the electronic device C and the first electronic device, the second electronic device, or the third electronic device, and the correspondence is not limited herein.
It is to be understood that the steps executed by each electronic device in the foregoing embodiments are only limited to specific scenarios, and each electronic device may have the capability of executing any step in the foregoing embodiments, and is not limited herein.
It is to be understood that, according to the requirements of an actual scenario, the electronic device in the embodiment of the present application may perform any one or more of the above-described metadata management method, the same-name file identification method, the data pre-reading method, and the method for maintaining data consistency, which is not limited herein.
For example, in conjunction with the description of the software structure diagram of the electronic device 100 shown in fig. 14, as shown in fig. 60, another software structure framework diagram of the electronic device in the embodiment of the present application is shown.
The application program layer of the electronic equipment can further comprise a metadata management module, a file identification module with the same name and a data consistency module. The inner core layer of the electronic device can also comprise a data pre-reading module.
The metadata management module can be used for executing the metadata management method in the embodiment of the application;
the homonymous file identification module is used for executing a homonymous file identification method in the embodiment of the application;
The data consistency module is used for executing the method for maintaining the data consistency in the embodiment of the application;
the data pre-reading module is used for executing the data pre-reading method in the embodiment of the application.
In some embodiments, the metadata management module, the homonym file identification module, the data consistency module, and the data pre-reading module may also be respectively located at other levels of the hierarchical architecture, which is not limited herein.
The above embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present application.
As used in the above embodiments, the term "when …" may be interpreted to mean "if …" or "after …" or "in response to determination …" or "in response to detection …", depending on the context. Similarly, depending on the context, the phrase "at the time of determination …" or "if (a stated condition or event) is detected" may be interpreted to mean "if the determination …" or "in response to the determination …" or "upon detection (a stated condition or event)" or "in response to detection (a stated condition or event)".
In the above embodiments, all or part of the implementation may be realized by software, hardware, firmware, or any combination thereof. When implemented in software, it may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. The procedures or functions described in accordance with the embodiments of the application are all or partially generated when the computer program instructions are loaded and executed on a computer. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (e.g., coaxial cable, fiber optic, digital subscriber line) or wirelessly (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that includes one or more available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk), among others.
One of ordinary skill in the art will appreciate that all or part of the processes in the methods of the above embodiments may be implemented by hardware related to instructions of a computer program, which may be stored in a computer-readable storage medium, and when executed, may include the processes of the above method embodiments. And the aforementioned storage medium includes: various media capable of storing program codes, such as ROM or RAM, magnetic or optical disks, etc.

Claims (28)

1. A metadata management method, comprising:
the method comprises the steps that a first electronic device comprises a first local file and a second local file in a first directory in a first local file system of the first electronic device, wherein the second local file comprises metadata information of the first local file;
when second electronic equipment receives a request for accessing a first directory in a second local file system of the second electronic equipment, the second electronic equipment sends the request for obtaining a second local file by a distributed file system to the first electronic equipment;
the first electronic equipment sends the second local file to the second electronic equipment;
the second electronic device caches the second local file in a first directory in a second local file system, and displays the first local file in the first directory in the second local file system according to the second local file
Wherein, the metadata information comprises at least one item of file name, file type, file size, modification time and access authority.
2. A metadata management method, comprising:
the method comprises the steps that a first electronic device comprises a first local file and a second local file in a first directory in a first local file system of the first electronic device, wherein the second local file comprises metadata information of the first local file;
when the first electronic device receives a request, sent by a second electronic device, for acquiring the second local file from a distributed file system, the first electronic device sends the second local file to the second electronic device, so that the first local file is displayed in a first directory in the second local file system of the second electronic device;
wherein, the metadata information comprises at least one item of file name, file type, file size, modification time and access authority.
3. The method of claim 2, further comprising:
when the first electronic device adds a third local file in the first directory in the first local file system, the first electronic device adds metadata information of the third local file in the second local file.
4. A method according to claim 2 or 3, characterized in that the method further comprises:
when the first electronic equipment deletes the first local file, the first electronic equipment deletes the metadata information of the first local file in the second local file.
5. The method according to any one of claims 2 to 4, further comprising:
when the first electronic equipment modifies the first local file, the first electronic equipment updates the metadata information of the first local file in the second local file according to the modification of the first local file.
6. The method of any of claims 2 to 5, wherein the first directory is one of a second directory; the method further comprises the following steps:
the first electronic device receives a request, sent by the second electronic device, for acquiring a fourth local file in a second directory in the first local file system by using the distributed file system; the fourth local file comprises metadata information of a first directory in the first local file system;
the first electronic equipment sends the fourth local file to the second electronic equipment;
The first electronic equipment receives a request, sent by the second electronic equipment, for acquiring the second local file in a first directory in the first local file system by using the distributed file system;
the first electronic equipment sends the second local file to the second electronic equipment; the second local file comprises metadata information of a fifth local file in a first directory in the first local file system;
and the first electronic equipment receives the access of the second electronic equipment to the fifth local file.
7. The method of any of claims 2 to 5, wherein the first directory is one of a second directory; the method further comprises the following steps:
the first electronic equipment receives a path of a fifth local file sent by the second electronic equipment; the fifth local file is a file in a first directory in the first local file system;
when the first electronic device determines that the fifth local file exists in the first directory in the first local file system, the first electronic device sends the second local file to the second electronic device; the second local file comprises metadata information of the fifth local file;
And the first electronic equipment receives the access of the second electronic equipment to the fifth local file.
8. The method according to any one of claims 2 to 7, further comprising:
the first electronic equipment receives an updated first local file sent by the second electronic equipment;
the first electronic equipment updates the second local file according to the updated first local file; the updated second local file comprises the metadata information of the updated first local file.
9. The method according to any one of claims 2 to 8, wherein the first electronic device includes a first read record, and the first read record includes a record of the second electronic device requesting the first electronic device to obtain a local file including metadata information.
10. The method according to any one of claims 2 to 9, further comprising:
and when the first electronic equipment sends the second local file to the second electronic equipment and the first electronic equipment updates the second local file, the first electronic equipment sends the updated second local file to the second electronic equipment.
11. A metadata management method, comprising:
when second electronic equipment receives a request for accessing a first directory in a second local file system of the second electronic equipment, the second electronic equipment sends a request for acquiring a second local file by a distributed file system to the first electronic equipment; the second local file comprises metadata information of a first local file in a first directory in a first local file system of the first electronic device;
the second electronic equipment receives the second local file sent by the first electronic equipment and caches the second local file in a first directory in a second local file system;
the second electronic equipment displays the first local file in a first directory in the second local file system according to the second local file
Wherein, the metadata information comprises at least one item of file name, file type, file size, modification time and access authority.
12. The method of claim 11, wherein the first directory is one of the second directories; the method further comprises the following steps:
when the second electronic device receives a request for accessing a fifth local file in a first directory in a second directory in a distributed file system, the second electronic device sends the request for obtaining a fourth local file in the second directory in the first local file system to the first electronic device; the fourth local file comprises metadata information of the first directory;
When the second electronic device determines that the received fourth local file contains the metadata information of the first directory, the second electronic device sends a request for acquiring a second local file in the first directory in the first local file system by the distributed file system to the first electronic device;
the second electronic equipment accesses the fifth local file according to the received second local file; the second local file includes metadata information of the fifth local file.
13. The method of claim 11, wherein the first directory is one of a second directory; the method further comprises the following steps:
when the second electronic device receives a request for accessing a fifth local file in a first directory in a second directory in a distributed file system, the second electronic device sends a path of the fifth local file to the first electronic device; the fifth local file is a file in a first directory in the first local file system;
and when the second electronic equipment receives the second local file sent by the first electronic equipment, the second electronic equipment accesses the fifth local file according to the second local file.
14. The method according to any one of claims 11 to 13, further comprising:
the second electronic device caches the first local file in a first directory in the second local file system;
and the second electronic equipment updates the first local file and sends the updated first local file to the first electronic equipment.
15. The method according to any one of claims 11 to 14, further comprising:
after the second electronic device receives the second local file sent by the first electronic device, the second electronic device receives the updated second local file sent by the first electronic device;
and the second electronic equipment displays the updated file in the first directory in the first local file system in the first directory in the second local file system according to the updated second local file.
16. The method according to any one of claims 11 to 15, further comprising:
and after the distributed file system stops mounting the second local file system, the second electronic equipment deletes the second local file cached in the first directory in the second local file system.
17. A first electronic device, wherein the first electronic device comprises: one or more processors and memory;
the memory coupled with the one or more processors, the memory to store computer program code, the computer program code including computer instructions, the one or more processors to invoke the computer instructions to cause the first electronic device to perform:
the method comprises the steps that a first local file and a second local file are included in a first directory in a first local file system of a first electronic device, and metadata information of the first local file is included in the second local file;
when a request of acquiring a second local file from a distributed file system sent by second electronic equipment is received, sending the second local file to the second electronic equipment, so that the first local file is displayed in a first directory in the second local file system of the second electronic equipment;
wherein, the metadata information comprises at least one item of file name, file type, file size, modification time and access authority.
18. The first electronic device of claim 17, wherein the first directory is one of a second directory; the one or more processors are further to invoke the computer instructions to cause the first electronic device to perform:
Receiving a request sent by the second electronic device that the distributed file system acquires a fourth local file in a second directory in the first local file system; the fourth local file comprises metadata information of a first directory in the first local file system;
sending the fourth local file to the second electronic device;
receiving a request sent by the second electronic device that the distributed file system acquires the second local file in the first directory in the first local file system;
sending the second local file to the second electronic device; the second local file comprises metadata information of a fifth local file in a first directory in the first local file system;
and receiving the access of the second electronic equipment to the fifth local file.
19. The first electronic device of claim 17, wherein the first directory is one of a second directory; the one or more processors are further to invoke the computer instructions to cause the first electronic device to perform:
receiving a path of a fifth local file sent by the second electronic equipment; the fifth local file is a file in a first directory in the first local file system;
When the fifth local file exists in the first directory in the first local file system, sending the second local file to the second electronic device; the second local file comprises metadata information of the fifth local file;
and receiving the access of the second electronic equipment to the fifth local file.
20. A chip system for application to a first electronic device, the chip system comprising one or more processors for invoking computer instructions to cause the first electronic device to perform the method of any of claims 2-11.
21. A computer program product comprising instructions for causing a first electronic device to perform the method according to any one of claims 2-11 when the computer program product is run on the first electronic device.
22. A computer-readable storage medium comprising instructions that, when executed on a first electronic device, cause the first electronic device to perform the method of any of claims 2-11.
23. A second electronic device, characterized in that the second electronic device comprises: one or more processors and memory;
The memory coupled with the one or more processors, the memory to store computer program code, the computer program code including computer instructions, the one or more processors to invoke the computer instructions to cause the second electronic device to perform:
when a request for accessing a first directory in a second local file system of second electronic equipment is received, sending a request for acquiring a second local file by a distributed file system to the first electronic equipment; the second local file comprises metadata information of a first local file in a first directory in a first local file system of the first electronic device;
receiving the second local file sent by the first electronic device, and caching the second local file into a first directory in a second local file system;
displaying the first local file in a first directory in the second local file system according to the second local file;
wherein, the metadata information comprises at least one item of file name, file type, file size, modification time and access authority.
24. The second electronic device of claim 23, wherein the first directory is one of a second directory; the one or more processors are further to invoke the computer instructions to cause the second electronic device to perform:
When a request for accessing a fifth local file in a first directory in a second directory in a distributed file system is received, sending a request for obtaining a fourth local file in the second directory in the first local file system from the distributed file system to the first electronic device; the fourth local file comprises metadata information of the first directory;
when the received fourth local file is determined to contain the metadata information of the first directory, sending a request of a distributed file system for acquiring a second local file in the first directory in the first local file system to the first electronic device;
accessing the fifth local file according to the received second local file; the second local file comprises metadata information of the fifth local file.
25. The second electronic device of claim 23, wherein the first directory is one of a second directory; the one or more processors are further to invoke the computer instructions to cause the second electronic device to perform:
when a request for accessing a fifth local file in a first directory in a second directory in a distributed file system is received, sending a path of the fifth local file to the first electronic device; the fifth local file is a file in a first directory in the first local file system;
And when the second local file sent by the first electronic equipment is received, accessing the fifth local file according to the second local file.
26. A chip system for application to a second electronic device, the chip system comprising one or more processors configured to invoke computer instructions to cause the second electronic device to perform the method of any one of claims 12-18.
27. A computer program product comprising instructions for causing a second electronic device to perform the method according to any of claims 12-18 when the computer program product is run on the second electronic device.
28. A computer-readable storage medium comprising instructions that, when executed on a second electronic device, cause the second electronic device to perform the method of any of claims 12-18.
CN202210259434.8A 2020-09-30 2020-11-30 Metadata management method and electronic equipment Pending CN114722021A (en)

Applications Claiming Priority (9)

Application Number Priority Date Filing Date Title
CN2020110629034 2020-09-30
CN202011066342 2020-09-30
CN202011062903 2020-09-30
CN2020110628879 2020-09-30
CN202011063023 2020-09-30
CN2020110630239 2020-09-30
CN202011062887 2020-09-30
CN2020110663425 2020-09-30
CN202011377326.8A CN114328377A (en) 2020-09-30 2020-11-30 Metadata management method and electronic equipment

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
CN202011377326.8A Division CN114328377A (en) 2020-09-30 2020-11-30 Metadata management method and electronic equipment

Publications (1)

Publication Number Publication Date
CN114722021A true CN114722021A (en) 2022-07-08

Family

ID=80949699

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210259434.8A Pending CN114722021A (en) 2020-09-30 2020-11-30 Metadata management method and electronic equipment

Country Status (2)

Country Link
CN (1) CN114722021A (en)
WO (1) WO2022068596A1 (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117493276B (en) * 2024-01-03 2024-04-09 柏科数据技术(深圳)股份有限公司 Reading method and device for Ceph file, server and storage medium

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160224650A1 (en) * 2015-02-02 2016-08-04 Synology Incorporated Method for performing file synchronization control, and associated apparatus
CN107451139A (en) * 2016-05-30 2017-12-08 北京三星通信技术研究有限公司 File resource methods of exhibiting, device and corresponding smart machine
CN109697016A (en) * 2017-10-20 2019-04-30 伊姆西Ip控股有限责任公司 Method and apparatus for improving the storage performance of container

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101354726B (en) * 2008-09-17 2010-09-29 中国科学院计算技术研究所 Method for managing memory metadata of cluster file system
CN102110146B (en) * 2011-02-16 2012-11-14 清华大学 Key-value storage-based distributed file system metadata management method
CN103559188B (en) * 2013-08-19 2016-12-28 曙光信息产业股份有限公司 Metadata management method and management system
US10108820B2 (en) * 2015-01-20 2018-10-23 Mediatek Inc. Snapshot data and hibernation data processing methods and devices
CN108920613A (en) * 2018-06-28 2018-11-30 郑州云海信息技术有限公司 A kind of metadata management method, system and equipment and storage medium
CN111030784A (en) * 2019-11-13 2020-04-17 泰康保险集团股份有限公司 Information synchronization method and device

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160224650A1 (en) * 2015-02-02 2016-08-04 Synology Incorporated Method for performing file synchronization control, and associated apparatus
CN107451139A (en) * 2016-05-30 2017-12-08 北京三星通信技术研究有限公司 File resource methods of exhibiting, device and corresponding smart machine
CN109697016A (en) * 2017-10-20 2019-04-30 伊姆西Ip控股有限责任公司 Method and apparatus for improving the storage performance of container

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
章隆兵等: "基于分布式共享存储系统的并行文件子系统DPFS", 《计算机研究与发展》 *

Also Published As

Publication number Publication date
WO2022068596A1 (en) 2022-04-07

Similar Documents

Publication Publication Date Title
WO2021083378A1 (en) Method for accelerating starting of application, and electronic device
WO2022257748A1 (en) Virtual memory management method and electronic device
US8504774B2 (en) Dynamic cache configuration using separate read and write caches
CN111143586B (en) Picture processing method and related device
CN115757193B (en) Memory management method and electronic equipment
JP7201827B2 (en) Image classification method and electronic device
WO2022100222A1 (en) Information retrieval method, apparatus and system, and storage medium
CN111615820B (en) Method and equipment for performing domain name resolution by sending key value to GRS server
WO2021185352A1 (en) Version upgrade method and related apparatus
WO2022068596A1 (en) Metadata management method, and electronic device
CN114328377A (en) Metadata management method and electronic equipment
CN114979457A (en) Image processing method and related device
EP4336357A1 (en) Message processing method and related apparatus
CN114489471B (en) Input and output processing method and electronic equipment
CN112416984A (en) Data processing method and device
CN113485969B (en) Storage fragmentation method and device, terminal and computer storage medium
CN112783418B (en) Method for storing application program data and mobile terminal
CN113721836A (en) Data deduplication method and device
CN114168115B (en) Communication system, application downloading method and device
WO2023001208A1 (en) Multi-file synchronization method and electronic device
WO2023072241A1 (en) Media file management method and related apparatus
CN116414782A (en) Method for identifying repeated file and electronic equipment
CN117520279A (en) File storage method and electronic equipment
CN118138677A (en) Image processing method, electronic equipment and system
CN116668997A (en) Positioning method, user interface and electronic equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination