WO2021226822A1 - 日志写入方法、装置、电子设备以及存储介质 - Google Patents

日志写入方法、装置、电子设备以及存储介质 Download PDF

Info

Publication number
WO2021226822A1
WO2021226822A1 PCT/CN2020/089766 CN2020089766W WO2021226822A1 WO 2021226822 A1 WO2021226822 A1 WO 2021226822A1 CN 2020089766 W CN2020089766 W CN 2020089766W WO 2021226822 A1 WO2021226822 A1 WO 2021226822A1
Authority
WO
WIPO (PCT)
Prior art keywords
log
file
interface
log file
storage system
Prior art date
Application number
PCT/CN2020/089766
Other languages
English (en)
French (fr)
Inventor
郭子亮
Original Assignee
深圳市欢太科技有限公司
Oppo广东移动通信有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市欢太科技有限公司, Oppo广东移动通信有限公司 filed Critical 深圳市欢太科技有限公司
Priority to PCT/CN2020/089766 priority Critical patent/WO2021226822A1/zh
Priority to CN202080095208.XA priority patent/CN115039082A/zh
Publication of WO2021226822A1 publication Critical patent/WO2021226822A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment

Definitions

  • This application relates to the field of computer data storage technology, and more specifically, to a log writing method, device, electronic equipment, and storage medium.
  • Log files are record files or collections of files used to record system operation events. They are being valued and relied on more and more. Many program operation problems can be solved by retrieving log files. The unified collection and analysis of log files can improve the program. Perform in-depth analysis of operational behavior. The related log writing method is limited by the writing and reading performance of the local disk.
  • this application proposes a log writing method, device, electronic equipment, and storage medium to improve the above-mentioned problems.
  • an embodiment of the present application provides a log writing method.
  • the method includes: obtaining a file operation of a business program; obtaining a first file operation corresponding to the file operation from a plurality of system interfaces configured in the log file system. Interface, wherein the system interface is an interface created based on a portable operating system interface standard; obtain a second interface corresponding to the first interface from a plurality of interfaces configured in the back-end log storage system; call the second interface
  • the log file corresponding to the file operation in the memory is written into the back-end log storage system.
  • an embodiment of the present application provides a log writing device.
  • the device includes: an operation acquisition unit for acquiring file operations of a business program; and a first interface acquisition unit for acquiring data from the log file system.
  • the first interface corresponding to the file operation is obtained from two system interfaces, where the system interface is an interface created based on the portable operating system interface standard; A second interface corresponding to the first interface is acquired among the interfaces; a writing unit is configured to call the second interface to write the log file corresponding to the file operation in the memory to the back-end log storage system.
  • an embodiment of the present application provides an electronic device, including one or more processors and a memory; one or more programs, wherein the one or more programs are stored in the memory and configured to Executed by the one or more processors, and the one or more programs are configured to execute the above-mentioned method.
  • an embodiment of the present application provides a computer-readable storage medium with program code stored in the computer-readable storage medium, wherein the above-mentioned method is executed when the program code is running.
  • the embodiments of the present application provide a log writing method, device, electronic equipment, and storage medium.
  • Obtain the file operation of the business program and obtain the first interface corresponding to the file operation from the multiple system interfaces configured in the log file system, where the system interface is an interface defined based on the portable operating system interface standard.
  • a second interface corresponding to the first interface is acquired from a plurality of interfaces configured in the log storage system, and the second interface is called to write the log file corresponding to the file operation in the memory to the back-end log storage system.
  • the generated log file can be directly written into the memory first, so that the log file system can call the second interface of the back-end log storage system to directly write the log file in the memory to the back-end log file storage system , So that the log file does not need to be placed on the disk, so that the log writing is no longer limited to the writing and reading performance of the local disk, and the high-speed writing of the log is realized.
  • Figure 1 shows an architecture diagram of a log writing system proposed by this application
  • FIG. 2 shows a flowchart of a log writing method proposed by an embodiment of the present application
  • FIG. 3 shows a schematic diagram of a scenario for invoking an interface proposed by an embodiment of the present application
  • FIG. 4 shows a flowchart of a log writing method proposed by another embodiment of the present application.
  • FIG. 5 shows a flowchart of a log writing method proposed by still another embodiment of the present application.
  • FIG. 6 shows a flowchart of a log writing method proposed by another embodiment of the present application.
  • FIG. 7 shows a structural block diagram of a log writing device proposed by an embodiment of the present application.
  • FIG. 8 shows a structural block diagram of a writing unit proposed by an embodiment of the present application.
  • FIG. 9 shows a structural block diagram of a log writing device proposed by an embodiment of the present application.
  • FIG. 10 shows a structural block diagram of an electronic device for executing the log writing method according to an embodiment of the present application in real time of the present application
  • FIG. 11 shows a storage unit used to save or carry program code for implementing the log writing method according to the embodiment of the present application in real time of the present application.
  • Log files are record files or collections of files used to record system operation events. They are being valued and relied on more and more. Many program operation problems can be solved by retrieving logs. The operation behavior of the program can be analyzed through unified collection and analysis of logs. Conduct an in-depth analysis.
  • the related log writing methods are secondary bypass reading, not only the performance is limited by the writing and reading performance of the local disk, but also increases the load pressure on the application log hard disk.
  • the inventor proposes to obtain the first interface corresponding to the file operation from the multiple system interfaces configured in the log file system by obtaining the file operation of the business program in this application, wherein the system interface is based on portable
  • the interface created by the operating system interface standard obtains the second interface corresponding to the first interface from the multiple interfaces configured in the back-end log storage system, and calls the second interface to store the log corresponding to the file operation in the memory
  • the file is written into the back-end log storage system, and the log file system calls the second interface of the back-end log storage system to directly write the log files in the memory to the back-end log file storage system, and does not need to be placed on the actual disk.
  • Log writing is no longer limited by the writing and reading performance of the local disk, and a method, device, electronic device and storage medium for high-speed writing of the log are realized.
  • the system 100 may include a log server 110 and a log storage 120.
  • the log server 110 may include a log file system 112 and a first interface module 114
  • the log storage terminal 120 may include a back-end log storage system 122 and a second interface module 124.
  • the first interface module 114 is configured with multiple system interfaces
  • the second interface module 124 is configured with multiple interfaces corresponding to the system interfaces.
  • the log file system 112 may communicate with the back-end log storage system 122 through the first interface module 114 and the second interface module 124.
  • a log writing method provided by an embodiment of the present application is applied to a log writing system, and the method includes:
  • Step S110 Obtain the file operation of the business program.
  • the file operations may include file opening operations, file reading operations, file writing operations, renaming operations, deleting operations, and file closing operations.
  • the log file system obtains the current file operations of the business program, such as opening files or reading and writing files.
  • Step S120 Obtain a first interface corresponding to the file operation from a plurality of system interfaces configured in the log file system, where the system interface is an interface defined based on a portable operating system interface standard.
  • the log file system is a distributed log file system.
  • the log file system is configured with multiple system interfaces, and the system interface is an interface related to the log file system defined based on the Portable Operating System Interface of Unix (POSIX) standard.
  • POSIX Portable Operating System Interface of Unix
  • the portable operating system interface standard related to the log file system defines a set of source code-level interfaces to enable business programs to operate files in a consistent manner.
  • a log file system that supports a portable operating system interface provides the implementation of corresponding functions and semantics corresponding to all POSIX file system interfaces.
  • the portable operating system interface defines multiple file operations, which can operate and control all aspects of files.
  • the multiple system interfaces provided by the log file system may include system interfaces such as open, read, write, rename, symlink, rm, and close.
  • system interfaces such as open, read, write, rename, symlink, rm, and close.
  • the portable operating system interface standard describes the calling service interface of the operating system, which is used to ensure that the compiled business program can be transplanted and run on multiple operating systems at the source code level.
  • the log file system obtains the system interface corresponding to the file operation of the business program from the multiple system interfaces configured in the system.
  • Step S130 Obtain a second interface corresponding to the first interface from a plurality of interfaces configured in the back-end log storage system.
  • the log file system may obtain an interface corresponding to the system interface from a plurality of interfaces pre-configured in the back-end log storage system according to the obtained system interface corresponding to the file operation corresponding to the business program.
  • the first interface module is configured with interface A, interface B, and interface C
  • the second interface module is configured with interface D, interface E, and interface F. If the file operation of the business program corresponds to the system The interface is interface A, and the log file system can find the interface corresponding to interface A from the multiple interfaces configured by the second interface module. If it is interface E, the log file system can call interface E to connect the files stored in the memory with the business program The log file corresponding to the operation is written to the back-end log storage system.
  • Step S140 Invoke the second interface to write the log file corresponding to the file operation in the memory to the back-end log storage system.
  • the log file system calls the second interface obtained through the above operation, and writes the log file generated in the memory of the log file system and corresponding to the file operation of the business program into the back-end log storage system.
  • the log writing method provided by the embodiment of the application obtains the file operation of the business program, and obtains the first interface corresponding to the file operation from the multiple system interfaces configured in the log file system, wherein the system interface is based on
  • the interface defined by the portable operating system interface standard acquires the second interface corresponding to the first interface from the multiple interfaces configured in the back-end log storage system, and calls the second interface to correspond to the file operation in the memory
  • the log file is written to the back-end log storage system.
  • the generated log file can be directly written into the memory first, so that the log file system can call the second interface of the back-end log storage system to directly write the log file in the memory to the back-end log file storage system , So that the log file does not need to be placed on the disk, so that the log writing is no longer limited to the writing and reading performance of the local disk, and the high-speed writing of the log is realized.
  • a log writing method provided by an embodiment of the present application is applied to a log writing system, and the method includes:
  • Step S210 Obtain the file operation of the business program.
  • the file operation of the business program is obtained from the system kernel of the system where the log file system is located, and the file operation of the business program is obtained by the system kernel from the mount path of the log file system.
  • the system where the log file system is located is an operating system, and optionally, it may be a Linux operating system.
  • the log file system runs based on the Linux operating system
  • the log file system is started and mounted to a certain directory
  • the business program operates files at the mount point of the mount path of the log file system, such as opening files or reading and writing Files, etc.
  • the Linux kernel transfers the file operations of the business program to the virtual file system
  • the virtual file system transfers the file operations of the business program to the log file system kernel module
  • the log file system kernel module transfers the file operations of the business program to the log file system user mode program .
  • the function of the Virtual File System is to use standard system calls to read and write different file systems located on different physical media, that is, to provide a unified operation interface and application programming interface for various file systems.
  • the virtual file system is a glue layer that allows system calls such as open(), read(), write(), etc. to work without caring about the underlying storage medium and file system type.
  • Each file system at the virtual file system layer is composed of mount points, super blocks, directories, and index nodes; when a file system is mounted, it is actually the process of creating these four data structures, so these four data The status of the structure is very important, and the relationship is very close. Since the virtual file system requires the actual file system to provide the above data structure, different file systems can access each other at the virtual file system layer.
  • Step S220 Obtain a first interface corresponding to the file operation from a plurality of system interfaces configured in the log file system, where the system interface is an interface created based on a portable operating system interface standard.
  • Step S230 Obtain a second interface corresponding to the first interface from a plurality of interfaces configured in the back-end log storage system.
  • Step S240 Create a log file corresponding to the file operation.
  • the log file is created at the mount point of the mount path of the log file system in accordance with the portable operating system interface standard.
  • create metadata information of the log file where the metadata information includes the file name, creation information, file size, and mounting path of the log file; write the metadata information to the log The target database in the file system.
  • Metadata is also called intermediary data and relay data. It is data describing data, mainly information describing data attributes, and can be used to support functions such as indicating storage location, historical data, resource search, and file recording.
  • the creation information may be information such as the creation time and creation path of the log file.
  • the writing the metadata information into the target database in the log file system may include: obtaining the log identifier of the log file from the back-end log storage system according to the mount path; The log identifier is written into the metadata information.
  • the target database is a key/value database
  • the key-value database is a database that stores data in key-value pairs, similar to a map in Java.
  • the entire database can be understood as a big map, and each key corresponds to a unique value.
  • the data structure of the key/value database is a hash table structure.
  • the log file system can obtain the identification of the log file by obtaining the metadata information of the log file in the target database, and the log data in the log file can be read through the identification of the log file. Further, it is possible to determine whether there is a problem with the operation of the business program by analyzing the read log data, and if there is a problem, it can be solved in time.
  • the log file system can pre-set the format of the metadata information of the log file, which can include the format of the file name, the file size of the log file and the creation time of the log file, etc.
  • the format of the metadata information of the newly created log file is compared with the format of the metadata information of the log file set in advance to determine whether the format of the metadata information of the newly created log file is wrong.
  • the log file system may compare the file name of the newly created log file read with the format of the file name preset in the log file system to determine whether the format of the file name is correct. The file name format is incorrect. Modify the file name format of the log file in time.
  • Step S250 Open the log file, and write log data into the log file.
  • Step S260 Write the log file to the back-end log storage system through the second interface.
  • the log file corresponding to the file operation of the business program can be called from multiple interfaces of the back-end log storage system and written to the back-end log storage system.
  • the log writing method provided by the embodiment of the application obtains the file operation of the business program, and obtains the first interface corresponding to the file operation from the multiple system interfaces configured in the log file system, wherein the system interface is based on
  • the interface defined by the portable operating system interface standard acquires the second interface corresponding to the first interface from the multiple interfaces configured in the back-end log storage system, creates a log file corresponding to the file operation, and opens the log File, write log data into the log file, and write the log file into the back-end log storage system through the second interface.
  • the generated log file can be directly written into the memory first, so that the log file system can call the second interface of the back-end log storage system to directly write the log file in the memory to the back-end log file storage system , So that the log file does not need to be placed on the disk, so that the log writing is no longer limited to the writing and reading performance of the local disk, and the high-speed writing of the log is realized.
  • the metadata information of the log file is created, the metadata information of the newly created log file is compared with the metadata information of the log file preset by the file system, and the metadata information that has errors is modified in time to ensure The correctness of the metadata information.
  • a log writing method provided by an embodiment of the present application is applied to a log writing system, and the method includes:
  • Step S310 Initialize the log file system.
  • the initializing the log file system may be understood as initializing the log file system, and the step of initializing the log file system may include: creating a metadata instance of the target database to be based on the The metadata instance determines the location of the log file in the log file system; creates a cache data instance of the target database to access the log data of the log file according to the cache data instance; initializes the log file The root directory to initialize the log file system.
  • the metadata instance is log file system metadata, which can be used to locate the location of the log file in the log file system.
  • the cached data instance can be understood as a handle for opening the log file, and the business program can access the log data in the log file through the cached data instance.
  • the root directory is created when the log file system is created, and its purpose is to store subdirectories (also called folders) or directory entries of files.
  • Step S320 Obtain the file operation of the business program.
  • Step S330 Obtain a first interface corresponding to the file operation from a plurality of system interfaces configured in the log file system, where the system interface is an interface created based on a portable operating system interface standard.
  • Step S340 Obtain a second interface corresponding to the first interface from a plurality of interfaces configured in the back-end log storage system.
  • Step S350 Obtain a target child thread from multiple child threads.
  • the step of obtaining the target sub-thread from multiple sub-threads includes: obtaining multiple sub-threads of the business program; obtaining sub-threads corresponding to file operations of the business program from the multiple sub-threads As the target child thread.
  • multiple sub-threads can be created according to the file operation of the business program, and different sub-threads can be used to perform different file operations. Further, the corresponding sub-thread can be determined according to the current file operation of the business program, and the corresponding The sub-thread is used as the target sub-thread to execute the subsequent operation of calling the second interface to write the log file corresponding to the file operation in the memory to the back-end log storage system.
  • Step S360 call the second interface based on the target sub-thread to write the log file corresponding to the file operation in the memory to the back-end log storage system; wherein the target sub-thread is controlled by a token bucket algorithm The writing speed of writing the log file corresponding to the file operation in the memory to the back-end log storage system.
  • the step of calling the second interface to write the log file corresponding to the file operation in the memory to the back-end log storage system includes: obtaining an interface calling request for calling the second interface, and The interface call request carries an encrypted signature; signature verification is performed on the interface call request; if the signature verification is passed, the second interface is called to write the log file corresponding to the file operation in the memory to the back-end log storage System; if the signature verification fails, stop calling the second interface to write the log file corresponding to the file operation in the memory to the back-end log storage system.
  • the request parameter may be an identifier corresponding to the interface, or the type identifier of the business program that calls the interface, or the like.
  • the log system program can use the second interface according to whether the identification corresponding to the verification interface is correct, and obtaining verification whether the type identification of the business program calling the interface conforms to the second interface.
  • Step S370 Return the first interface to the business program according to the mounting path, so that the business program can determine whether the file operation corresponding to the first interface is completed.
  • the first interface corresponding to the second interface is mounted according to the log file system
  • the original path is returned to the business program.
  • the business program can determine whether the file operation of the business program corresponding to the first interface is completed according to whether the first interface is received. The file operation corresponding to the first interface is completed, and if the first interface is not received, it is determined that the file operation corresponding to the first interface is not completed.
  • the log writing method provided by the embodiment of the application obtains the file operation of the business program, and obtains the first interface corresponding to the file operation from the multiple system interfaces configured in the log file system, wherein the system interface is based on
  • the interface defined by the portable operating system interface standard acquires the second interface corresponding to the first interface from the multiple interfaces configured in the back-end log storage system, acquires the target sub-thread from the multiple sub-threads, and calls based on the target sub-thread
  • the second interface writes the log file corresponding to the file operation in the memory to the back-end log storage system.
  • the generated log file can be directly written into the memory first, so that the log file system can call the second interface of the back-end log storage system to directly write the log file in the memory to the back-end log file storage system , So that the log file does not need to be placed on the disk, so that the log writing is no longer limited to the writing and reading performance of the local disk, and the high-speed writing of the log is realized. Further, when calling the second interface, first verify whether the calling request is legal. If it is legal, call the second interface to write the log file corresponding to the file operation in the memory to the back-end log storage system, which improves the interface The security of the calling process.
  • a log writing method provided by an embodiment of the present application is applied to a log writing system, and the method includes:
  • Step S410 Obtain a target child thread from multiple child threads.
  • the system can automatically create a main thread of the log file system. Furthermore, multiple sub-threads can be created through the main thread. Further, the multiple sub-threads can be assigned tasks. Different sub-threads can handle different tasks or the same tasks. The processing tasks are generally assigned by the system. Instructions for logical processing of specific files. When the system needs to process multiple files, it will generate multiple sub-threads for parallel processing based on the main thread. This parallel processing of multiple sub-threads can speed up the processing speed, but in order to achieve load balancing, there will be a certain limit of sub-threads created. The preset number can generally be determined according to parameters such as the amount of processed data or CPU performance.
  • the creating a preset number of sub-threads further includes: dividing processing tasks, and allocating execution tasks to each sub-thread. For example, there are 10 files in the log file system that need to be processed, and the log file system can create 5 threads according to the number of files that need to be processed, and each thread calculates 2 files. In the process of allocating the amount of execution tasks, the processing amount can be equally or unequally allocated.
  • the log file system may obtain the sub-thread corresponding to the current file operation of the business program from the multiple sub-threads determined through the foregoing operations, and use the sub-thread as the target sub-thread.
  • the multiple sub-threads can be named, and the names of the multiple sub-threads can be stored in the target database, and the task progress of the multiple sub-threads performing file operations can be stored in real time, and the relationship between the sub-threads and the task progress can be established.
  • the mapping relationship, and then the corresponding task progress can be obtained by looking up the child thread name.
  • the task progress of the file operation corresponding to the subthread name can be searched in the target database.
  • Step S420 Obtain the file operation of the business program based on the target sub-thread.
  • the current file operation of the business program is obtained at the mount point of the mount path of the log file system based on the target sub-thread obtained by the above method.
  • Step S430 Obtain a first interface corresponding to the file operation from a plurality of system interfaces configured in the log file system based on the target sub-thread.
  • the log file system user mode program is based on the target sub-thread to obtain the first interface corresponding to the file operation.
  • the file operation of the business program is obtained by the mount path of the log file system.
  • Step S440 Obtain a second interface corresponding to the first interface from a plurality of interfaces configured in the back-end log storage system based on the target sub-thread.
  • Step S450 Call the second interface based on the target sub-thread to write the log file corresponding to the file operation in the memory to the back-end log storage system; wherein the target sub-thread is controlled by a token bucket algorithm The writing speed of writing the log file corresponding to the file operation in the memory to the back-end log storage system.
  • the token bucket algorithm is the most commonly used algorithm in network traffic shaping (Traffic Shaping) and Rate Limiting (Rate Limiting). Typically, the token bucket algorithm can be used to control the number of data sent to the network and allow burst data to be sent. Therefore, in order to ensure the stability of the log file system and the back-end log storage system, the token bucket algorithm can be used to control the target sub-thread to write the log file corresponding to the file operation of the business program in the memory to the back-end log storage system. ⁇ Into the speed.
  • multiple sub-threads are limited to the same token bucket algorithm.
  • the token bucket algorithm can be used to specify which sub-thread has the right to write to the log file.
  • the log writing method provided by an embodiment of the application obtains a target sub-thread from multiple sub-threads, obtains file operations of a business program based on the target sub-thread, and obtains from multiple system interfaces configured in the log file system based on the target sub-thread.
  • the first interface corresponding to the file operation wherein the system interface is an interface defined based on a portable operating system interface standard, and based on the target sub-thread, the first interface is obtained from multiple interfaces configured by the back-end log storage system.
  • the second interface corresponding to the interface calls the second interface based on the target sub-thread to write the log file corresponding to the file operation in the memory to the back-end log storage system.
  • the generated log file can be directly written into the memory first, so that the log file system can call the second interface of the back-end log storage system to directly write the log file in the memory to the back-end log file storage system , So that the log file does not need to be placed on the disk, so that the log writing is no longer limited to the writing and reading performance of the local disk, and the high-speed writing of the log is realized.
  • the token bucket algorithm is used to limit the writing speed of the log file, which ensures the stability of the log file system and the back-end log storage system.
  • a log writing device 500 provided by an embodiment of the present application, the device 500 includes:
  • the operation obtaining unit 510 is configured to obtain file operations of the business program.
  • the operation acquiring unit 510 is further configured to acquire the file operation of the business program from the system kernel of the system where the log file system is located, and the file operation of acquiring the business program is mounted by the system kernel from the log file system. The path is obtained.
  • the first interface acquiring unit 520 is configured to acquire a first interface corresponding to the file operation from multiple system interfaces configured by the log file system, where the system interface is an interface created based on a portable operating system interface standard.
  • the second interface acquiring unit 530 is configured to acquire a second interface corresponding to the first interface from a plurality of interfaces configured in the back-end log storage system.
  • the writing unit 540 is configured to call the second interface to write the log file corresponding to the file operation in the memory to the back-end log storage system.
  • the writing unit 540 is also used to obtain an interface invocation request for invoking the second interface, the interface invocation request carries an encrypted signature; to perform signature verification on the interface invocation request;
  • the second interface writes the log file corresponding to the file operation in the memory to the back-end log storage system; if the signature verification fails, stop calling the second interface to perform the operation with the file in the memory
  • the corresponding log file is written into the back-end log storage system.
  • a writing unit 540 provided by an embodiment of the present application, and the writing unit 540 includes:
  • the file creation module 542 is used to create a log file corresponding to the file operation.
  • the file creation module 542 is further configured to create the log file at the mount point of the mount path of the log file system according to the portable operating system interface standard.
  • the file creation module 542 is also used to create metadata information of the log file, where the metadata information includes the file name, creation information, file size, and mounting path of the log file; The metadata information is written into the target database in the log file system.
  • the file creation module 542 is further configured to obtain the log identifier of the log file from the back-end log storage system according to the mount path; write the log identifier into the metadata information .
  • the data writing module 544 is used to open the log file and write log data into the log file.
  • the log writing module 546 is configured to write the log file into the back-end log storage system through the second interface.
  • a log writing device 600 provided by an embodiment of the present application, the device 600 includes:
  • the initialization unit 610 is used to initialize the log file system. Specifically, create a metadata instance of the target database to determine the location of the log file in the log file system according to the metadata instance; create a cache data instance of the target database to determine the location of the log file in the log file system according to the metadata instance; The data instance accesses the log data of the log file; and initializes the root directory of the log file to initialize the log file system.
  • the operation obtaining unit 620 is configured to obtain file operations of the business program. Specifically, it is used to obtain the file operation of the business program based on the target sub-thread.
  • the first interface acquiring unit 630 is configured to acquire a first interface corresponding to the file operation from a plurality of system interfaces configured by the log file system, where the system interface is an interface created based on a portable operating system interface standard.
  • the second interface acquiring unit 640 is configured to acquire a second interface corresponding to the first interface from a plurality of interfaces configured in the back-end log storage system.
  • it is configured to obtain a second interface corresponding to the first interface from a plurality of interfaces configured in the back-end log storage system based on the target sub-thread.
  • the writing unit 650 is configured to call the second interface to write the log file corresponding to the file operation in the memory to the back-end log storage system.
  • the second interface is configured to call the second interface based on the target sub-thread to write the log file corresponding to the file operation in the memory to the back-end log storage system; wherein, the target is controlled by a token bucket algorithm
  • the sub-thread writes the log file corresponding to the file operation in the memory to the writing speed of the back-end log storage system.
  • the determining unit 660 is configured to return the first interface to the business program according to the mounting path, so that the business program determines whether the file operation corresponding to the first interface is completed.
  • an embodiment of the present application also provides another electronic device 200 that can execute the aforementioned log writing method.
  • the electronic device 200 includes one or more (only one shown in the figure) processor 102, a memory 104, and a network module 106 that are coupled to each other.
  • the memory 104 stores a program that can execute the content in the foregoing embodiment, and the processor 102 can execute the program stored in the memory 104.
  • the processor 102 may include one or more processing cores.
  • the processor 102 uses various interfaces and lines to connect various parts of the entire electronic device 200, and executes by running or executing instructions, programs, code sets, or instruction sets stored in the memory 104, and calling data stored in the memory 104.
  • the processor 102 may use at least one of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA).
  • DSP Digital Signal Processing
  • FPGA Field-Programmable Gate Array
  • PDA Programmable Logic Array
  • the processor 102 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), a modem, and the like.
  • the CPU mainly processes the operating system, user interface, and application programs; the GPU is used for rendering and drawing of display content; the modem is used for processing wireless communication. It can be understood that the above-mentioned modem may not be integrated into the processor 102, but may be implemented by a communication chip alone.
  • the memory 104 may include random access memory (RAM) or read-only memory (Read-Only Memory).
  • the memory 104 may be used to store instructions, programs, codes, code sets or instruction sets.
  • the memory 104 may include a storage program area and a storage data area, where the storage program area may store instructions for implementing the operating system and instructions for implementing at least one function (such as touch function, sound playback function, image playback function, etc.) , Instructions used to implement the following various method embodiments, etc.
  • the data storage area can also store data (such as phone book, audio and video data, chat record data) created by the terminal 100 during use.
  • the network module 106 is used to receive and send electromagnetic waves, and realize the mutual conversion between electromagnetic waves and electrical signals, so as to communicate with a communication network or other devices, such as with an audio playback device.
  • the network module 106 may include various existing circuit elements for performing these functions, for example, an antenna, a radio frequency transceiver, a digital signal processor, an encryption/decryption chip, a subscriber identity module (SIM) card, a memory, etc. .
  • SIM subscriber identity module
  • the network module 106 can communicate with various networks, such as the Internet, an intranet, and a wireless network, or communicate with other devices through a wireless network.
  • the aforementioned wireless network may include a cellular telephone network, a wireless local area network, or a metropolitan area network.
  • the network module 106 can exchange information with the base station.
  • FIG. 11 shows a structural block diagram of a computer-readable storage medium provided by an embodiment of the present application.
  • the computer-readable medium 800 stores program code, and the program code can be invoked by a processor to execute the method described in the foregoing method embodiment.
  • the computer-readable storage medium 800 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read Only Memory), EPROM, hard disk, or ROM.
  • the computer-readable storage medium 800 includes a non-transitory computer-readable storage medium.
  • the computer-readable storage medium 800 has storage space for the program code 810 for executing any method steps in the above-mentioned methods. These program codes can be read from or written into one or more computer program products.
  • the program code 810 may be compressed in a suitable form, for example.
  • a log writing method, device, electronic device, and storage medium provided in the present application are used to obtain file operations of a business program, and obtain a first interface corresponding to the file operation from a plurality of system interfaces configured in the log file system, wherein The system interface is an interface defined based on a portable operating system interface standard.
  • a second interface corresponding to the first interface is obtained from a plurality of interfaces configured in the back-end log storage system, and the second interface is called to store in the memory
  • the log file corresponding to the file operation is written into the back-end log storage system.
  • the generated log file can be directly written into the memory first, so that the log file system can call the second interface of the back-end log storage system to directly write the log file in the memory to the back-end log file storage system , So that the log file does not need to be placed on the disk, so that the log writing is no longer limited to the writing and reading performance of the local disk, and the high-speed writing of the log is realized.

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Hardware Design (AREA)
  • Quality & Reliability (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Debugging And Monitoring (AREA)

Abstract

一种日志写入方法、装置、电子设备以及存储介质。所述方法包括：获取业务程序的文件操作（S110）；从日志文件系统配置的多个系统接口中获取与所述文件操作对应的第一接口，其中所述系统接口为基于可移植操作系统接口标准定义的接口（S120）；从后端日志存储系统配置的多个接口中获取与所述第一接口对应的第二接口（S130）；调用所述第二接口将内存中与所述文件操作对应的日志文件写入所述后端日志存储系统（S140）。通过上述方法使得日志文件系统可以调用后端日志存储系统的第二接口直接将内存中的日志文件写入后端日志文件存储系统，使得日志文件不需要落盘到磁盘上，以便日志写入不再受限于本地磁盘的写入和读取性能，实现了日志的高速写入。

Description

日志写入方法、装置、电子设备以及存储介质 技术领域
本申请涉及计算机数据存储技术领域，更具体地，涉及一种日志写入方法、装置、电子设备以及存储介质。
背景技术
日志文件是用于记录系统操作事件的记录文件或文件集合，越来越被重视和依赖，通过检索日志文件可以解决非常多的程序运作问题，通过对日志文件的统一采集、分析可以对程序的运作行为进行深入的分析。相关的日志写入方法受限于本地磁盘的写入和读取性能。
发明内容
鉴于上述问题，本申请提出了一种日志写入方法、装置、电子设备以及存储介质，以改善上述问题。
第一方面，本申请实施例提供了一种日志写入方法，所述方法包括：获取业务程序的文件操作；从日志文件系统配置的多个系统接口中获取与所述文件操作对应的第一接口，其中所述系统接口为基于可移植操作系统接口标准创建的接口；从后端日志存储系统配置的多个接口中获取与所述第一接口对应的第二接口；调用所述第二接口将内存中与所述文件操作对应的日志文件写入所述后端日志存储系统。
第二方面，本申请实施例提供了一种日志写入装置，所述装置包括：操作获取单元，用于获取业务程序的文件操作；第一接口获取单元，用于从日志文件系统配置的多个系统接口中获取与所述文件操作对应的第一接口，其中所述系统接口为基于可移植操作系统接口标准创建的接口；第二接口获取单元，用于从后端日志存储系统配置的多个接口中获取与所述第一接口对应的第二接口；写入单元，用于调用所述第二接口将内存中与所述文件操作对应的日志文件写入所述后端日志存储系统。
第三方面，本申请实施例提供了一种电子设备，包括一个或多个处理器以及存储器；一个或多个程序，其中所述一个或多个程序被存储在所述存储器中并被配置为由所述一个或多个处理器执行，所述一个或多个程序配置用于执行上述的方法。
第四方面，本申请实施例提供了一种计算机可读存储介质，所述计算机可读存储介质中存储有程序代码，其中，在所述程序代码运行时执行上述的方法。
本申请实施例提供了一种日志写入的方法、装置、电子设备以及存储介质。获取业务程序的文件操作，从日志文件系统配置的多个系统接口中获取与所述文件操作对应的第一接口，其中所述系统接口为基于可移植操作系统接口标准定义的接口，从后端日志存储系统配置的多个接口中获取与所述第一接口对应的第二接口，调用所述第二接口将内存中与所述文件操作对应的日志文件写入所述后端日志存储系统。通过上述方法，可以将所生成的日志文件先直接写入到内存中，进而使得日志文件系统可以调用后端日志存储系统的第二接口直接将内存中的日志文件写入后端日志文件存储系统，使得日志文件不需要落盘到磁盘上，以便日志写入不再受限于本地磁盘的写入和读取性能，实现了日志的高速写入。
附图说明
为了更清楚地说明本申请实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1示出了本申请提出的一种日志写入系统架构图；
图2示出了本申请一实施例提出的一种日志写入方法的流程图；
图3示出了本申请一实施例提出的一种调用接口的场景示意图；
图4示出了本申请另一实施例提出的一种日志写入方法的流程图；
图5示出了本申请再一实施例提出的一种日志写入方法的流程图；
图6示出了本申请又一实施例提出的一种日志写入方法的流程图；
图7示出了本申请实施例提出的一种日志写入装置的结构框图；
图8示出了本申请实施例提出的一种写入单元的结构框图；
图9示出了本申请实施例提出的一种日志写入装置的结构框图；
图10示出了本申请实时中的用于执行根据本申请实施例的日志写入方法的电子设备的结构框图；
图11示出了本申请实时中的用于保存或者携带实现根据本申请实施例的日志写入方法的程序代码的存储单元。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
日志文件是用于记录系统操作事件的记录文件或文件集合，越来越被重视和依赖，通过检索日志可以解决非常多的程序运作问题，通过对日志的统一采集、分析可以对程序的运作行为进行深入的分析。
而发明人在对相关的日志写入方法的研究中发现，相关的日志写入方法对日志的格式和路径都有较高的要求，需要提前将日志路径配置到采集端的配置文件中，十分不灵活；另外，相关的日志写入方法对日志采集的实时性也比较差，会为了提高采集性能配置较高的汇聚时间和条目数，一般会每5秒或1000条日志才将日志写入后端存储系统。相关的日志写入方式均为二次旁路读取，不仅性能受限于本地磁盘的写入和读取性能，且对应用程序的日志硬盘增加了负载压力。
因此，发明人提出了本申请中的通过获取业务程序的文件操作，从日志文件系统配置的多个系统接口中获取与所述文件操作对应的第一接口，其中所述系统接口为基于可移植操作系统接口标准创建的接口，从后端日志存储系统配置的多个接口中获取与所述第一接口对应的第二接口，调用所述第二接口将内存中与所述文件操作对应的日志文件写入所述后端日志存储系统，日志文件系统调用后端日志存储系统的第二接口直接将内存中的日志文件写入后端日志文件存储系统，不需要落盘到实际的磁盘上，日志写入不再受限于本地磁盘的写入和读取性能，实现了日志的高速写入的方法、装置、电子设备以及存储介质。
请参阅图1，本申请实施例提供的一种日志写入系统100，所述系统100可以包括日志服务端110以及日志存储端120。其中，日志服务端110可以包括日志文件系统112以及第一接口模块114，日志存储端120可以包括后端日志存储系统122以及第二接口模块124。进一步的，所述第一接口模块114中配置有多个系统接口，所述第二接口模块124中配置有多个与系统接口对应的接口。所述日志文件系统112可以通过第一接口模块114和第二 接口模块124与后端日志存储系统122进行通信。
请参阅图2，本申请实施例提供的一种日志写入方法，应用于日志写入系统，所述方法包括：
步骤S110：获取业务程序的文件操作。
作为一种方式，所述文件操作可以包括打开文件操作、读取文件操作、写入文件操作、重命名操作、删除操作以及关闭文件操作。日志文件系统获取业务程序当前的文件操作，比如打开文件或者读写文件。
步骤S120：从日志文件系统配置的多个系统接口中获取与所述文件操作对应的第一接口，其中所述系统接口为基于可移植操作系统接口标准定义的接口。
作为一种方式，所述日志文件系统为分布式日志文件系统。所述日志文件系统中配置有多个系统接口，所述系统接口为基于可移植操作系统接口(Portable Operating System Interface of Unix,POSIX)标准定义的与日志文件系统相关的接口。与日志文件系统相关的可移植操作系统接口标准定义了一组源代码级别的接口用以使业务程序可以以一致的方法操作文件。而一个支持可移植操作系统接口的日志文件系统则提供了对应所有POSIX文件系统接口的相应功能和语义的实现。可移植操作系统接口定义了多个文件操作，可以操作和控制文件的方方面面。该日志文件系统提供的多个系统接口可以包括open、read、write、rename、symlink、rm以及close等系统接口。其中，可移植操作系统接口标准描述了操作系统的调用服务接口，用于保证编制的业务程序可以在源代码一级上在多种操作系统上移植运行。
具体的，日志文件系统从系统配置的多个系统接口中获取与业务程序的文件操作对应的系统接口。
步骤S130：从后端日志存储系统配置的多个接口中获取与所述第一接口对应的第二接口。
作为一种方式，后端日志存储系统中预先配置有多个接口，所述多个接口与日志文件系统中的多个系统接口一一对应。日志文件系统可以根据获取的与业务程序对应的文件操作对应的系统接口，从后端日志存储系统预先配置的多个接口中获取与所述系统接口对应的接口。示例性的，如图3所示，第一接口模块中配置有接口A、接口B以及接口C，第二接口模块中配置有接口D、接口E以及接口F，如果业务程序的文件操作对应系统接口为接口A，日志文件系统可以从第二接口模块配置的多个接口中找到与接口A对应的接口，假如为接口E，日志文件系统可以调用接口E将内存中存储的与业务程序的文件操作对应的日志文件写入后端日志存储系统中。
步骤S140：调用所述第二接口将内存中与所述文件操作对应的日志文件写入所述后端日志存储系统。
作为一种方式，日志文件系统调用通过上述操作获得的第二接口，将日志文件系统的内存中生成的与业务程序的文件操作对应的日志文件写入后端日志存储系统。
本申请实施例提供的一种日志写入方法，获取业务程序的文件操作，从日志文件系统配置的多个系统接口中获取与所述文件操作对应的第一接口，其中所述系统接口为基于可移植操作系统接口标准定义的接口，从后端日志存储系统配置的多个接口中获取与所述第一接口对应的第二接口，调用所述第二接口将内存中与所述文件操作对应的日志文件写入所述后端日志存储系统。通过上述方法，可以将所生成的日志文件先直接写入到内存中，进而使得日志文件系统可以调用后端日志存储系统的第二接口直接将内存中的日志文件写入后端日志文件存储系统，使得日志文件不需要落盘到磁盘上，以便日志写入不再受限于本地磁盘的写入和读取性能，实现了日志的高速写入。
请参阅图4，本申请实施例提供的一种日志写入方法，应用于日志写入系统，所述方法包括：
步骤S210：获取业务程序的文件操作。
作为一种方式，从所述日志文件系统所在系统的系统内核获取业务程序的文件操作，所述业务程序的文件操作由所述系统内核从所述日志文件系统的挂载路径获取到。其中，所述所述日志文件系统所在系统为操作系统，可选的，可以为Linux操作系统。
具体的，所述日志文件系统基于Linux操作系统运行，日志文件系统启动并挂载到某个目录下，业务程序在日志文件系统的挂载路径的挂载点操作文件，比如打开文件或者读写文件等。Linux内核将业务程序的文件操作转交给虚拟文件系统，虚拟文件系统将业务程序的文件操作转交给日志文件系统内核模块，日志文件系统内核模块将业务程序的文件操作转交给日志文件系统用户态程序。
其中，虚拟文件系统(Virtual File System,VFS)的作用就是采用标准的系统调用读写位于不同物理介质上的不同文件系统，即为各类文件系统提供了一个统一的操作界面和应用编程接口。虚拟文件系统是一个可以让open()、read()、write()等系统调用不用关心底层的存储介质和文件系统类型就可以工作的粘合层。每个文件系统在虚拟文件系统层都是由挂载点、超级块、目录和索引节点组成；当挂载一个文件系统时，实际也就是创建这四个数据结构的过程，因此这四个数据结构的地位很重要，关系也很紧密。由于虚拟文件系统要求实际的文件系统必须提供以上数据结构，所以不同的文件系统在虚拟文件系统层可以互相访问。
步骤S220：从日志文件系统配置的多个系统接口中获取与所述文件操作对应的第一接口，其中所述系统接口为基于可移植操作系统接口标准创建的接口。
步骤S230：从后端日志存储系统配置的多个接口中获取与所述第一接口对应的第二接口。
步骤S240：创建与所述文件操作对应的日志文件。
作为一种方式，在所述日志文件系统的挂载路径的挂载点，按照所述可移植操作系统接口标准创建所述日志文件。
可选的，创建所述日志文件的元数据信息，其中所述元数据信息包括所述日志文 件的文件名、创建信息、文件大小以及挂载路径；将所述元数据信息写入所述日志文件系统中的目标数据库。
其中，元数据又称为中介数据、中继数据，为描述数据的数据，主要是描述数据属性的信息，可以用来支持指示存储位置、历史数据、资源查找、文件记录等功能。所述创建信息可以为日志文件的创建时间、创建路径等信息。
进一步的，所述将所述元数据信息写入所述日志文件系统中的目标数据库可以包括：根据所述挂载路径从所述后端日志存储系统中获取所述日志文件的日志标识；将所述日志标识写入到所述元数据信息中。
其中，所述目标数据库为key/value数据库，Key-value数据库是一种以键值对存储数据的一种数据库，类似Java中的map。可以将整个数据库理解为一个大的map，每个键都会对应一个唯一的值。所述key/value数据库的数据结构为哈希表结构。
进一步的，日志文件系统可以通过获取目标数据库中的日志文件的元数据信息获得日志文件的标识，通过日志文件的标识可以读取日志文件中的日志数据。进一步的，可以通过对读取的日志数据进行分析，确定业务程序的运作是否存在问题，若存在问题，可以及时解决。日志文件系统中可以预先设置日志文件的元数据信息的格式，可以包括文件名的格式、日志文件的文件大小以及日志文件的创建时间等，在创建新的日志文件的元数据信息时，可以通过将新创建的日志文件的元数据信息的格式与预先设置的日志文件的元数据信息的格式进行比较，以判断新创建的日志文件的元数据信息的格式是否发生错误。示例性的，日志文件系统可以将读取到的新创建的日志文件的文件名与日志文件系统中预先设置的文件名的格式进行比较，判断文件名的格式是否正确，若所述日志文件的文件名的格式不正确，及时修改日志文件的文件名格式。
步骤S250：打开所述日志文件，将日志数据写入所述日志文件。
作为一种方式，可以操作业务程序在日志文件系统的挂载点，按照可移植操作系统接口标准打开上述创建的日志文件，将日志文件系统内存中生成的日志数据写入所述日志文件。
步骤S260：将所述日志文件通过所述第二接口写入所述后端日志存储系统。
作为一种方式，可以从后端日志存储系统的多个接口中调用与业务程序的文件操作对应的日志文件写入后端日志存储系统。
本申请实施例提供的一种日志写入方法，获取业务程序的文件操作，从日志文件系统配置的多个系统接口中获取与所述文件操作对应的第一接口，其中所述系统接口为基于可移植操作系统接口标准定义的接口，从后端日志存储系统配置的多个接口中获取与所述第一接口对应的第二接口，创建与所述文件操作对应的日志文件，打开所述日志文件，将日志数据写入所述日志文件，将所述日志文件通过所述第二接口写入所述后端日志存储系统。通过上述方法，可以将所生成的日志文件先直接写入到内存中，进而使得日志文件系统可以调用后端日志存储系统的第二接口直接将内存中的日 志文件写入后端日志文件存储系统，使得日志文件不需要落盘到磁盘上，以便日志写入不再受限于本地磁盘的写入和读取性能，实现了日志的高速写入。进一步的，在创建日志文件的元数据信息时，将新创建的日志文件的元数据信息与文件系统预先设置的日志文件的元数据信息进行比较，对于发生错误的元数据信息及时进行修改，保证了元数据信息的正确性。
请参阅图5，本申请实施例提供的一种日志写入方法，应用于日志写入系统，所述方法包括：
步骤S310：初始化所述日志文件系统。
作为一种方式，所述初始化所述日志文件系统可以理解为对日志文件系统进行初始化设置，对日志文件系统进行初始化设置的步骤可以包括：创建所述目标数据库的元数据实例，以根据所述元数据实例确定所述日志文件在所述日志文件系统中的位置；创建所述目标数据库的缓存数据实例，以根据所述缓存数据实例访问所述日志文件的日志数据；初始化所述日志文件的根目录，以初始化所述日志文件系统。
具体的，所述元数据实例为日志文件系统元数据，可以用于定位日志文件在日志文件系统中的位置。所述缓存数据实例可以理解为一个打开日志文件的句柄，业务程序可以通过缓存数据实例访问日志文件中的日志数据。所述根目录在日志文件系统创建时即已被创建，其目的就是存储子目录(也称为文件夹)或文件的目录项。
步骤S320：获取业务程序的文件操作。
步骤S330：从日志文件系统配置的多个系统接口中获取与所述文件操作对应的第一接口，其中所述系统接口为基于可移植操作系统接口标准创建的接口。
步骤S340：从后端日志存储系统配置的多个接口中获取与所述第一接口对应的第二接口。
步骤S350：从多个子线程中获取目标子线程。
作为一种方式，所述从多个子线程中获取目标子线程的步骤包括：获取所述业务程序的多个子线程；从所述多个子线程中获取与所述业务程序的文件操作对应的子线程作为所述目标子线程。
具体的，可以根据业务程序的文件操作创建多个子线程，不同的子线程可以用于执行不同的文件操作，进一步的，可以根据业务程序当前的文件操作确定对应的子线程，将所述对应的子线程作为目标子线程，用于执行后续的调用第二接口将内存中与所述文件操作对应的日志文件写入后端日志存储系统的操作。
步骤S360：基于所述目标子线程调用所述第二接口将内存中与所述文件操作对应的日志文件写入所述后端日志存储系统；其中，通过令牌桶算法控制所述目标子线程将内存中与所述文件操作对应的日志文件写入所述后端日志存储系统的写入速度。
作为一种方式，调用所述第二接口将内存中与所述文件操作对应的日志文件写入所述后端日志存储系统的步骤包括：获取调用所述第二接口的接口调用请求，所述接 口调用请求携带加密的签名；对所述接口调用请求进行签名验证；若签名验证通过，则调用所述第二接口将内存中与所述文件操作对应的日志文件写入所述后端日志存储系统；若所述签名验证不通过，则停止调用所述第二接口将内存中与所述文件操作对应的日志文件写入所述后端日志存储系统。
具体的，为了防止接口调用的过程中被黑客恶意篡改，调用接口请求都需要携带签名。后端日志存储系统将根据请求参数，对签名进行验证，签名不符合的请求将会被拒绝。
进一步的，所述请求参数可以为接口对应的标识，或者调用所述接口的业务程序的类型标识等。日志系统程序可以根据验证接口对应的标识是否正确，获取验证调用所述接口的业务程序的类型标识是否符合等来抵用第二接口。
步骤S370：将所述第一接口根据所述挂载路径返回给所述业务程序，以用于所述业务程序判断与所述第一接口对应的文件操作是否完成。
作为一种方式，在基于目标子线程调用第二接口将内存中与文件操作对应的日志文件写入后端日志存储系统后，将与第二接口对应的第一接口根据日志文件系统的挂载路径原路返回给业务程序，业务程序可以根据是否接收到所述第一接口，来判断业务程序的与第一接口对应的文件操作是否完成，若接收到所述第一接口，则确定与所述第一接口对应的文件操作完成，若没有接收到所述第一接口，则确定与所述第一接口对应的文件操作没有完成。
本申请实施例提供的一种日志写入方法，获取业务程序的文件操作，从日志文件系统配置的多个系统接口中获取与所述文件操作对应的第一接口，其中所述系统接口为基于可移植操作系统接口标准定义的接口，从后端日志存储系统配置的多个接口中获取与所述第一接口对应的第二接口，从多个子线程中获取目标子线程，基于目标子线程调用所述第二接口将内存中与所述文件操作对应的日志文件写入所述后端日志存储系统。通过上述方法，可以将所生成的日志文件先直接写入到内存中，进而使得日志文件系统可以调用后端日志存储系统的第二接口直接将内存中的日志文件写入后端日志文件存储系统，使得日志文件不需要落盘到磁盘上，以便日志写入不再受限于本地磁盘的写入和读取性能，实现了日志的高速写入。进一步的，在调用第二接口时，先通过验证调用请求是否合法，若合法，则调用所述第二接口将内存中的与文件操作对应的日志文件写入后端日志存储系统，提高了接口调用过程的安全性。
请参阅图6，本申请实施例提供的一种日志写入方法，应用于日志写入系统，所述方法包括：
步骤S410：从多个子线程中获取目标子线程。
作为一种方式，在启动日志文件系统时，系统可以自动创建一个日志文件系统的主线程。进而可以通过主线程创建多个子线程，进一步的，可以对所述多个子线程进行任务分配，不同的子线程可以处理不同的任务也可以处理相同的任务，所述处理任 务一般为系统下达的对特定文件进行逻辑处理的指令。当系统需要处理多个文件时，会根据主线程生成多个子线程进行并行处理，这种多个子线程并行处理能够加快处理速度，但为了实现负载均衡，创建的子线程会有一定限量。所述预设数量一般可根据处理的数据量或CPU性能等参数来确定。
可选的，所述创建预设数量的子线程还包括：划分处理任务，为每个子线程分配执行任务量。例如日志文件系统中有10个文件需要进行处理，日志文件系统可以根据需要处理的文件数量创建5个线程，每个线程计算2个文件。在分配执行任务量的过程中可以均等分配处理量也可以不均等分配处理量。
进一步的，日志文件系统可以从通过上述操作确定的多个子线程中获取与业务程序当前的文件操作对应的子线程，将所述子线程作为目标子线程。
进一步的，还可以为所述多个子线程命名，并将所述多个子线程的名称存入目标数据库中，并实时存入多个子线程执行文件操作的任务进度，可以建立子线程与任务进度的映射关系，进而可以通过查找子线程名获取对应的任务进度。示例性的，可以通过获取目标子线程的名称，进而可以在目标数据库中查找所述子线程名对应的文件操作的任务进度。
步骤S420：基于所述目标子线程获取业务程序的文件操作。
作为一种方式，基于通过上述方法获得的目标子线程在日志文件系统的挂载路径的挂载点获取业务程序当前的文件操作。
步骤S430：基于所述目标子线程从日志文件系统配置的多个系统接口中获取与所述文件操作对应的第一接口。
作为一种方式，在基于所述目标子线程从日志文件系统配置的多个系统接口中获取与所述文件操作对应的第一接口之前，日志文件系统用户态程序基于所述目标子线程从所述日志文件系统的挂载路径获取到业务程序的文件操作。
步骤S440：基于所述目标子线程从后端日志存储系统配置的多个接口中获取与所述第一接口对应的第二接口。
步骤S450：基于所述目标子线程调用所述第二接口将内存中与所述文件操作对应的日志文件写入所述后端日志存储系统；其中，通过令牌桶算法控制所述目标子线程将内存中与所述文件操作对应的日志文件写入所述后端日志存储系统的写入速度。
作为一种方式，在网络中传输数据时，为了防止网络拥塞，需要限制流出网络的流量，使流量以比较均匀的速度向外发送。令牌桶算法是网络流量整形(Traffic Shaping)和速率限制(Rate Limiting)中最常使用的一种算法。典型情况下，令牌桶算法可以用来控制发送到网络上的数据的数目，并允许突发数据的发送。因此，为了保证日志文件系统和后端日志存储系统的稳定性，可以通过令牌桶算法来控制目标子线程将内存中与业务程序的文件操作对应的日志文件写入后端日志存储系统的写入速度。
进一步的，多个子线程受限于同一个令牌桶算法，在有多个子线程的情况下，可以通 过令牌桶算法规定具体哪一个子线程具有写入日志文件的权限。
本申请实施例提供的一种日志写入方法，从多个子线程中获取目标子线程，基于目标子线程获取业务程序的文件操作，基于目标子线程从日志文件系统配置的多个系统接口中获取与所述文件操作对应的第一接口，其中所述系统接口为基于可移植操作系统接口标准定义的接口，基于目标子线程从后端日志存储系统配置的多个接口中获取与所述第一接口对应的第二接口，基于目标子线程调用所述第二接口将内存中与所述文件操作对应的日志文件写入所述后端日志存储系统。通过上述方法，可以将所生成的日志文件先直接写入到内存中，进而使得日志文件系统可以调用后端日志存储系统的第二接口直接将内存中的日志文件写入后端日志文件存储系统，使得日志文件不需要落盘到磁盘上，以便日志写入不再受限于本地磁盘的写入和读取性能，实现了日志的高速写入。进一步的，通过令牌桶算法限制日志文件的写入速度，保证了日志文件系统和后端日志存储系统的稳定性。
请参阅图7，本申请实施例提供的一种日志写入装置500，所述装置500包括：
操作获取单元510，用于获取业务程序的文件操作。
所述操作获取单元510，还用于从所述日志文件系统所在系统的系统内核获取业务程序的文件操作，所述获取业务程序的文件操作由所述系统内核从所述日志文件系统的挂载路径获取到。
第一接口获取单元520，用于从日志文件系统配置的多个系统接口中获取与所述文件操作对应的第一接口，其中所述系统接口为基于可移植操作系统接口标准创建的接口。
第二接口获取单元530，用于从后端日志存储系统配置的多个接口中获取与所述第一接口对应的第二接口。
写入单元540，用于调用所述第二接口将内存中与所述文件操作对应的日志文件写入所述后端日志存储系统。
所述写入单元540，还用于获取调用所述第二接口的接口调用请求，所述接口调用请求携带加密的签名；对所述接口调用请求进行签名验证；若签名验证通过，则调用所述第二接口将内存中与所述文件操作对应的日志文件写入所述后端日志存储系统；若所述签名验证不通过，则停止调用所述第二接口将内存中与所述文件操作对应的日志文件写入所述后端日志存储系统。
请参阅图8，本申请实施例提供的一种写入单元540，所述写入单元540包括：
文件创建模块542，用于创建与所述文件操作对应的日志文件。
所述文件创建模块542，还用于在所述日志文件系统的挂载路径的挂载点，按照所述可移植操作系统接口标准创建所述日志文件。
进一步的，所述文件创建模块542，还用于创建所述日志文件的元数据信息，其中所述元数据信息包括所述日志文件的文件名、创建信息、文件大小以及挂载路径；将 所述元数据信息写入所述日志文件系统中的目标数据库。
进一步的，所述文件创建模块542还用于根据所述挂载路径从所述后端日志存储系统中获取所述日志文件的日志标识；将所述日志标识写入到所述元数据信息中。
数据写入模块544，用于打开所述日志文件，将日志数据写入所述日志文件。
日志写入模块546，用于将所述日志文件通过所述第二接口写入所述后端日志存储系统。
请参阅图9，本申请实施例提供的一种日志写入装置600，所述装置600包括：
初始化单元610，用于初始化所述日志文件系统。具体的，创建所述目标数据库的元数据实例，以根据所述元数据实例确定所述日志文件在所述日志文件系统中的位置；创建所述目标数据库的缓存数据实例，以根据所述缓存数据实例访问所述日志文件的日志数据；初始化所述日志文件的根目录，以初始化所述日志文件系统。
操作获取单元620，用于获取业务程序的文件操作。具体的，用于基于所述目标子线程获取业务程序的文件操作。
进一步的，用于获取所述业务程序的多个子线程；从所述多个子线程中获取与所述业务程序的文件操作对应的子线程作为所述目标子线程。
第一接口获取单元630，用于从日志文件系统配置的多个系统接口中获取与所述文件操作对应的第一接口，其中所述系统接口为基于可移植操作系统接口标准创建的接口。
具体的，用于基于所述目标子线程从日志文件系统配置的多个系统接口中获取与所述文件操作对应的第一接口
第二接口获取单元640，用于从后端日志存储系统配置的多个接口中获取与所述第一接口对应的第二接口。
具体的，用于基于所述目标子线程从后端日志存储系统配置的多个接口中获取与所述第一接口对应的第二接口。
写入单元650，用于调用所述第二接口将内存中与所述文件操作对应的日志文件写入所述后端日志存储系统。
具体的，用于基于所述目标子线程调用所述第二接口将内存中与所述文件操作对应的日志文件写入所述后端日志存储系统；其中，通过令牌桶算法控制所述目标子线程将内存中与所述文件操作对应的日志文件写入所述后端日志存储系统的写入速度。
判断单元660，用于将所述第一接口根据所述挂载路径返回给所述业务程序，以用于所述业务程序判断与所述第一接口对应的文件操作是否完成。
需要说明的是，本申请中装置实施例与前述方法实施例是相互对应的，装置实施例中具体的原理可以参见前述方法实施例中的内容，此处不再赘述。
下面将结合图10对本申请提供的一种电子设备进行说明。
请参阅图10，基于上述的日志写入方法、装置，本申请实施例还提供的另一种可以执 行前述日志写入方法的电子设备200。电子设备200包括相互耦合的一个或多个(图中仅示出一个)处理器102、存储器104以及网络模块106。其中，该存储器104中存储有可以执行前述实施例中内容的程序，而处理器102可以执行该存储器104中存储的程序。
其中，处理器102可以包括一个或者多个处理核。处理器102利用各种接口和线路连接整个电子设备200内的各个部分，通过运行或执行存储在存储器104内的指令、程序、代码集或指令集，以及调用存储在存储器104内的数据，执行电子设备100的各种功能和处理数据。可选地，处理器102可以采用数字信号处理(Digital Signal Processing，DSP)、现场可编程门阵列(Field－Programmable Gate Array，FPGA)、可编程逻辑阵列(Programmable Logic Array，PLA)中的至少一种硬件形式来实现。处理器102可集成中央处理器(Central Processing Unit，CPU)、图像处理器(Graphics Processing Unit，GPU)和调制解调器等中的一种或几种的组合。其中，CPU主要处理操作系统、用户界面和应用程序等；GPU用于负责显示内容的渲染和绘制；调制解调器用于处理无线通信。可以理解的是，上述调制解调器也可以不集成到处理器102中，单独通过一块通信芯片进行实现。
存储器104可以包括随机存储器(Random Access Memory，RAM)，也可以包括只读存储器(Read-Only Memory)。存储器104可用于存储指令、程序、代码、代码集或指令集。存储器104可包括存储程序区和存储数据区，其中，存储程序区可存储用于实现操作系统的指令、用于实现至少一个功能的指令(比如触控功能、声音播放功能、图像播放功能等)、用于实现下述各个方法实施例的指令等。存储数据区还可以存储终端100在使用中所创建的数据(比如电话本、音视频数据、聊天记录数据)等。
所述网络模块106用于接收以及发送电磁波，实现电磁波与电信号的相互转换，从而与通讯网络或者其他设备进行通讯，例如和音频播放设备进行通讯。所述网络模块106可包括各种现有的用于执行这些功能的电路元件，例如，天线、射频收发器、数字信号处理器、加密/解密芯片、用户身份模块(SIM)卡、存储器等等。所述网络模块106可与各种网络如互联网、企业内部网、无线网络进行通讯或者通过无线网络与其他设备进行通讯。上述的无线网络可包括蜂窝式电话网、无线局域网或者城域网。例如，网络模块106可以与基站进行信息交互。
请参考图11，其示出了本申请实施例提供的一种计算机可读存储介质的结构框图。该计算机可读介质800中存储有程序代码，所述程序代码可被处理器调用执行上述方法实施例中所描述的方法。
计算机可读存储介质800可以是诸如闪存、EEPROM(电可擦除可编程只读存储器)、EPROM、硬盘或者ROM之类的电子存储器。可选地，计算机可读存储介质800包括非易失性计算机可读介质(non-transitory computer-readable storage medium)。计算机可读存储介质800具有执行上述方法中的任何方法步骤的程序代码810的存储空间。这些程序代码可以从一个或者多个计算机程序产品中读出或者写入到这一个或者多个计算机程序产品中。程序代码810可以例如以适当形式进行压缩。
本申请提供的一种日志写入方法、装置、电子设备以及存储介质，获取业务程序的文件操作，从日志文件系统配置的多个系统接口中获取与所述文件操作对应的第一接口，其中所述系统接口为基于可移植操作系统接口标准定义的接口，从后端日志存储系统配置的多个接口中获取与所述第一接口对应的第二接口，调用所述第二接口将内存中与所述文件操作对应的日志文件写入所述后端日志存储系统。通过上述方法，可以将所生成的日志文件先直接写入到内存中，进而使得日志文件系统可以调用后端日志存储系统的第二接口直接将内存中的日志文件写入后端日志文件存储系统，使得日志文件不需要落盘到磁盘上，以便日志写入不再受限于本地磁盘的写入和读取性能，实现了日志的高速写入。
最后应说明的是：以上实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述实施例对本申请进行了详细的说明，本领域的普通技术人员当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不驱使相应技术方案的本质脱离本申请各实施例技术方案的精神和范围。

Claims (20)

  1. 一种日志写入方法，其特征在于，所述方法包括：
    获取业务程序的文件操作；
    从日志文件系统配置的多个系统接口中获取与所述文件操作对应的第一接口，其中所述系统接口为基于可移植操作系统接口标准定义的接口；
    从后端日志存储系统配置的多个接口中获取与所述第一接口对应的第二接口；
    调用所述第二接口将内存中与所述文件操作对应的日志文件写入所述后端日志存储系统。
  2. 根据权利要求1所述的方法，其特征在于，所述获取业务程序的文件操作，包括：
    从所述日志文件系统所在系统的系统内核获取业务程序的文件操作，所述业务程序的文件操作由所述系统内核从所述日志文件系统的挂载路径获取到。
  3. 根据权利要求1所述的方法，其特征在于，所述调用所述第二接口将内存中与所述文件操作对应的日志文件写入所述后端日志存储系统，包括：
    创建与所述文件操作对应的日志文件；
    打开所述日志文件，将日志数据写入所述日志文件；
    将所述日志文件通过所述第二接口写入所述后端日志存储系统。
  4. 根据权利要求3所述的方法，其特征在于，所述创建与所述文件操作对应的日志文件包括：
    在所述日志文件系统的挂载路径的挂载点，按照所述可移植操作系统接口标准创建所述日志文件。
  5. 根据权利要求1-4任一所述方法，其特征在于，所述方法还包括：
    创建所述日志文件的元数据信息，其中所述元数据信息包括所述日志文件的文件名、创建信息、文件大小以及挂载路径；
    将所述元数据信息写入所述日志文件系统中的目标数据库。
  6. 根据权利要求5所述的方法，其特征在于，所述将所述元数据信息写入所述目标数据库，包括：
    根据所述挂载路径从所述后端日志存储系统中获取所述日志文件的日志标识；
    将所述日志标识写入到所述元数据信息中。
  7. 根据权利要求6所述的方法，其特征在于，所述获取业务程序的文件操作，之前还包括：
    初始化所述日志文件系统；
    所述初始化所述日志文件系统，包括：
    创建所述目标数据库的元数据实例，以根据所述元数据实例确定所述日志文件在所述日志文件系统中的位置；
    创建所述目标数据库的缓存数据实例，以根据所述缓存数据实例访问所述日志文件的日志数据；
    初始化所述日志文件的根目录，以初始化所述日志文件系统。
  8. 根据权利要求2-7任一所述的方法，其特征在于，所述调用所述第二接口将内存中与所述文件操作对应的日志文件写入所述后端日志存储系统之后，还包括：
    将所述第一接口根据所述挂载路径返回给所述业务程序，以用于所述业务程序判断与所述第一接口对应的文件操作是否完成。
  9. 根据权利要求5-7任一所述的方法，其特征在于，所述目标数据库为key/value数据库，所述key/value数据库的数据结构为哈希表结构。
  10. 根据权利要求1-9任一所述的方法，其特征在于，所述调用所述第二接口将内存中与所述文件操作对应的日志文件写入所述后端日志存储系统，包括：
    从多个子线程中获取目标子线程；
    基于所述目标子线程调用所述第二接口将内存中与所述文件操作对应的日志文件写入所述后端日志存储系统；
    其中，通过令牌桶算法控制所述目标子线程将内存中与所述文件操作对应的日志文件写入所述后端日志存储系统的写入速度。
  11. 根据权利要求1所述的方法，其特征在于，所述获取业务程序的文件操作之前，还包括：
    从多个子线程中获取目标子线程；
    所述获取业务程序的文件操作，包括：
    基于所述目标子线程获取业务程序的文件操作；
    所述从日志文件系统配置的多个系统接口中获取与所述文件操作对应的第一接口，其中所述系统接口为基于可移植操作系统接口标准创建的接口，包括：
    基于所述目标子线程从日志文件系统配置的多个系统接口中获取与所述文件操作对应的第一接口；
    所述从后端日志存储系统配置的多个接口中获取与所述第一接口对应的第二接口，包括：
    基于所述目标子线程从后端日志存储系统配置的多个接口中获取与所述第一接口对应的第二接口；
    所述调用所述第二接口将内存中与所述文件操作对应的日志文件写入所述后端日志存储系统，包括：
    基于所述目标子线程调用所述第二接口将内存中与所述文件操作对应的日志文件写入所述后端日志存储系统；
    其中，通过令牌桶算法控制所述目标子线程将内存中与所述文件操作对应的日志文件写入所述后端日志存储系统的写入速度。
  12. 根据权利要求10或11所述的方法，其特征在于，所述从多个子线程中获取目标子线程，包括：
    获取所述业务程序的多个子线程；
    从所述多个子线程中获取与所述业务程序的文件操作对应的子线程作为所述目标子线程。
  13. 根据权利要求1-12任一所述的方法，其特征在于，所述调用所述第二接口将内存中与所述文件操作对应的日志文件写入所述后端日志存储系统，包括：
    获取调用所述第二接口的接口调用请求，所述接口调用请求携带加密的签名；
    对所述接口调用请求进行签名验证；
    若签名验证通过，则调用所述第二接口将内存中与所述文件操作对应的日志文件写入所述后端日志存储系统。
  14. 根据权利要求13所述的方法，其特征在于，所述方法还包括：
    若所述签名验证不通过，则停止调用所述第二接口将内存中与所述文件操作对应的日志文件写入所述后端日志存储系统。
  15. 根据权利要求1-14任一所述的方法，其特征在于，所述日志文件系统为分布式日志文件系统。
  16. 一种日志写入装置，其特征在于，所述装置包括：
    操作获取单元，用于获取业务程序的文件操作；
    第一接口获取单元，用于从日志文件系统配置的多个系统接口中获取与所述文件操作对应的第一接口，其中所述系统接口为基于可移植操作系统接口标准定义的接口；
    第二接口获取单元，用于从后端日志存储系统配置的多个接口中获取与所述第一接口对应的第二接口；
    写入单元，用于调用所述第二接口将内存中与所述文件操作对应的日志文件写入所述后端日志存储系统。
  17. 根据权利要求16所述的装置，其特征在于，所述操作获取单元还用于从所述日志文件系统所在系统的系统内核获取业务程序的文件操作，所述获取业务程序的文件操作由所述系统内核从所述日志文件系统的挂载路径获取到。
  18. 根据权利要求16所述的装置，其特征在于，所述写入单元包括：
    文件创建模块，用于创建与所述文件操作对应的日志文件；
    数据写入模块，用于打开所述日志文件，将日志数据写入所述日志文件；
    日志写入模块，用于将所述日志文件通过所述第二接口写入所述后端日志存储系统。
  19. 一种电子设备，其特征在于，包括一个或多个处理器以及存储器；一个或多 个程序被存储在所述存储器中并被配置为由所述一个或多个处理器执行权利要求1-15任一所述的方法。
  20. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质中存储有程序代码，其中，在所述程序代码被处理器运行时执行权利要求1-15任一所述的方法。
PCT/CN2020/089766 2020-05-12 2020-05-12 日志写入方法、装置、电子设备以及存储介质 WO2021226822A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2020/089766 WO2021226822A1 (zh) 2020-05-12 2020-05-12 日志写入方法、装置、电子设备以及存储介质
CN202080095208.XA CN115039082A (zh) 2020-05-12 2020-05-12 日志写入方法、装置、电子设备以及存储介质

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2020/089766 WO2021226822A1 (zh) 2020-05-12 2020-05-12 日志写入方法、装置、电子设备以及存储介质

Publications (1)

Publication Number Publication Date
WO2021226822A1 true WO2021226822A1 (zh) 2021-11-18

Family

ID=78525554

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/089766 WO2021226822A1 (zh) 2020-05-12 2020-05-12 日志写入方法、装置、电子设备以及存储介质

Country Status (2)

Country Link
CN (1) CN115039082A (zh)
WO (1) WO2021226822A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113885809A (zh) * 2021-12-07 2022-01-04 云和恩墨(北京)信息技术有限公司 数据管理系统及方法
CN114564460A (zh) * 2022-02-25 2022-05-31 苏州浪潮智能科技有限公司 基于分布式存储系统的参数调优方法、装置、设备及介质
CN118012732A (zh) * 2024-04-08 2024-05-10 合众新能源汽车股份有限公司 一种日志管理方法、装置及电子设备

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090222618A1 (en) * 2008-02-29 2009-09-03 Samsung Electronics Co., Ltd. Memory system and block merge method
CN102521114A (zh) * 2011-11-30 2012-06-27 华中科技大学 一种虚拟化环境下的文件系统日志存储系统
CN108170768A (zh) * 2017-12-25 2018-06-15 腾讯科技（深圳）有限公司 数据库同步方法、装置及可读介质
CN109582658A (zh) * 2018-12-03 2019-04-05 郑州云海信息技术有限公司 一种分布式文件系统实现数据一致性的方法及装置

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090222618A1 (en) * 2008-02-29 2009-09-03 Samsung Electronics Co., Ltd. Memory system and block merge method
CN102521114A (zh) * 2011-11-30 2012-06-27 华中科技大学 一种虚拟化环境下的文件系统日志存储系统
CN108170768A (zh) * 2017-12-25 2018-06-15 腾讯科技（深圳）有限公司 数据库同步方法、装置及可读介质
CN109582658A (zh) * 2018-12-03 2019-04-05 郑州云海信息技术有限公司 一种分布式文件系统实现数据一致性的方法及装置

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113885809A (zh) * 2021-12-07 2022-01-04 云和恩墨(北京)信息技术有限公司 数据管理系统及方法
CN113885809B (zh) * 2021-12-07 2022-03-18 云和恩墨(北京)信息技术有限公司 数据管理系统及方法
CN114564460A (zh) * 2022-02-25 2022-05-31 苏州浪潮智能科技有限公司 基于分布式存储系统的参数调优方法、装置、设备及介质
CN114564460B (zh) * 2022-02-25 2024-01-19 苏州浪潮智能科技有限公司 基于分布式存储系统的参数调优方法、装置、设备及介质
CN118012732A (zh) * 2024-04-08 2024-05-10 合众新能源汽车股份有限公司 一种日志管理方法、装置及电子设备
CN118012732B (zh) * 2024-04-08 2024-06-28 合众新能源汽车股份有限公司 一种日志管理方法、装置及电子设备

Also Published As

Publication number Publication date
CN115039082A (zh) 2022-09-09

Similar Documents

Publication Publication Date Title
WO2021226822A1 (zh) 日志写入方法、装置、电子设备以及存储介质
EP3933580B1 (en) Method and device for deploying virtual machine or container
US11157445B2 (en) Indexing implementing method and system in file storage
US11296940B2 (en) Centralized configuration data in a distributed file system
US9940331B1 (en) Proactive scavenging of file system snaps
CN109145051A (zh) 分布式数据库的数据汇总方法及装置和电子设备
CN108427677B (zh) 一种对象访问方法、装置及电子设备
US11625192B2 (en) Peer storage compute sharing using memory buffer
US11444785B2 (en) Establishment of trusted communication with container-based services
WO2014166446A1 (zh) 文件访问处理方法、系统及计算机存储介质
US10262024B1 (en) Providing consistent access to data objects transcending storage limitations in a non-relational data store
CN108846129B (zh) 存储数据访问方法、装置及存储介质
EP3497586A1 (en) Discovery of calling application for control of file hydration behavior
US11157456B2 (en) Replication of data in a distributed file system using an arbiter
CN113722114A (zh) 一种数据服务的处理方法、装置、计算设备及存储介质
WO2016101662A1 (zh) 一种数据处理方法及相关服务器
US11853284B2 (en) In-place updates with concurrent reads in a decomposed state
CN111803917A (zh) 资源的处理方法和装置
CN107181773A (zh) 分布式存储系统的数据存储及数据管理方法、设备
WO2021254214A1 (zh) 安卓应用程序迁移的方法和装置
CN115098297B (zh) 一种云原生存储数据卷的一致性快照生成方法和系统
WO2019109538A1 (zh) 一种分布式数据存储方法及装置
CN114895852A (zh) 数据处理方法、系统、装置、存储介质以及电子设备
US11526446B1 (en) Modifying caching amongst services from a history of requests and responses
CN110554915A (zh) 一种信息处理的方法、装置、计算机存储介质及终端

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20935134

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 18.04.2023)

122 Ep: pct application non-entry in european phase

Ref document number: 20935134

Country of ref document: EP

Kind code of ref document: A1