CN115167784A - Data writing method, device, equipment and storage medium - Google Patents

Data writing method, device, equipment and storage medium Download PDF

Info

Publication number
CN115167784A
CN115167784A CN202211075892.2A CN202211075892A CN115167784A CN 115167784 A CN115167784 A CN 115167784A CN 202211075892 A CN202211075892 A CN 202211075892A CN 115167784 A CN115167784 A CN 115167784A
Authority
CN
China
Prior art keywords
party
random access
access memory
controller
nonvolatile random
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202211075892.2A
Other languages
Chinese (zh)
Inventor
钟戟
彭云武
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Suzhou Inspur Intelligent Technology Co Ltd
Original Assignee
Suzhou Inspur Intelligent Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Suzhou Inspur Intelligent Technology Co Ltd filed Critical Suzhou Inspur Intelligent Technology Co Ltd
Priority to CN202211075892.2A priority Critical patent/CN115167784A/en
Publication of CN115167784A publication Critical patent/CN115167784A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0602Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/061Improving I/O performance
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0602Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0625Power saving in storage systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0628Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0629Configuration or reconfiguration of storage systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0668Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671In-line storage system
    • G06F3/0673Single storage device
    • G06F3/0679Non-volatile semiconductor memory device, e.g. flash memory, one time programmable memory [OTP]

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Techniques For Improving Reliability Of Storages (AREA)
  • Memory System Of A Hierarchy Structure (AREA)

Abstract

The application relates to the technical field of storage, and discloses a data writing method, a device, equipment and a storage medium, which comprises the following steps: writing the target data into a cache space of a third-party nonvolatile random access memory connected with a target controller of the storage array; the target data is data input into the target controller; when the storage array is abnormally powered off, the standby power module arranged in the storage array only supplies power to the control component in the controller where the standby power module is located and the third-party nonvolatile random access memory, so that the control component stores the target data written in the cache space of the third-party nonvolatile random access memory into the nonvolatile medium of the third-party nonvolatile random access memory. When the power failure occurs abnormally in the storage array, the standby power module only needs to supply power to the third-party nonvolatile random access memory and the control assembly, high-efficiency data writing under the miniaturization of the standby power module is achieved, and meanwhile data reliability is guaranteed.

Description

Data writing method, device and equipment and storage medium
Technical Field
The present invention relates to the field of storage technologies, and in particular, to a data writing method, apparatus, device, and storage medium.
Background
The data reliability of the memory array is extremely important as a device for storing data. When the ac power input by the storage array device is abnormally powered down, the data in the memory of the storage controller is not written into the nonvolatile medium and may be lost. Generally, a BBU (Battery Backup Unit) in a storage array device provides a Backup power supply to ensure that a processor of the storage array stores data in a memory to a nonvolatile medium, such as a hard Disk, a Solid-State Disk (SSD), and the like.
In the prior art, if the storage array device is abnormally powered down, the standby power module supplies power to the minimum unit of each controller, and the key devices comprise: processors, memory sticks, SSD system disks, complex Programmable Logic Devices (CPLDs), and the like. Along with the high performance design of the storage array controller, the standby power consumption is larger and larger, the energy required to be provided by the standby power module is larger and larger, the cell energy density of the standby power module is not obviously improved, the number of cells required by the standby power module is increased, the size of the standby power module is larger and larger, and the design of a storage array product is difficult to meet.
Therefore, how to miniaturize the standby power module in the memory array is a technical problem to be solved by those skilled in the art.
Disclosure of Invention
In view of this, the present invention provides a data writing method, apparatus, device and storage medium, which implement high-efficiency data writing with a small standby power module and ensure data reliability. The specific scheme is as follows:
a first aspect of the present application provides a data writing method, including:
writing the target data into a cache space of a third-party nonvolatile random access memory connected with a target controller of the storage array; wherein the target data is data input into the target controller;
when the storage array is abnormally powered off, the standby power module arranged in the storage array only supplies power to the control component in the controller where the standby power module is located and the third-party nonvolatile random access memory, so that the control component stores the target data written in the cache space of the third-party nonvolatile random access memory into the nonvolatile medium of the third-party nonvolatile random access memory.
Optionally, before writing the target data into a cache space of a third-party nonvolatile random access memory connected to the storage array, the method further includes:
controlling a server on the side of the target controller to input the target data to the target controller through an I/O (input/output) interface and writing the target data into a cache space of the target controller;
after the target data is written into the cache space of the third-party nonvolatile random access memory connected with the target controller of the storage array, the method further comprises the following steps:
and modifying the data type of the target data in the cache space of the target controller so that the target data written into the cache space of the target controller can be cleared or replaced.
Optionally, after writing the target data into the cache space of the target controller, the method further includes:
and controlling the target controller to judge the I/O operation corresponding to the target data, and if the I/O operation is a write operation, executing the step of writing the target data into the cache space of the third-party nonvolatile random access memory.
Optionally, after writing the target data into a cache space of a third-party nonvolatile random access memory connected to the storage array, the method further includes:
and controlling the target controller to analyze the target data to obtain management information of the target data, and sending the management information to other controllers in the storage array so as to index corresponding data from the third-party nonvolatile random access memory through the other controllers according to the management information.
Optionally, the sending the management information to other controllers in the storage array includes:
and sending the management information to the cache spaces of other controllers in the storage array.
Optionally, after sending the management information to other controllers in the storage array, the method further includes:
and controlling the target controller to send confirmation information to the server so that the server confirms that the writing operation is completed after receiving the confirmation information.
Optionally, after the target controller is controlled to analyze the target data to obtain the management information of the target data, the method further includes:
and writing the management information into a cache space of the third-party nonvolatile random access memory, so that when the control component stores the target data into the nonvolatile medium due to abnormal power failure of the storage array, the control component simultaneously stores the management information written into the cache space of the third-party nonvolatile random access memory into the nonvolatile medium.
Optionally, the data writing method further includes:
setting a plurality of third-party nonvolatile random access memories, and writing the target data in cache spaces of the plurality of third-party nonvolatile random access memories;
when the storage array is abnormally powered off, the control assembly in the controller of the power backup module and the at least one third-party nonvolatile random access memory are only powered by the plurality of power backup modules, so that the control assembly stores the target data written in the cache space of the at least one third-party nonvolatile random access memory into the nonvolatile medium of the third-party nonvolatile random access memory.
Optionally, the storage array is a dual-controller storage array, and the third-party nonvolatile random access memory is a dual-port nonvolatile random access memory; and two ports of the third-party nonvolatile random access memory are respectively connected with two controllers in the storage array.
Optionally, each controller in the storage array is provided with the standby power module;
the power supply for only the control component in the controller where the power supply module is located and the third-party nonvolatile random access memory through the power supply module arranged in the storage array comprises:
determining one controller from all controllers in the storage array as a standby power controller, and only supplying power to the control component in the standby power controller and the third-party nonvolatile random access memory through the standby power module in the standby power controller.
Optionally, the controlling component stores the target data written in the cache space of the third-party nonvolatile random access memory into a nonvolatile medium of the third-party nonvolatile random access memory, and further includes:
the control component generates a first storage instruction and issues the first storage instruction to a main control chip in the third-party nonvolatile random access memory, so that the main control chip stores the target data written in the cache space of the third-party nonvolatile random access memory into the nonvolatile medium of the third-party nonvolatile random access memory according to the first storage instruction.
Optionally, after the controlling component stores the target data written in the cache space of the third-party nonvolatile random access memory into the nonvolatile medium of the third-party nonvolatile random access memory, the controlling component further includes:
the control component generates a closing instruction and executes the closing instruction, so that the power standby module in the power standby controller stops supplying power to the control component and the third-party nonvolatile random access memory.
A second aspect of the present application provides a data writing apparatus comprising:
the first cache module is used for writing the target data into a cache space of a third-party nonvolatile random access memory connected with a target controller of the storage array; wherein the target data is data input into the target controller;
and the standby power cache module is used for supplying power to the control component in the controller of the standby power module and the third-party nonvolatile random access memory only through the standby power module arranged in the storage array when the storage array is abnormally powered down, so that the control component stores the target data written in the cache space of the third-party nonvolatile random access memory into the nonvolatile medium of the third-party nonvolatile random access memory.
A third aspect of the application provides an electronic device comprising a processor and a memory; wherein the memory is used for storing a computer program which is loaded and executed by the processor to implement the aforementioned data writing method.
A fourth aspect of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored, and when the computer-executable instructions are loaded and executed by a processor, the foregoing data writing method is implemented.
In the method, target data are written into a cache space of a third-party nonvolatile random access memory connected with a target controller of a storage array; wherein the target data is data input into the target controller; when the storage array is abnormally powered off, the standby power module arranged in the storage array only supplies power to the control component in the controller where the standby power module is located and the third-party nonvolatile random access memory, so that the control component stores the target data written in the cache space of the third-party nonvolatile random access memory into the nonvolatile medium of the third-party nonvolatile random access memory. It can be seen that, this application introduces the third party nonvolatile random access memory who has buffer memory and nonvolatile medium simultaneously, when utilizing storage array write in data, write in the buffer memory of third party nonvolatile random access memory simultaneously, when the storage array takes place the unusual power failure, the power supply module only needs to supply power for third party nonvolatile random access memory and control assembly, write in the data in the buffer memory into nonvolatile medium again under control assembly's instruction, the power supply consumption of power supply module is not influenced by the high performance design of storage array controller, can reduce the power supply consumption, realize the miniaturization of power supply module, guarantee data write in efficiency and data reliability simultaneously.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
FIG. 1 is a flow chart of a data writing method provided in the present application;
FIG. 2 is a diagram illustrating a specific data writing method according to the present application;
FIG. 3 is a diagram of a data write architecture for a dual controller memory array according to the present application;
FIG. 4 is a flow chart of a specific data writing method provided herein;
FIG. 5 is a flow chart illustrating a prior art I/O data write operation of a dual-steering storage array according to the present application;
FIG. 6 is a flow chart of an I/O data write operation of a dual-control storage array according to an embodiment of the present application;
FIG. 7 is a schematic structural diagram of a data writing apparatus according to the present application;
fig. 8 is a structural diagram of a data writing electronic device according to the present application.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
At present, if the memory array device is abnormally powered down, a built-in power backup module (BBU) of the memory array device generally supplies power to the minimum unit of each controller to provide a backup power supply, so as to ensure that a processor of the memory array stores data in a memory to a nonvolatile medium. However, with the high performance design of the storage array controller, the standby power consumption is larger and larger, the energy required to be provided by the standby power module is larger and larger, the cell energy density of the standby power module is not significantly improved, the number of cells required by the standby power module is increased, so that the size of the standby power module is larger and larger, and the design of the storage array product is difficult to meet. In order to overcome the technical defects, the application provides a data writing scheme, when the storage array is abnormally powered down, the standby power module only needs to supply power to the third-party nonvolatile random access memory and the control assembly, high-efficiency data writing under miniaturization of the standby power module is achieved, and meanwhile data reliability is guaranteed.
Fig. 1 is a flowchart of a data writing method according to an embodiment of the present disclosure. Referring to fig. 1, the data writing method includes:
s11: writing the target data into a cache space of a third-party nonvolatile random access memory connected with a target controller of the storage array; wherein the target data is data input into the target controller.
In this embodiment, the storage array includes a plurality of controllers, a data disk SSD, and a third-party nonvolatile random access memory. The third-party nonvolatile random access memory is connected with the plurality of controllers, so that data can be synchronously stored through the third-party nonvolatile random access memory. The third-party nonvolatile Random Access Memory is denoted as NVRAM (NON-Volatile Random Access Memory), and belongs to a Random Access Memory capable of maintaining data after power failure. The third-party nonvolatile random access memory comprises a cache (such as DRAM) and a nonvolatile medium (such as Flash), wherein the DRAM is arranged to improve the cache efficiency.
In this embodiment, when a server inputs target data to a target controller of a storage array, the target data is written into a cache space of the third-party nonvolatile random access memory connected to the target controller. Wherein the target data is data input into the target controller of the storage array, typically I/O data.
Specifically, the memory array is a dual-controller memory array, and the third-party nonvolatile random access memory is a dual-port nonvolatile random access memory. And two ports of the third-party nonvolatile random access memory are respectively connected with two controllers in the storage array. That is, the storage array includes dual controllers, the design of the storage array system supports the dual controllers to access the dual-port NVRAM disk respectively, and the DRAM capacity of the NVRAM disk is large enough (for example, larger than 16 GB), which meets the write cache space requirement of the storage array. Under the condition, the controller and the processor can directly read and write the DRAM space of the NVRAM disk, the processor operates on a remote memory, the I/O read-write operation has extremely low time delay and the performance is consistent with that of the traditional storage array.
S12: when the storage array is abnormally powered off, the standby power module arranged in the storage array only supplies power to the control component in the controller where the standby power module is located and the third-party nonvolatile random access memory, so that the control component stores the target data written in the cache space of the third-party nonvolatile random access memory into the nonvolatile medium of the third-party nonvolatile random access memory.
In this embodiment, the power supply of the standby power module of the storage array crosses the backplane and can supply power to the NVRAM disk. When the storage array is abnormally powered down, only the control component in the controller where the power backup module is located and the third-party nonvolatile random access memory are powered through the power backup module arranged in the storage array, so that the control component stores the target data written in the cache space of the third-party nonvolatile random access memory into the nonvolatile medium of the third-party nonvolatile random access memory.
It can be understood that the controller of the storage array is provided with a power backup module, and the power backup module in any one of the controllers can be selected to supply power according to actual conditions, which is not limited in this embodiment. If the storage array is abnormally powered down, the standby power module only needs to supply power to the NVRAM disk and the control components of the controller, such as the CPLD and the like, and the CPLD informs the NVRAM disk to write the target data (I/O data) written in the DRAM into the Flash of the NVRAM disk.
Further, in order to improve the system storage fault tolerance rate and avoid data loss caused by damage to the NVRAM disk, in this embodiment, a plurality of third-party nonvolatile random access memories may be set, and the target data is written in the cache spaces of the plurality of third-party nonvolatile random access memories. When the storage array is abnormally powered off, the control assembly in the controller of the power backup module and the at least one third-party nonvolatile random access memory are only powered by the plurality of power backup modules, so that the control assembly stores the target data written in the cache space of the at least one third-party nonvolatile random access memory into the nonvolatile medium of the third-party nonvolatile random access memory.
As can be seen, in the embodiment of the present application, target data is written into a cache space of a third-party nonvolatile random access memory connected to a target controller of a storage array; wherein the target data is data input into a target controller of the storage array; when the storage array is abnormally powered off, the standby power module arranged in the storage array only supplies power to the control component in the controller where the standby power module is located and the third-party nonvolatile random access memory, so that the control component stores the target data written in the cache space of the third-party nonvolatile random access memory into the nonvolatile medium of the third-party nonvolatile random access memory. The embodiment of the application introduces the third-party nonvolatile random access memory with the cache and the nonvolatile medium, when data are written in by using the storage array, the data are written in the cache of the third-party nonvolatile random access memory simultaneously, when the abnormal power failure occurs in the storage array, the standby power module only needs to supply power for the third-party nonvolatile random access memory and the control component, the data in the cache are written in the nonvolatile medium again under the indication of the control component, the standby power consumption of the standby power module is not influenced by the high-performance design of the storage array controller, the standby power consumption can be reduced, the miniaturization of the standby power module is realized, and the data writing efficiency and the data reliability are ensured simultaneously.
Fig. 2 is a flowchart of a specific data writing method according to an embodiment of the present disclosure. Referring to fig. 2, the data writing method includes:
s21: and a server at the target controller side of the control storage array inputs the target data to the target controller through an I/O interface and writes the target data into a cache space of the target controller.
In this embodiment, the target data input into the storage array is generally written by a server on the host side, and first, the server on the target controller side that controls the storage array inputs the target data into the target controller through an I/O interface. Fig. 3 is a diagram of a data writing architecture of a dual-controller storage array according to this embodiment, which includes two controllers (controller 0 and controller 1), two NVRAM disks (NVRAM disk 0 and NVRAM disk 1), and multiple (X) I/O cards. In fig. 3, the server 0 is a server on the target controller side, and the controller 0 is the target storage. And directly writing the target data input into the target memory into the cache space of the target controller. That is, the host side server writes the I/O data into the DRAM (DDR) of the controller 0, and the controller 0 analyzes the type, use, updated or saved address, and the like of the I/O data;
s22: and controlling the target controller to judge the I/O operation corresponding to the target data, and if the I/O operation is a write operation, writing the target data into a cache space of a third-party nonvolatile random access memory connected with the target controller of the storage array.
S23: and modifying the data type of the target data in the cache space of the target controller so that the target data written in the cache space of the target controller can be cleared or replaced.
In this embodiment, after the target data is written into the cache space of the target controller, the target controller is controlled to determine the I/O operation corresponding to the target data, and if the I/O operation is a write operation, the target data is written into the cache space of the third-party nonvolatile random access memory connected to the target controller of the storage array. If the I/O operation is determined to be of the Write type, controller 0 retains the target data written in its DRAM and writes to the DRAM of the NVRAM disk.
In this embodiment, the data type of the target data in the cache space of the target controller is modified at the same time, so that the target data written in the cache space of the target controller can be removed or replaced. For example, controller 0 changes the I/O data in its DRAM to a Read type, after which the target data of the Read type may be cleared or replaced.
S24: and controlling the target controller to analyze the target data to obtain management information of the target data, and sending the management information to other controllers in the storage array so as to index corresponding data from the third-party nonvolatile random access memory according to the management information through the other controllers.
In this embodiment, in order to synchronize the target data to other controllers in the storage array, the target data needs to be analyzed to obtain management information of the target data. Namely, the target controller is controlled to analyze the target data to obtain the management information of the target data. Specifically, after writing the I/O data to the DRAM of the controller 0, the controller 0 needs to analyze the I/O data type, the purpose, the updated or saved address, and the like, which is the management information. The management information plays a role of data indexing, and is sent to other controllers in the storage array, so that corresponding data can be indexed from the third-party nonvolatile random access memory through the other controllers according to the management information.
S25: and writing the management information into a cache space of the third-party nonvolatile random access memory, so that when the control component stores the target data into the nonvolatile medium due to abnormal power failure of the storage array, the control component simultaneously stores the management information written into the cache space of the third-party nonvolatile random access memory into the nonvolatile medium.
In this embodiment, in order to avoid that the management information in each controller DRAM is lost when the power failure occurs abnormally, the management information may be written into a cache space of the third-party nonvolatile random access memory, so that when the control component stores the target data into the nonvolatile medium when the power failure occurs abnormally in the storage array, the control component simultaneously stores the management information written into the cache space of the third-party nonvolatile random access memory into the nonvolatile medium. When the server needs to read the target data after power-on recovery, the data in the nonvolatile medium and the corresponding management information are written into a DRAM of the NVRAM disk, and then the management information in the DRAM of the NVRAM disk is written into a controller of the server input read operation. It is to be noted that the order of execution of steps S22-S26 is not fixed.
S26: and controlling the target controller to send confirmation information to the server so that the server confirms that the writing operation is completed after receiving the confirmation information.
In this embodiment, after the management information is written into another controller of the storage array, the target controller needs to be further controlled to send confirmation information to the server, so that the server confirms that the writing operation is completed after receiving the confirmation information. That is, in the complete storage array I/O data write operation flow, the controller 0 needs to send information to the server on the host side to confirm that the I/O write operation is completed, and the server confirms that the write operation is completed.
S27: when the storage array is abnormally powered off, the standby power module arranged in the storage array only supplies power to the control component in the controller where the standby power module is located and the third-party nonvolatile random access memory, so that the control component stores the target data written in the cache space of the third-party nonvolatile random access memory into the nonvolatile medium of the third-party nonvolatile random access memory.
In this embodiment, as to the specific process of the step S27, reference may be made to corresponding contents disclosed in the foregoing embodiments, and details are not repeated here.
Therefore, in the embodiment of the present application, a server on a target controller side of a storage array is controlled to input the target data to the target controller through an I/O interface, and the target data is written into a cache space of the target controller. And controlling the target controller to analyze the target data to obtain management information of the target data, controlling the target controller to judge the I/O operation corresponding to the target data according to the management information, and writing the target data into a cache space of a third-party nonvolatile random access memory connected with the target controller of the storage array if the target controller is a write operation. And sending the management information to other controllers in the storage array so as to index corresponding data from the third-party nonvolatile random access memory according to the management information through the other controllers. On the basis, modifying the data type of the target data in the cache space of the target controller so that the target data written into the cache space of the target controller can be cleared or replaced. And finally, controlling the target controller to send confirmation information to the server so that the server confirms that the writing operation is completed after receiving the confirmation information. The scheme has the advantages of high writing process performance, low time delay and high reliability of data redundancy backup.
Fig. 4 is a flowchart of a specific data writing method according to an embodiment of the present application. Referring to fig. 4, the data writing method includes:
s31: writing the target data into a cache space of a third-party nonvolatile random access memory connected with a target controller of the storage array; wherein the target data is data input into the target controller.
For the specific process of the step S31, reference may be made to the corresponding contents disclosed in the foregoing embodiments, and details are not repeated here.
S32: when the storage array is abnormally powered down, one controller is determined from all controllers in the storage array to serve as a standby power controller, and only a control component in the standby power controller and the third-party nonvolatile random access memory are powered up through the standby power module in the standby power controller.
In this embodiment, each controller in the storage array is provided with the standby power module. When the storage array is abnormally powered down, one controller is determined from all controllers in the storage array to serve as a standby power controller, and a control assembly in the standby power controller and the third-party nonvolatile random access memory are powered up through the standby power module in the standby power controller. Generally, a controller that accepts I/O data of the host-side server, such as the controller 0 in fig. 3, may be selected, that is, the controller 0 in fig. 3 may be used as the standby power controller, and the CPLD component and the NVRAM disk in the standby power controller are powered through the standby power module of the controller 0.
S33: the control component generates a first storage instruction and issues the first storage instruction to a main control chip in the third-party nonvolatile random access memory, so that the main control chip stores the target data written in the cache space of the third-party nonvolatile random access memory into a nonvolatile medium of the third-party nonvolatile random access memory according to the first storage instruction.
In this embodiment, after power is supplied, the control component generates a first storage instruction and issues the first storage instruction to a main control chip in the third-party nonvolatile random access memory, where the main control chip may be an SSD main control chip. And the SSD master stores the target data written in the cache space of the third-party nonvolatile random access memory into a nonvolatile medium of the third-party nonvolatile random access memory according to the first storage instruction. Namely, if the storage array is abnormally powered down, the standby power module only needs to supply power to the NVRAM disk and the CPLD of the controller in the steps, and the CPLD informs the NVRAM disk through the system and writes the I/O data in the DRAM into the Flash of the NVRAM.
S34: the control component generates a closing instruction and executes the closing instruction, so that the power standby module in the power standby controller stops supplying power to the control component and the third-party nonvolatile random access memory.
In this embodiment, after the write I/O data is safely stored, the standby power module is turned off to stop supplying power. Specifically, the control component generates a shutdown instruction and executes the shutdown instruction, so that the standby power module in the standby power controller stops supplying power to the control component and the third-party nonvolatile random access memory, that is, the system shuts down the standby power module through the CPLD.
In this embodiment, when the storage array is abnormally powered down, one controller is determined from all the controllers in the storage array to serve as a standby power controller, and a control component in the standby power controller and the third-party nonvolatile random access memory are powered up through the standby power module in the standby power controller. After power is supplied, the control component generates a first storage instruction and issues the first storage instruction to a main control chip in the third-party nonvolatile random access memory, so that the main control chip stores the target data written in the cache space of the third-party nonvolatile random access memory into a nonvolatile medium of the third-party nonvolatile random access memory according to the first storage instruction. After data is safely written in, the control component generates a closing instruction and executes the closing instruction, so that the power standby module in the power standby controller stops supplying power to the control component and the third-party nonvolatile random access memory. The system starts the standby power module to be a third-party nonvolatile random access memory and closes the standby power module through a control component in the controller, so that the standby power controllability is improved.
The following is a description of a comparison between the effect of the conventional storage array I/O data writing operation and the effect of the storage array I/O data writing operation based on the above scheme, and the advantages of the scheme of the present application are highlighted.
FIG. 5 is a flow chart of a prior art I/O data write operation for a dual-steering storage array. After judging that the I/O is the Write type, the controller 0 is kept in the DRAM of the controller 0, and then the I/O data is written into the DRAM of the controller 0 through the PCIe mirror image channel to make data redundancy backup. Under the design, the single control supports 2 processors, 32 DDR5 memory banks and an SSD system disk, and the standby power consumption is larger and larger (as shown in Table 1). When the single controller is powered on, the power consumption is 1000W, and if a 18650 ternary lithium battery cell is selected, 12 batteries are required to form 4 strings and 3 parallel.
TABLE 1
Device with a metal layer Single power consumption (W) Number of Power consumption (W)
Processor with a memory having a plurality of memory cells 350 2 700
Memory bank 10 32 320
SSD 15 1 15
CPLD 3 1 3
Is totaled 1038
FIG. 6 is a flow chart of the I/O data write operation of the dual-control storage array according to the present application. The host side server writes the I/O data into the DRAM of the controller 0, and the controller 0 analyzes the type, use, updated or saved address, etc. of the I/O data; if the I/O is judged to be of the Write type, the controller 0 is kept in the DRAM of the controller and is written into the DRAM of the NVRAM disk; the I/O data in controller 0 ERAM becomes Read type and may be cleared or replaced; controller 0 synchronizes management information to controller 1, such as information on the type of I/O data, data size, address, location address saved in NVRAM, etc., ensuring that controller 1 can directly retrieve the I/O data in the NVRAM disk. The controller 0 sends information to the server at the host side to confirm that the I/O write operation is completed; the server acknowledges the end of the write operation. In the process, the double-control data mirroring synchronization operation is removed from the storage array I/O writing operation flow, and the storage array I/O writing operation flow is directly written into the NVRAM disk DRAM to complete the I/O writing operation. When the storage array is abnormally powered off, the standby power module only needs to supply power to the NVRAM and the CPLD, so that data in NVRAM DRAM are safely written into a Flash medium of the NVRAM, the standby power module of the storage array controller is miniaturized, the number of battery cells is reduced, the weight of the whole machine is reduced, more space is left, the heat dissipation of a system is optimized, or the controller supports a processor with a higher specification, I/O cards with more number and the like.
In the scheme of the application, the standby power module only needs to supply power to less devices such as an NVRAM disk, a CPLD (complex programmable logic device) of the controller and the like, so that the standby power module can be miniaturized, and the mass of the whole machine is reduced. As shown in table 2, the power consumption of the single controller during standby power is less than 100W, and if a 18650 ternary lithium battery cell is selected, only 3 nodes are needed to form 3 strings and 1 parallel; the number of the battery cores is reduced by 75%, and the volume, the weight and the cost of the standby power module are greatly reduced.
TABLE 2
Device with a metal layer Single power consumption (W) Number of Power consumption (W)
NVRAM 20 2 40
CPLD 3 1 3
Total up to 43
It is understood that the cells store charge, are sensitive to temperature and humidity, and are hazardous. If the number of the battery cells of the standby power module is large, the failure rate is high. When the ternary lithium cell exceeds 12 knots, there is a risk of a rated charge exceeding 100Wh, which may not allow air transport. After the power supply module is miniaturized, the number of the battery cores is reduced, the failure rate is reduced, and the risk of air transportation control is avoided. After the size of the standby power module is reduced, the heat dissipation of the storage array can be optimized, more space is reserved to support a processor with larger power consumption, more I/O cards are supported, and the like, and the product specification is improved.
Referring to fig. 7, an embodiment of the present application further discloses a data writing apparatus, which includes:
the first cache module 11 is configured to write target data into a cache space of a third-party nonvolatile random access memory connected to a target controller of the storage array; wherein the target data is data input into the target controller;
the backup power cache module 12 is configured to, when the storage array is abnormally powered down, supply power to only the control component in the controller where the backup power module is located and the third-party nonvolatile random access memory through the backup power module arranged in the storage array, so that the control component stores the target data written in the cache space of the third-party nonvolatile random access memory into the nonvolatile medium of the third-party nonvolatile random access memory.
As can be seen, in the embodiment of the present application, target data is written into a cache space of a third-party nonvolatile random access memory connected to a target controller of a storage array; wherein the target data is data input into the target controller; when the storage array is abnormally powered off, the standby power module arranged in the storage array only supplies power to the control component in the controller where the standby power module is located and the third-party nonvolatile random access memory, so that the control component stores the target data written in the cache space of the third-party nonvolatile random access memory into the nonvolatile medium of the third-party nonvolatile random access memory. The embodiment of the application introduces the third-party nonvolatile random access memory with the cache and the nonvolatile medium, when data are written in by using the storage array, the data are written in the cache of the third-party nonvolatile random access memory simultaneously, when the abnormal power failure occurs in the storage array, the standby power module only needs to supply power for the third-party nonvolatile random access memory and the control component, the data in the cache are written in the nonvolatile medium again under the indication of the control component, the standby power consumption of the standby power module is not influenced by the high-performance design of the storage array controller, the standby power consumption can be reduced, the miniaturization of the standby power module is realized, and the data writing efficiency and the data reliability are ensured simultaneously.
In some embodiments, the data writing apparatus further includes:
the second cache module is used for controlling a server on the side of the target controller to input the target data to the target controller through an I/O (input/output) interface and writing the target data into a cache space of the target controller;
an operation type judgment module, configured to control the target controller to judge an I/O operation corresponding to the target data, and if the operation is a write operation, perform a step of writing the target data into a cache space of the third-party nonvolatile random access memory;
the data type modification module is used for modifying the data type of the target data so that the target data written into the cache space of the target controller can be removed or replaced;
the data analysis module is used for controlling the target controller to analyze the target data to obtain management information of the target data;
the first sending module is used for sending the management information to other controllers in the storage array so as to index corresponding data from the third-party nonvolatile random access memory through the other controllers according to the management information;
a second sending module, configured to write the management information into a cache space of the third-party nonvolatile random access memory, so that when the control component stores the target data into the nonvolatile medium due to abnormal power failure of the storage array, the control component simultaneously stores the management information written into the cache space of the third-party nonvolatile random access memory into the nonvolatile medium;
the write feedback module is used for controlling the target controller to send confirmation information to the server so that the server confirms that the write operation is completed after receiving the confirmation information;
the memory backup module is used for setting a plurality of third-party nonvolatile random access memories and writing the target data into cache spaces of the third-party nonvolatile random access memories; when the storage array is abnormally powered off, the control assembly in the controller of the power backup module and the at least one third-party nonvolatile random access memory are only powered by the plurality of power backup modules, so that the control assembly stores the target data written in the cache space of the at least one third-party nonvolatile random access memory into the nonvolatile medium of the third-party nonvolatile random access memory.
In some embodiments, the first sending module is specifically configured to send the management information to cache spaces of other controllers in the storage array.
In some specific embodiments, the backup cache module 12 specifically includes:
a standby power module selection unit, configured to determine one controller from all controllers in the storage array as a standby power controller, and supply power to only the control component in the standby power controller and the third-party nonvolatile random access memory through the standby power module in the standby power controller;
and the cache unit is used for generating a first storage instruction by the control component and sending the first storage instruction to a main control chip in the third-party nonvolatile random access memory, so that the main control chip stores the target data written in the cache space of the third-party nonvolatile random access memory into the nonvolatile medium of the third-party nonvolatile random access memory according to the first storage instruction.
In some embodiments, the data writing apparatus further includes:
and the power supply stopping module is used for generating a closing instruction by the control component and executing the closing instruction so as to enable the power supply stopping module in the power supply controller to stop supplying power to the control component and the third-party nonvolatile random access memory.
Further, the embodiment of the application also provides electronic equipment. FIG. 8 is a block diagram illustrating an electronic device 20 according to an exemplary embodiment, and nothing in the figure should be taken as a limitation on the scope of use of the present application.
Fig. 8 is a schematic structural diagram of an electronic device 20 according to an embodiment of the present disclosure. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input output interface 25, and a communication bus 26. The memory 22 is used for storing a computer program, and the computer program is loaded and executed by the processor 21 to implement the relevant steps in the data writing method disclosed in any of the foregoing embodiments. In addition, the electronic device 20 in the present embodiment may specifically be 8.
In this embodiment, the power supply 23 is configured to provide a working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and an external device, and a communication protocol followed by the communication interface is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited herein; the input/output interface 25 is configured to obtain external input data or output data to the outside, and a specific interface type thereof may be selected according to specific application requirements, which is not specifically limited herein.
In addition, the storage 22 is used as a carrier for storing resources, and may be a read-only memory, a random access memory, a magnetic disk or an optical disk, etc., and the resources stored thereon may include an operating system 221, a computer program 222, data 223, etc., and the storage manner may be a transient storage or a permanent storage.
The operating system 221 is configured to manage and control each hardware device and the computer program 222 on the electronic device 20, so as to implement the operation and processing of the mass data 223 in the memory 22 by the processor 21, and may be Windows Server, netware, unix, linux, or the like. The computer program 222 may further include a computer program that can be used to perform other specific tasks in addition to the computer program that can be used to perform the data writing method by the electronic device 20 disclosed in any of the foregoing embodiments. Data 223 may include target data collected by electronic device 20.
Further, an embodiment of the present application further discloses a storage medium, in which a computer program is stored, and when the computer program is loaded and executed by a processor, the steps of the data writing method disclosed in any of the foregoing embodiments are implemented.
The embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same or similar parts among the embodiments are referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description.
Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising a …" does not exclude the presence of another identical element in a process, method, article, or apparatus that comprises the element.
The data writing method, apparatus, device and storage medium provided by the present invention are described in detail above, and the principle and implementation of the present invention are explained herein by applying specific examples, and the description of the above examples is only used to help understanding the method and core idea of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, the specific embodiments and the application range may be changed, and in summary, the content of the present specification should not be construed as a limitation to the present invention.

Claims (15)

1. A method of writing data, comprising:
writing the target data into a cache space of a third-party nonvolatile random access memory connected with a target controller of the storage array; wherein the target data is data input into the target controller;
when the storage array is abnormally powered off, the standby power module arranged in the storage array only supplies power to the control component in the controller where the standby power module is located and the third-party nonvolatile random access memory, so that the control component stores the target data written in the cache space of the third-party nonvolatile random access memory into the nonvolatile medium of the third-party nonvolatile random access memory.
2. The data writing method according to claim 1, wherein before writing the target data into the cache space of the third-party nonvolatile random access memory connected to the storage array, the method further comprises:
controlling a server on the side of the target controller to input the target data to the target controller through an I/O (input/output) interface and writing the target data into a cache space of the target controller;
after the target data is written into the cache space of the third-party nonvolatile random access memory connected with the target controller of the storage array, the method further comprises the following steps:
and modifying the data type of the target data in the cache space of the target controller so that the target data written into the cache space of the target controller can be cleared or replaced.
3. The data writing method according to claim 2, wherein after the writing the target data into the cache space of the target controller, further comprising:
and controlling the target controller to judge the I/O operation corresponding to the target data, and if the I/O operation is a write operation, executing the step of writing the target data into the cache space of the third-party nonvolatile random access memory.
4. The data writing method according to claim 2, wherein after the writing the target data into the cache space of the target controller, further comprising:
and controlling the target controller to analyze the target data to obtain management information of the target data, and sending the management information to other controllers in the storage array so as to index corresponding data from the third-party nonvolatile random access memory through the other controllers according to the management information.
5. The data writing method according to claim 4, wherein the sending the management information to other controllers in the storage array comprises:
and sending the management information to the cache spaces of other controllers in the storage array.
6. The data writing method according to claim 4, wherein after sending the management information to other controllers in the storage array, further comprising:
and controlling the target controller to send confirmation information to the server so that the server confirms that the writing operation is completed after receiving the confirmation information.
7. The data writing method according to claim 4, wherein after the target controller is controlled to analyze the target data to obtain the management information of the target data, the method further comprises:
and writing the management information into a cache space of the third-party nonvolatile random access memory, so that when the control component stores the target data into the nonvolatile medium due to abnormal power failure of the storage array, the control component simultaneously stores the management information written into the cache space of the third-party nonvolatile random access memory into the nonvolatile medium.
8. The data writing method according to claim 1, further comprising:
setting a plurality of third-party nonvolatile random access memories, and writing the target data in cache spaces of the plurality of third-party nonvolatile random access memories;
when the storage array is abnormally powered off, the control assembly in the controller of the power backup module and the at least one third-party nonvolatile random access memory are only powered by the plurality of power backup modules, so that the control assembly stores the target data written in the cache space of the at least one third-party nonvolatile random access memory into the nonvolatile medium of the third-party nonvolatile random access memory.
9. The data writing method according to claim 1, wherein the memory array is a dual controller memory array, and the third-party nonvolatile random access memory is a dual port nonvolatile random access memory; and two ports of the third-party nonvolatile random access memory are respectively connected with two controllers in the storage array.
10. The data writing method according to any one of claims 1 to 9, wherein the standby power module is provided in each controller in the storage array;
the power supply for only the control component in the controller where the power supply module is located and the third-party nonvolatile random access memory through the power supply module arranged in the storage array comprises:
determining one controller from all controllers in the storage array as a standby power controller, and only supplying power to the control component in the standby power controller and the third-party nonvolatile random access memory through the standby power module in the standby power controller.
11. The data writing method according to claim 10, wherein the control component stores the target data written in the cache space of the third-party nonvolatile random access memory into a nonvolatile medium of the third-party nonvolatile random access memory, further comprising:
the control component generates a first storage instruction and issues the first storage instruction to a main control chip in the third-party nonvolatile random access memory, so that the main control chip stores the target data written in the cache space of the third-party nonvolatile random access memory into the nonvolatile medium of the third-party nonvolatile random access memory according to the first storage instruction.
12. The data writing method according to claim 10, wherein after the control component stores the target data written in the cache space of the third-party nonvolatile random access memory into the nonvolatile medium of the third-party nonvolatile random access memory, the method further comprises:
the control component generates a closing instruction and executes the closing instruction, so that the power standby module in the power standby controller stops supplying power to the control component and the third-party nonvolatile random access memory.
13. A data writing apparatus, comprising:
the first cache module is used for writing the target data into a cache space of a third-party nonvolatile random access memory connected with a target controller of the storage array; wherein the target data is data input into the target controller;
and the standby power cache module is used for supplying power to the control component in the controller of the standby power module and the third-party nonvolatile random access memory only through the standby power module arranged in the storage array when the storage array is abnormally powered down, so that the control component stores the target data written in the cache space of the third-party nonvolatile random access memory into the nonvolatile medium of the third-party nonvolatile random access memory.
14. An electronic device, comprising a processor and a memory; wherein the memory is used for storing a computer program which is loaded and executed by the processor to implement the data writing method according to any one of claims 1 to 12.
15. A computer-readable storage medium storing computer-executable instructions which, when loaded and executed by a processor, implement a data writing method as claimed in any one of claims 1 to 12.
CN202211075892.2A 2022-09-05 2022-09-05 Data writing method, device, equipment and storage medium Pending CN115167784A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211075892.2A CN115167784A (en) 2022-09-05 2022-09-05 Data writing method, device, equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211075892.2A CN115167784A (en) 2022-09-05 2022-09-05 Data writing method, device, equipment and storage medium

Publications (1)

Publication Number Publication Date
CN115167784A true CN115167784A (en) 2022-10-11

Family

ID=83482125

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211075892.2A Pending CN115167784A (en) 2022-09-05 2022-09-05 Data writing method, device, equipment and storage medium

Country Status (1)

Country Link
CN (1) CN115167784A (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115629713A (en) * 2022-12-06 2023-01-20 苏州浪潮智能科技有限公司 Data synchronization method, device, equipment and medium
CN117472294A (en) * 2023-12-28 2024-01-30 合肥康芯威存储技术有限公司 Memory and data processing method thereof
WO2024193142A1 (en) * 2023-03-22 2024-09-26 华为技术有限公司 Storage apparatus and method, device, and storage system
CN118708125A (en) * 2024-08-27 2024-09-27 苏州元脑智能科技有限公司 Write cache space management method, device, medium and computer program product

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104077246A (en) * 2014-07-02 2014-10-01 浪潮(北京)电子信息产业有限公司 Device for realizing volatile memory backup
CN109085909A (en) * 2018-07-25 2018-12-25 郑州云海信息技术有限公司 A kind of distribution of disk array standby method for electrically and device
CN111679794A (en) * 2020-06-17 2020-09-18 北京中存超为科技有限公司 Method and device for data synchronization in multi-control storage system
CN111880636A (en) * 2020-07-30 2020-11-03 北京浪潮数据技术有限公司 Power-off protection method of storage array and related device

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104077246A (en) * 2014-07-02 2014-10-01 浪潮(北京)电子信息产业有限公司 Device for realizing volatile memory backup
CN109085909A (en) * 2018-07-25 2018-12-25 郑州云海信息技术有限公司 A kind of distribution of disk array standby method for electrically and device
CN111679794A (en) * 2020-06-17 2020-09-18 北京中存超为科技有限公司 Method and device for data synchronization in multi-control storage system
CN111880636A (en) * 2020-07-30 2020-11-03 北京浪潮数据技术有限公司 Power-off protection method of storage array and related device

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115629713A (en) * 2022-12-06 2023-01-20 苏州浪潮智能科技有限公司 Data synchronization method, device, equipment and medium
WO2024193142A1 (en) * 2023-03-22 2024-09-26 华为技术有限公司 Storage apparatus and method, device, and storage system
CN117472294A (en) * 2023-12-28 2024-01-30 合肥康芯威存储技术有限公司 Memory and data processing method thereof
CN117472294B (en) * 2023-12-28 2024-04-09 合肥康芯威存储技术有限公司 Memory and data processing method thereof
CN118708125A (en) * 2024-08-27 2024-09-27 苏州元脑智能科技有限公司 Write cache space management method, device, medium and computer program product

Similar Documents

Publication Publication Date Title
CN115167784A (en) Data writing method, device, equipment and storage medium
US8707096B2 (en) Storage system, data backup method, and system restarting method of a storage system incorporating volatile and nonvolatile memory devices
JP4996693B2 (en) System for optimizing the performance and reliability of a storage controller cache offload circuit
US9927999B1 (en) Trim management in solid state drives
US9042197B2 (en) Power fail protection and recovery using low power states in a data storage device/system
CN106066890B (en) Distributed high-performance database all-in-one machine system
CN101937376B (en) A kind of data managing method and data storage device
US9696922B2 (en) Storage system
CN103049220B (en) Storage controlling method, memory control device and solid-state memory system
CN102436419A (en) Non-volatile memory systems and methods of managing power of the same
CN101710252B (en) Power supply method and power supply device of storage system
US20120159004A1 (en) Redundant solid state disk system via interconnect cards
US8832355B2 (en) Storage device, storage controlling device, and storage controlling method
CN110488673A (en) A kind of data processing module and data processing method of low-power consumption mode
CN112114741B (en) Storage System
CN105808462B (en) FPGA (field programmable Gate array) -based simulation memory, realization method of simulation memory and computer
US20190340089A1 (en) Method and apparatus to provide uninterrupted operation of mission critical distributed in-memory applications
CN107122316A (en) A kind of SOC is for method for electrically and SOC
CN102541458A (en) Method for increasing data writing speed of electronic hard disk
CN101364138B (en) Power supply management method and device of solid hard disk storage system
JP2014532929A (en) Embedded memory and embedded storage system
CN110989921B (en) Configurable memory array system, control method thereof and communication equipment
CN113535666A (en) Data writing method and device, database system and storage medium
CN107391316A (en) A kind of computing device and its application method based on non-volatile memory
US9697097B2 (en) Storage system and method for controlling storage system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20221011