CN114077639A

CN114077639A - Data writing method and device, electronic equipment and storage medium

Info

Publication number: CN114077639A
Application number: CN202010843797.7A
Authority: CN
Inventors: 赵凯
Original assignee: Beijing ByteDance Network Technology Co Ltd
Current assignee: Beijing ByteDance Network Technology Co Ltd
Priority date: 2020-08-20
Filing date: 2020-08-20
Publication date: 2022-02-22

Abstract

The embodiment of the disclosure provides a data writing method and device, electronic equipment and a storage medium. The method comprises the following steps: when a data writing request is received, acquiring scene information of the data writing request and target architecture configuration information corresponding to the scene information, wherein the target architecture configuration information is recorded in a configuration file; loading a target data storage architecture according to the target architecture configuration information; and controlling a storage node in the target data storage architecture to write target data, wherein the target data is the data requested to be written by the data write request. By adopting the technical scheme, the configuration information of the data storage architecture used by the server for writing data in different application scenes is recorded in the configuration file of the server instead of the program code of the data writing service provided by the server, so that the program code of the data writing service provided by the server can be simplified, and the maintenance difficulty of the program code is reduced.

Description

Data writing method and device, electronic equipment and storage medium

Technical Field

The present disclosure relates to data writing technologies, and in particular, to a data writing method and apparatus, an electronic device, and a storage medium.

Background

When the user needs to write data, the server needs to provide a data writing service for the user.

In a server providing a service with a high Query Per Second (QPS), the bottom layer needs to be supported by various data storage services (such as MySQL, Redis, MongoDB, local cache, etc.). Because the performance of different storage services is different, and the positions and roles of the same data storage service in the data storage architectures of different data writing scenarios may also be different, the data storage architecture of each data writing scenario needs to be preset.

In the prior art, a data storage architecture of some common scenarios is specified in a program code of a data writing service mainly at a program writing stage. However, when the data storage architecture is more hierarchical and complex, this arrangement may make the program code cumbersome and difficult to maintain.

Disclosure of Invention

The embodiment of the disclosure provides a data writing method and device, electronic equipment and a storage medium, so as to simplify program codes of a server for providing data writing services and reduce the maintenance difficulty of the program codes.

In a first aspect, an embodiment of the present disclosure provides a data writing method, including:

when a data writing request is received, acquiring scene information of the data writing request and target architecture configuration information corresponding to the scene information, wherein the target architecture configuration information is recorded in a configuration file;

loading a target data storage architecture according to the target architecture configuration information;

and controlling a storage node in the target data storage architecture to write target data, wherein the target data is the data requested to be written by the data write request.

In a second aspect, an embodiment of the present disclosure further provides a data writing apparatus, including:

the information acquisition module is used for acquiring scene information of a data writing request and target architecture configuration information corresponding to the scene information when the data writing request is received, and the target architecture configuration information is recorded in a configuration file;

the architecture loading module is used for loading a target data storage architecture according to the target architecture configuration information;

and the data writing module is used for controlling a storage node in the target data storage architecture to write target data, wherein the target data is data requested to be written by the data writing request.

In a third aspect, an embodiment of the present disclosure further provides an electronic device, including:

one or more processors;

a memory for storing one or more programs,

when executed by the one or more processors, cause the one or more processors to implement a data writing method as described in embodiments of the disclosure.

In a fourth aspect, the disclosed embodiments also provide a computer-readable storage medium, on which a computer program is stored, which when executed by a processor implements the data writing method according to the disclosed embodiments.

According to the data writing method, the data writing device, the electronic device and the storage medium provided by the embodiment of the disclosure, when a data writing request is received, scene information of the data writing request and target architecture configuration information corresponding to the scene information and recorded in a configuration file are obtained, a target data storage architecture is loaded according to the target architecture configuration information, and then a storage node in the target data storage architecture is controlled to write target data requested to be written by the data writing request. By adopting the technical scheme, the configuration information of the data storage architecture used by the server for writing data in different application scenes is recorded in the configuration file of the server instead of the program code of the data writing service provided by the server, so that the program code of the data writing service provided by the server can be simplified, and the maintenance difficulty of the program code is reduced. And the storage architecture is configured in a scene in the configuration file, so that different data storage architectures are set for different application scenes, and the expandability of the program code can be enhanced.

Drawings

The above and other features, advantages and aspects of various embodiments of the present disclosure will become more apparent by referring to the following detailed description when taken in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numbers refer to the same or similar elements. It should be understood that the drawings are schematic and that elements and features are not necessarily drawn to scale.

Fig. 1 is a schematic flow chart of a data writing method according to an embodiment of the disclosure;

fig. 2 is a schematic flow chart of another data writing method according to an embodiment of the disclosure;

fig. 3 is a schematic structural diagram of a linked list corresponding to a data storage architecture according to an embodiment of the present disclosure;

fig. 4 is a schematic structural diagram of a linked list corresponding to an updated data storage architecture according to an embodiment of the present disclosure;

fig. 5 is a schematic structural diagram of a linked list corresponding to another updated data storage architecture according to the embodiment of the present disclosure;

fig. 6 is a block diagram of a data writing apparatus according to an embodiment of the disclosure;

fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure.

Detailed Description

Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it is to be understood that the present disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein, but rather are provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the disclosure are for illustration purposes only and are not intended to limit the scope of the disclosure.

It should be understood that the various steps recited in the method embodiments of the present disclosure may be performed in a different order, and/or performed in parallel. Moreover, method embodiments may include additional steps and/or omit performing the illustrated steps. The scope of the present disclosure is not limited in this respect.

The term "include" and variations thereof as used herein are open-ended, i.e., "including but not limited to". The term "based on" is "based, at least in part, on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Relevant definitions for other terms will be given in the following description.

It should be noted that the terms "first", "second", and the like in the present disclosure are only used for distinguishing different devices, modules or units, and are not used for limiting the order or interdependence relationship of the functions performed by the devices, modules or units. [ ordinal numbers ]

It is noted that references to "a", "an", and "the" modifications in this disclosure are intended to be illustrative rather than limiting, and that those skilled in the art will recognize that "one or more" may be used unless the context clearly dictates otherwise.

The names of messages or information exchanged between devices in the embodiments of the present disclosure are for illustrative purposes only, and are not intended to limit the scope of the messages or information.

Fig. 1 is a schematic flow chart of a data writing method according to an embodiment of the disclosure. The method may be performed by a data writing apparatus, wherein the apparatus may be implemented by software and/or hardware, may be configured in an electronic device, and may typically be configured in a server. Optionally, the data writing method provided by the embodiment of the present disclosure is suitable for a case of writing data based on a received data writing request, and is particularly suitable for a case of writing data in a data storage architecture in which data storage is performed in a key value pair storage manner to a storage node after the data writing request is received. As shown in fig. 1, a data writing method provided by an embodiment of the present disclosure may include:

s101, when a data writing request is received, scene information of the data writing request and target architecture configuration information corresponding to the scene information are obtained, and the target architecture configuration information is recorded in a configuration file.

The data writing request may be a data writing request that is sent to the local terminal by the terminal device when the user has a data writing requirement and is used for requesting the local terminal to write a certain piece of data, such as a data deleting request for requesting to delete a certain piece of data, a data updating request for requesting to update a certain piece of data, or a data inserting request for requesting to insert a certain piece of data. The data write request may or may not carry an identification of the data requested to be written, such as a data ID. The context information of the data write request may be understood as information describing an application context of the data write request, such as an identity (e.g., a context ID) of the application context, or context parameters such as an upstream service party, a service cluster, and/or a sending time of the data write request. The target architecture configuration information is configuration information of a target data storage architecture suitable for an application scenario to which the data write request belongs, and when the application scenarios are different, storage architectures adopted in writing data may be different. The configuration file may be understood as a file for recording the architecture configuration information of each data storage architecture configured by the server to implement writing of data, and the architecture configuration information recorded in the configuration file may be set by a developer or an operation and maintenance person of the server before the server operates and/or updated during the server operates.

Specifically, after receiving a data write-in request sent by a terminal device, a server analyzes the data write-in request to obtain scene parameters of an application scene of the data write-in request, and further, when the scene parameters of each application scene and the architecture configuration information of the corresponding application scene are correspondingly recorded in a configuration file, the server can query and obtain the architecture configuration information corresponding to the scene parameters according to the corresponding relationship between each scene parameter and each architecture configuration information recorded in the configuration file, and use the architecture configuration information as target architecture configuration information; when the configuration file correspondingly records the identity of each application scene and the architecture configuration information of the corresponding application scene, the identity of the application scene corresponding to the scene parameter can be obtained by querying according to a preset corresponding relation table of the scene parameter and the identity of the scene, and the architecture configuration information of the application scene to which the data write request belongs is obtained by querying according to the architecture configuration information corresponding to the identity of each application scene recorded in the configuration file and serves as the target architecture configuration information.

And S102, loading a target data storage framework according to the target framework configuration information.

The target data storage architecture may be understood as a storage architecture adopted when writing data requested to be written by the data write request, and correspondingly, the target architecture configuration information is the architecture configuration information of the target data storage architecture. The target data storage architecture may include a plurality of storage nodes capable of providing data storage services, and node types of the storage nodes may be the same or different, for example, the target data storage architecture may be provided with a plurality of storage nodes of the same type, or only with storage nodes of different types. The target data storage architecture can be abstracted into a linked list, that is, the target data storage architecture can be presented in a memory in the form of a linked list, and correspondingly, each storage node in the target data storage architecture can be abstracted into a node in the linked list. The node type of the storage node can be Redis, MySQL or MongoDB, and correspondingly, one or more Redis databases, one or more MySQL databases and/or one or more MongoDB databases and the like can be arranged in the target data storage node.

Specifically, the server reads the target architecture configuration information in the configuration file, and loads the target architecture configuration information into the memory according to the read target architecture configuration information to obtain a data storage architecture corresponding to the target architecture configuration information, that is, a target data storage architecture.

In this embodiment, the server may execute the operation of loading the target data storage architecture each time a data write request is received; the operation of loading the target data storage architecture may be executed only when the data storage architecture corresponding to the application scenario to which the received data write request belongs is not loaded in the memory or the architecture configuration information of the data storage architecture corresponding to the application scenario to which the received data write request belongs loaded in the memory is different from the acquired target architecture configuration information, which is not limited in this embodiment.

S103, controlling a storage node in the target data storage architecture to write target data, wherein the target data is data requested to be written by the data write request.

The target data is the data requested to be written by the data writing request, namely when the data writing request is a data deleting request, the target data is the data requested to be deleted by the data writing request; when the data write request is a data update request, the target data is data after the update data requested by the data write request is updated, and at this time, the data write request may further include data to be updated before the update data requested by the data write request is updated; when the data write request is a data insert request, the target data is the data requested to be inserted by the data write request.

Specifically, the server analyzes the received data writing request in advance, and determines target data of the data writing request. Therefore, after the target data storage nodes are loaded, the server can simultaneously control all the storage nodes in the target data storage architecture to write target data; alternatively, the storage nodes in the target storage architecture are controlled to write the target data randomly or in a certain order (e.g., in an order from high to low of the hierarchy of the storage nodes in the target data storage architecture).

For example, when the data write request is a data delete request, the data write request may carry target data of which the deletion request is requested, or may not carry the target data of which the deletion request is requested, but only carry a data identifier of the target data of which the deletion request is requested; correspondingly, when the server controls a certain storage node to write the target data, the server may control the storage node to determine the storage location of the target data in the storage node according to the target data carried in the data write request or the data identifier of the target data, and delete the target data stored in the storage node.

When the data write request is a data update request, the data write request may carry data to be updated before update of target data requested to be updated and the updated target data, or carry a data identifier of the target data requested to be updated and the updated target data; correspondingly, when controlling a certain storage node to write target data, the server may control the storage node to determine a storage location of the data to be updated in the storage node according to the data to be updated or the data identifier carried in the data write request, delete the original data, and write the target data into the storage location.

When the data write request is a data insertion request, the data write request may carry target data requested to be inserted; accordingly, when the server controls a certain storage node to write the target data, the server may obtain a storage location capable of storing the target data, and write the target data in the storage location.

In the data writing method provided in this embodiment, when a data writing request is received, scene information of the data writing request and target architecture configuration information corresponding to the scene information and described in a configuration file are acquired, and a target data storage architecture is loaded according to the target architecture configuration information, so that a storage node in the target data storage architecture is controlled to write target data requested to be written by the data writing request. By adopting the technical scheme, the configuration information of the data storage architecture used by the server when the server writes data in different application scenes is recorded in the configuration file of the server instead of the data writing service program code provided by the server, so that the program code of the data writing service provided by the server can be simplified, and the maintenance difficulty of the program code is reduced. And the storage architecture is configured in a scene in the configuration file, so that different data storage architectures are set for different application scenes, and the expandability of the program code can be enhanced.

Fig. 2 is a schematic flow chart of a data writing method provided in an embodiment of the present disclosure, and the scheme in this embodiment may be combined with one or more of the alternatives in the above embodiments. In this embodiment, optionally, a root node of the target data storage architecture is determined as a current node; judging whether the current node is a non-virtual node or not; if the current node is a non-virtual node, determining a target data writing function of the current node according to the target architecture configuration information, wherein the target data writing function corresponds to the data writing request; calling the target data writing function to control the current node to write target data; and determining the next node as the current node, and returning to execute the operation of judging whether the current node is a non-virtual node until the next node does not exist.

Optionally, before determining the target data writing function of the current node according to the target architecture configuration information, the method further includes: determining that the current node is not provided with a bottom pocket identifier, wherein the bottom pocket identifier is used for indicating a local terminal to directly control the bottom pocket node of the current node to write target data; correspondingly, the method further comprises the following steps: and if the current node is provided with a bottom pocket identifier, determining the bottom pocket node of the current node recorded in the target architecture configuration information as the current node, and returning to execute the operation of judging whether the current node is a non-virtual node.

Optionally, the data writing method provided in this embodiment further includes: and if the current node is a virtual node, determining a next write-in node of the current node recorded in the target architecture configuration information as the current node, and returning to execute the operation of judging whether the current node is a non-virtual node.

Optionally, before the loading the target data storage architecture according to the target architecture configuration information, the method further includes: determining that the original data storage architecture is not loaded in the memory or the original architecture configuration information of the original data storage architecture loaded in the memory is different from the target architecture configuration information, wherein the original data storage architecture corresponds to the scene information.

Correspondingly, as shown in fig. 2, the data writing method provided in this embodiment may include:

s201, when a data writing request is received, scene information of the data writing request and target architecture configuration information corresponding to the scene information are obtained, and the target architecture configuration information is recorded in a configuration file.

S202, determining that the original data storage architecture is not loaded in the memory or the original architecture configuration information of the original data storage architecture loaded in the memory is different from the target architecture configuration information, wherein the original data storage architecture corresponds to the scene information.

Correspondingly, if the original configuration information of the original data storage architecture loaded in the memory is the same as the target architecture configuration information, the original data storage architecture may be determined as the target data storage architecture, and the target data requested to be written by the data write request is written based on the target data storage architecture.

The memory can be understood as a memory space of the server, that is, an internal memory of the computer device bearing the server; the original architecture configuration information may be understood as a data storage architecture corresponding to an application scenario of the data write request loaded before the current time.

In this embodiment, the server reloads the target data storage architecture only when the original data storage architecture corresponding to the application scenario of the data write request is not loaded in the memory (i.e., the original data storage architecture corresponding to the scenario information of the data write request is not loaded) or when the original architecture configuration information of the original data storage architecture loaded in the memory is different from the target architecture configuration information recorded in the configuration file; when the original data storage architecture is loaded in the memory and the original architecture configuration information of the original data storage architecture is the same as the target architecture configuration information recorded in the configuration file, the operation of loading the target data storage architecture is not executed, but the storage nodes in the original data storage architecture are directly controlled to write data, so that the operation required to be executed during data writing is further simplified, and the response speed of the server is improved.

Specifically, after acquiring scene information and target architecture configuration information of a data write request, a server judges whether an original data storage architecture corresponding to the scene information exists in a memory, and if not, loads a target data storage architecture according to the target architecture configuration information; if the configuration information of the original data storage architecture is the same as the configuration information of the target architecture, directly determining the original data storage architecture as the target data storage architecture; if not, loading a target data storage framework according to the target framework configuration information, and deleting the original data storage framework from the memory after all original data writing requests which are received before the data writing requests and are written based on the original data storage framework are processed, so as to avoid occupation of useless data storage frameworks on the memory.

In one embodiment, a developer or an operation and maintenance person of the server may modify configuration information of the data storage architecture recorded in a configuration file of the server. Therefore, when the server receives the configuration updating instruction, the target architecture configuration information can be updated according to the configuration updating instruction.

Specifically, when monitoring that a developer or an operation and maintenance person triggers an operation of modifying architecture configuration information (such as target architecture configuration information) in a configuration file, the server determines that a configuration update instruction is received, and updates the architecture configuration information in the configuration file based on an input operation of the developer or the operation and maintenance person, such as adding, deleting or modifying the architecture configuration information recorded in the configuration file, thereby realizing the addition, deletion or modification of a data storage architecture corresponding to the server.

In this embodiment, when a server developer and an operation and maintenance worker encounter an emergency (e.g., an access volume surge, a Redis avalanche, MySQL blocking, or the like), the configuration information recorded in the configuration file may be changed, so as to change the corresponding data storage architecture of the server, thereby implementing a hot update of the data storage architecture of the server, satisfying requirements for fast emergency degradation, current limiting, offloading, cleaning, side effects, and the like of storage nodes in the data storage architecture, and converging management of the architecture information of the data storage architecture, disaster tolerance, degradation, current limiting, offloading, cleaning, side effects, and the like of the data storage architecture.

For example, it is assumed that a linked list abstracted from a data storage structure of the server is shown in fig. 3, the data storage architecture includes three storage nodes, MySQL, Redis and MongoDB, the MongoDB is a backup node of MySQL, and the Redis is located at the next layer of MySQL. When the server writes data, firstly controlling MySQL to write the data, and if the data is successfully written, continuously controlling Redis to write the data; and if the writing fails, controlling the MongoDB to write the data. If MySQL breaks down at a certain moment, a large amount of slow SQL is caused, and if the MySQL writes data continuously under control, a large amount of data fails to be written, at this moment, the operation and maintenance personnel can update the data storage architecture by changing the architecture configuration information of the data storage architecture, move MySQL out of the data storage architecture (as shown in fig. 4), or perform disaster recovery switching, and switch MySQL into MongoDB (as shown in fig. 5).

It is understood that, the embodiment may also determine whether the original architecture configuration information of the original storage architecture is the same as the target architecture configuration information directly according to the loading time of the original storage architecture and the updating time of the target architecture configuration information without acquiring the original architecture configuration information of the original data storage architecture, for example, the server may record the update time of the target architecture configuration information when updating the target architecture configuration information, and recording the loading time of the original data storage architecture when the original data storage architecture is loaded, thereby, after determining that the original data storage structure corresponding to the scene information of the data write request is loaded in the memory, it may be determined whether the update time of the target architecture configuration information is later than the load time of the original data storage architecture, and if so, determining that the original architecture configuration information of the original data storage architecture is different from the target architecture configuration information; if not, determining that the original architecture configuration information of the original data storage architecture is the same as the target architecture configuration information.

S203, loading a target data storage framework according to the target framework configuration information.

And S204, determining the root node of the target data storage architecture as the current node.

The current node can be understood as a storage node in the target data storage architecture, which needs to be controlled to write the target data currently.

S205, judging whether the current node is a non-virtual node, if so, executing S206 or S207; if not, S209 is executed.

In this embodiment, in order to facilitate control over each storage node, each storage node may be set as a normal node or a virtual node according to whether each storage node has a data writing function. The common node is a storage node with a data writing function; the virtual node is a storage node without a data writing function. Thus, the target architecture configuration information may include configuration information of each storage node in the target storage architecture, i.e., node configuration information. The node configuration information of a certain storage node may record node type information and a next write node of the storage node. When a certain storage node cannot normally provide a data writing function, a developer or an operation and maintenance person of the server may set the storage node as a virtual node, that is, change the type information of the storage node recorded in the node configuration information of the storage node into the virtual node. Correspondingly, if the node type of a certain storage node is a virtual node, it means that the storage node cannot normally provide data writing service. The next writing node may be understood as a node that needs to continue to write the target data when the current node is a virtual node or when the target data written by the current node is not abnormal.

Specifically, after determining the current node, the server determines whether the current node is a virtual node according to the node configuration information of the current node. If the current node is a virtual node, further judging whether the current node has a next write-in node according to the node configuration information of the current node, if so, determining the next write-in node of the current node as the current node, and returning to execute S205; and if the next write-in node does not exist, determining that the target data is completely written, and ending the operation. Correspondingly, if the current node is not a virtual node, whether the current node is provided with the bottom pocket identifier or not can be continuously judged, and corresponding operation is executed.

And S206, if the current node is provided with a bottom pocket identifier, determining the bottom pocket node of the current node recorded in the target architecture configuration information as the current node, and returning to execute the step S205, wherein the bottom pocket identifier is used for indicating a local terminal to directly control the bottom pocket node of the current node to write target data.

In this embodiment, the node configuration information of each storage node in the target data storage architecture may further include a bottom node and bottom identification information of the corresponding storage node. The bottom pocket node of the storage node can be understood as a storage node for further writing data when the server fails to write data in the storage node (namely, an exception occurs in the process of writing data), that is, when the server controls the storage node to write data abnormally, the server further controls the bottom pocket node of the storage node to write data; the bottom pocket identification information can be understood as information indicating whether the server directly skips the storage node and controls the bottom pocket node of the storage node to write data, namely when the bottom pocket identification information of the storage node is not empty, namely when the storage node is provided with the bottom pocket identification, the server directly jumps the storage node and directly controls the bottom pocket node of the storage node to write data, thereby realizing thermal degradation of the data storage architecture in an emergency.

Specifically, after determining that the current node is not a virtual node, the server determines whether the bottom pocket identification information in the node configuration information of the current node is empty, and if yes, determines that the bottom pocket identification is not set in the current node and executes subsequent operations; if not, determining that the current node is provided with the pocket bottom identifier, determining the pocket bottom node of the current node recorded in the node configuration information of the current node as the current node, and returning to execute the step S205.

It can be understood that, this embodiment does not limit the execution sequence of the two determination operations of determining whether the current node is a virtual node and determining whether the current node is provided with the pocket bottom identifier, for example, after the current node is determined in this embodiment, it may also be determined whether the current node is provided with the pocket bottom identifier first, if so, the pocket bottom node of the current node recorded in the node configuration information of the current node is directly determined as the current node, and the operation of determining whether the current node is provided with the pocket bottom identifier is returned to be executed; if not, further judging whether the current node is a virtual node, if so, determining the next write-in node of the current node as the current node, and returning to execute the operation of judging whether the current node is provided with the bottom pocket identifier; and if the current node is not the virtual node, controlling the current node to write the target data.

S207, if the current node is not provided with the bottom pocket identification, determining a target data writing function of the current node according to the target architecture configuration information, wherein the target data writing function corresponds to the data writing request.

In this embodiment, the node configuration information of each storage node in the target data storage architecture may further include an object name of a data access object of the corresponding storage node. The data access object is an interface object for accessing a corresponding storage node (such as a corresponding database), and object information of the data access object can be set in advance by a server or a database developer and stored in the server or the corresponding storage node, and can include an object name and a data writing function of the data access object; and may further include a pointer type of the data access object. The object name of the data access object is the unique identification of the data access object when the server writes data; the data writing function may be used to write data into the corresponding storage node, such as deleting or updating the data stored in the corresponding storage node or inserting new data into the corresponding storage node, and accordingly, the data writing function may be a data deleting function, a data updating function, or a data inserting function; the pointer type may be used to characterize the specific type of data written by the data access object.

Specifically, when the current node does not set the pocket identifier, it means that the server can control the current node to write the target data without skipping the pocket node of the current node which directly controls the current node to perform the data write operation. Therefore, when the current node is not provided with the bottom-of-pocket identifier, the server may determine the data access object of the current node according to the name of the data access object of the current node recorded in the node configuration information of the current node, and further determine the data write-in function, which is recorded in the object information of the data access object and corresponds to the request of the data write-in request, as the data write-in function of the current node when processing the data write-in request, and if the data write-in request is data deletion, take the data deletion function in the data access object of the current node as the data write-in function; when the data writing request is a data updating request, taking a data updating function of a data access object of the current node as a data writing function; and when the data writing request is a data inserting request, taking the data inserting function of the data access object of the current node as a data writing function.

And S208, calling the target data writing function to control the current node to write the target data.

Specifically, the server sends the target data and/or the identity of the target data to the current node; correspondingly, the current node executes the data writing function of the current node according to the target data and/or the identity of the target data.

In an embodiment, the data writing method provided in this embodiment may further include: and if the current node is provided with a cleaning identifier, determining a target node corresponding to the cleaning identifier, and controlling the target node to delete stored original data, wherein the original data correspond to the target data.

The original data may be data related to the target data and having an operation error after the target data is changed, such as data calculated based on the target data, data to be updated before updating corresponding to the target data, or data that needs to be further deleted in other nodes after the target data is deleted.

In the foregoing embodiment, the node configuration information of the storage node in the target data storage architecture may further include cleaning identification information of the storage node, so as to clean data corresponding to the target data in other nodes. The cleaning identification information can be set by a server developer or an operation and maintenance person according to needs. When the node name of the target storage node is recorded in the cleaning identification information in the node configuration information of the current node, it indicates that the current node is provided with the cleaning identification, and after the current node successfully writes the target data, the target storage node is also required to be controlled to clean the original data corresponding to the target data, for example, a data deletion function in a data access object of the target storage node is called, so that the target storage node is controlled to delete the original data corresponding to the target data and stored in the target storage node. That is, when the current node is provided with the clear identifier, the server may create a new thread for execution after determining that the target data is successfully written into the current node, and execute the data deletion function of the target storage node through the new thread.

In an embodiment, the data writing method provided in this embodiment may further include: and if the current node is provided with the side effect identification, executing the side effect function of the current node.

In the above embodiment, the node configuration information of the storage node in the target data storage architecture may further include side effect identification information of the storage node, thereby implementing the execution of the side effect. The side effect identification information can be set by a server developer or an operation and maintenance person according to needs. When a side effect function to be executed is recorded in the side effect identification information in the node configuration information of the current node, it indicates that the current node is provided with the side effect identification, and after the current node successfully writes the target data, the side effect function also needs to be executed. That is, when the current node has the side effect identifier, the server may create a new thread to execute the side effect function of the current node after determining that the target data is successfully written into the current node.

S209, judging whether a next node exists, if so, executing S210; if not, the operation is ended.

Wherein the next node can be understood as the next node that controls writing of the target data.

In this embodiment, it is possible to directly take the next write node of the current node as the next node without considering whether the current node is abnormal when writing the target data; it is also possible to consider whether or not an abnormality occurs in the current node when the target data is written, and when the current node is a virtual node, or when the current node is not a virtual node and an abnormality does not occur in the target data writing, take the next write node of the current node as the next node, and when the current node is not a virtual node and an abnormality occurs in the target data writing, take the bottom node of the current node as the next node, so as to improve convenience in subsequent query of the target data, which will be described below as an example.

Wherein the next node can be understood as the next node that controls writing of the target data. Whether the process of writing the target data into the current node is abnormal or not can be determined according to the returned information of the current node. When the current node is provided with the asynchronous writing identifier, that is, the asynchronous execution information in the node configuration information of the current node is not empty, the process of controlling the current node to write the target data can be executed asynchronously, and at the moment, it can be directly judged that no abnormality occurs in the process of writing the current node data into the target data.

And S210, determining the next node as the current node, and returning to execute S205.

Specifically, if the current node is a virtual node, or the current node is not a virtual node and no exception occurs when the target data is written, determining a next write node of the current node recorded in the target architecture configuration information as the current node; and if the current node is not the virtual node and the data target data is abnormal, determining the bottom node of the current node as the current node.

In one embodiment, when the current node is not a virtual node, the determining the next node as the current node includes: if the current node is not abnormal when the target data is written in, determining a next writing node of the current node recorded in the target architecture configuration information as the current node; and if the current node is abnormal when the target data is written in and the current node is not in a writing stop state, determining the bottom pocket node of the current node recorded in the target architecture configuration information as the current node.

In the above embodiment, when determining that an abnormality occurs in the current node in the process of writing data, the server determines whether the current node is in a writing stop state, if yes, determines that the current node is in the writing stop state, and stops the storage node in the control target storage architecture from writing the target data; if not, determining that the current node is not in the writing stop state, determining the bottom node of the current node recorded in the target architecture configuration information as the current node, and returning to execute the step S205. The fast failure information of the storage node may be recorded in the node configuration information of the storage node.

Therefore, by setting node types and bottom-of-pocket nodes for storage nodes in the data storage structure, recording architecture configuration information of the data storage architecture through a configuration file, wherein the architecture configuration information comprises node names of the storage nodes and node names of the bottom-of-pocket nodes in the data storage architecture; when the target architecture configuration information of the target data storage architecture is determined to be changed, the target data storage architecture is loaded according to the target architecture configuration information, and the storage nodes in the loaded target data storage architecture are controlled to write data, so that the program codes of the server for providing writing service can be simplified, and the maintenance difficulty of the program codes is reduced; the data storage system can also realize the functions of hot updating of the data storage architecture, disaster tolerance and degradation of the data storage architecture and the like, can converge the schemes of the data storage architecture, the disaster tolerance and the degradation of the data storage architecture and the like at one place for management, and reduces the management difficulty of the data storage architecture.

Fig. 6 is a block diagram of a data writing apparatus according to an embodiment of the disclosure. The apparatus may be implemented by software and/or hardware, may be configured in an electronic device, and typically may be configured in a server, and may write data by performing a data writing method. As shown in fig. 6, the data writing apparatus provided by the embodiment of the present disclosure may include an information obtaining module 301, a framework loading module 302, and a data writing module 303, wherein,

an information obtaining module 301, configured to, when a data write request is received, obtain scene information of the data write request and target architecture configuration information corresponding to the scene information, where the target architecture configuration information is recorded in a configuration file;

a framework loading module 302, configured to load a target data storage framework according to the target framework configuration information;

a data writing module 303, configured to control a storage node in the target data storage architecture to write target data, where the target data is data requested to be written by the data writing request.

In the data writing device provided in this embodiment, when receiving a data writing request, the ranking information obtaining module obtains scene information of the data writing request and target architecture configuration information corresponding to the scene information and described in a configuration file, and loads a target data storage architecture according to the target architecture configuration information through the architecture loading module, and then controls a storage node in the target data storage architecture to write target data requested to be written by the data writing request through the data writing module. By adopting the technical scheme, the configuration information of the data storage architecture used by the server when the server writes data in different application scenes is recorded in the configuration file of the server instead of the data writing service program code provided by the server, so that the program code of the data writing service provided by the server can be simplified, and the maintenance difficulty of the program code is reduced. And the storage architecture is configured in a scene in the configuration file, so that different data storage architectures are set for different application scenes, and the expandability of the program code can be enhanced.

In the above scheme, the data writing module 303 may include: a node determining unit, configured to determine a root node of the target data storage architecture as a current node; a virtual node judging unit, configured to judge whether the current node is a non-virtual node; a function determining unit, configured to determine, when the current node is a non-virtual node, a target data writing function of the current node according to the target architecture configuration information, where the target data writing function corresponds to the data writing request; the data writing unit is used for calling the target data writing function so as to control the current node to write target data; and the return unit is used for determining the next node as the current node and returning to the virtual node judgment unit.

In the above scheme, the next node is a next write-in node or a bottom node of the current node; the return unit may be operable to: when the current node is not abnormal when the target data is written in, determining a next write-in node of the current node recorded in the target architecture configuration information as the current node; and when the current node is abnormal when the target data is written in and the current node is not in a writing stop state, determining a bottom pocket node of the current node recorded in the target architecture configuration information as the current node.

In the foregoing scheme, the data writing module 303 may further include: a bottom pocket identifier determining unit, configured to determine that a bottom pocket identifier is not set in the current node before determining a target data write-in function of the current node according to the target architecture configuration information, where the bottom pocket identifier is used to instruct a local terminal to directly control a bottom pocket node of the current node to write in target data; correspondingly, the pocket bottom identification determination unit is further configured to: and when the current node is provided with the bottom pocket identification, determining the bottom pocket node of the current node recorded in the target architecture configuration information as the current node, and returning to the virtual node judgment unit.

In the foregoing scheme, the data writing module 303 may further include: and a write node determining unit, configured to determine, when the current node is a virtual node, a next write node of the current node recorded in the target architecture configuration information as the current node, and return the current node to the virtual node determining unit.

In the foregoing scheme, the data writing module 303 may further include: and the cleaning unit is used for determining a target node corresponding to the cleaning identifier when the current node is provided with the cleaning identifier, and controlling the target node to delete stored original data, wherein the original data correspond to the target data.

In the foregoing scheme, the data writing module 303 may further include: and the side effect unit is used for executing the side effect function of the current node when the current node is provided with the side effect identifier.

Optionally, the data writing device provided in this embodiment may further include: and the architecture determination module is used for determining that the original data storage architecture is not loaded in the memory or the original architecture configuration information of the original data storage architecture loaded in the memory is different from the target architecture configuration information before the target data storage architecture is loaded according to the target architecture configuration information, wherein the original data storage architecture corresponds to the scene information.

Optionally, the data writing device provided in this embodiment may further include: and the information updating module is used for updating the target architecture configuration information according to the configuration updating instruction when the configuration updating instruction is received.

The data writing device provided by the embodiment of the disclosure can execute the data writing method provided by any embodiment of the disclosure, and has the corresponding functional module and the beneficial effect of executing the data writing method. For details of the technology not described in detail in this embodiment, reference may be made to a data writing method provided in any embodiment of the present disclosure.

Referring now to FIG. 7, a block diagram of an electronic device (e.g., server) 700 suitable for use in implementing embodiments of the present disclosure is shown. The electronic device shown in fig. 7 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.

As shown in fig. 7, the electronic device 700 may include a processing means (e.g., central processing unit, graphics processor, etc.) 701 that may perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)702 or a program loaded from a storage means 706 into a Random Access Memory (RAM) 703. In the RAM 703, various programs and data necessary for the operation of the electronic apparatus 700 are also stored. The processing device 701, the ROM 702, and the RAM 703 are connected to each other by a bus 704. An input/output (I/O) interface 705 is also connected to bus 704.

Generally, the following devices may be connected to the I/O interface 705: input devices 706 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, etc.; an output device 707 including, for example, a Liquid Crystal Display (LCD), a speaker, and the like; storage 706 including, for example, magnetic tape, hard disk, etc.; and a communication device 709. The communication means 709 may allow the electronic device 700 to communicate wirelessly or by wire with other devices to exchange data. While fig. 7 illustrates an electronic device 700 having various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided.

In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a non-transitory computer readable medium, the computer program containing program code for performing the method illustrated by the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication means 709, or may be installed from the storage means 706, or may be installed from the ROM 702. The computer program, when executed by the processing device 701, performs the above-described functions defined in the methods of the embodiments of the present disclosure.

It should be noted that the computer readable medium in the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In contrast, in the present disclosure, a computer readable signal medium may comprise a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.

In some embodiments, the clients, servers may communicate using any currently known or future developed network Protocol, such as HTTP (HyperText Transfer Protocol), and may interconnect with any form or medium of digital data communication (e.g., a communications network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the Internet (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed network.

The computer readable medium may be embodied in the electronic device; or may exist separately without being assembled into the electronic device.

The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: when a data writing request is received, acquiring scene information of the data writing request and target architecture configuration information corresponding to the scene information, wherein the target architecture configuration information is recorded in a configuration file;

Computer program code for carrying out operations for the present disclosure may be written in any combination of one or more programming languages, including but not limited to an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The units described in the embodiments of the present disclosure may be implemented by software or hardware. Wherein the names of the modules do not in some cases constitute a limitation of the unit itself.

The functions described herein above may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), systems on a chip (SOCs), Complex Programmable Logic Devices (CPLDs), and the like.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

Example 1 provides a data writing method according to one or more embodiments of the present disclosure, including:

Example 2 in accordance with one or more embodiments of the present disclosure, the method of example 1,

the controlling storage nodes in the target data storage architecture to write target data comprises:

determining a root node of the target data storage architecture as a current node;

judging whether the current node is a non-virtual node or not;

if the current node is a non-virtual node, determining a target data writing function of the current node according to the target architecture configuration information, wherein the target data writing function corresponds to the data writing request;

calling the target data writing function to control the current node to write target data;

and determining the next node as the current node, and returning to execute the operation of judging whether the current node is a non-virtual node until the next node does not exist.

Example 3 the method of example 2, the next node being a next write node or a pocket bottom node of the current node, according to one or more embodiments of the present disclosure; the determining the next node as the current node includes:

if the current node is not abnormal when the target data is written in, determining a next writing node of the current node recorded in the target architecture configuration information as the current node;

and if the current node is abnormal when the target data is written in and the current node is not in a writing stop state, determining the bottom pocket node of the current node recorded in the target architecture configuration information as the current node.

Example 4 the method of example 2, prior to the determining the target data writing function for the current node according to the target architecture configuration information, further comprising, in accordance with one or more embodiments of the present disclosure:

determining that the current node is not provided with a bottom pocket identifier, wherein the bottom pocket identifier is used for indicating a local terminal to directly control the bottom pocket node of the current node to write target data;

correspondingly, the method further comprises the following steps:

and if the current node is provided with a bottom pocket identifier, determining the bottom pocket node of the current node recorded in the target architecture configuration information as the current node, and returning to execute the operation of judging whether the current node is a non-virtual node.

Example 5 the method of example 2, in accordance with one or more embodiments of the present disclosure, further comprising:

and if the current node is a virtual node, determining a next write-in node of the current node recorded in the target architecture configuration information as the current node, and returning to execute the operation of judging whether the current node is a non-virtual node.

Example 6 the method of any of examples 2-5, in accordance with one or more embodiments of the present disclosure, further comprising:

and if the current node is provided with a cleaning identifier, determining a target node corresponding to the cleaning identifier, and controlling the target node to delete stored original data, wherein the original data correspond to the target data.

Example 7 the method of any of examples 2-5, in accordance with one or more embodiments of the present disclosure, further comprising:

and if the current node is provided with the side effect identification, executing the side effect function of the current node.

Example 8 the method of any of examples 1-5, in accordance with one or more embodiments of the present disclosure, prior to the loading the target data storage architecture according to the target architecture configuration information, further comprising:

determining that the original data storage architecture is not loaded in the memory or the original architecture configuration information of the original data storage architecture loaded in the memory is different from the target architecture configuration information, wherein the original data storage architecture corresponds to the scene information.

Example 9 the method of example 8, in accordance with one or more embodiments of the present disclosure, further comprising:

and when a configuration updating instruction is received, updating the target architecture configuration information according to the configuration updating instruction.

Example 10 provides, in accordance with one or more embodiments of the present disclosure, a data writing apparatus comprising:

Example 11 provides, in accordance with one or more embodiments of the present disclosure, an electronic device, comprising:

one or more processors;

a memory for storing one or more programs,

when executed by the one or more processors, cause the one or more processors to implement the data writing method of any of examples 1-9.

Example 12 provides a computer-readable storage medium having stored thereon a computer program that, when executed by a processor, implements the data writing method of any of examples 1-9, in accordance with one or more embodiments of the present disclosure.

The foregoing description is only exemplary of the preferred embodiments of the disclosure and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the disclosure herein is not limited to the particular combination of features described above, but also encompasses other embodiments in which any combination of the features described above or their equivalents does not depart from the spirit of the disclosure. For example, the above features and (but not limited to) the features disclosed in this disclosure having similar functions are replaced with each other to form the technical solution.

Further, while operations are depicted in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Likewise, while several specific implementation details are included in the above discussion, these should not be construed as limitations on the scope of the disclosure. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination.

Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

1. A method of writing data, comprising:

2. The method of claim 1, wherein the controlling storage nodes in the target data storage architecture to write target data comprises:

judging whether the current node is a non-virtual node or not;

3. The method of claim 2, wherein the next node is a next write node or a bottom pocket node of the current node; the determining the next node as the current node includes:

4. The method of claim 2, further comprising, prior to said determining a target data write function for the current node based on the target architecture configuration information:

correspondingly, the method further comprises the following steps:

5. The method of claim 2, further comprising:

6. The method of any of claims 2-5, further comprising:

7. The method of any of claims 2-5, further comprising:

8. The method of any of claims 1-5, further comprising, prior to said loading a target data storage architecture according to said target architecture configuration information:

9. The method of claim 8, further comprising:

10. A data writing apparatus, comprising:

11. An electronic device, comprising:

one or more processors;

a memory for storing one or more programs,

when executed by the one or more processors, cause the one or more processors to implement a data writing method as recited in any one of claims 1-9.

12. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out a data writing method according to any one of claims 1 to 9.