CN117312440A - Service data processing method and device - Google Patents

Service data processing method and device Download PDF

Info

Publication number
CN117312440A
CN117312440A CN202210715756.9A CN202210715756A CN117312440A CN 117312440 A CN117312440 A CN 117312440A CN 202210715756 A CN202210715756 A CN 202210715756A CN 117312440 A CN117312440 A CN 117312440A
Authority
CN
China
Prior art keywords
service data
storage
service
storage cluster
data processing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210715756.9A
Other languages
Chinese (zh)
Inventor
刘荣华
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Wodong Tianjun Information Technology Co Ltd
Original Assignee
Beijing Wodong Tianjun Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Wodong Tianjun Information Technology Co Ltd filed Critical Beijing Wodong Tianjun Information Technology Co Ltd
Priority to CN202210715756.9A priority Critical patent/CN117312440A/en
Publication of CN117312440A publication Critical patent/CN117312440A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • G06F16/275—Synchronous replication
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • G06F16/273—Asynchronous replication or reconciliation

Landscapes

  • Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Computing Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention discloses a business data processing method and device, and relates to the technical field of computers. One embodiment of the method comprises the following steps: receiving a service data processing request sent by a service system, and determining a storage service mode of a database system; wherein the service data processing request includes at least one of a service data storage request and a service data reading request; if the storage service mode is a multi-storage service mode, executing at least one of a service data storage process and a service data reading process according to the service data processing request; and sending the service data processing result to the service system. The embodiment improves the service data processing efficiency, supports the rapid transfer switching of single-point faults, expands the application scene of service data processing and improves the user experience.

Description

Service data processing method and device
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a method and an apparatus for processing service data.
Background
With the increasing frequency and complexity of access requirements of service data of an information system, single-point failover and recovery processing of a data storage cluster becomes more important, and how to process single-point failover and recovery of the storage cluster in the information system becomes a problem to be solved.
The prior art has at least the following problems:
in the existing service data processing method, when a certain storage cluster fails, after the failure transfer is started, the service data cannot be automatically synchronized from the stored storage cluster to the storage cluster with failure recovery, meanwhile, the service data which is not synchronized during the failure transfer cannot be accessed, and the failure transfer among heterogeneous storage clusters is not supported, so that the service data processing efficiency is low, the application scene is narrower, and the user experience is poor.
Disclosure of Invention
In view of this, the embodiments of the present invention provide a service data processing method and apparatus, which can adjust a storage service mode according to a functional state of a storage cluster, call an API interface of the storage cluster in the storage service mode to execute service data processing, so as to improve service data processing efficiency, support fast transfer and switching of single point failure, expand an application scenario of service data processing, and improve user experience.
To achieve the above object, according to an aspect of an embodiment of the present invention, there is provided a service data processing method, including:
receiving a service data processing request sent by a service system, and determining a storage service mode of a database system; the database system comprises a main storage cluster and at least one auxiliary storage cluster, and the service data processing request comprises at least one of a service data storage request and a service data reading request;
If the storage service mode is a multi-storage service mode, executing at least one of a service data storage process and a service data reading process according to the service data processing request; wherein executing the business data storage process includes: calling an API (application program interface) of a main storage cluster to store the service data object into the main storage cluster according to the service data object indicated by the service data storage request, and asynchronously storing the service data object stored into the main storage cluster into at least one auxiliary storage cluster; the performing of the service data reading process includes: calling an API interface of the main storage cluster to read corresponding service data according to the service data number indicated by the service data reading request;
and sending the service data processing result to the service system.
Further, before the step of determining the storage service mode of the database system, the method further comprises:
accessing storage clusters in the database system to determine whether storage functions of the plurality of storage clusters are available;
and determining a storage service mode according to the judging result.
Further, the step of storing the business data object in the main storage cluster further comprises:
if the business data object is stored in the main storage cluster, receiving storage failure feedback;
And storing the business data object to any one of the auxiliary storage clusters, and asynchronously storing the business data object to other auxiliary storage clusters after the business data object is successfully stored.
Further, the method further comprises:
periodically accessing the main storage cluster, and judging whether the main storage cluster recovers the storage function;
if yes, asynchronously storing the business data objects stored in the secondary storage cluster into the primary storage cluster.
Further, according to the service data number indicated by the service data reading request, calling the API interface of the main storage cluster to read the corresponding service data, and further including:
if the API interface of the main storage cluster is called to read the corresponding service data, receiving a reading failure feedback;
and calling an API interface of any pair of storage clusters to read corresponding service data until receiving the successful reading feedback or the complete failed reading feedback.
Further, if the storage service mode is a single storage service mode, the method further includes:
determining a target storage cluster corresponding to the single storage service mode;
calling an API interface of a target storage cluster to store the service data object into the target storage cluster according to the service data object indicated by the service data storage request; and/or
And calling an API interface of the target storage cluster to read corresponding service data according to the service data number indicated by the service data reading request.
Further, in the case that the storage service mode is adjusted to the multiple storage service mode, the method further includes:
and asynchronously storing the business data objects stored to the target storage cluster into other storage clusters of the database system.
According to still another aspect of the embodiment of the present invention, there is provided a service data processing apparatus including:
the storage service mode determining module is used for receiving a service data processing request sent by the service system and determining a storage service mode of the database system; the database system comprises a main storage cluster and at least one auxiliary storage cluster, and the service data processing request comprises at least one of a service data storage request and a service data reading request;
the business data processing module is used for: if the storage service mode is a multi-storage service mode, executing at least one of a service data storage process and a service data reading process according to the service data processing request; wherein executing the business data storage process includes: calling an API (application program interface) of a main storage cluster to store the service data object into the main storage cluster according to the service data object indicated by the service data storage request, and asynchronously storing the service data object stored into the main storage cluster into at least one auxiliary storage cluster; the performing of the service data reading process includes: calling an API interface of the main storage cluster to read corresponding service data according to the service data number indicated by the service data reading request;
And the sending module is used for sending the service data processing result to the service system.
According to another aspect of an embodiment of the present invention, there is provided an electronic device for processing service data, including:
one or more processors;
storage means for storing one or more programs,
the one or more programs, when executed by the one or more processors, cause the one or more processors to implement any of the traffic data processing methods described above.
According to still another aspect of an embodiment of the present invention, there is provided a computer readable medium having stored thereon a computer program which, when executed by a processor, implements a service data processing method as any one of the above.
One embodiment of the above invention has the following advantages or benefits: because the service data processing request sent by the service system is received, the storage service mode of the database system is determined; the database system comprises a main storage cluster and at least one auxiliary storage cluster, and the service data processing request comprises at least one of a service data storage request and a service data reading request; if the storage service mode is a multi-storage service mode, executing at least one of a service data storage process and a service data reading process according to the service data processing request; wherein executing the business data storage process includes: calling an API (application program interface) of a main storage cluster to store the service data object into the main storage cluster according to the service data object indicated by the service data storage request, and asynchronously storing the service data object stored into the main storage cluster into at least one auxiliary storage cluster; the performing of the service data reading process includes: calling an API interface of the main storage cluster to read corresponding service data according to the service data number indicated by the service data reading request; the technical means for sending the service data processing result to the service system overcomes the technical problems that in the existing service data processing method, after the fault transfer is started, the service data cannot be automatically synchronized from a stored storage cluster to a storage cluster with fault recovery, the service data which is not synchronized in the fault transfer process cannot be accessed, the fault transfer among heterogeneous storage clusters is not supported, the fault tolerance and the robustness of the service data processing of a database system are poor, the service data processing efficiency is low, the application scene is narrow and the user experience is poor, and further the technical problems that the storage service mode can be adjusted according to the functional state of the storage cluster, the API interface of the storage cluster in the storage service mode is called to execute the service data processing, the service data processing efficiency is improved, the fault tolerance and the robustness of the database system are improved, the quick transfer switching of single-point faults is supported, the high availability of the system storage is improved, the application scene of the service data processing is expanded, and the technical effect of the user experience is improved are further achieved.
Further effects of the above-described non-conventional alternatives are described below in connection with the embodiments.
Drawings
The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:
fig. 1 is a schematic diagram of a main flow of a service data processing method according to an embodiment of the present invention;
FIG. 2a is a schematic diagram of a main flow of a method for processing business data in a dual storage service mode according to another embodiment of the present invention;
FIG. 2b is a schematic diagram of a main flow of a method for processing business data in a single storage service mode according to still another embodiment of the present invention;
fig. 3 is a schematic diagram of main modules of a service data processing apparatus according to an embodiment of the present invention;
FIG. 4 is an exemplary system architecture diagram in which embodiments of the present invention may be applied;
fig. 5 is a schematic diagram of a computer system suitable for use in implementing an embodiment of the invention.
Detailed Description
Exemplary embodiments of the present invention will now be described with reference to the accompanying drawings, in which various details of the embodiments of the present invention are included to facilitate understanding, and are to be considered merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
Fig. 1 is a schematic diagram of a main flow of a service data processing method according to an embodiment of the present invention; as shown in fig. 1, the service data processing method provided by the embodiment of the present invention mainly includes:
step S101, receiving a business data processing request sent by a business system, and determining a storage service mode of a database system; wherein the database system comprises a primary storage cluster and at least one secondary storage cluster, and the service data processing request comprises at least one of a service data storage request and a service data reading request.
Further, according to an embodiment of the present invention, before the step of determining the storage service mode of the database system, the method further includes:
accessing storage clusters in the database system to determine whether storage functions of the plurality of storage clusters are available;
and determining a storage service mode according to the judging result.
The single storage service mode comprises a single master storage service mode and a single slave storage service mode, and is storage service used for executing business data only by a single master storage cluster or a single slave storage cluster in the database storage system;
the multiple storage service mode refers to a mode in which there are a plurality (greater than or equal to two) of storage clusters in the database storage system for executing storage service data.
According to a specific implementation manner of the embodiment of the present invention, after receiving a service processing request, a storage cluster in a database system may be accessed, and a storage service mode may be determined according to a determination result. The storage clusters in the database system can be accessed periodically, and the storage service mode is determined according to the judgment result. For example, for a dual storage cluster database system, if all storage clusters in the database storage system are accessed and then fed back as all available, then the storage service mode is determined to be a multiple storage service mode. If any storage cluster is fed back to be in a completely unavailable state, after waiting for a period of time (for example, twice the data synchronization time length is used for ensuring that the data which is just stored in the storage cluster with the fault is synchronized to other storage clusters from the corresponding memory synchronization queues, and the time length can be adjusted according to the actual situation), the storage service mode is adjusted to be a single storage service mode. Through the arrangement, the storage service mode is determined by combining the access results (judging whether the access results are available) of all the storage clusters in the database system, the problem of single-point failure caused by failure of a certain storage cluster is avoided, and the service data processing efficiency is improved.
Step S102, if the storage service mode is a multi-storage service mode, executing at least one of a business data storage process and a business data reading process according to the business data processing request; wherein executing the business data storage process includes: calling an API (application program interface) of a main storage cluster to store the service data object into the main storage cluster according to the service data object indicated by the service data storage request, and asynchronously storing the service data object stored into the main storage cluster into at least one auxiliary storage cluster; the performing of the service data reading process includes: and calling an API interface of the main storage cluster to read corresponding service data according to the service data number indicated by the service data reading request.
According to the embodiment of the invention, in the multi-storage service mode, for the service data storage request, the API interface of the main storage cluster is preferentially called to execute the writing operation of the service data object. According to a specific implementation manner of the embodiment of the present invention, a synchronization schedule may be created in the primary storage cluster, the service data key and the service type are written into the synchronization schedule, and after the service data object is written into the primary storage cluster, the synchronization schedule is fed back to the information system, so that the information system uses the service data batch writing function to implement asynchronous writing operation of the secondary storage cluster. On the other hand, in the multi-storage service mode, for the service reading request, the main storage cluster can be directly accessed according to the service data number, so as to realize the reading operation of the service data. The storage cluster after executing the steps sends storage result feedback/reading result feedback to the database system; if the storage result feedback/reading result feedback is successful, the current business data processing is completed; if the feedback is failure, switching the storage clusters, and calling an API interface of another storage cluster to continue to execute corresponding service processing.
Further, according to an embodiment of the present invention, the step of storing the service data object in the main storage cluster further includes:
if the business data object is stored in the main storage cluster, receiving storage failure feedback;
and storing the business data object to any one of the auxiliary storage clusters, and asynchronously storing the business data object to other auxiliary storage clusters after the business data object is successfully stored.
According to the embodiment of the invention, in the multi-storage service mode, when the writing of the main storage cluster fails, service switching is directly carried out, any one of the auxiliary storage clusters is called to write the corresponding service data object, and after the storage is completed, the service data object is asynchronously stored in other auxiliary storage clusters.
Preferably, the method further comprises:
periodically accessing the main storage cluster, and judging whether the main storage cluster recovers the storage function;
if yes, asynchronously storing the business data objects stored in the secondary storage cluster into the primary storage cluster.
Through the arrangement, after the main storage cluster with the previous faults recovers the storage function, the business data objects stored in the auxiliary storage clusters are asynchronously stored in the main storage clusters, so that the data consistency of all the storage clusters in the database system is ensured. According to a specific implementation manner of the embodiment of the present invention, if a storage failure exists in a main storage cluster, a multi-storage service mode is adjusted to a single-storage service mode, and after determining that the main storage cluster recovers a storage function and completing synchronization of corresponding service data objects, the single-storage service mode is also required to be recovered to the multi-storage service mode. By the arrangement, the fault tolerance and the robustness of the database system are improved, and the high availability of system storage is improved.
Further, according to an embodiment of the present invention, the step of calling the API interface of the main storage cluster to read the corresponding service data according to the service data number indicated by the service data reading request further includes:
if the API interface of the main storage cluster is called to read the corresponding service data, receiving a reading failure feedback;
and calling an API interface of any pair of storage clusters to read corresponding service data until receiving the successful reading feedback or the complete failed reading feedback.
According to the embodiment of the invention, in the multi-storage service mode, for the service reading request, if the main storage cluster feeds back the reading failure, the API interface of any pair of storage clusters can be called to read corresponding service data until receiving the reading success feedback or all the reading failure feedback. Further, according to an embodiment of the present invention, if the first feedback reading fails, a second reading may be performed.
Preferably, according to an embodiment of the present invention, if the storage service mode is a single storage service mode, the method further includes:
determining a target storage cluster corresponding to the single storage service mode;
calling an API interface of a target storage cluster to store the service data object into the target storage cluster according to the service data object indicated by the service data storage request; and/or
And calling an API interface of the target storage cluster to read corresponding service data according to the service data number indicated by the service data reading request.
According to the embodiment of the invention, if the storage service mode is a single storage service mode, only one storage cluster is used for the access (storage and reading) operation of the service data in the current mode, the corresponding target storage cluster can be directly determined, and the API interface of the target storage cluster is called to execute the corresponding service data processing.
Further, according to an embodiment of the present invention, in a case where the storage service mode is adjusted from the single storage service mode to the multiple storage service mode, the method further includes:
and asynchronously storing the business data objects stored to the target storage cluster into other storage clusters of the database system.
According to the embodiment of the invention, when the access function of the database system determines that all the access functions of the storage clusters are normal, the storage service mode is adjusted to the multi-storage service mode, and the data consistency among a plurality of storage clusters in the database system is guaranteed by the arrangement.
Step S103, sending the service data processing result to the service system.
If the storage or the reading is successful, sending a successful service data processing result to a service system; and if the storage or all the reading fails, sending a service data processing failure result to the service system.
According to the technical scheme of the embodiment of the invention, the storage service mode of the database system is determined by adopting the service data processing request sent by the receiving service system; the database system comprises a main storage cluster and at least one auxiliary storage cluster, and the service data processing request comprises at least one of a service data storage request and a service data reading request; if the storage service mode is a multi-storage service mode, executing at least one of a service data storage process and a service data reading process according to the service data processing request; wherein executing the business data storage process includes: calling an API (application program interface) of a main storage cluster to store the service data object into the main storage cluster according to the service data object indicated by the service data storage request, and asynchronously storing the service data object stored into the main storage cluster into at least one auxiliary storage cluster; the performing of the service data reading process includes: calling an API interface of the main storage cluster to read corresponding service data according to the service data number indicated by the service data reading request; the technical means for sending the service data processing result to the service system overcomes the technical problems that in the existing service data processing method, after the fault transfer is started, the service data cannot be automatically synchronized from a stored storage cluster to a storage cluster with fault recovery, the service data which is not synchronized in the fault transfer process cannot be accessed, the fault transfer among heterogeneous storage clusters is not supported, the fault tolerance and the robustness of the service data processing of a database system are poor, the service data processing efficiency is low, the application scene is narrow and the user experience is poor, and further the technical problems that the storage service mode can be adjusted according to the functional state of the storage cluster, the API interface of the storage cluster in the storage service mode is called to execute the service data processing, the service data processing efficiency is improved, the fault tolerance and the robustness of the database system are improved, the quick transfer switching of single-point faults is supported, the high availability of the system storage is improved, the application scene of the service data processing is expanded, and the technical effect of the user experience is improved are further achieved.
Fig. 2a and 2b are schematic diagrams of main flows of a service data processing method according to still another embodiment of the present invention; as shown in fig. 2a and fig. 2b, the service data processing method provided by the embodiment of the present invention is applicable to a database system with dual storage clusters:
fig. 2a shows a method for processing service data in a dual storage service mode, which mainly includes steps S201 to S209, and mainly includes:
step S201, receiving a service data processing request sent by a service system, and accessing a storage cluster in a database system. Wherein the service data processing request includes at least one of a service data storage request and a service data read request. In an embodiment of the present invention, a database system includes a primary storage cluster and a secondary storage cluster,
according to a specific implementation manner of the embodiment of the invention, after receiving the service processing request, a storage cluster in the database system is accessed, and a storage service mode is determined according to the access feedback result. The storage clusters in the database system can be accessed periodically, and the storage service mode is determined according to the judgment result.
Step S202, a storage service mode is determined according to the access feedback result. If the feedback results accessed by the dual storage clusters are all available, determining the dual storage service mode; and if the access feedback result of any storage cluster is in an unavailable state, determining that the storage cluster is in a single storage service mode.
For example, for a dual storage cluster database system, if feedback is all available after accessing a dual storage cluster in the database storage system, then the storage service mode is determined to be a dual storage service mode. If any storage cluster is fed back to be in a completely unavailable state, after waiting for a period of time (for example, twice the data synchronization time length is used for ensuring that the data which is just stored in the storage cluster with the fault is synchronized to other storage clusters from the corresponding memory synchronization queues, and the time length can be adjusted according to the actual situation), the storage service mode is adjusted to be a single storage service mode. Through the arrangement, the storage service mode is determined by combining the access results (judging whether the access results are available) of all the storage clusters in the database system, the problem of single-point failure caused by failure of a certain storage cluster is avoided, and the service data processing efficiency is improved.
Step S203, calling the API interface of the main storage cluster to store the business data object into the main storage cluster according to the business data object indicated by the business data storage request, and asynchronously storing the business data object stored into the main storage cluster into the auxiliary storage cluster.
According to the embodiment of the invention, in the dual storage service mode, for the service data storage request, the API interface of the main storage cluster is preferentially called to execute the writing operation of the service data object. According to a specific implementation manner of the embodiment of the present invention, a synchronization schedule may be created in the primary storage cluster, the service data key and the service type are written into the synchronization schedule, and after the service data object is written into the primary storage cluster, the synchronization schedule is fed back to the information system, so that the information system uses the service data batch writing function to implement asynchronous writing operation of the secondary storage cluster.
Step S204, if the business data object is stored in the main storage cluster, receiving the storage failure feedback; the business data object is stored to the secondary storage cluster.
According to the embodiment of the invention, in the multi-storage service mode, when the main storage cluster fails to write, service switching is directly performed, and the auxiliary storage cluster is called to write corresponding service data objects.
Step S205, periodically accessing the main storage cluster, and judging whether the main storage cluster recovers the storage function; if yes, asynchronously storing the business data objects stored in the secondary storage cluster into the primary storage cluster.
Through the arrangement, after the main storage cluster with the previous faults recovers the storage function, the business data objects stored in the auxiliary storage clusters are asynchronously stored in the main storage clusters, so that the data consistency of all the storage clusters in the database system is ensured.
Step S206, the storage processing result of the service data is sent to the service system.
Step S207, calling the API interface of the main storage cluster to read the corresponding service data according to the service data number indicated by the service data reading request.
According to the embodiment of the invention, in the dual storage service mode, for the service reading request, the main storage cluster can be directly accessed according to the service data number, so that the service data reading operation is realized.
Step S208, if the API interface of the main storage cluster is called to read the corresponding service data, receiving a reading failure feedback; and calling an API (application program interface) of the secondary storage cluster to read corresponding service data.
According to the embodiment of the invention, in the dual storage service mode, when the main storage cluster fails to write, service switching is directly performed, and the auxiliary storage cluster is called to write corresponding service data objects.
Step S209, the reading processing result of the service data is sent to the service system.
Fig. 2b shows a method for processing service data in a single storage service mode, which mainly includes steps S201, S210-S211, and mainly includes:
step S210, determining that the storage service mode is a single storage service mode according to the access feedback result, and determining a target storage cluster corresponding to the single storage service mode.
According to the embodiment of the invention, if any storage cluster is fed back to be in a completely unavailable state, after waiting for a period of time (for example, twice the data synchronization time length is used to ensure that the data which is just stored in the storage cluster with the failure is synchronized to other storage clusters from the corresponding memory synchronization queues, and the time length can be adjusted according to the actual situation), the storage service mode is adjusted to be a single storage service mode.
Step S211, calling the API interface of the target storage cluster to store the business data object into the target storage cluster according to the business data object indicated by the business data storage request.
Step S212, the storage processing result of the service data is sent to the service system.
Step S213, calling the API interface of the target storage cluster to read the corresponding service data according to the service data number indicated by the service data reading request.
According to a specific implementation manner of the embodiment of the present invention, if the storage service mode is a single storage service mode, only one storage cluster is used for access (storage and reading) operation of service data in the current mode, and a corresponding target storage cluster can be directly determined, and an API interface of the target storage cluster is called to execute corresponding service data processing.
In step S214, when the storage service mode is adjusted to the multiple storage service mode, the business data objects stored in the target storage cluster are asynchronously stored in other storage clusters of the database system.
Through the arrangement, data consistency among a plurality of storage clusters in the database system is guaranteed.
Step S215, the read processing result of the service data is sent to the service system.
According to the technical scheme of the embodiment of the invention, the storage service mode of the database system is determined by adopting the service data processing request sent by the receiving service system; the database system comprises a main storage cluster and at least one auxiliary storage cluster, and the service data processing request comprises at least one of a service data storage request and a service data reading request; if the storage service mode is a multi-storage service mode, executing at least one of a service data storage process and a service data reading process according to the service data processing request; wherein executing the business data storage process includes: calling an API (application program interface) of a main storage cluster to store the service data object into the main storage cluster according to the service data object indicated by the service data storage request, and asynchronously storing the service data object stored into the main storage cluster into at least one auxiliary storage cluster; the performing of the service data reading process includes: calling an API interface of the main storage cluster to read corresponding service data according to the service data number indicated by the service data reading request; the technical means for sending the service data processing result to the service system overcomes the technical problems that in the existing service data processing method, after the fault transfer is started, the service data cannot be automatically synchronized from a stored storage cluster to a storage cluster with fault recovery, the service data which is not synchronized in the fault transfer process cannot be accessed, the fault transfer among heterogeneous storage clusters is not supported, the fault tolerance and the robustness of the service data processing of a database system are poor, the service data processing efficiency is low, the application scene is narrow and the user experience is poor, and further the technical problems that the storage service mode can be adjusted according to the functional state of the storage cluster, the API interface of the storage cluster in the storage service mode is called to execute the service data processing, the service data processing efficiency is improved, the fault tolerance and the robustness of the database system are improved, the quick transfer switching of single-point faults is supported, the high availability of the system storage is improved, the application scene of the service data processing is expanded, and the technical effect of the user experience is improved are further achieved.
Fig. 3 is a schematic diagram of main modules of a service data processing apparatus according to an embodiment of the present invention; as shown in fig. 3, the service data processing apparatus 300 provided in the embodiment of the present invention mainly includes:
the storage service mode determining module 301 is configured to receive a service data processing request sent by a service system, and determine a storage service mode of a database system; wherein the database system comprises a primary storage cluster and at least one secondary storage cluster, and the service data processing request comprises at least one of a service data storage request and a service data reading request.
Further, according to an embodiment of the present invention, the storage service mode determining module 301 is further configured to:
accessing storage clusters in the database system to determine whether storage functions of the plurality of storage clusters are available;
and determining a storage service mode according to the judging result.
According to a specific implementation manner of the embodiment of the present invention, after receiving a service processing request, a storage cluster in a database system may be accessed, and a storage service mode may be determined according to a determination result. The storage clusters in the database system can be accessed periodically, and the storage service mode is determined according to the judgment result. For example, for a dual storage cluster database system, if all storage clusters in the database storage system are accessed and then fed back as all available, then the storage service mode is determined to be a multiple storage service mode. If any storage cluster is fed back to be in a completely unavailable state, after waiting for a period of time (for example, twice the data synchronization time length is used for ensuring that the data which is just stored in the storage cluster with the fault is synchronized to other storage clusters from the corresponding memory synchronization queues, and the time length can be adjusted according to the actual situation), the storage service mode is adjusted to be a single storage service mode. Through the arrangement, the storage service mode is determined by combining the access results (judging whether the access results are available) of all the storage clusters in the database system, the problem of single-point failure caused by failure of a certain storage cluster is avoided, and the service data processing efficiency is improved.
A service data processing module 302, configured to: if the storage service mode is a multi-storage service mode, executing at least one of a service data storage process and a service data reading process according to the service data processing request; wherein executing the business data storage process includes: calling an API (application program interface) of a main storage cluster to store the service data object into the main storage cluster according to the service data object indicated by the service data storage request, and asynchronously storing the service data object stored into the main storage cluster into at least one auxiliary storage cluster; the performing of the service data reading process includes: and calling an API interface of the main storage cluster to read corresponding service data according to the service data number indicated by the service data reading request.
According to the embodiment of the invention, in the multi-storage service mode, for the service data storage request, the API interface of the main storage cluster is preferentially called to execute the writing operation of the service data object. According to a specific implementation manner of the embodiment of the present invention, a synchronization schedule may be created in the primary storage cluster, the service data key and the service type are written into the synchronization schedule, and after the service data object is written into the primary storage cluster, the synchronization schedule is fed back to the information system, so that the information system uses the service data batch writing function to implement asynchronous writing operation of the secondary storage cluster. On the other hand, in the multi-storage service mode, for the service reading request, the main storage cluster can be directly accessed according to the service data number, so as to realize the reading operation of the service data.
Further, according to an embodiment of the present invention, the service data processing module 302 is further configured to:
if the business data object is stored in the main storage cluster, receiving storage failure feedback;
and storing the business data object to any one of the auxiliary storage clusters, and asynchronously storing the business data object to other auxiliary storage clusters after the business data object is successfully stored.
According to the embodiment of the invention, in the multi-storage service mode, when the writing of the main storage cluster fails, service switching is directly carried out, any one of the auxiliary storage clusters is called to write the corresponding service data object, and after the storage is completed, the service data object is asynchronously stored in other auxiliary storage clusters.
Preferably, the service data processing module 302 is further configured to:
periodically accessing the main storage cluster, and judging whether the main storage cluster recovers the storage function;
if yes, asynchronously storing the business data objects stored in the secondary storage cluster into the primary storage cluster.
Through the arrangement, after the main storage cluster with the previous faults recovers the storage function, the business data objects stored in the auxiliary storage clusters are asynchronously stored in the main storage clusters, so that the data consistency of all the storage clusters in the database system is ensured.
Further, according to an embodiment of the present invention, the service data processing module 302 is further configured to:
if the API interface of the main storage cluster is called to read the corresponding service data, receiving a reading failure feedback;
and calling an API interface of any pair of storage clusters to read corresponding service data until receiving the successful reading feedback or the complete failed reading feedback.
According to the embodiment of the invention, in the multi-storage service mode, for the service reading request, if the main storage cluster feeds back the reading failure, the API interface of any pair of storage clusters can be called to read corresponding service data until receiving the reading success feedback or all the reading failure feedback. Further, according to an embodiment of the present invention, if the first feedback reading fails, a second reading may be performed.
Preferably, according to an embodiment of the present invention, if the storage service mode is a single storage service mode, the service data processing module 302 is further configured to:
determining a target storage cluster corresponding to the single storage service mode;
calling an API interface of a target storage cluster to store the service data object into the target storage cluster according to the service data object indicated by the service data storage request; and/or
And calling an API interface of the target storage cluster to read corresponding service data according to the service data number indicated by the service data reading request.
According to the embodiment of the invention, if the storage service mode is a single storage service mode, only one storage cluster is used for the access (storage and reading) operation of the service data in the current mode, the corresponding target storage cluster can be directly determined, and the API interface of the target storage cluster is called to execute the corresponding service data processing.
Further, in the case that the storage service mode is adjusted from the single storage service mode to the multiple storage service mode according to the embodiment of the present invention, the above-mentioned business data processing module 302 is further configured to:
and asynchronously storing the business data objects stored to the target storage cluster into other storage clusters of the database system.
Through the arrangement, data consistency among a plurality of storage clusters in the database system is guaranteed.
And the sending module 303 is configured to send the service data processing result to the service system.
According to the technical scheme of the embodiment of the invention, the storage service mode of the database system is determined by adopting the service data processing request sent by the receiving service system; the database system comprises a main storage cluster and at least one auxiliary storage cluster, and the service data processing request comprises at least one of a service data storage request and a service data reading request; if the storage service mode is a multi-storage service mode, executing at least one of a service data storage process and a service data reading process according to the service data processing request; wherein executing the business data storage process includes: calling an API (application program interface) of a main storage cluster to store the service data object into the main storage cluster according to the service data object indicated by the service data storage request, and asynchronously storing the service data object stored into the main storage cluster into at least one auxiliary storage cluster; the performing of the service data reading process includes: calling an API interface of the main storage cluster to read corresponding service data according to the service data number indicated by the service data reading request; the technical means for sending the service data processing result to the service system overcomes the technical problems that in the existing service data processing method, after the fault transfer is started, the service data cannot be automatically synchronized from a stored storage cluster to a storage cluster with fault recovery, the service data which is not synchronized in the fault transfer process cannot be accessed, the fault transfer among heterogeneous storage clusters is not supported, the fault tolerance and the robustness of the service data processing of a database system are poor, the service data processing efficiency is low, the application scene is narrow and the user experience is poor, and further the technical problems that the storage service mode can be adjusted according to the functional state of the storage cluster, the API interface of the storage cluster in the storage service mode is called to execute the service data processing, the service data processing efficiency is improved, the fault tolerance and the robustness of the database system are improved, the quick transfer switching of single-point faults is supported, the high availability of the system storage is improved, the application scene of the service data processing is expanded, and the technical effect of the user experience is improved are further achieved.
Fig. 4 shows an exemplary system architecture 400 to which the traffic data processing method or traffic data processing apparatus of an embodiment of the present invention may be applied.
As shown in fig. 4, a system architecture 400 may include terminal devices 401, 402, 403, a network 404, and a server 405 (this architecture is merely an example, and the components contained in a particular architecture may be tailored to the application specific case). The network 404 is used as a medium to provide communication links between the terminal devices 401, 402, 403 and the server 405. The network 404 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.
A user may interact with the server 405 via the network 404 using the terminal devices 401, 402, 403 to receive or send messages or the like. Various communication client applications may be installed on the terminal devices 401, 402, 403, such as a business data processing class application, a web browser application, a search class application, a data processing class tool, a mailbox client, social platform software, etc. (as examples only).
The terminal devices 401, 402, 403 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smartphones, tablets, laptop and desktop computers, and the like.
The server 405 may be a server providing various services, such as a server (by way of example only) that utilizes terminal devices 401, 402, 403 (performs business data processing/performs data processing) for a user. The server may analyze and process the received data, such as the service data processing request, and feedback the processing result (e.g., service data processing result feedback information—only an example) to the terminal device.
It should be noted that, the service data processing method provided in the embodiment of the present invention is generally executed by the server 405, and accordingly, the service data processing device is generally disposed in the server 405.
It should be understood that the number of terminal devices, networks and servers in fig. 4 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
Referring now to FIG. 5, there is illustrated a schematic diagram of a computer system 500 suitable for use in implementing a terminal device or server in accordance with an embodiment of the present invention. The terminal device or server shown in fig. 5 is only an example, and should not impose any limitation on the functions and scope of use of the embodiments of the present invention.
As shown in fig. 5, the computer system 500 includes a Central Processing Unit (CPU) 501, which can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 502 or a program loaded from a storage section 508 into a Random Access Memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the system 500 are also stored. The CPU 501, ROM 502, and RAM 503 are connected to each other through a bus 504. An input/output (I/O) interface 505 is also connected to bus 504.
The following components are connected to the I/O interface 505: an input section 506 including a keyboard, a mouse, and the like; an output portion 507 including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker, and the like; a storage portion 508 including a hard disk and the like; and a communication section 509 including a network interface card such as a LAN card, a modem, or the like. The communication section 509 performs communication processing via a network such as the internet. The drive 510 is also connected to the I/O interface 505 as needed. A removable medium 511 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 510 as needed so that a computer program read therefrom is mounted into the storage section 508 as needed.
In particular, according to embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication portion 509, and/or installed from the removable media 511. The above-described functions defined in the system of the present invention are performed when the computer program is executed by a Central Processing Unit (CPU) 501.
The computer readable medium shown in the present invention may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules involved in the embodiments of the present invention may be implemented in software or in hardware. The described modules may also be provided in a processor, for example, as: a processor includes a storage service mode determination module, a traffic data processing module, and a transmission module. The names of these modules do not in any way constitute a limitation of the module itself, and for example, the storage service mode determining module may also be described as "a module for receiving a service data processing request sent by a service system, determining a storage service mode of a database system".
As another aspect, the present invention also provides a computer-readable medium that may be contained in the apparatus described in the above embodiments; or may be present alone without being fitted into the device. The computer readable medium carries one or more programs which, when executed by a device, cause the device to include: receiving a service data processing request sent by a service system, and determining a storage service mode of a database system; the database system comprises a main storage cluster and at least one auxiliary storage cluster, and the service data processing request comprises at least one of a service data storage request and a service data reading request; if the storage service mode is a multi-storage service mode, executing at least one of a service data storage process and a service data reading process according to the service data processing request; wherein executing the business data storage process includes: calling an API (application program interface) of a main storage cluster to store the service data object into the main storage cluster according to the service data object indicated by the service data storage request, and asynchronously storing the service data object stored into the main storage cluster into at least one auxiliary storage cluster; the performing of the service data reading process includes: calling an API interface of the main storage cluster to read corresponding service data according to the service data number indicated by the service data reading request; and sending the service data processing result to the service system.
According to the technical scheme of the embodiment of the invention, the storage service mode of the database system is determined by adopting the service data processing request sent by the receiving service system; the database system comprises a main storage cluster and at least one auxiliary storage cluster, and the service data processing request comprises at least one of a service data storage request and a service data reading request; if the storage service mode is a multi-storage service mode, executing at least one of a service data storage process and a service data reading process according to the service data processing request; wherein executing the business data storage process includes: calling an API (application program interface) of a main storage cluster to store the service data object into the main storage cluster according to the service data object indicated by the service data storage request, and asynchronously storing the service data object stored into the main storage cluster into at least one auxiliary storage cluster; the performing of the service data reading process includes: calling an API interface of the main storage cluster to read corresponding service data according to the service data number indicated by the service data reading request; the technical means for sending the service data processing result to the service system overcomes the technical problems that in the existing service data processing method, after the fault transfer is started, the service data cannot be automatically synchronized from a stored storage cluster to a storage cluster with fault recovery, the service data which is not synchronized in fault transfer can not be accessed, the fault transfer among heterogeneous storage clusters is not supported, the service data processing efficiency is low, the application scene is narrow, and the user experience is poor are solved, and further the technical effects that the storage service mode can be adjusted according to the functional state of the storage cluster, the API interface of the storage cluster under the storage service mode is called to execute the service data processing, the service data processing efficiency is improved, the quick transfer switching of single-point faults is supported, the application scene of the service data processing is expanded, and the user experience is improved are achieved.
The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives can occur depending upon design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.

Claims (10)

1. A method for processing service data, comprising:
receiving a service data processing request sent by a service system, and determining a storage service mode of a database system; the database system comprises a main storage cluster and at least one auxiliary storage cluster, and the service data processing request comprises at least one of a service data storage request and a service data reading request;
if the storage service mode is a multi-storage service mode, executing at least one of a service data storage process and a service data reading process according to the service data processing request; wherein executing the business data storage processing includes: calling an API (application program interface) of the main storage cluster to store the service data object into the main storage cluster according to the service data object indicated by the service data storage request, and asynchronously storing the service data object stored into the main storage cluster into the at least one auxiliary storage cluster; the executing of the service data reading process includes: calling an API interface of the main storage cluster to read corresponding service data according to the service data number indicated by the service data reading request;
And sending a service data processing result to the service system.
2. The business data processing method according to claim 1, wherein before the step of determining the storage service mode of the database system, the method further comprises:
accessing storage clusters in the database system to determine whether storage functions of the plurality of storage clusters are available;
and determining the storage service mode according to the judging result.
3. The method of claim 1, wherein the step of storing the business data object into the primary storage cluster further comprises:
if the business data object is stored in the main storage cluster, receiving storage failure feedback;
and storing the service data object to any one of the auxiliary storage clusters, and asynchronously storing the service data object to other auxiliary storage clusters after successful storage.
4. A business data processing method according to claim 3, characterized in that the method further comprises:
periodically accessing the main storage cluster, and judging whether the main storage cluster recovers a storage function or not;
and if yes, asynchronously storing the business data objects stored in the secondary storage cluster into the primary storage cluster.
5. The service data processing method according to claim 1, wherein the step of calling the API interface of the main storage cluster to read the corresponding service data according to the service data number indicated by the service data reading request further comprises:
if the API interface of the main storage cluster is called to read the corresponding service data, receiving a reading failure feedback;
and calling the API interface of any pair of storage clusters to read corresponding service data until receiving the successful reading feedback or the complete failed reading feedback.
6. The traffic data processing method according to claim 1, wherein if the storage service mode is a single storage service mode, the method further comprises:
determining a target storage cluster corresponding to the single storage service mode;
calling an API interface of the target storage cluster to store the service data object into the target storage cluster according to the service data object indicated by the service data storage request; and/or
And calling an API interface of the target storage cluster to read corresponding service data according to the service data number indicated by the service data reading request.
7. The traffic data processing method according to claim 6, wherein in the case where the storage service mode is adjusted to a multiple storage service mode, the method further comprises:
And asynchronously storing the business data objects stored to the target storage cluster into other storage clusters of the database system.
8. A traffic data processing apparatus, comprising:
the storage service mode determining module is used for receiving a service data processing request sent by the service system and determining a storage service mode of the database system; the database system comprises a main storage cluster and at least one auxiliary storage cluster, and the service data processing request comprises at least one of a service data storage request and a service data reading request;
the business data processing module is used for: if the storage service mode is a multi-storage service mode, executing at least one of a service data storage process and a service data reading process according to the service data processing request; wherein executing the business data storage processing includes: calling an API (application program interface) of the main storage cluster to store the service data object into the main storage cluster according to the service data object indicated by the service data storage request, and asynchronously storing the service data object stored into the main storage cluster into the at least one auxiliary storage cluster; the executing of the service data reading process includes: calling an API interface of the main storage cluster to read corresponding service data according to the service data number indicated by the service data reading request;
And the sending module is used for sending the service data processing result to the service system.
9. An electronic device for processing business data, comprising:
one or more processors;
storage means for storing one or more programs,
when executed by the one or more processors, causes the one or more processors to implement the method of any of claims 1-7.
10. A computer readable medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the method according to any of claims 1-7.
CN202210715756.9A 2022-06-23 2022-06-23 Service data processing method and device Pending CN117312440A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210715756.9A CN117312440A (en) 2022-06-23 2022-06-23 Service data processing method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210715756.9A CN117312440A (en) 2022-06-23 2022-06-23 Service data processing method and device

Publications (1)

Publication Number Publication Date
CN117312440A true CN117312440A (en) 2023-12-29

Family

ID=89260871

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210715756.9A Pending CN117312440A (en) 2022-06-23 2022-06-23 Service data processing method and device

Country Status (1)

Country Link
CN (1) CN117312440A (en)

Similar Documents

Publication Publication Date Title
CN110609872B (en) Method and apparatus for synchronizing node data
CN111181765A (en) Task processing method and device
CN107818027B (en) Method and device for switching main name node and standby name node and distributed system
CN107526838B (en) Method and device for database cluster capacity expansion
CN113760522A (en) Task processing method and device
CN112084254A (en) Data synchronization method and system
CN113760924B (en) Distributed transaction processing method and device
CN117312440A (en) Service data processing method and device
CN114265605A (en) Version rollback method and device for functional component of business system
CN113760469A (en) Distributed computing method and device
CN113761075A (en) Method, device, equipment and computer readable medium for switching databases
CN113742376A (en) Data synchronization method, first server and data synchronization system
CN112883103A (en) Method and device for data transfer between clusters
CN115981828B (en) Service message processing method and device
CN111277632B (en) Method and device for managing applications in system cluster
CN111767113A (en) Method and device for realizing container eviction
CN116028245B (en) Service message processing method and device based on message middleware
CN115827174B (en) Task processing method and device based on multiple instances
CN112948430B (en) Date data query method and device
CN112749042B (en) Application running method and device
US12019652B2 (en) Method and device for synchronizing node data
CN112583614B (en) Host cluster management method, device and computer readable medium
CN115033600A (en) Data processing method and data processing device
CN114925305A (en) Page loading method and device
CN113760533A (en) Service processing method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination