WO2023185802A1 - 数据处理方法及装置 - Google Patents

数据处理方法及装置 Download PDF

Info

Publication number
WO2023185802A1
WO2023185802A1 PCT/CN2023/084287 CN2023084287W WO2023185802A1 WO 2023185802 A1 WO2023185802 A1 WO 2023185802A1 CN 2023084287 W CN2023084287 W CN 2023084287W WO 2023185802 A1 WO2023185802 A1 WO 2023185802A1
Authority
WO
WIPO (PCT)
Prior art keywords
image data
target image
data
serial number
processing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2023/084287
Other languages
English (en)
French (fr)
Inventor
李伟
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Alibaba China Co Ltd
Alibaba Cloud Computing Ltd
Original Assignee
Alibaba China Co Ltd
Alibaba Cloud Computing Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Alibaba China Co Ltd, Alibaba Cloud Computing Ltd filed Critical Alibaba China Co Ltd
Publication of WO2023185802A1 publication Critical patent/WO2023185802A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W64/00—Locating users or terminals or network equipment for network management purposes, e.g. mobility management
    • H04W64/006—Locating users or terminals or network equipment for network management purposes, e.g. mobility management with additional information processing, e.g. for direction or speed determination

Definitions

  • the embodiments of the present application relate to the field of data processing technology, and in particular, to a data processing method and device.
  • the process of factory digitization involves the realization of many businesses.
  • the collected field data can be backed up, that is, the data source can be connected to the servers corresponding to at least two data acquisition systems.
  • the server corresponding to each data acquisition system can obtain field data and send the obtained field data to the business party for processing.
  • the business party processes the received field data, since it contains duplicate field data, it needs to deduplicate the received field data first, and then perform subsequent processing on the field data after the deduplication process, so as to realize related services. .
  • Embodiments of the present application provide a data processing method and device to improve the accuracy of deduplication processing and ensure the normal realization of services.
  • embodiments of the present application provide a data processing method, applied to a first server, and the method includes:
  • the data is obtained by the second server processing the target image data associated with the serial number.
  • processing the target image data associated with the serial number to obtain first target image data, so as to implement the to-be-executed service based on the first target image data and the second target image data includes:
  • a target image data and the second target image data are deduplicated, and the to-be-executed business is implemented based on the target image data after the deduplication process.
  • processing the target image data associated with the serial number to obtain first target image data, so as to implement the to-be-executed service based on the first target image data and the second target image data includes:
  • the first target image data is sent to a third-party business platform, so that the third-party business platform implements the business to be executed according to the first target image data and the second target image data, wherein:
  • the second target image data is sent by the second server to the third-party business platform.
  • processing the target image data associated with the serial number to obtain the first target image data includes:
  • the step of setting the bitmap corresponding to the serial number according to the first target image data and the target image data associated with the serial number includes:
  • the bit corresponding to the failed data in the bitmap is set to zero, and the bit corresponding to the successfully processed data in the bitmap is set to one.
  • the resource lock permission is released so that the second server obtains the resource lock permission, and processes the target image data associated with the serial number according to the bitmap after the setting process to obtain the first Two target image data, including:
  • the second server obtains the resource lock permission, and obtains the target sub-mirror data from the target image data associated with the serial number to obtain the second target image data, wherein the associated The target image data of the serial number is obtained by the second server obtaining the target image data corresponding to the business to be executed from the data source according to the data collection request, and associating the serial number with the target image data, so target submirror
  • the data is data obtained from the target image data associated with the sequence number and corresponding to when the bits in the bitmap are zero.
  • the method also includes:
  • the target sub-image data is set as the second target image data and sent to the third-party service platform.
  • releasing the resource lock permission includes:
  • the resource lock permission is released.
  • the receiving data collection request includes:
  • the control rhythm module generates a frame synchronization signal every preset time period, wherein the frame synchronization signal includes a sequence number;
  • Receive the frame synchronization signal and generate a data collection request including the sequence number according to the frame synchronization signal.
  • embodiments of the present application provide a data processing device, including:
  • a receiving module configured to receive a data collection request, where the data collection request contains a serial number
  • a processing module configured to obtain target image data corresponding to the business to be executed from the data source according to the data collection request, and associate the serial number with the target image data;
  • the processing module is also used to process the target image data associated with the serial number to obtain the first target image data, so as to realize the business to be executed according to the first target image data and the second target image data,
  • the second target image data is obtained by processing the target image data associated with the serial number by the second server.
  • Embodiments of the present application provide a data processing method and device.
  • the first server can first receive a data collection request containing a serial number, and then obtain the target corresponding to the business to be executed from the data source according to the data collection request. Mirror data, and associate the serial number with the target mirror data.
  • the first server can also process the target mirror data associated with the serial number to obtain the first target mirror data.
  • the second server can process the target mirror data associated with the serial number, Obtain the second target image data, and then implement the business to be executed based on the first target image data and the second target image data.
  • FIG 1 is an application schematic diagram of the data processing process in the prior art of this application
  • Figure 2 is a schematic diagram of the architecture of the application system of the data processing method provided by the embodiment of the present application;
  • Figure 3 is a schematic flow chart of the data processing method provided by the embodiment of the present application.
  • FIG. 4 is an application schematic diagram of the data processing process in another prior art of this application.
  • FIG. 5 is a schematic diagram of the application of the data processing method provided by the embodiment of the present application.
  • Figure 6 is a schematic structural diagram of a data processing device provided by an embodiment of the present application.
  • FIG. 7 is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present application.
  • field data is generally collected through a processor, and then the collected field data is transmitted to the corresponding server of the data acquisition system for processing. And then realize related business.
  • field data is generally collected through PLC (Programmable Logic Controller) and then transmitted to the data acquisition system.
  • PLC Programmable Logic Controller
  • the data acquisition system is responsible for data calculation, storage, distribution, etc.
  • the northbound data mining system is a variety of business platforms. It generally receives data through the pull or push mode of the message queue, and can also query data.
  • the southbound data source part is abnormal, data loss will occur. If an exception occurs during data transmission within the data processing system, data loss will occur. The northbound business system lost data when acquiring data. If an exception occurs when the data processing system stores data, it will also cause the loss of data used by the northbound business system. If an exception occurs in the northbound business system, you will want to continue consuming data from the abnormal point after the exception is recovered, otherwise an exception will occur. Therefore, in order to solve the aforementioned problems and take into account the cost issue in the industrial field, the dual-machine backup method is more appropriate in terms of the balance between cost and availability.
  • the traditional high-availability solution for dual-machine backup can be divided into two parts: data and program: program high availability is to make the program stateless, store the status data in the database, and achieve high availability of the program through clustering.
  • program high availability is to make the program stateless, store the status data in the database, and achieve high availability of the program through clustering.
  • the program also exists in a mirror-like manner, that is, the program on one machine is the same as the program on the other machine. If one machine becomes abnormal, the other machine can completely replace it and continue working.
  • High data availability is generally solved through dual-machine database backup and shared storage. After one database becomes abnormal, the other database also has full data.
  • the dual-machine edge data acquisition system will store two copies of data at the same time and distribute the two-way data. It may cause a waste of storage resources, which is equivalent to storing two copies of one data. Moreover, both machines need to be backed up simultaneously, which puts pressure on both network IO and disk IO. If the business platform is only connected to one edge data acquisition system, there will be a switching time for failover when an exception occurs, and real-time hot backup is not possible. If the business platform is connected to two edge data acquisition systems, there will be two channels of duplicate data, and the business side will need to deduplicate them by themselves. For two-channel data, if deduplication is performed, since there may be multiple field data received, there is still a problem in identifying which data is the mirrored data of the same point.
  • Figure 1 is an application schematic diagram of the data processing process in the prior art of this application.
  • two channels of mirror data can be collected from the mirror data source through a programmable logic controller.
  • the systems (can also correspond to two servers) are edge data collection system A (i.e., the first server) and edge data collection system B (i.e., the second server).
  • 1, 2, and 3 represent the order in which data sources are generated, 1 ',2',3' are mirrored data.
  • the data is obtained and processed in edge data acquisition system A and edge data acquisition system B.
  • the order of each data may change. If the data received by the business platform is out of order, it cannot effectively identify which data is duplicate data, reducing the accuracy of deduplication processing and thus reducing the cost of data processing.
  • the effectiveness of backup affects the normal implementation of business.
  • this application associates the same serial number with the same data, so that after receiving duplicate data sent by multiple servers, it can effectively identify which data is duplicate data based on the serial number, and perform deduplication operations. , achieving the technical effect of not only improving the accuracy of deduplication processing, but also improving the effectiveness of data backup, thereby ensuring the normal realization of business.
  • FIG 2 is a schematic architectural diagram of the application system of the data processing method provided by the embodiment of the present application.
  • the application system may include a first server 101, a second server 102, a programmable logic controller 103, Mirror data source 104.
  • the first server 101 and the second server 102 can be servers corresponding to the edge data collection system, that is, they can implement data calculation, storage, distribution and other tasks that the edge data collection system is responsible for.
  • the programmable logic controller 103 is a device for connecting industrial equipment at industrial sites. It collects data through industrial protocols in the south direction and connects to edge data acquisition systems in the north direction.
  • the first server 101 and the second server 102 can respectively receive a data collection request containing a serial number, and then collect the target image data corresponding to the business to be executed from the image data source 104 through the programmable logic controller 103, and associate the serial number with Target image data. For example, if the sequence number is 8, the identifiers of the target image data obtained by the first server 101 and the second server 102 this time are both set to 8.
  • the application system can also include a third-party business platform 105.
  • the first target image data and the second target image data can be obtained, and then the first target image data and the second target image data can be obtained.
  • the first target image data and the second target image data are sent to the third-party service platform 105, so that the third-party service platform 105 implements the business to be executed based on the first target image data and the second target image data.
  • the third-party business platform 105 can implement the to-be-executed business based on the first target image data and the second target image data using existing methods, which will not be limited in detail here.
  • the application system may also include a metronome (or beat model) 106, which is a timing synchronization module that periodically sends a frame synchronization signal to mark the mirror data.
  • the function of the metronome is to establish a mapping relationship for the data of the mirror data source, that is, Data generated under the same beat have the same beat sequence number, and frame synchronization signals can be sent periodically.
  • the frame synchronization signals carry an incremental sequence number.
  • the two-channel data acquisition system can generate a data acquisition request and send it to The data source collects data, and the collected data uses a serial number for subsequent transmission.
  • the metronome can identify two interrelated mirror data generated by the mirror data source, that is, data with the same serial number can be considered as mirror data. .
  • Figure 3 is a schematic flowchart of a data processing method provided by an embodiment of the present application.
  • the method of this embodiment can be executed by the first server 101.
  • the method in this embodiment may include:
  • S301 Receive a data collection request, where the data collection request contains a serial number.
  • the data collection request periodically sent by the beat module can be received, and then the corresponding target image data is collected according to the data collection request, and related services are implemented based on the target image data.
  • receiving a data collection request may specifically include:
  • the control rhythm module generates a frame synchronization signal every preset time period, wherein the frame synchronization signal includes a sequence number.
  • Receive the frame synchronization signal and generate a data collection request including the sequence number according to the frame synchronization signal.
  • the beat module can generate a frame synchronization signal every preset time period.
  • the collection driver of the first server can generate a data collection request including a serial number.
  • the serial number can be Arabic numerals, or any one or more of uppercase letters, lowercase letters, and special symbols. You only need to ensure that the serial number generated is different each time.
  • S302 Obtain the target image data corresponding to the business to be executed from the data source according to the data collection request, and associate the serial number with the target image data.
  • the first server may obtain target mirroring data corresponding to the business to be executed from the data source according to the data collection request, where the data source may be a mirroring data source. After obtaining the target image data, you can associate the serial number with the target image data.
  • the second server can also receive the data collection request, and then obtain the target image data corresponding to the business to be executed from the data source according to the data collection request, and after obtaining the target image data, combine the serial number with the target image data Make an association. Through the serial number, a mapping relationship can be established between the target image data obtained by the first server and the target image data obtained by the second server, that is, the serial numbers of the data generated at the same beat are the same.
  • S303 Process the target image data associated with the serial number to obtain the first target image data, so as to implement the business to be executed based on the first target image data and the second target image data, where the second target image data is the second server pair. Obtained by processing the target image data associated with the serial number.
  • the first server can process the target image data associated with the serial number to obtain the first target image data.
  • the second server can also process the target image data associated with the serial number. Perform processing to obtain the second target image data.
  • the processing process can be storage processing, distribution processing, deduplication processing, etc.
  • the processing procedures performed by the first server and the second server may be the same or different.
  • the first server can implement the service to be executed based on the first target image data and the second target image data.
  • the second server can also implement the service to be executed based on the first target image data and the second target image data.
  • the first target image data and the second target image data can also be sent to a third-party business platform, so that the third-party business platform can implement the business to be executed based on the first target image data and the second target image data.
  • the first server can first receive a data collection request containing a serial number, and then obtain the target image data corresponding to the business to be executed from the data source according to the data collection request, and associate the serial number with the target image data.
  • the server can also process the target image data associated with the serial number to obtain the first target image data.
  • the second server can process the target image data associated with the serial number to obtain the second target image data, and then based on the first target image data And the second target mirror data realizes the business to be executed, by associating the same sequence number with the same data, so that after receiving the duplicate data sent by the multi-channel server, which data can be effectively identified as duplicate data according to the sequence number, and Performing deduplication operations improves the accuracy of deduplication processing, thereby improving the effectiveness of data backup and ensuring the normal realization of business.
  • the target image data associated with the serial number is processed to obtain the first target image data, so as to realize the service to be executed according to the first target image data and the second target image data.
  • Can include:
  • the target image data associated with the serial number is stored and processed to obtain the first target image data.
  • a target image data and the second target image data are deduplicated, and the to-be-executed business is implemented based on the target image data after the deduplication process.
  • the same batch of target image data obtained by different servers has been associated with the same serial number, which can determine which target image data is duplicate data.
  • the third-party business platform can compare the first target image data and the second target image data based on the serial number associated with the first target image data and the serial number associated with the second target image data.
  • the target image data is deduplicated to obtain deduplicated target image data that does not contain duplicate image data, and the business to be executed is implemented based on the deduplicated target image data.
  • first target image data and the second target image data can also be deduplicated through the first server or the second server, and the business to be executed can be implemented based on the target image data after the deduplication process.
  • the third-party business platform can directly process the first target image data based on the serial number associated with the first target image data and the serial number associated with the second target image data.
  • the target image data and the second target image data are deduplicated, which improves the efficiency and accuracy of the deduplication process.
  • the target image data associated with the serial number is processed to obtain the first target image data, so as to realize the service to be executed according to the first target image data and the second target image data.
  • Can include:
  • the target image data associated with the serial number is processed to obtain the first target image data.
  • the bitmap corresponding to the serial number is set according to the first target image data and the target image data associated with the serial number.
  • the first target image data is sent to a third-party business platform, so that the third-party business platform implements the business to be executed according to the first target image data and the second target image data, wherein:
  • the second target image data is sent by the second server to the third-party business platform.
  • the third-party business platform receives After duplicate data, deduplication operation needs to be performed first, and then the relevant pending business can be implemented. However, each serial number can only be associated with one data. If there are multiple data, different serial numbers need to be assigned. Otherwise, the deduplication operation of duplicate data cannot be achieved, and the data processing process is cumbersome and increases the processing time.
  • FIG. 4 is an application schematic diagram of the data processing process in another prior art of the present application.
  • edge data acquisition systems which can also correspond to two servers
  • edge data acquisition system A i.e. the first One server
  • edge data collection system B i.e. the second server
  • edge data collection system A is running, edge data collection system B remains on standby, and then switches to edge data collection system B after a problem occurs in edge data collection system A.
  • this method relies on dual-machine health monitoring.
  • Dual-machine switching is performed when monitoring abnormalities occur. If some problems cannot be detected, the backup function cannot be performed. For example: one machine loses packets but is not detected. If the two machines are not switched at this time, the packet loss exception will still occur, resulting in the inability to fully utilize the resources of the two machines. One machine is running, and the resources of the other machine are basically in a silent state. , and the business platform needs to sense the status of the two machines when connecting the two machines. When an exception occurs, the two machines are switched and the business platform is also switched. At this time, the failover takes time and hot backup cannot be performed in real time. If one mirror data source is only connected to one edge data acquisition channel, and when a problem occurs, another mirror source is connected, there will also be a problem of long failover time.
  • two servers alternately process the data of the same frame synchronization signal to avoid concurrent processing of the same target image data.
  • you can determine whether there is any loss of mirrored data before and after. or other anomalies, and repair the anomalies (for example, when there is data loss, redistribute or store the lost data. Or, when there is no data loss, only distribute a complete target image data to the third-party business platform to achieve data removal. (re-operation), which improves the timeliness of abnormal data recovery.
  • multiple target image data of the same batch can be assigned the same serial number. There is no need to be limited to assigning different serial numbers to each data, which simplifies the business implementation process.
  • the role of the alternate loader is to process the data of the same frame synchronization signal alternately by multiple servers to avoid concurrent processing of the same mirror data.
  • the components of the alternate loader include resource locks and bitmaps (also called bitmaps).
  • the resource lock is a token that is processed alternately by each server. Whoever gets this token has the authority to process the mirror data of a frame synchronization signal. The party that does not get it needs to wait for the other party to release the lock.
  • the target mirror data carries the frame synchronization signal for transmission. Whichever server obtains the mirror data first will obtain the mutex lock of the frame synchronization signal.
  • the server that obtains the resource lock starts to initiate operations such as data calculation, storage, and distribution. When another server receives the target image data, it will wait until the previous server is unlocked or released over time before the latter server can continue to process the target image data.
  • the second server can first determine whether the bitmap corresponding to the serial number already exists. If it does not exist, the second server will create a bitmap corresponding to the current serial number, and determine whether the bitmap corresponding to the serial number already exists.
  • the target image data sets the bitmap corresponding to the serial number. Subsequently, if the first server resumes operation, the target image data associated with the serial number can be processed according to the bitmap after the setting process to obtain the second Target image data. That is, the identities of the two servers are not fixed. It is not always the first server that obtains the resource lock.
  • the second server compares the information. The first server and the second server have equal opportunities to obtain the resource lock. The second server can also obtain the resource lock. The resource lock is obtained first, and the first server compares the information.
  • the two servers can process the target image data alternately, avoiding concurrent processing of the same image data, and can first determine whether there is any loss or other abnormality before and after the image data, and repair the abnormality. , which improves the accuracy of data processing and ensures the normal realization of business.
  • the target image data associated with the serial number is processed to obtain the first target image data, which may specifically include:
  • the processing process can be only storage processing, or only data distribution processing, or it can also include both storage processing and data distribution processing, and different processing processes each correspond to a single bitmap.
  • the second server the principle is similar and will not be limited in detail here.
  • the data distribution processing can be that the first server or the second server distributes data to the third-party business platform through a message queue.
  • users can also specify the data of concern and distribute it on demand.
  • setting the bitmap corresponding to the serial number according to the first target image data and the target image data associated with the serial number may specifically include:
  • Data that fails to be processed is determined according to the first target image data and the target image data associated with the serial number.
  • the bit corresponding to the failed data in the bitmap is set to zero, and the bit corresponding to the successfully processed data in the bitmap is set to one.
  • the data that failed to be processed may be determined first. For example, it may be a storage failure, or Distribute the data that failed, then set the bit corresponding to the data to zero, and set the bit corresponding to other data to 1.
  • different sequence numbers can correspond to different bitmaps. After the serial number is associated with the target image data, a bitmap corresponding to the serial number can be generated, and the number of bits in the bitmap is the same as the number of data contained in the target image data.
  • release the resource lock permission so that the second server obtains the resource lock permission, and processes the target image data associated with the serial number according to the bitmap after the setting process to obtain the second target image data specifically can include:
  • the target image data of the serial number is obtained by the second server obtaining the target image data corresponding to the business to be executed from the data source according to the data collection request, and associating the serial number with the target image data, so
  • the target sub-image data is data obtained from the target image data associated with the sequence number and corresponds to when the bit bit in the bitmap is zero.
  • the first server after the first server completes processing the target image data, it can release the resource lock permission.
  • the second server obtains the resource lock permission, and then determines whether the bitmap of the target image data already exists. If it exists, compare it with the one it received. If there is a difference between the data and the set data in the bitmap, the different data will be reprocessed to ensure that the other server can remedy the situation when the previous server has an abnormality.
  • each bit in the bitmap corresponding to the data identity ID (which can be a serial number) is set.
  • the second server After the second server obtains the resource lock authority, it can first determine whether there is a bitmap corresponding to the serial number. If it exists, compare the data it received with the set data in the bitmap. If there is a difference, then correct the difference. The data is rewritten.
  • the method may also include:
  • the target sub-image data is set as the second target image data and sent to the third-party service platform.
  • the target sub-mirror data After obtaining the target sub-mirror data, you can first determine the number of data contained in the target sub-mirror data. If the number of data contained in the target sub-mirror data is zero, it indicates that there is no abnormal data when the first server processes the data, and you can generate If no exception is found, the resource lock permission is released, thereby reminding the user that no data processing exception has occurred. This can alert managers in real time and improve the timeliness of data processing.
  • the target sub-mirror data can be set as the second target mirror data and sent to the third-party business platform, that is, Abnormal data is re-sent to the third-party business platform to ensure data integrity.
  • this application not only improves the timeliness of data processing, but also ensures the integrity of the data.
  • releasing the resource lock permission may include:
  • the resource lock permission is released.
  • releasing the resource lock permission may include two situations, one is after the first server completes the setting process, and the other is after the first server's setting process exceeds the preset time length threshold.
  • the preset duration threshold can be customized and set according to the actual application scenario.
  • Figure 5 is an application schematic diagram of the data processing method provided by the embodiment of the present application.
  • edge data acquisition systems which can also correspond to two servers
  • the data processing process may include data storage and data distribution, and data storage corresponds to a one-bit bitmap, and data distribution corresponds to a one-bit bitmap.
  • two sets of edge data acquisition systems collect a pair of target image data from the image data source.
  • the frame synchronization signal sequence is 8 (that is, the sequence number is 8), and the target image data contains 3 data They are A, B and C respectively.
  • the edge data acquisition system performs data storage, it first obtains the resource lock of the storage alternating loader, and the acquiring party initiates the writing operation of this frame synchronization mirror data.
  • the edge data acquisition system A on the left first obtains the resource lock permission, and when writing the target image data with the frame synchronization sequence number 8, it finds that the writing of data B is abnormal, then the bitmap with the sequence number 8 (also called For the bitmap), set the bits of data B to 0. Because A and C are set successfully, set the corresponding bits of A and C to 1.
  • the edge data acquisition system B on the right obtains the resource lock permission.
  • the edge data acquisition system A on the left first obtains the resource lock permission, and when distributing the target image data with serial number 8, it is found that the data distribution of A fails, or the response is abnormal, then the corresponding data of data A is set in the bitmap with serial number 8. The bit is 0 because B and C are successfully distributed and their corresponding bits are set to 1.
  • the edge data acquisition system B on the right obtained the resource lock permission. It determined that the bitmap with serial number 8 already existed, and then initiated a comparison. After comparison, it was found that the bitmap bit of A in the mirror data it received was abnormal. Then reinitiate the distribution of A data, and release the resource lock and bitmap with sequence number 8 after the operation is completed.
  • the pull mode of data distribution is similar to this use and will not be discussed again here.
  • the abnormal switching time is 0, and the frame synchronization alternate loading method is realized through the metronome and alternate loader, which solves the problem of switching between dual-machine cold backup or warm backup solutions in the traditional field.
  • the problem of long time is solved, the problem of data loss during abnormal switching is solved, and the integrity of data under dual machines is truly realized.
  • Figure 6 is a schematic structural diagram of the data processing device provided by the embodiment of this application. As shown in Figure 6, it may include:
  • the receiving module 601 is used to receive a data collection request, where the data collection request contains a serial number.
  • the receiving module 601 is also used to:
  • the control rhythm module generates a frame synchronization signal every preset time period, wherein the frame synchronization signal includes a sequence number.
  • Receive the frame synchronization signal and generate a data collection request including the sequence number according to the frame synchronization signal.
  • the processing module 602 is configured to obtain target image data corresponding to the business to be executed from the data source according to the data collection request, and associate the serial number with the target image data.
  • the processing module 602 is also used to process the target image data associated with the serial number to obtain the first target image data, so as to realize the business to be executed according to the first target image data and the second target image data. , wherein the second target image data is obtained by processing the target image data associated with the serial number by the second server.
  • processing module 602 is also used to:
  • the target image data associated with the serial number is stored and processed to obtain the first target image data.
  • a target image data and the second target image data are deduplicated, and the to-be-executed business is implemented based on the target image data after the deduplication process.
  • processing module 602 is also used to:
  • the target image data associated with the serial number is processed to obtain the first target image data.
  • the bitmap corresponding to the serial number is set according to the first target image data and the target image data associated with the serial number.
  • the first target image data is sent to a third-party business platform, so that the third-party business platform implements the business to be executed according to the first target image data and the second target image data, wherein:
  • the second target image data is sent by the second server to the third-party business platform.
  • processing module 602 is also used to:
  • processing module 602 is also used to:
  • Data that fails to be processed is determined according to the first target image data and the target image data associated with the serial number.
  • the bit corresponding to the failed data in the bitmap is set to zero, and the bit corresponding to the successfully processed data in the bitmap is set to one.
  • processing module 602 is also used to:
  • the target image data of the serial number is obtained by the second server obtaining the target image data corresponding to the business to be executed from the data source according to the data collection request, and associating the serial number with the target image data, so
  • the target sub-image data is data obtained from the target image data associated with the sequence number and corresponds to when the bit bit in the bitmap is zero.
  • processing module 602 is also used to:
  • the target sub-image data is set as the second target image data and sent to the third-party service platform.
  • processing module 602 is also used to:
  • the resource lock permission is released.
  • the device provided by the embodiment of the present application can implement the above-mentioned method as shown in the embodiment shown in Figure 3. Its implementation principles and technical effects are similar and will not be described again here.
  • FIG 7 is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present application.
  • a device 700 provided by this embodiment includes a processor 701 and a memory communicatively connected to the processor. Among them, the processor 701 and the memory 702 are connected through a bus 703.
  • the processor 701 executes the computer execution instructions stored in the memory 702, so that the processor 701 executes the method in the above method embodiment.
  • the processor may be a central processing unit (English: Central Processing Unit, referred to as: CPU), or other general-purpose processors, digital signal processors (English: Digital Signal Processor) Signal Processor (DSP for short), Application Specific Integrated Circuit (English: Application Specific Integrated Circuit (ASIC for short)), etc.
  • a general purpose processor may be a microprocessor or the processor may be any conventional processor, etc.
  • the memory may include high-speed RAM memory and may also include non-volatile storage NVM, such as at least one disk memory.
  • the bus can be an Industry Standard Architecture (Industry Standard Architecture, ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc.
  • ISA Industry Standard Architecture
  • PCI Peripheral Component Interconnect
  • EISA Extended Industry Standard Architecture
  • the bus can be divided into address bus, data bus, control bus, etc.
  • the bus in the drawings of this application is not limited to only one bus or one type of bus.
  • Embodiments of the present application also provide a computer-readable storage medium.
  • Computer-executable instructions are stored in the computer-readable storage medium.
  • the processor executes the computer-executable instructions, the data processing method of the above method embodiment is implemented.
  • An embodiment of the present application also provides a computer program product, which includes a computer program.
  • the computer program When the computer program is executed by a processor, it implements the data processing method as described above.
  • the above-mentioned computer-readable storage medium can be implemented by any type of volatile or non-volatile storage device or their combination, such as static random access memory (SRAM), electrically erasable memory.
  • SRAM static random access memory
  • EEPROM Programmable read-only memory
  • EPROM erasable programmable read-only memory
  • PROM programmable read-only memory
  • ROM read-only memory
  • Readable storage media can be any available media that can be accessed by a general purpose or special purpose computer.
  • An exemplary readable storage medium is coupled to the processor such that the processor can read information from the readable storage medium and write information to the readable storage medium.
  • the readable storage medium may also be an integral part of the processor.
  • the processor and readable storage medium may be located in Application Specific Integrated Circuits (ASICs for short).
  • ASICs Application Specific Integrated Circuits
  • the processor and the readable storage medium may also exist as discrete components in the device.
  • the aforementioned program can be stored in a computer-readable storage medium.
  • the steps including the above-mentioned method embodiments are executed; and the aforementioned storage media include: ROM, RAM, magnetic disks, optical disks and other media that can store program codes.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

本申请提供了一种数据处理方法及装置，方法应用于第一服务器，包括：接收数据采集请求，其中，数据采集请求中包含序列号（S301），根据数据采集请求从数据源中获取与待执行业务对应的目标镜像数据，并关联序列号与目标镜像数据（S302），对关联序列号的目标镜像数据进行处理，得到第一目标镜像数据，以根据第一目标镜像数据以及第二目标镜像数据实现待执行业务，其中，第二目标镜像数据为第二服务器对关联序列号的目标镜像数据进行处理得到的（S303）。本申请提高了去重处理的准确性，进而提高了数据备份的有效性，保证了业务的正常实现。

Description

数据处理方法及装置
本申请要求于2022年03月29日提交中国专利局、申请号为202210326055.6、申请名称为“数据处理方法及装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请实施例涉及数据处理技术领域，尤其涉及一种数据处理方法及装置。
背景技术
随着网络技术的发展，在数字化背景下，开启了工厂数字化的进程。
现有技术中，工厂数字化进程中会涉及到众多业务的实现，为了保证业务的正常实现，可以对采集到的现场数据进行备份，即将数据源连接到至少两个数采系统对应的服务器中。每个数采系统对应的服务器均可以获取现场数据，并将获取到的现场数据均发送至业务方进行处理。业务方在对接收到的现场数据进行处理时，由于包含重复的现场数据，需要先对接收到的现场数据进行去重处理，再对去重处理之后的现场数据进行后续处理，进而实现相关业务。
然而，在对接收到的现场数据进行去重处理时，由于接收到的现场数据可能有多个，无法有效识别哪些数据是重复数据，降低了去重处理的准确性，进而降低了数据备份的有效性，影响了业务的正常实现。
发明内容
本申请实施例提供一种数据处理方法及装置，以提高去重处理的准确性，保证业务的正常实现。
第一方面，本申请实施例提供一种数据处理方法，应用于第一服务器，所述方法包括：
接收数据采集请求，其中，所述数据采集请求中包含序列号；
根据所述数据采集请求从数据源中获取与待执行业务对应的目标镜像数据，并关联所述序列号与所述目标镜像数据；
对关联所述序列号的目标镜像数据进行处理，得到第一目标镜像数据，以根据所述第一目标镜像数据以及第二目标镜像数据实现所述待执行业务，其中，所述第二目标镜像数据为所述第二服务器对关联所述序列号的目标镜像数据进行处理得到的。
可选的，所述对关联所述序列号的目标镜像数据进行处理，得到第一目标镜像数据，以根据所述第一目标镜像数据以及第二目标镜像数据实现所述待执行业务，包括：
对关联所述序列号的目标镜像数据进行存储处理，得到第一目标镜像数据；
将所述第一目标镜像数据分发至第三方业务平台，以使所述第三方业务平台根据所述第一目标镜像数据关联的序列号和第二目标镜像数据关联的序列号，对所述第一目标镜像数据以及所述第二目标镜像数据进行去重处理，并根据去重处理之后的目标镜像数据实现所述待执行业务。
可选的，所述对关联所述序列号的目标镜像数据进行处理，得到第一目标镜像数据，以根据所述第一目标镜像数据以及第二目标镜像数据实现所述待执行业务，包括：
在获得资源锁权限之后，对关联所述序列号的目标镜像数据进行处理，得到第一目标镜像数据；
根据所述第一目标镜像数据以及关联所述序列号的目标镜像数据对所述序列号对应的比特位图进行置位处理；
释放所述资源锁权限，以使所述第二服务器获得资源锁权限，并根据置位处理之后的比特位图对关联所述序列号的目标镜像数据进行处理，得到第二目标镜像数据；
将所述第一目标镜像数据发送至第三方业务平台，以使所述第三方业务平台根据所述第一目标镜像数据和所述第二目标镜像数据实现所述待执行业务，其中，所述第二目标镜像数据为所述第二服务器发送至所述第三方业务平台的。
可选的，所述对关联所述序列号的目标镜像数据进行处理，得到第一目标镜像数据，包括：
对关联所述序列号的目标镜像数据进行存储处理和/或数据分发处理，得到第一目标镜像数据，其中，所述存储处理与所述分发处理对应不同的比特位图。
可选的，所述根据所述第一目标镜像数据以及关联所述序列号的目标镜像数据对所述序列号对应的比特位图进行置位处理，包括：
根据所述第一目标镜像数据以及关联所述序列号的目标镜像数据确定处理失败的数据；
将所述比特位图中处理失败的数据对应的比特位设置为零，并将所述比特位图中处理成功的数据对应的比特位设置为一。
可选的，所述释放所述资源锁权限，以使所述第二服务器获得资源锁权限，并根据置位处理之后的比特位图对关联所述序列号的目标镜像数据进行处理，得到第二目标镜像数据，包括：
释放所述资源锁权限，以使所述第二服务器获取资源锁权限，并从关联所述序列号的目标镜像数据中获取目标子镜像数据，得到第二目标镜像数据，其中，所述关联所述序列号的目标镜像数据为所述第二服务器根据所述数据采集请求从数据源中获取与待执行业务对应的目标镜像数据，并关联所述序列号与所述目标镜像数据得到的，所述目标子镜像 数据为从关联所述序列号的目标镜像数据中获取的与比特位图中比特位为零时对应的数据。
可选的，所述方法还包括：
确定所述目标子镜像数据的数量；
若所述目标子镜像数据的数量为零，则生成未发现异常提示，并释放所述资源锁权限；
若所述目标子镜像数据的数量不为零，则将所述目标子镜像数据设置为第二目标镜像数据发送至所述第三方业务平台。
可选的，所述释放所述资源锁权限，包括：
在置位处理完成之后，或处理时长超过预设时长阈值之后，释放所述资源锁权限。
可选的，所述接收数据采集请求，包括：
控制节拍模块每隔预设时长生成一帧同步信号，其中，所述帧同步信号中包含序列号；
接收所述帧同步信号，并根据所述帧同步信号生成包含所述序列号的数据采集请求。
第二方面，本申请实施例提供一种数据处理装置，包括：
接收模块，用于接收数据采集请求，其中，所述数据采集请求中包含序列号；
处理模块，用于根据所述数据采集请求从数据源中获取与待执行业务对应的目标镜像数据，并关联所述序列号与所述目标镜像数据；
所述处理模块，还用于对关联所述序列号的目标镜像数据进行处理，得到第一目标镜像数据，以根据所述第一目标镜像数据以及第二目标镜像数据实现所述待执行业务，其中，所述第二目标镜像数据为第二服务器对关联所述序列号的目标镜像数据进行处理得到的。
本申请实施例提供了一种数据处理方法及装置，采用上述方案后，第一服务器可以先接收包含序列号的数据采集请求，然后根据数据采集请求从数据源中获取与待执行业务对应的目标镜像数据，并关联序列号与目标镜像数据，第一服务器还可以对关联序列号的目标镜像数据进行处理，得到第一目标镜像数据，第二服务器可以对关联序列号的目标镜像数据进行处理，得到第二目标镜像数据，再根据第一目标镜像数据以及第二目标镜像数据实现待执行业务，通过为同一数据关联相同的序列号的方式，使得在接收到多路服务器发送的重复数据之后，可以根据序列号有效的识别哪些数据为重复数据，提高了去重处理的准确性，进而提高了数据备份的有效性，保证了业务的正常实现。
附图说明
为了更清楚地说明本申请实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。
图1为本申请现有技术中数据处理过程的应用示意图；
图2为本申请实施例提供的数据处理方法的应用系统的架构示意图；
图3为本申请实施例提供的数据处理方法的流程示意图；
图4为本申请另一现有技术中数据处理过程的应用示意图；
图5为本申请实施例提供的数据处理方法的应用示意图；
图6为本申请实施例提供的数据处理装置的结构示意图；
图7为本申请实施例提供的电子设备的硬件结构示意图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
本申请的说明书和权利要求书及上述附图中的术语“第一”、“第二”、“第三”“第四”等(如果存在)是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换，以便这里描述的本申请的实施例还能够包括除了图示或描述的那些实例以外的其他顺序实例。此外，术语“包括”和“具有”以及他们的任何变形，意图在于覆盖不排他的包含，例如，包含了一系列步骤或单元的过程、方法、系统、产品或设备不必限于清楚地列出的那些步骤或单元，而是可包括没有清楚地列出的或对于这些过程、方法、产品或设备固有的其它步骤或单元。
现有技术中，工厂数字化进程中会涉及到众多业务的实现，在实现相关业务时，一般是通过处理器采集现场数据，然后将采集的现场数据传输到数采系统对应的服务器中进行处理，进而实现相关的业务。工业场景下一般是通过PLC(Programmable Logic Controller，可编程逻辑控制器)采集现场数据，然后传输到数采系统中，数采系统负责数据的计算、存储，分发等工作。数采系统北向是各种业务平台，一般通过消息队列的pull或push模式接收数据，同时也可以进行数据的查询。然后，在保证数据从采集、计算、存储、分发以及查询等一系列链路上的高可用性的过程中，可能出现以下问题：如果南向数据源部分出现异常，则会出现丢数据的问题。如果数据处理系统内部在传输数据时发生异常即会出现数据的丢失。北向的业务系统在获取数据时出现丢数据的情况。如果数据处理系统在存储数据时发生异常，也会造成北向业务系统使用数据的丢失。如果北向业务系统出现异常，在异常恢复后会希望从异常点开始继续消费数据，否则会出现异常。因此，为了解决前述问题，且兼顾工业领域成本问题，双机备份的方式在成本和可用性的平衡上为较合适的。
传统的双机备份的高可用方案可以分为数据和程序两部分：程序高可用是将程序无状态化，状态数据存入数据库，程序通过集群方式实现高可用。双机情况下程序也是类似镜像方式存在，即一台机器的程序和另一台机器的程序是相同的，一台机器异常后，另一个台机器可以完全替代继续工作。数据高可用一般通过数据库双机互备，共享存储来解决。 一个数据库异常后，另一个数据库也有全量数据。如果将镜像数据源分别链接到双机边缘数采，同时运行，则在数据处理过程中，双机边缘数采系统会同时存储两份数据，分发两路数据。可能会造成存储资源浪费的问题，即相当于一个数据存储了两份。而且双机还要同步备份，这样对网络IO和磁盘IO都是压力。业务平台如果仅对接一路边缘数采系统，在出现异常时故障切换会有切换时间，不能实时热备。业务平台如果对接两路边缘数采系统，则会出现两路重复数据，需要业务方自己去重。对于两路数据，如果进行去重，由于接收到的现场数据可能有多个，对于识别哪些数据是同一个点位的镜像数据仍然存在问题。
示例性的，图1为本申请现有技术中数据处理过程的应用示意图，如图1所示，可以通过可编程逻辑控制器从镜像数据源中采集两路镜像数据，有两个边缘数采系统(也可以对应两个服务器)，分别为边缘数采系统A(即第一服务器)和边缘数采系统B(即第二服务器)，1、2、3代表数据源产生的先后顺序，1',2',3'为镜像的数据，实际数据中是没有此顺序标号来标识数据之间的顺序关系的，在边缘数采系统A和边缘数采系统B获取数据以及对数据进行处理和传输的过程中，各数据之间的顺序可能会发生变更，若业务平台接收到的数据为乱序的，无法有效识别哪些数据是重复数据，降低了去重处理的准确性，进而降低了数据备份的有效性，影响了业务的正常实现。
基于上述技术问题，本申请通过为同一数据关联相同的序列号的方式，使得在接收到多路服务器发送的重复数据之后，可以根据序列号有效的识别哪些数据为重复数据，并进行去重操作，达到了既提高了去重处理的准确性，又提高了数据备份的有效性，进而保证了业务的正常实现的技术效果。
图2为本申请实施例提供的数据处理方法的应用系统的架构示意图，如图2所示，在该应用系统中，可以包括第一服务器101、第二服务器102、可编程逻辑控制器103、镜像数据源104。第一服务器101以及第二服务器102可以为边缘数采系统对应的服务器，即可以实现边缘数采系统负责的数据计算、存储、分发等工作。可编程逻辑控制器103是工业现场对接工业设备的装置，其南向通过工业协议采集数据，北向对接边缘数采系统。
第一服务器101和第二服务器102可以分别接收包含序列号的数据采集请求，然后通过可编程逻辑控制器103从镜像数据源104中采集与待执行业务对应的目标镜像数据，并关联序列号与目标镜像数据。示例性的，若序列号为8，则将第一服务器101和第二服务器102本次获取的目标镜像数据的标识均设置为8。
此外，应用系统还可以包含第三方业务平台105，在第一服务器101和第二服务器102均分别对目标镜像数据进行处理之后，可以得到第一目标镜像数据和第二目标镜像数据，然后可以将第一目标镜像数据和第二目标镜像数据发送至第三方业务平台105，以使第三方业务平台105根据第一目标镜像数据和第二目标镜像数据实现待执行业务。其中，第三方业务平台105根据第一目标镜像数据和第二目标镜像数据实现待执行业务可以采用现有的方式实现，在此不再详细进行限定。
另外，应用系统还可以包含节拍器(或称为节拍模型)106，为定时同步模块，其通过周期发送帧同步信号标记镜像数据，节拍器的作用为对镜像数据源的数据建立映射关系，即在同一节拍下产生的数据其节拍序号相同，可以周期性的发送帧同步信号，帧同步信号携带一个递增序列号，两路数采系统在收到帧同步信号之后可以生成数据采集请求，并向数据源进行数据的采集，采集的数据使用一个序列号进行后续的传输，通过节拍器可以标识镜像数据源产生的两个相互关联的镜像数据，即具有相同序列号的数据是可以认为是镜像数据。
下面以具体地实施例对本申请的技术方案进行详细说明。下面这几个具体的实施例可以相互结合，对于相同或相似的概念或过程可能在某些实施例不再赘述。
图3为本申请实施例提供的数据处理方法的流程示意图，本实施例的方法可以由第一服务器101执行。如图3所示，本实施例的方法，可以包括：
S301：接收数据采集请求，其中，数据采集请求中包含序列号。
在本实施例中，可以接收节拍模块周期性发送的数据采集请求，然后根据数据采集请求采集对应的目标镜像数据，并根据目标镜像数据实现相关业务。
进一步的，所述接收数据采集请求，具体可以包括：
控制节拍模块每隔预设时长生成一帧同步信号，其中，所述帧同步信号中包含序列号。
接收所述帧同步信号，并根据所述帧同步信号生成包含所述序列号的数据采集请求。
具体的，节拍模块可以每隔预设时长生成一帧同步信号，第一服务器的采集驱动在接收到帧同步信号之后，可以生成包含序列号的数据采集请求。其中，序列号可以为阿拉伯数字，还可以为大写字母，小写字母、特殊符号中的任意一种或多种，仅需保证每次生成的序列号不同即可。
S302：根据数据采集请求从数据源中获取与待执行业务对应的目标镜像数据，并关联序列号与目标镜像数据。
在本实施例中，第一服务器在接收到数据采集请求之后，可以根据数据采集请求从数据源中获取与待执行业务对应的目标镜像数据，其中，数据源可以为镜像数据源。获取到目标镜像数据之后，可以将序列号与目标镜像数据进行关联。同理，第二服务器也可以接收到数据采集请求，然后根据数据采集请求从数据源中获取与待执行业务对应的目标镜像数据，并在获取到目标镜像数据之后，将序列号与目标镜像数据进行关联。通过序列号，可以将第一服务器获取到的目标镜像数据与第二服务器获取到的目标镜像数据建立映射关系，即在同一节拍下产生的数据其序列号相同。此外，第二服务器可以有一个或多个。在本实施例中，第二服务器为一个。
S303：对关联序列号的目标镜像数据进行处理，得到第一目标镜像数据，以根据第一目标镜像数据以及第二目标镜像数据实现待执行业务，其中，第二目标镜像数据为第二服务器对关联序列号的目标镜像数据进行处理得到的。
在本实施例中，在目标镜像数据关联序列号之后，第一服务器可以对关联序列号的目标镜像数据进行处理，得到第一目标镜像数据，第二服务器也可以对关联序列号的目标镜像数据进行处理，得到第二目标镜像数据。其中，处理过程可以为存储处理，分发处理、去重处理等。第一服务器与第二服务器进行的处理过程可以相同，也可以不同。
处理完成之后，第一服务器可以根据第一目标镜像数据以及第二目标镜像数据实现待执行业务。此外，第二服务器也可以根据第一目标镜像数据以及第二目标镜像数据实现待执行业务。或者，还可以将第一目标镜像数据以及第二目标镜像数据发送至第三方业务平台，使得第三方业务平台根据第一目标镜像数据以及第二目标镜像数据实现待执行业务。
采用上述方案后，第一服务器可以先接收包含序列号的数据采集请求，然后根据数据采集请求从数据源中获取与待执行业务对应的目标镜像数据，并关联序列号与目标镜像数据，第一服务器还可以对关联序列号的目标镜像数据进行处理，得到第一目标镜像数据，第二服务器可以对关联序列号的目标镜像数据进行处理，得到第二目标镜像数据，再根据第一目标镜像数据以及第二目标镜像数据实现待执行业务，通过为同一数据关联相同的序列号的方式，使得在接收到多路服务器发送的重复数据之后，可以根据序列号有效的识别哪些数据为重复数据，并进行去重操作，提高了去重处理的准确性，进而提高了数据备份的有效性，保证了业务的正常实现。
基于图3的方法，本说明书实施例还提供了该方法的一些具体实施方案，下面进行说明。
在另一实施例中，对关联所述序列号的目标镜像数据进行处理，得到第一目标镜像数据，以根据所述第一目标镜像数据以及第二目标镜像数据实现所述待执行业务，具体可以包括：
对关联所述序列号的目标镜像数据进行存储处理，得到第一目标镜像数据。
将所述第一目标镜像数据分发至第三方业务平台，以使所述第三方业务平台根据所述第一目标镜像数据关联的序列号和第二目标镜像数据关联的序列号，对所述第一目标镜像数据以及所述第二目标镜像数据进行去重处理，并根据去重处理之后的目标镜像数据实现所述待执行业务。
具体的，不同服务器获取的同一批次的目标镜像数据均已经与同一序列号进行关联，即可以确定哪些目标镜像数据为重复数据。在不同服务器将目标镜像数据分发至第三方业务平台之后，第三方业务平台可以根据第一目标镜像数据关联的序列号和第二目标镜像数据关联的序列号，对第一目标镜像数据以及第二目标镜像数据进行去重处理，得到不包含重复镜像数据的去重处理之后的目标镜像数据，并根据去重处理之后的目标镜像数据实现待执行业务。
此外，还可以通过第一服务器或者第二服务器对第一目标镜像数据以及第二目标镜像数据进行去重处理，并根据去重处理之后的目标镜像数据实现待执行业务。
综上，通过将不同服务器获取的目标镜像数据与同一序列号进行关联，使得第三方业务平台可以根据第一目标镜像数据关联的序列号和第二目标镜像数据关联的序列号，直接对第一目标镜像数据以及第二目标镜像数据进行去重处理，提高了去重处理的效率与准确性。
在另一实施例中，对关联所述序列号的目标镜像数据进行处理，得到第一目标镜像数据，以根据所述第一目标镜像数据以及第二目标镜像数据实现所述待执行业务，具体可以包括：
在获得资源锁权限之后，对关联所述序列号的目标镜像数据进行处理，得到第一目标镜像数据。
根据所述第一目标镜像数据以及关联所述序列号的目标镜像数据对所述序列号对应的比特位图进行置位处理。
释放所述资源锁权限，以使所述第二服务器获得资源锁权限，并根据置位处理之后的比特位图对关联所述序列号的目标镜像数据进行处理，得到第二目标镜像数据。
将所述第一目标镜像数据发送至第三方业务平台，以使所述第三方业务平台根据所述第一目标镜像数据和所述第二目标镜像数据实现所述待执行业务，其中，所述第二目标镜像数据为所述第二服务器发送至所述第三方业务平台的。
在本实施例中，在将不同服务器获取的同一批次的目标镜像数据与同一序列号进行关联之后，若服务器都将关联之后的目标镜像数据发送至第三方业务平台，第三方业务平台接收到重复数据之后需要先进行去重操作，然后再实现相关的待执行业务。然而，每个序列号仅能关联一个数据，若有多个数据，需要分配不同的序列号，否则也无法实现重复数据的去重操作，数据处理过程繁琐，且增加了处理时长。
现有技术中，还可以将镜像数据源对接两路边缘数采系统(即两路服务器)，但是一路边缘数采系统运行，另一路边缘数采系统保持待机，等到一路出现问题后再进行切换。图4为本申请另一现有技术中数据处理过程的应用示意图，如图4所示，有两个边缘数采系统(也可以对应两个服务器)，分别为边缘数采系统A(即第一服务器)和边缘数采系统B(即第二服务器)，边缘数采系统A运行，边缘数采系统B保持待机，等到边缘数采系统A出现问题后，再切换至边缘数采系统B。然而，该方式需要依赖双机健康监测，监测异常时进行双机切换，如果有些问题监测不到则导致无法起到备份的作用。例如：一台机器丢包但是没有被检测到，此时如果不切换双机，丢包异常依然会发生，导致无法充分利用双机资源，一台机器运行，一台机器资源在基本处于静默状态，且业务平台在对接双机时需要感知双机状态，当异常发生时，双机切换，业务平台也要切换，此时故障切换需要时间，不能实时进行热备份。如果将镜像数据源的一个只对接一路边缘数采，当出现问题时在连接另一路镜像源，同样也会有故障切换时间长的问题。
本申请通过设置一交替加载器，针对同一个帧同步信号的数据，两个服务器交替进行处理，避免并发处理同一个目标镜像数据。交替处理时可以判断镜像数据前后是否有丢失 或其它异常，并针对异常进行修复(如，有数据丢失时，重新分发或存储丢失的数据。或者，没有数据丢失时，仅分发一份完整目标镜像数据至第三方业务平台，实现数据的去重操作)，提高了异常数据补救的及时性。且同一批次的多个目标镜像数据可以分配相同的序列号，无需再局限于为每个数据分配不同的序列号，简化了业务实现的处理过程。
其中，交替加载器的作用为针对同一个帧同步信号的数据，多个服务器交替进行处理，避免并发处理同一个镜像数据。交替处理时可以先判断镜像数据前后是否有丢失或其它异常，针对异常进行修复。交替加载器的组成包括资源锁与比特位图(也可称为bitmap)。资源锁为各服务器交替处理的令牌，谁得到此令牌即具有对一个帧同步信号的镜像数据的处理权限，没有得到的一方需要等待另一方释放此锁。目标镜像数据携带帧同步信号进行传输，哪个服务器先得到此镜像数据后，获取此帧同步信号的互斥锁，得到资源锁的服务器开始发起数据的计算、存储以及分发等操作。另外的服务器后收到目标镜像数据，则进行等待，直到前一个服务器解锁，或者超时释放之后，后一台服务器才可以继续对目标镜像数据进行处理。
此外，第二服务器在获得资源锁权限之后，可以先判断是否已经存在序列号对应的比特位图，若不存在，则第二服务器会创建当前序列号对应的比特位图，并根据关联序列号的目标镜像数据对序列号对应的比特位图进行置位处理，后续若第一服务器恢复运行之后，可以根据置位处理之后的比特位图对关联序列号的目标镜像数据进行处理，得到第二目标镜像数据。即两个服务器身份不是固定的，并不是一直都是第一服务器获取资源锁，第二服务器比对信息，第一服务器和第二服务器获取资源锁的机会是均等的，也可以由第二服务器先获取资源锁，由第一服务器进行信息的比对。
综上，通过交替加载器的设置，使得两个服务器可以交替对目标镜像数据进行处理，避免并发处理同一个镜像数据，且可以先判断镜像数据前后是否有丢失或其它异常，并针对异常进行修复，提高了数据处理的准确性，进而保证了业务的正常实现。
进一步的，对关联所述序列号的目标镜像数据进行处理，得到第一目标镜像数据，具体可以包括：
对关联所述序列号的目标镜像数据进行存储处理和/或数据分发处理，得到第一目标镜像数据，其中，所述存储处理与所述分发处理对应不同的比特位图。
具体的，处理过程可以仅为存储处理，或者仅为数据分发处理，还可既包含存储处理又包含数据分发处理，且不同的处理过程均单独对应一个比特位图。对于第二服务器，原理类似，在此不再详细进行限定。
此外，数据分发处理可以为第一服务器或第二服务器通过消息队列方式向第三方业务平台分发数据。其中，用户还可以指定关注的数据，按需进行分发。
进一步的，根据所述第一目标镜像数据以及关联所述序列号的目标镜像数据对所述序列号对应的比特位图进行置位处理，具体可以包括：
根据所述第一目标镜像数据以及关联所述序列号的目标镜像数据确定处理失败的数据。
将所述比特位图中处理失败的数据对应的比特位设置为零，并将所述比特位图中处理成功的数据对应的比特位设置为一。
具体的，在根据第一目标镜像数据以及关联序列号的目标镜像数据对序列号对应的比特位图进行置位处理时，可以先确定处理失败的数据，示例性的，可以为存储失败，或者分发失败的数据，然后将该数据对应的比特位设置为零，其他数据对应的比特位设置为1。
其中，不同的序列号可以对应不同的比特位图。在序列号与目标镜像数据关联之后，可以生成该序列号对应的比特位图，且该比特位图中比特位的个数与目标镜像数据中包含的数据个数相同。
另外，释放所述资源锁权限，以使所述第二服务器获得资源锁权限，并根据置位处理之后的比特位图对关联所述序列号的目标镜像数据进行处理，得到第二目标镜像数据，具体可以包括：
释放所述资源锁权限，以使所述第二服务器获取资源锁权限，并从关联所述序列号的目标镜像数据中获取目标子镜像数据，得到第二目标镜像数据，其中，所述关联所述序列号的目标镜像数据为所述第二服务器根据所述数据采集请求从数据源中获取与待执行业务对应的目标镜像数据，并关联所述序列号与所述目标镜像数据得到的，所述目标子镜像数据为从关联所述序列号的目标镜像数据中获取的与比特位图中比特位为零时对应的数据。
具体的，第一服务器在处理完成目标镜像数据之后，可以释放资源锁权限，第二服务器获得资源锁权限，然后判断目标镜像数据的比特位图是否已经存在，若存在，则对比其接收到的数据和比特位图中的置位数据，如果有差异，则针对差异的数据进行再处理，以此来保证在前一服务器出现异常时另一服务器可以进行补救。示例性的，以存储为例，当第一服务器调用数据库接口完成写入操作后，则将此数据身份ID(可以为序列号)对应的比特位图中的各比特位进行置位。第二服务器获得资源锁权限之后，可以先判断是否存在该序列号对应的比特位图，若存在，则对比其接收到的数据和比特位图中的置位数据，如果有差异，则针对差异的数据重新进行写入操作。
综上，通过资源锁权限与比特位图的结合使用，可以实现镜像数据流的差异性对比，并针对差异数据进行实时补救，既实现了两个服务器实时热备份，一个服务器断开另一个服务器可以无缝切换的效果，且在异常切换时可以实现数据基本不丢失，还不依赖保活和健康检测，避免了现有技术中需要监测异常，并在监测到异常时进行切换，导致切换时间内无法提供服务或切换时间内会有数据丢失的情况。
此外，所述方法还可以包括：
确定所述目标子镜像数据的数量。
若所述目标子镜像数据的数量为零，则生成未发现异常提示，并释放所述资源锁权限。
若所述目标子镜像数据的数量不为零，则将所述目标子镜像数据设置为第二目标镜像数据发送至所述第三方业务平台。
具体的，在得到目标子镜像数据之后，可以先确定目标子镜像数据中包含的数据数量，若包含的数据数量为零，则表明第一服务器在处理数据时，没有出现异常数据，则可以生成未发现异常提示，并释放所述资源锁权限，进而提醒用户未发生数据处理异常，可以实时提醒管理人员，提高了处理数据的及时性。
若目标子镜像数据中包含的数据数量不为零，则表明第一服务器在处理数据时出现了异常数据，则可以将目标子镜像数据设置为第二目标镜像数据发送至第三方业务平台，即将出现异常的数据重新发送至第三方业务平台，保证了数据的完整性。
综上，本申请既提高了处理数据的及时性，又保证了数据的完整性。
另外，释放所述资源锁权限，可以包括：
在置位处理完成之后，或处理时长超过预设时长阈值之后，释放所述资源锁权限。
具体的，释放资源锁权限可以包含两种情况，一种为第一服务器完成置位处理之后，另一种为第一服务器的置位处理过程超过了预设时长阈值之后。其中，预设时长阈值可以根据实际应用场景自定义进行设置。
示例性的，图5为本申请实施例提供的数据处理方法的应用示意图，如图5所示，有两个边缘数采系统(也可以对应两个服务器)，分别为边缘数采系统A(即第一服务器)和边缘数采系统B(即第二服务器)，且两个边缘数采系统均保持运行状态。数据处理过程可以包含数据存储以及数据分发，且数据存储对应一比特位图，数据分发对应一比特位图。通过节拍器发送的帧同步信号，两套边缘数采系统从镜像数据源采集一对目标镜像数据，如下图帧同步信号序列为8(即序列号为8)，目标镜像数据中包含3个数据分别为A，B，C。边缘数采系统在做数据存储时，先获取存储交替加载器的资源锁，获取的一方发起此帧同步镜像数据的写入操作。例如左边的边缘数采系统A先获取资源锁权限，在写入帧同步序列号为8的目标镜像数据时发现数据B的写入异常，则在序列号为8的比特位图(也可称为bitmap)中将数据B的比特位设置为0，因为A和C设置成功则设置A和C对应的比特位为1。右边的边缘数采系统B后获取资源锁权限，在确定序列号为8的bitmap已经存在之后，则发起比对，经过比较发现其接收到的镜像数据中的B在bitmap中异常，则其重新发起B数据的写入操作，操作完成后释放序列号为8的资源锁和bitmap。另外，边缘数采系统在做数据分发时，此处以push类型的消息队列为例，先获取分发交替加载器的资源锁，获取的一方发起帧同步镜像数据的分发。例如，左边的边缘数采系统A先获取资源锁权限，在分发序列号为8的目标镜像数据时发现A数据分发失败，或者响应异常，则在序列号为8的bitmap中设置数据A对应的比特位为0，因为B和C分发成功则将其对应的比特位设置为1。右边的边缘数采系统B后获取资源锁权限，其确定序列号为8的bitmap已经存在，则发起比对，经过比较发现其接收到的镜像数据中A的bitmap位异常， 则重新发起A数据的分发，操作完成后释放序列号为8的资源锁和bitmap。数据分发的pull模式与此使用类似，在此不再重复进行论述。
综上，通过双机实时热备份的方式，实现异常切换时间为0，通过节拍器以及交替加载器等实现了帧同步交替加载方式，解决了传统领域中双机冷备或者温备方案中切换时间较长的问题，解决了异常切换期间丢数据问题，真正实现了双机下数据的完整性。
基于同样的思路，本说明书实施例还提供了上述方法对应的装置，图6为本申请实施例提供的数据处理装置的结构示意图，如图6所示，可以包括：
接收模块601，用于接收数据采集请求，其中，所述数据采集请求中包含序列号。
在本实施例中，所述接收模块601，还用于：
控制节拍模块每隔预设时长生成一帧同步信号，其中，所述帧同步信号中包含序列号。
接收所述帧同步信号，并根据所述帧同步信号生成包含所述序列号的数据采集请求。
处理模块602，用于根据所述数据采集请求从数据源中获取与待执行业务对应的目标镜像数据，并关联所述序列号与所述目标镜像数据。
所述处理模块602，还用于对关联所述序列号的目标镜像数据进行处理，得到第一目标镜像数据，以根据所述第一目标镜像数据以及第二目标镜像数据实现所述待执行业务，其中，所述第二目标镜像数据为第二服务器对关联所述序列号的目标镜像数据进行处理得到的。
在另一实施例中，所述处理模块602，还用于：
对关联所述序列号的目标镜像数据进行存储处理，得到第一目标镜像数据。
将所述第一目标镜像数据分发至第三方业务平台，以使所述第三方业务平台根据所述第一目标镜像数据关联的序列号和第二目标镜像数据关联的序列号，对所述第一目标镜像数据以及所述第二目标镜像数据进行去重处理，并根据去重处理之后的目标镜像数据实现所述待执行业务。
在另一实施例中，所述处理模块602，还用于：
在获得资源锁权限之后，对关联所述序列号的目标镜像数据进行处理，得到第一目标镜像数据。
根据所述第一目标镜像数据以及关联所述序列号的目标镜像数据对所述序列号对应的比特位图进行置位处理。
释放所述资源锁权限，以使所述第二服务器获得资源锁权限，并根据置位处理之后的比特位图对关联所述序列号的目标镜像数据进行处理，得到第二目标镜像数据。
将所述第一目标镜像数据发送至第三方业务平台，以使所述第三方业务平台根据所述第一目标镜像数据和所述第二目标镜像数据实现所述待执行业务，其中，所述第二目标镜像数据为所述第二服务器发送至所述第三方业务平台的。
进一步的，所述处理模块602，还用于：
对关联所述序列号的目标镜像数据进行存储处理和/或数据分发处理，得到第一目标镜像数据，其中，所述存储处理与所述分发处理对应不同的比特位图。
此外，所述处理模块602，还用于：
根据所述第一目标镜像数据以及关联所述序列号的目标镜像数据确定处理失败的数据。
将所述比特位图中处理失败的数据对应的比特位设置为零，并将所述比特位图中处理成功的数据对应的比特位设置为一。
此外，所述处理模块602，还用于：
释放所述资源锁权限，以使所述第二服务器获取资源锁权限，并从关联所述序列号的目标镜像数据中获取目标子镜像数据，得到第二目标镜像数据，其中，所述关联所述序列号的目标镜像数据为所述第二服务器根据所述数据采集请求从数据源中获取与待执行业务对应的目标镜像数据，并关联所述序列号与所述目标镜像数据得到的，所述目标子镜像数据为从关联所述序列号的目标镜像数据中获取的与比特位图中比特位为零时对应的数据。
此外，所述处理模块602，还用于：
确定所述目标子镜像数据的数量。
若所述目标子镜像数据的数量为零，则生成未发现异常提示，并释放所述资源锁权限。
若所述目标子镜像数据的数量不为零，则将所述目标子镜像数据设置为第二目标镜像数据发送至所述第三方业务平台。
另外，所述处理模块602，还用于：
在置位处理完成之后，或处理时长超过预设时长阈值之后，释放所述资源锁权限。
本申请实施例提供的装置，可以实现上述如图3所示的实施例的方法，其实现原理和技术效果类似，此处不再赘述。
图7为本申请实施例提供的电子设备的硬件结构示意图，如图7所示，本实施例提供的设备700包括：处理器701，以及与所述处理器通信连接的存储器。其中，处理器701、存储器702通过总线703连接。
在具体实现过程中，处理器701执行所述存储器702存储的计算机执行指令，使得处理器701执行上述方法实施例中的方法。
处理器701的具体实现过程可参见上述方法实施例，其实现原理和技术效果类似，本实施例此处不再赘述。
在上述的图7所示的实施例中，应理解，处理器可以是中央处理单元(英文：Central Processing Unit，简称：CPU)，还可以是其他通用处理器、数字信号处理器(英文：Digital Signal Processor，简称：DSP)、专用集成电路(英文：Application Specific Integrated Circuit，简称：ASIC)等。通用处理器可以是微处理器或者该处理器也可以是任何常规 的处理器等。结合发明所公开的方法的步骤可以直接体现为硬件处理器执行完成，或者用处理器中的硬件及软件模块组合执行完成。
存储器可能包含高速RAM存储器，也可能还包括非易失性存储NVM，例如至少一个磁盘存储器。
总线可以是工业标准体系结构(Industry Standard Architecture，ISA)总线、外部设备互连(Peripheral Component Interconnect，PCI)总线或扩展工业标准体系结构(Extended Industry Standard Architecture，EISA)总线等。总线可以分为地址总线、数据总线、控制总线等。为便于表示，本申请附图中的总线并不限定仅有一根总线或一种类型的总线。
本申请实施例还提供一种计算机可读存储介质，所述计算机可读存储介质中存储有计算机执行指令，当处理器执行所述计算机执行指令时，实现上述方法实施例的数据处理方法。
本申请实施例还提供一种计算机程序产品，包括计算机程序，所述计算机程序被处理器执行时，实现如上所述的数据处理方法。
上述的计算机可读存储介质，上述可读存储介质可以是由任何类型的易失性或非易失性存储设备或者它们的组合实现，如静态随机存取存储器(SRAM)，电可擦除可编程只读存储器(EEPROM)，可擦除可编程只读存储器(EPROM)，可编程只读存储器(PROM)，只读存储器(ROM)，磁存储器，快闪存储器，磁盘或光盘。可读存储介质可以是通用或专用计算机能够存取的任何可用介质。
一种示例性的可读存储介质耦合至处理器，从而使处理器能够从该可读存储介质读取信息，且可向该可读存储介质写入信息。当然，可读存储介质也可以是处理器的组成部分。处理器和可读存储介质可以位于专用集成电路(Application Specific Integrated Circuits，简称：ASIC)中。当然，处理器和可读存储介质也可以作为分立组件存在于设备中。
本领域普通技术人员可以理解：实现上述各方法实施例的全部或部分步骤可以通过程序指令相关的硬件来完成。前述的程序可以存储于一计算机可读取存储介质中。该程序在执行时，执行包括上述各方法实施例的步骤；而前述的存储介质包括：ROM、RAM、磁碟或者光盘等各种可以存储程序代码的介质。
最后应说明的是：以上各实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述各实施例对本申请进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分或者全部技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本申请各实施例技术方案的范围。

Claims (10)

  1. 一种数据处理方法，其特征在于，应用于第一服务器，所述方法包括：
    接收数据采集请求，其中，所述数据采集请求中包含序列号；
    根据所述数据采集请求从数据源中获取与待执行业务对应的目标镜像数据，并关联所述序列号与所述目标镜像数据；
    对关联所述序列号的目标镜像数据进行处理，得到第一目标镜像数据，以根据所述第一目标镜像数据以及第二目标镜像数据实现所述待执行业务，其中，所述第二目标镜像数据为第二服务器对关联所述序列号的目标镜像数据进行处理得到的。
  2. 根据权利要求1所述的方法，其特征在于，所述对关联所述序列号的目标镜像数据进行处理，得到第一目标镜像数据，以根据所述第一目标镜像数据以及第二目标镜像数据实现所述待执行业务，包括：
    对关联所述序列号的目标镜像数据进行存储处理，得到第一目标镜像数据；
    将所述第一目标镜像数据分发至第三方业务平台，以使所述第三方业务平台根据所述第一目标镜像数据关联的序列号和第二目标镜像数据关联的序列号，对所述第一目标镜像数据以及所述第二目标镜像数据进行去重处理，并根据去重处理之后的目标镜像数据实现所述待执行业务。
  3. 根据权利要求1所述的方法，其特征在于，所述对关联所述序列号的目标镜像数据进行处理，得到第一目标镜像数据，以根据所述第一目标镜像数据以及第二目标镜像数据实现所述待执行业务，包括：
    在获得资源锁权限之后，对关联所述序列号的目标镜像数据进行处理，得到第一目标镜像数据；
    根据所述第一目标镜像数据以及关联所述序列号的目标镜像数据对所述序列号对应的比特位图进行置位处理；
    释放所述资源锁权限，以使所述第二服务器获得资源锁权限，并根据置位处理之后的比特位图对关联所述序列号的目标镜像数据进行处理，得到第二目标镜像数据；
    将所述第一目标镜像数据发送至第三方业务平台，以使所述第三方业务平台根据所述第一目标镜像数据和所述第二目标镜像数据实现所述待执行业务，其中，所述第二目标镜像数据为所述第二服务器发送至所述第三方业务平台的。
  4. 根据权利要求3所述的方法，其特征在于，所述对关联所述序列号的目标镜像数据进行处理，得到第一目标镜像数据，包括：
    对关联所述序列号的目标镜像数据进行存储处理和/或数据分发处理，得到第一目标镜像数据，其中，所述存储处理与所述分发处理对应不同的比特位图。
  5. 根据权利要求3所述的方法，其特征在于，所述根据所述第一目标镜像数据以及关联所述序列号的目标镜像数据对所述序列号对应的比特位图进行置位处理，包括：
    根据所述第一目标镜像数据以及关联所述序列号的目标镜像数据确定处理失败的数据；
    将所述比特位图中处理失败的数据对应的比特位设置为零，并将所述比特位图中处理成功的数据对应的比特位设置为一。
  6. 根据权利要求5所述的方法，其特征在于，所述释放所述资源锁权限，以使所述第二服务器获得资源锁权限，并根据置位处理之后的比特位图对关联所述序列号的目标镜像数据进行处理，得到第二目标镜像数据，包括：
    释放所述资源锁权限，以使所述第二服务器获取资源锁权限，并从关联所述序列号的目标镜像数据中获取目标子镜像数据，得到第二目标镜像数据，其中，所述关联所述序列号的目标镜像数据为所述第二服务器根据所述数据采集请求从数据源中获取与待执行业务对应的目标镜像数据，并关联所述序列号与所述目标镜像数据得到的，所述目标子镜像数据为从关联所述序列号的目标镜像数据中获取的比特位图中比特位为零时对应的数据。
  7. 根据权利要求6所述的方法，其特征在于，所述方法还包括：
    确定所述目标子镜像数据的数量；
    若所述目标子镜像数据的数量为零，则生成未发现异常提示，并释放所述资源锁权限；
    若所述目标子镜像数据的数量不为零，则将所述目标子镜像数据设置为第二目标镜像数据发送至所述第三方业务平台。
  8. 根据权利要求3所述的方法，其特征在于，所述释放所述资源锁权限，包括：
    在置位处理完成之后，或处理时长超过预设时长阈值之后，释放所述资源锁权限。
  9. 根据权利要求1-8任一项所述的方法，其特征在于，所述接收数据采集请求，包括：
    控制节拍模块每隔预设时长生成一帧同步信号，其中，所述帧同步信号中包含序列号；
    接收所述帧同步信号，并根据所述帧同步信号生成包含所述序列号的数据采集请求。
  10. 一种数据处理装置，其特征在于，所述装置包括：
    接收模块，用于接收数据采集请求，其中，所述数据采集请求中包含序列号；
    处理模块，用于根据所述数据采集请求从数据源中获取与待执行业务对应的目标镜像数据，并关联所述序列号与所述目标镜像数据；
    所述处理模块，还用于对关联所述序列号的目标镜像数据进行处理，得到第一目标镜像数据，以根据所述第一目标镜像数据以及第二目标镜像数据实现所述待执行业务，其中，所述第二目标镜像数据为第二服务器对关联所述序列号的目标镜像数据进行处理得到的。
PCT/CN2023/084287 2022-03-29 2023-03-28 数据处理方法及装置 Ceased WO2023185802A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210326055.6 2022-03-29
CN202210326055.6A CN114554593B (zh) 2022-03-29 2022-03-29 数据处理方法及装置

Publications (1)

Publication Number Publication Date
WO2023185802A1 true WO2023185802A1 (zh) 2023-10-05

Family

ID=81666110

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/084287 Ceased WO2023185802A1 (zh) 2022-03-29 2023-03-28 数据处理方法及装置

Country Status (2)

Country Link
CN (1) CN114554593B (zh)
WO (1) WO2023185802A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117271007A (zh) * 2023-11-21 2023-12-22 恒生电子股份有限公司 业务处理方法及系统

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114554593B (zh) * 2022-03-29 2025-08-01 阿里巴巴（中国）有限公司 数据处理方法及装置
CN119696929B (zh) * 2025-02-24 2025-08-26 推动时代科技(北京)有限公司 镜像数据的获取和传输方法及系统、介质、计算机设备

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050256972A1 (en) * 2004-05-11 2005-11-17 Hewlett-Packard Development Company, L.P. Mirroring storage interface
US20190081862A1 (en) * 2017-09-13 2019-03-14 Verizon Digital Media Services Inc. Rapid Configuration Propagation in a Distributed Multi-Tenant Platform
US20190207812A1 (en) * 2016-11-24 2019-07-04 Tencent Technology (Shenzhen) Company Limited Hybrid cloud network configuration management
CN111274050A (zh) * 2020-01-19 2020-06-12 深圳华锐金融技术股份有限公司 业务数据转发方法、装置、计算机设备和存储介质
WO2020207267A1 (zh) * 2019-04-08 2020-10-15 阿里巴巴集团控股有限公司 网络系统、镜像管理方法、设备及存储介质
CN114554593A (zh) * 2022-03-29 2022-05-27 阿里巴巴（中国）有限公司 数据处理方法及装置

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10013313B2 (en) * 2014-09-16 2018-07-03 Actifio, Inc. Integrated database and log backup
CN104537112B (zh) * 2015-01-20 2017-07-14 成都携恩科技有限公司 一种安全云计算的方法
CN110912955B (zh) * 2018-09-17 2022-04-05 阿里巴巴集团控股有限公司 容器镜像下载、上传方法及装置
CN111983948A (zh) * 2020-07-07 2020-11-24 宝能(广州)汽车研究院有限公司 多传感器数据同步方法及其设备

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050256972A1 (en) * 2004-05-11 2005-11-17 Hewlett-Packard Development Company, L.P. Mirroring storage interface
US20190207812A1 (en) * 2016-11-24 2019-07-04 Tencent Technology (Shenzhen) Company Limited Hybrid cloud network configuration management
US20190081862A1 (en) * 2017-09-13 2019-03-14 Verizon Digital Media Services Inc. Rapid Configuration Propagation in a Distributed Multi-Tenant Platform
WO2020207267A1 (zh) * 2019-04-08 2020-10-15 阿里巴巴集团控股有限公司 网络系统、镜像管理方法、设备及存储介质
CN111274050A (zh) * 2020-01-19 2020-06-12 深圳华锐金融技术股份有限公司 业务数据转发方法、装置、计算机设备和存储介质
CN114554593A (zh) * 2022-03-29 2022-05-27 阿里巴巴（中国）有限公司 数据处理方法及装置

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
CAN-YANG ZHANG, LIU XIAO-JIE: "Virtual machine image clustering deduplication algorithm based on improved Simhash ", JOURNAL OF SICHUAN UNIVERSITY(NATURAL SCIENCE EDITION), vol. 57, no. 1, 8 January 2020 (2020-01-08), pages 57 - 65, XP093094784 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117271007A (zh) * 2023-11-21 2023-12-22 恒生电子股份有限公司 业务处理方法及系统
CN117271007B (zh) * 2023-11-21 2024-02-13 恒生电子股份有限公司 业务处理方法及系统

Also Published As

Publication number Publication date
CN114554593A (zh) 2022-05-27
CN114554593B (zh) 2025-08-01

Similar Documents

Publication Publication Date Title
WO2023185802A1 (zh) 数据处理方法及装置
US6687849B1 (en) Method and apparatus for implementing fault-tolerant processing without duplicating working process
US8874508B1 (en) Systems and methods for enabling database disaster recovery using replicated volumes
CN112506702B (zh) 数据中心容灾方法、装置、设备及存储介质
WO2021135530A1 (zh) 数据恢复方法、装置、电子设备及介质
US20170168756A1 (en) Storage transactions
US20080209145A1 (en) Techniques for asynchronous data replication
US8127174B1 (en) Method and apparatus for performing transparent in-memory checkpointing
US10725980B2 (en) Highly available cluster agent for backup and restore operations
WO2017067484A1 (zh) 一种虚拟化数据中心调度系统和方法
CN110781157A (zh) 一种基于nas的备份与恢复方法和装置
CN112307013B (zh) 用于管理应用系统的方法、设备和计算机程序产品
CN113190620B (zh) Redis集群之间数据的同步方法、装置、设备及存储介质
WO2017118080A1 (zh) 一种中央处理器cpu热移除、热添加方法及装置
CN113626252A (zh) 一种基于集群的城市级容灾方法、装置、电子设备及介质
CN106815094B (zh) 一种用于实现主备同步模式下事务提交的方法与设备
US20150213102A1 (en) Synchronous data replication in a content management system
CN117499210A (zh) 一种双活集群仲裁方法、装置、计算机设备和存储介质
CN107623581B (zh) 服务列表生成方法、装置及系统，获取、上报方法及装置
CN115396296B (zh) 业务处理方法、装置、电子设备及计算机可读存储介质
CN103793296A (zh) 一种在集群中用于辅助备份复制计算机系统的方法
CN110502460A (zh) 数据处理的方法和节点
CN105760456B (zh) 一种保持数据一致性的方法和装置
CN114490712B (zh) PostgreSQL数据库的服务状态确定系统和方法
CN118193280A (zh) 数据恢复方法、装置、存储设备及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23778167

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 23778167

Country of ref document: EP

Kind code of ref document: A1