CN115037729B - Data aggregation method, device, electronic equipment and computer readable medium - Google Patents
Data aggregation method, device, electronic equipment and computer readable medium Download PDFInfo
- Publication number
- CN115037729B CN115037729B CN202210437349.6A CN202210437349A CN115037729B CN 115037729 B CN115037729 B CN 115037729B CN 202210437349 A CN202210437349 A CN 202210437349A CN 115037729 B CN115037729 B CN 115037729B
- Authority
- CN
- China
- Prior art keywords
- data
- operation time
- segment
- time
- identifier
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000002776 aggregation Effects 0.000 title claims abstract description 68
- 238000004220 aggregation Methods 0.000 title claims abstract description 68
- 238000000034 method Methods 0.000 title claims abstract description 65
- 239000012634 fragment Substances 0.000 claims abstract description 40
- 230000004044 response Effects 0.000 claims abstract description 11
- 238000003860 storage Methods 0.000 claims description 19
- 238000004590 computer program Methods 0.000 claims description 18
- 230000008859 change Effects 0.000 claims description 12
- 230000010354 integration Effects 0.000 claims description 10
- 238000012545 processing Methods 0.000 claims description 10
- 230000001174 ascending effect Effects 0.000 claims description 7
- 238000011084 recovery Methods 0.000 claims description 7
- 230000008569 process Effects 0.000 abstract description 11
- 238000010586 diagram Methods 0.000 description 16
- 238000004891 communication Methods 0.000 description 7
- 238000011161 development Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 6
- 230000005540 biological transmission Effects 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 4
- 230000002123 temporal effect Effects 0.000 description 4
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000003068 static effect Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 239000000835 fiber Substances 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 230000006399 behavior Effects 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000005520 cutting process Methods 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000009191 jumping Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/12—Applying verification of the received information
- H04L63/123—Applying verification of the received information received data contents, e.g. message integrity
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/14—Session management
- H04L67/143—Termination or inactivation of sessions, e.g. event-controlled end of session
- H04L67/145—Termination or inactivation of sessions, e.g. event-controlled end of session avoiding end of session, e.g. keep-alive, heartbeats, resumption message or wake-up for inactive or interrupted session
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/14—Session management
- H04L67/146—Markers for unambiguous identification of a particular session, e.g. session cookie or URL-encoding
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Cardiology (AREA)
- Computer Hardware Design (AREA)
- General Engineering & Computer Science (AREA)
- Medical Informatics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a data aggregation method, a data aggregation device, electronic equipment and a computer readable medium, and relates to the technical field of big data acquisition. One embodiment of the method comprises the following steps: in response to the converging task of the current data segment, the following steps are circularly executed until the data segment of the data set is determined to meet the set condition, and the final data set is output: acquiring reference operation time and reference data identification corresponding to the last converged data segment, and determining data items with operation time equal to the reference operation time in the current data segment; acquiring data items with the operation time equal to the reference operation time and the data identification after the reference data identification from the current data segment, and taking the data items with the operation time later than the reference operation time as the data segments which are converged at this time; and integrating the data fragments converged at this time into a data set according to the set data set rule. The data set changes dynamically during the aggregation process of this embodiment.
Description
Technical Field
The present invention relates to the field of big data acquisition technologies, and in particular, to a data aggregation method, a device, an electronic apparatus, and a computer readable medium.
Background
In the design and development work of the cross-province government platform, the primary problem faced by the development of business logic is how to aggregate the transaction data of a plurality of provinces. The conventional data aggregation scheme adopts a data snapshot mode, namely, a data set at a certain moment is extracted to serve as temporary static data, and finally needed data is aggregated through data cleaning and loading of the temporary static data.
The data snapshot mode acquires complete static data at a certain moment, and the data set affected by user behavior in a specific item is generally in dynamic change, and the data snapshot mode ignores the dynamic change of the data set.
Disclosure of Invention
In view of this, the embodiments of the present invention provide a data aggregation method, apparatus, electronic device, and computer readable medium, where the method obtains an operation time and a data identifier of a data segment that is aggregated last time, and obtains a data segment that is aggregated this time from a current data segment based on the obtained operation time and the data identifier, so as to integrate the data segment that is aggregated this time into a data set, where the data set dynamically changes during the aggregation process, and a complete data set can be obtained.
To achieve the above object, according to an aspect of an embodiment of the present invention, there is provided a data aggregation method.
The data aggregation method of the embodiment of the invention comprises the following steps: in response to the converging task of the current data segment, the following steps are circularly executed until the data segment of the data set is determined to meet the set condition, and the final data set is output:
Acquiring the operation time and the data identifier of the last converged data segment as corresponding reference operation time and reference data identifier, and determining the data item with the operation time equal to the reference operation time in the current data segment;
Acquiring data items with the operation time equal to the reference operation time and the data identification after the reference data identification from the current data segment, and taking the data items with the operation time later than the reference operation time as the data segments which are gathered at this time;
And integrating the data fragments converged at this time into the data set according to a set data set rule.
Optionally, after the step of acquiring the operation time and the data identifier of the last aggregated data segment as the corresponding reference operation time and the reference data identifier, the method further includes:
Judging whether a data item with the operation time equal to the reference operation time exists in the current data segment;
And if no data item with the operation time equal to the reference operation time exists in the current data segment, acquiring the data item with the operation time later than the reference operation time from the current data segment as the data segment which is converged at this time.
Optionally, the data set rule is an ascending order rule with time as a main part and data identification as a secondary part;
the integrating the data segments converged at the time into the data set comprises the following steps: and arranging the data fragments converged at the time to the last of the data set.
Optionally, the method further comprises: and when the convergence task is interrupted, determining that the data segments of the data set do not meet the set conditions, and resuming the execution of the convergence task.
Optionally, the setting conditions are: the number of the data fragments of the data set is equal to a set number threshold.
Optionally, the method obtains a final data set by taking the reference operation time and the reference data identifier as an incoming aggregation interface and circularly calling the aggregation interface.
Optionally, the current data segment is transaction data of a government transaction transmitted through an HTTP network.
To achieve the above object, according to another aspect of the embodiments of the present invention, there is provided a data aggregation apparatus.
The data aggregation device of the embodiment of the invention comprises: the response module is used for responding to the converging task of the current data segment, circularly executing the processing procedures of the determining module, the first obtaining module and the integrating module until the data segment of the data set is determined to meet the set condition, and outputting the final data set; wherein,
The determining module is used for acquiring the operation time and the data identifier of the last converged data segment as corresponding reference operation time and reference data identifier and determining the data item with the operation time equal to the reference operation time in the current data segment;
The first obtaining module is configured to obtain, from the current data segment, a data item having an operation time equal to the reference operation time and a data identifier after the reference data identifier, and a data item having an operation time later than the reference operation time as a data segment that is collected at this time;
the integration module is used for integrating the data fragments converged at this time into the data set according to a set data set rule.
Optionally, the apparatus further comprises: the judging module is used for judging whether a data item with the operation time equal to the reference operation time exists in the current data segment;
And the second acquisition module is used for acquiring the data item with the operation time later than the reference operation time from the current data segment as the data segment which is converged at this time if the data item with the operation time equal to the reference operation time does not exist in the current data segment.
Optionally, the data set rule is an ascending order rule with time as a main part and data identification as a secondary part;
the integration module is further configured to arrange the data segments that are collected this time to a last of the data set.
Optionally, the apparatus further comprises: and the recovery module is used for determining that the data segments of the data set do not meet the set conditions when the convergence task is interrupted, and recovering to execute the convergence task.
Optionally, the setting conditions are: the number of the data fragments of the data set is equal to a set number threshold.
Optionally, the device obtains a final data set by taking the reference operation time and the reference data identifier as an incoming aggregation interface and circularly calling the aggregation interface.
Optionally, the current data segment is transaction data of a government transaction transmitted through an HTTP network.
To achieve the above object, according to still another aspect of the embodiments of the present invention, there is provided an electronic device.
An electronic device according to an embodiment of the present invention includes: one or more processors; and the storage device is used for storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors are enabled to realize a data aggregation method of the embodiment of the invention.
To achieve the above object, according to still another aspect of the embodiments of the present invention, there is provided a computer-readable medium.
A computer readable medium of an embodiment of the present invention has stored thereon a computer program which, when executed by a processor, implements a data aggregation method of an embodiment of the present invention.
To achieve the above object, according to still another aspect of embodiments of the present invention, there is provided a computer program product.
A computer program product of an embodiment of the present invention includes a computer program that, when executed by a processor, implements a data aggregation method of an embodiment of the present invention.
One embodiment of the above invention has the following advantages or benefits: the data fragments which are converged at this time are integrated into the data set by acquiring the operation time and the data identification of the data fragments which are converged at last time and acquiring the data fragments which are converged at this time from the current data fragments based on the operation time and the data identification, and the data set dynamically changes in the converging process, so that a complete data set can be obtained.
The method comprises the steps of dividing a current data segment by operation time preferentially, and dividing the current data segment by operation time and data identification when the change of a data item cannot be reflected by the operation time, so as to obtain a data segment suitable for the current convergence. The ascending order arrangement rule with time as the main and data mark as the secondary is used as the data set rule, so that the obtained data set can describe the change of time and space.
After the convergence task is interrupted, the convergence task can be automatically restored, and the noninductive convergence is realized. By setting the condition of cycle jump, the recovery capability of incremental aggregation and interrupt of the aggregation task can be well supported, and the pressure of the aggregation task on the server is reduced. The convergence logic is realized in a convergence interface mode, so that the realization is simple and quick, the development workload is low, and the development efficiency is high. The convergence of government affair data can be automatically realized through HTTP network communication, the applicability is wide, and different development environments and security strategies can be supported.
Further effects of the above-described non-conventional alternatives are described below in connection with the embodiments.
Drawings
The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:
FIG. 1 is a schematic diagram of main steps of a data aggregation method according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of the main flow of a data aggregation method according to another embodiment of the present invention;
FIG. 3 is a schematic diagram of a system architecture of a data aggregation method according to still another embodiment of the present invention;
FIG. 4 is a schematic diagram of the main flow of a data aggregation method according to still another embodiment of the present invention;
FIG. 5 is a schematic diagram of the main modules of a data aggregation device according to an embodiment of the present invention;
FIG. 6 is an exemplary system architecture diagram in which embodiments of the present invention may be applied;
fig. 7 is a schematic diagram of a computer system suitable for use in implementing an embodiment of the invention.
Detailed Description
Exemplary embodiments of the present invention will now be described with reference to the accompanying drawings, in which various details of the embodiments of the present invention are included to facilitate understanding, and are to be considered merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
The technical scheme of the application obtains, stores, uses, processes and the like the data, which all meet the relevant regulations of national laws and regulations.
As described in the background, the conventional data aggregation scheme is a data snapshot mode. A snapshot is a complete, available copy of a given data set that includes an image of the corresponding data at some point in time (i.e., the point in time at which the copy began). The snapshot may be a copy of the data it represents, or may be a copy of the data.
The running environment of the cross-provincial government platform is a network interface based on an HTTP transmission protocol, the data fragments have the upper limit of transmission content, and the network interface is required to be called for multiple times in a time-sharing way to obtain complete data to be aggregated. Therefore, the data snapshot mode can only acquire data fragments with indefinite time, limited space (namely content) and meeting the set rule, the data fragments in dynamic change cannot be integrated, the response time of the data snapshot mode is long, and the development cost is high.
In order to solve the above-mentioned problems, the present embodiment provides a new data aggregation method, which is suitable for integrating data segments in dynamic changes, and can obtain a complete data set. The following detailed description refers to the accompanying drawings.
Fig. 1 is a schematic diagram of main steps of a data aggregation method according to an embodiment of the present invention. As shown in fig. 1, the data aggregation method in the embodiment of the present invention mainly includes the following steps:
Step S101: and responding to the converging task of the current data segment, acquiring the operation time and the data identifier of the last converging data segment as corresponding reference operation time and reference data identifier, and determining that the data item with the operation time equal to the reference operation time exists in the current data segment. The current data segment may be a portion of data that is intercepted from the target data (e.g., network data) and may include one or more data items. The time when the data item changes, referred to herein as the operation time, is recorded.
After receiving the converging task for the current data segment, acquiring the operation time and the data identifier of the last converging data segment, taking the acquired operation time as the reference operation time, and taking the acquired data identifier as the reference data identifier. And searching out the data items with the operation time equal to the reference operation time based on the operation time of each data item in the current data segment.
Step S102: and acquiring data items with the operation time equal to the reference operation time and the data identification after the reference data identification from the current data segment, and taking the data items with the operation time later than the reference operation time as the data segments which are gathered at this time. The data identifier, i.e. the unique identifier of the data item, may be used to indicate its storage location in the target data. In an embodiment, the larger the data identifier, the more backward the data item is in the storage location of the target data.
When the acquisition operation time is equal to the reference operation time and the data identifier is a data item after the reference data identifier, the data item with the operation time equal to the reference operation time can be acquired from the current data segment, and then the data item with the data identifier larger than the reference data identifier is further acquired from the data items. The data item whose operation time is later than the reference operation time is acquired, i.e. the data item generated after the reference operation time is acquired from the current data piece.
Step S103: and integrating the data fragments converged at this time into the data set according to a set data set rule. The data set rules are used to describe the way in which data segments are integrated, and in an embodiment may be arrangement rules describing temporal and spatial (i.e., content) variations. And according to the data set rule, arranging the data fragments converged at this time into the data set.
Step S104: judging whether the data segments of the data set meet the set conditions, and if the data segments of the data set do not meet the set conditions, executing the step S101; otherwise, step S105 is performed. The condition of jumping out of the loop is preset, after each integration of the data segments into the data set, it is determined whether the data segments of the data set satisfy the set condition, and if the set condition is not satisfied, steps S101 to S104 are executed in a loop.
Step S105: and outputting the final data set. And if the data fragments of the data set meet the set conditions, indicating that the integration of the data set is completed, and outputting the final data set.
Fig. 2 is a schematic diagram of the main flow of a data aggregation method according to a further embodiment of the present invention. As shown in fig. 2, the data aggregation method in the embodiment of the present invention mainly includes the following steps:
Step S201: and responding to the converging task of the current data segment, and acquiring the operation time and the data identifier of the last converging data segment as corresponding reference operation time and reference data identifier. It should be noted that, if the task is executed for the first time, the operation time and the data identifier of the last aggregated data segment are empty. If the task is not executed for the first time, the operation time of the last aggregated data segment can be the latest operation time of each data item of the data segment, and the data identifier can be the maximum data identifier of each data item of the data segment.
Step S202: judging whether a data item with the operation time equal to the reference operation time exists in the current data segment, and if the data item with the operation time equal to the reference operation time does not exist in the current data segment, executing step S203; otherwise, step S204 is performed. The time of the transmission change of the data item is recorded through the operation time, and the time recording is characterized by only going backwards, so that the recording mode can accurately reflect the generation of the data item and the sequence of the change time.
In the embodiment, the data time of the data provider can be used as the operation time, so that the problem of variability of different service systems due to different times is avoided.
Step S203: and acquiring a data item with the operation time later than the reference operation time from the current data segment as the data segment converged at this time, and executing step S205. The step adopts the operation time to segment the current data segment, and only data items after the reference operation time are acquired.
Step S204: and acquiring the data items with the operation time equal to the reference operation time and the data identification after the reference data identification from the current data segment, and taking the data items with the operation time later than the reference operation time as the data segments which are converged at this time, and executing step S205. The method comprises the steps of cutting a current data segment by adopting operation time and data identification together, obtaining a data item which is equal to reference operation time and has the data identification larger than the reference data identification, and obtaining the data item after the reference operation time.
When the operation time is adopted to divide the current data segment, the limit of the space quantity is used as the boundary of the data segment. In the case where there is a batch operation for the service system of the data provider, it may happen that the data segment boundary portion data is a portion data of pieces of data at a certain time.
At this time, the operation time of the data item existing in the data segment is the same as the operation time of the data segment converged last time. Therefore, the constraint rule of the data mark as the boundary of the data segment converged next time can be added in the data segment converged this time. And after the data processing at the same time is completed, namely the operation time of the data item is different from the reference operation time, the data item after the reference operation time is continuously acquired.
Step S205: and integrating the data fragments converged at this time into a data set according to the set data set rule. The acquisition of a data set has three elements: temporal, spatial (i.e., content) and data collection rules. In this embodiment, the data set rule uses an ordering rule that can describe temporal and spatial variations, such as an ascending ordering rule that is time-based, with data identified as secondary. And according to the data set rule, arranging the data fragments converged at this time to the last of the data set.
In the aggregation process of the embodiment, whether acquired or not acquired, the newly changed data are arranged at the end of the data set, so that the data changed in the aggregation process can be acquired in the current aggregation task, and the timeliness of the aggregated data is improved.
Step S206: judging whether the data segments of the data set meet the set conditions, and if the data segments of the data set do not meet the set conditions, executing step S201; otherwise, step S207 is performed. Since the current data segment needs to be sliced every cycle, a skip-out condition can be set up.
In an embodiment, the condition may be: the number of data fragments of the data set is equal to a set number threshold. If the number of data segments of the data set is smaller than the set number threshold, steps S201 to S206 are repeatedly performed. The condition can well support increment aggregation, and reduce the pressure of an aggregation task on a server.
Step S207: and outputting the final data set. And if the number of the data fragments of the data set is equal to the set threshold value, indicating that the integration of the data set is finished, and outputting a final data set.
In a preferred embodiment, when the convergence task is interrupted, it is determined that the data segment of the data set does not meet the set condition, and execution of the convergence task is resumed. The embodiment enables the data convergence scheme to have the task interruption recovery capability, and realizes the noninductive convergence of the interrupted convergence task after recovery.
Fig. 3 is a schematic system architecture of a data aggregation method according to still another embodiment of the present invention. As shown in fig. 3, the data aggregation method of the present embodiment is applied to item data aggregation for government items of a plurality of provinces. The government affairs can be all levels of government, all relevant departments and public institutions for providing administrative service matters for business transaction staff, such as business license transaction, tax registration, identity card transaction and the like. The transaction data of the government transaction may include transaction transactor information and transaction related information.
The transaction data convergence flow direction of the cross-provincial government affair platform is as follows: the government affair item manager of each province adds a general office item label to the government affair item, wherein the general office item label is used for defining which data contents of the item data need to be aggregated, developing an aggregation interface according to aggregation logic appointed by a cross-province government affair platform, and registering to an open platform. The general office platform item directory system realizes the integration and convergence of all the provincial item data by calling the convergence interface.
Fig. 4 is a schematic diagram of a main flow of a data aggregation method according to still another embodiment of the present invention. As shown in fig. 4, the data aggregation method of the present embodiment is applied to item data aggregation of government items of a plurality of provinces, and includes the steps of:
Step S401: and responding to the converging task of the current data segment, and acquiring the operation time and the data identifier of the last converging data segment as corresponding reference operation time and reference data identifier. In this embodiment, the current data segment is the transaction data of the government transaction transmitted through the HTTP network. HTTP is an abbreviation of HyperText Transfer Protocol, hypertext transfer protocol.
Step S402: and taking the reference operation time and the reference data identifier as an incoming call to the convergence interface so as to call the convergence interface. It should be noted that, if the aggregation task is executed for the first time, the reference operation time and the reference data identifier do not need to be input.
Step S403: and executing the convergence logic defined in the convergence interface to obtain the data fragments required by the current convergence. The convergence logic is the implementation process of step S102 or step S202 to step S204.
The data set time corresponding to the item data of each province is in dynamic change, and only HTTP network transmission can be used, and the capacity upper limit exists, so in the embodiment, when the convergence interface is called circularly each time, the operation time and the data identifier of the last converged data segment are transmitted, and the current data segment is segmented by adopting the operation time and the data identifier. Each loop invokes the convergence interface a new cut and sort of the current data segment.
Step S404: and integrating the data fragments collected at this time into a data set according to a data set rule which is arranged in an ascending order with the recording time as a main part and the data mark as a secondary part.
Step S405: judging whether the data segments of the data set meet the set conditions, and if the data segments of the data set do not meet the set conditions, executing step S401; otherwise, step S406 is performed.
Step S406: and outputting the final data set. In this embodiment, a plurality of data segments in time sharing are integrated by a data set rule, and a complete data set is obtained.
The data aggregation method of the embodiment is based on an HTTP network, and can realize data aggregation as long as the network is connected. In addition, the embodiment collects the data fragments suitable for the current convergence by acquiring the operation time and the data identification of the data fragments converged last time through each call, has wide applicability and can support different development environments and different security strategies. In addition, the embodiment is simple and quick in implementation, and the workload of developers is low.
Fig. 5 is a schematic diagram of main modules of a data aggregation apparatus according to an embodiment of the present invention. As shown in fig. 5, a data aggregation apparatus 500 according to an embodiment of the present invention mainly includes:
and the response module 501 is configured to respond to the task of converging the current data segment, and circularly execute the processing procedures of the determining module, the first obtaining module and the integrating module until it is determined that the data segment of the data set meets the set condition, and output the final data set.
The current data segment may be a portion of data that is intercepted from the target data (e.g., network data) and may include one or more data items. After each execution of the determining module, the first acquiring module and the integrating module, judging whether the data fragments of the data set meet the set conditions, and executing the determining module, the first acquiring module and the integrating module again when the set conditions are not met; and when the set condition is met, indicating that the integration of the data sets is completed, and outputting a final data set.
The determining module 502 is configured to obtain an operation time and a data identifier of a last aggregated data segment as a corresponding reference operation time and a reference data identifier, and determine that a data item with an operation time equal to the reference operation time exists in the current data segment.
After receiving the converging task for the current data segment, acquiring the operation time and the data identifier of the last converging data segment, taking the acquired operation time as the reference operation time, and taking the acquired data identifier as the reference data identifier. And searching out the data items with the operation time equal to the reference operation time based on the operation time of each data item in the current data segment.
A first obtaining module 503, configured to obtain, from the current data segment, a data item having an operation time equal to the reference operation time and a data identifier after the reference data identifier, and a data item having an operation time later than the reference operation time as a data segment that is assembled this time.
When the acquisition operation time is equal to the reference operation time and the data identifier is a data item after the reference data identifier, the data item with the operation time equal to the reference operation time can be acquired from the current data segment, and then the data item with the data identifier larger than the reference data identifier is further acquired from the data items. The data item whose operation time is later than the reference operation time is acquired, i.e. the data item generated after the reference operation time is acquired from the current data piece.
And the integrating module 504 is configured to integrate the data segments that are aggregated at this time into the data set according to a set data set rule. The data set rules are used to describe the way in which data segments are integrated, and in an embodiment may be arrangement rules describing temporal and spatial (i.e., content) variations. And according to the data set rule, arranging the data fragments converged at this time into the data set.
In addition, the data aggregation apparatus 500 according to the embodiment of the present invention may further include: a judgment module, a second acquisition module, and a recovery module (not shown in fig. 5). The judging module is used for judging whether the current data segment has a data item with the operation time equal to the reference operation time.
And the second acquisition module is used for acquiring the data item with the operation time later than the reference operation time from the current data segment as the data segment which is converged at this time if the data item with the operation time equal to the reference operation time does not exist in the current data segment. And the recovery module is used for determining that the data segments of the data set do not meet the set conditions when the convergence task is interrupted, and recovering to execute the convergence task.
From the above description, it can be seen that by acquiring the operation time and the data identifier of the last aggregated data segment, and acquiring the current aggregated data segment from the current data segment based on the acquired operation time and the data identifier, the current aggregated data segment is further integrated into the data set, the data set dynamically changes during the aggregation process, and a complete data set can be obtained.
Fig. 6 illustrates an exemplary system architecture 600 to which the data aggregation method or data aggregation apparatus of embodiments of the present invention may be applied.
As shown in fig. 6, the system architecture 600 may include terminal devices 601, 602, 603, a network 604, and a server 605. The network 604 is used as a medium to provide communication links between the terminal devices 601, 602, 603 and the server 605. The network 604 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.
A user may interact with the server 605 via the network 604 using the terminal devices 601, 602, 603 to receive or send messages, etc. The terminal devices 601, 602, 603 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smartphones, tablets, laptop and desktop computers, and the like.
The server 605 may be a server providing various services, such as a background management server processing an aggregate task transmitted by a user using the terminal devices 601, 602, 603. The background management server can acquire the operation time and the data identification of the last converged data segment, determine the current converged data segment, integrate the current converged data segment into a data set, and feed back the processing result (such as a final data set) to the terminal equipment.
It should be noted that, the data aggregation method provided in the embodiment of the present invention is generally executed by the server 605, and accordingly, the data aggregation device is generally disposed in the server 605.
It should be understood that the number of terminal devices, networks and servers in fig. 6 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
According to embodiments of the present invention, the present invention also provides an electronic device, a computer-readable medium and a computer program product.
The electronic device of the present invention includes: one or more processors; and the storage device is used for storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors are enabled to realize a data aggregation method of the embodiment of the invention.
The computer readable medium of the present invention has stored thereon a computer program which, when executed by a processor, implements a data aggregation method of an embodiment of the present invention.
The computer program product of the present invention comprises a computer program which, when executed by a processor, implements a data aggregation method of an embodiment of the present invention.
Referring now to FIG. 7, there is illustrated a schematic diagram of a computer system 700 suitable for use in implementing an electronic device of an embodiment of the present invention. The terminal device shown in fig. 7 is only an example, and should not impose any limitation on the functions and the scope of use of the embodiment of the present invention.
As shown in fig. 7, the computer system 700 includes a Central Processing Unit (CPU) 701, which can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 702 or a program loaded from a storage section 708 into a Random Access Memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the system 700 are also stored. The CPU 701, ROM 702, and RAM 703 are connected to each other through a bus 704. An input/output (I/O) interface 705 is also connected to bus 704.
The following components are connected to the I/O interface 705: an input section 706 including a keyboard, a mouse, and the like; an output portion 707 including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, a speaker, and the like; a storage section 708 including a hard disk or the like; and a communication section 709 including a network interface card such as a LAN card, a modem, or the like. The communication section 709 performs communication processing via a network such as the internet. The drive 710 is also connected to the I/O interface 705 as needed. A removable medium 711 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 710 as necessary, so that a computer program read therefrom is mounted into the storage section 708 as necessary.
In particular, according to embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication portion 709, and/or installed from the removable medium 711. The above-described functions defined in the system of the present invention are performed when the computer program is executed by a Central Processing Unit (CPU) 701.
The computer readable medium shown in the present invention may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules involved in the embodiments of the present invention may be implemented in software or in hardware. The described modules may also be provided in a processor, for example, as: a processor includes a response module, a determination module, a first acquisition module, and an integration module. The names of these modules do not constitute a limitation on the module itself in some cases, for example, the response module may also be described as "a module that, in response to the task of aggregation of the current data segment, performs processing of the determining module, the first obtaining module, and the integrating module in a loop until it is determined that the data segment of the data set satisfies the set condition, and outputs the final data set".
As another aspect, the present invention also provides a computer-readable medium that may be contained in the apparatus described in the above embodiments; or may be present alone without being fitted into the device. The computer readable medium carries one or more programs which, when executed by a device, cause the device to include: in response to the converging task of the current data segment, the following steps are circularly executed until the data segment of the data set is determined to meet the set condition, and the final data set is output: acquiring the operation time and the data identifier of the last converged data segment as corresponding reference operation time and reference data identifier, and determining the data item with the operation time equal to the reference operation time in the current data segment; acquiring data items with the operation time equal to the reference operation time and the data identification after the reference data identification from the current data segment, and taking the data items with the operation time later than the reference operation time as the data segments which are gathered at this time; and integrating the data fragments converged at this time into the data set according to a set data set rule.
According to the technical scheme of the embodiment of the invention, the data fragments which are converged at this time are acquired from the current data fragments by acquiring the operation time and the data identification of the data fragments which are converged at last, and based on the acquired operation time and the data identification, so that the data fragments which are converged at this time are integrated into the data set, the data set dynamically changes in the converging process, and the complete data set can be acquired.
The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives can occur depending upon design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.
Claims (17)
1. A method of data aggregation, comprising:
in response to the converging task of the current data segment, the following steps are circularly executed until the data segment of the data set is determined to meet the set condition, and the final data set is output:
Acquiring the operation time and the data identifier of the last converged data segment as corresponding reference operation time and reference data identifier, and determining the data item with the operation time equal to the reference operation time in the current data segment; the current data segment is a part of data intercepted from the target data and comprises one or more data items, and when the data items change, the time of the change at the time is recorded and is called operation time; and the last operation time of the data fragments gathered last time is the latest operation time of each data item of the data fragments; the data identifier is a unique identifier of the data item and is used for representing the storage position of the data item in the target data, and the larger the data identifier is, the more the data identifier is in the storage position of the target data; the data identifier of the last converged data segment is the maximum data identifier of each data item of the data segment;
Acquiring data items with the operation time equal to the reference operation time and the data identification after the reference data identification from the current data segment, and taking the data items with the operation time later than the reference operation time as the data segments which are gathered at this time;
And integrating the data fragments converged at this time into the data set according to a set data set rule.
2. The method of claim 1, wherein after the step of obtaining the operation time and the data identification of the last aggregated data piece as the corresponding reference operation time and reference data identification, the method further comprises:
Judging whether a data item with the operation time equal to the reference operation time exists in the current data segment;
And if no data item with the operation time equal to the reference operation time exists in the current data segment, acquiring the data item with the operation time later than the reference operation time from the current data segment as the data segment which is converged at this time.
3. The method of claim 1, wherein the data set rule is an ascending order rule with time primary and data identification secondary;
the integrating the data segments converged at the time into the data set comprises the following steps:
and arranging the data fragments converged at the time to the last of the data set.
4. The method according to claim 1, wherein the method further comprises:
and when the convergence task is interrupted, determining that the data segments of the data set do not meet the set conditions, and resuming the execution of the convergence task.
5. The method of claim 4, wherein the set condition is: the number of the data fragments of the data set is equal to a set number threshold.
6. The method according to any of claims 1 to 5, characterized in that the method obtains a final data set by taking the reference operation time and the reference data identity as an incoming aggregation interface and looping the aggregation interface.
7. The method of claim 6, wherein the current data segment is transaction data of a government transaction transmitted over an HTTP network.
8. A data aggregation apparatus, comprising:
The response module is used for responding to the converging task of the current data segment, circularly executing the processing procedures of the determining module, the first obtaining module and the integrating module until the data segment of the data set is determined to meet the set condition, and outputting the final data set; wherein,
The determining module is used for acquiring the operation time and the data identifier of the last converged data segment as corresponding reference operation time and reference data identifier and determining the data item with the operation time equal to the reference operation time in the current data segment; the current data segment is a part of data intercepted from the target data and comprises one or more data items, and when the data items change, the time of the change at the time is recorded and is called operation time; and the last operation time of the data fragments gathered last time is the latest operation time of each data item of the data fragments; the data identifier is a unique identifier of the data item and is used for representing the storage position of the data item in the target data, and the larger the data identifier is, the more the data identifier is in the storage position of the target data; the data identifier of the last converged data segment is the maximum data identifier of each data item of the data segment;
The first obtaining module is configured to obtain, from the current data segment, a data item having an operation time equal to the reference operation time and a data identifier after the reference data identifier, and a data item having an operation time later than the reference operation time as a data segment that is collected at this time;
the integration module is used for integrating the data fragments converged at this time into the data set according to a set data set rule.
9. The apparatus of claim 8, wherein the apparatus further comprises:
the judging module is used for judging whether a data item with the operation time equal to the reference operation time exists in the current data segment;
And the second acquisition module is used for acquiring the data item with the operation time later than the reference operation time from the current data segment as the data segment which is converged at this time if the data item with the operation time equal to the reference operation time does not exist in the current data segment.
10. The apparatus of claim 8, wherein the data set rule is an ascending order rule with time primary and data identification secondary;
the integration module is further configured to arrange the data segments that are collected this time to a last of the data set.
11. The apparatus of claim 8, wherein the apparatus further comprises: a recovery module for
And when the convergence task is interrupted, determining that the data segments of the data set do not meet the set conditions, and resuming the execution of the convergence task.
12. The apparatus of claim 11, wherein the set condition is: the number of the data fragments of the data set is equal to a set number threshold.
13. The apparatus according to any of claims 8 to 12, wherein the apparatus obtains a final data set by taking the reference operation time and the reference data identity as an incoming aggregation interface and looping through the aggregation interface.
14. The apparatus of claim 13, wherein the current data segment is transaction data of a government transaction transmitted over an HTTP network.
15. An electronic device, comprising:
one or more processors;
storage means for storing one or more programs,
When executed by the one or more processors, causes the one or more processors to implement the method of any of claims 1-7.
16. A computer readable medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the method according to any of claims 1-7.
17. A computer program product comprising a computer program, characterized in that the program, when executed by a processor, implements the method according to any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210437349.6A CN115037729B (en) | 2022-04-21 | 2022-04-21 | Data aggregation method, device, electronic equipment and computer readable medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210437349.6A CN115037729B (en) | 2022-04-21 | 2022-04-21 | Data aggregation method, device, electronic equipment and computer readable medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115037729A CN115037729A (en) | 2022-09-09 |
| CN115037729B true CN115037729B (en) | 2024-05-28 |
Family
ID=83118583
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210437349.6A Active CN115037729B (en) | 2022-04-21 | 2022-04-21 | Data aggregation method, device, electronic equipment and computer readable medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115037729B (en) |
Citations (30)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5649195A (en) * | 1995-05-22 | 1997-07-15 | International Business Machines Corporation | Systems and methods for synchronizing databases in a receive-only network |
| JP2006155676A (en) * | 2006-03-17 | 2006-06-15 | Fujitsu Ltd | Database backup method for designated time and database backup device |
| CN1961313A (en) * | 2004-06-25 | 2007-05-09 | 苹果电脑有限公司 | Methods and systems for indexing files and adding associated metadata to index and metadata databases |
| CN101291209A (en) * | 2007-04-17 | 2008-10-22 | 大唐移动通信设备有限公司 | Method and apparatus for data synchronization |
| CN102831214A (en) * | 2006-10-05 | 2012-12-19 | 斯普兰克公司 | Time series search engine |
| CN103530290A (en) * | 2012-07-03 | 2014-01-22 | 深圳市腾讯计算机系统有限公司 | Method and system for data migration among databases |
| CN104270424A (en) * | 2014-09-17 | 2015-01-07 | 深圳创维数字技术有限公司 | Database synchronization method, server and system |
| CN106101256A (en) * | 2016-07-07 | 2016-11-09 | 百度在线网络技术（北京）有限公司 | Method and apparatus for synchrodata |
| CN106372664A (en) * | 2016-08-30 | 2017-02-01 | 成都路行通信息技术有限公司 | Picture aggregation method and device |
| CN106874389A (en) * | 2017-01-11 | 2017-06-20 | 腾讯科技（深圳）有限公司 | The moving method and device of data |
| CN107491458A (en) * | 2016-06-13 | 2017-12-19 | 阿里巴巴集团控股有限公司 | A kind of method and apparatus and system of storage time sequence data |
| US10216695B1 (en) * | 2017-09-21 | 2019-02-26 | Palantir Technologies Inc. | Database system for time series data storage, processing, and analysis |
| CN109558442A (en) * | 2018-11-19 | 2019-04-02 | 中国科学院信息工程研究所 | A kind of real-time assemblage method of data and system |
| CN109710698A (en) * | 2018-12-28 | 2019-05-03 | 北京明朝万达科技股份有限公司 | A kind of data assemblage method, device, electronic equipment and medium |
| CN110377784A (en) * | 2019-07-24 | 2019-10-25 | 广州酷狗计算机科技有限公司 | Sing single update method, device, terminal and storage medium |
| CN110765204A (en) * | 2019-09-30 | 2020-02-07 | 武汉达梦数据库有限公司 | Method and device for processing incremental synchronous abnormal interrupt condition |
| US10657018B1 (en) * | 2019-08-26 | 2020-05-19 | Coupang Corp. | Systems and methods for dynamic aggregation of data and minimization of data loss |
| CN111506581A (en) * | 2020-06-17 | 2020-08-07 | 北京北龙超级云计算有限责任公司 | Data aggregation method and server |
| CN111756794A (en) * | 2020-05-06 | 2020-10-09 | 上海明略人工智能(集团)有限公司 | Data synchronization method and mobile terminal |
| CN112291185A (en) * | 2019-07-24 | 2021-01-29 | 中国移动通信集团贵州有限公司 | Method and device for collecting network data |
| CN112307057A (en) * | 2020-10-27 | 2021-02-02 | 北京健康之家科技有限公司 | Data processing method and device, electronic equipment and computer storage medium |
| CN113192233A (en) * | 2021-04-29 | 2021-07-30 | 北京车和家信息技术有限公司 | Data acquisition method, device, equipment and medium |
| CN113420048A (en) * | 2021-05-19 | 2021-09-21 | 中交公规土木大数据信息技术（北京）有限公司 | Data aggregation method and device, electronic equipment and storage medium |
| CN113535856A (en) * | 2021-07-29 | 2021-10-22 | 上海哔哩哔哩科技有限公司 | Data synchronization method and system |
| CN113656511A (en) * | 2021-10-20 | 2021-11-16 | 天津南大通用数据技术股份有限公司 | Heterogeneous database increment synchronization method and system based on source database non-outage |
| CN113791955A (en) * | 2021-09-17 | 2021-12-14 | 济南浪潮数据技术有限公司 | Data aggregation device and method for monitoring system and server |
| CN113886408A (en) * | 2021-09-29 | 2022-01-04 | 北京字节跳动网络技术有限公司 | Data uploading method and device, computer equipment and storage medium |
| CN114116811A (en) * | 2022-01-29 | 2022-03-01 | 北京优特捷信息技术有限公司 | Log processing method, device, equipment and storage medium |
| CN114185920A (en) * | 2021-11-17 | 2022-03-15 | 南京星云数字技术有限公司 | Log data processing method and device, computer equipment and storage medium |
| CN114238523A (en) * | 2021-12-17 | 2022-03-25 | 蚂蚁区块链科技(上海)有限公司 | Data synchronization method and device |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP5774513B2 (en) * | 2012-02-07 | 2015-09-09 | 株式会社日立ソリューションズ | File list generation method and system, program, and file list generation device |
| US20190370800A1 (en) * | 2018-05-31 | 2019-12-05 | Visa International Service Association | Method, System, and Computer Program Product for Aggregating Data from a Plurality of Sources |
-
2022
- 2022-04-21 CN CN202210437349.6A patent/CN115037729B/en active Active
Patent Citations (30)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5649195A (en) * | 1995-05-22 | 1997-07-15 | International Business Machines Corporation | Systems and methods for synchronizing databases in a receive-only network |
| CN1961313A (en) * | 2004-06-25 | 2007-05-09 | 苹果电脑有限公司 | Methods and systems for indexing files and adding associated metadata to index and metadata databases |
| JP2006155676A (en) * | 2006-03-17 | 2006-06-15 | Fujitsu Ltd | Database backup method for designated time and database backup device |
| CN102831214A (en) * | 2006-10-05 | 2012-12-19 | 斯普兰克公司 | Time series search engine |
| CN101291209A (en) * | 2007-04-17 | 2008-10-22 | 大唐移动通信设备有限公司 | Method and apparatus for data synchronization |
| CN103530290A (en) * | 2012-07-03 | 2014-01-22 | 深圳市腾讯计算机系统有限公司 | Method and system for data migration among databases |
| CN104270424A (en) * | 2014-09-17 | 2015-01-07 | 深圳创维数字技术有限公司 | Database synchronization method, server and system |
| CN107491458A (en) * | 2016-06-13 | 2017-12-19 | 阿里巴巴集团控股有限公司 | A kind of method and apparatus and system of storage time sequence data |
| CN106101256A (en) * | 2016-07-07 | 2016-11-09 | 百度在线网络技术（北京）有限公司 | Method and apparatus for synchrodata |
| CN106372664A (en) * | 2016-08-30 | 2017-02-01 | 成都路行通信息技术有限公司 | Picture aggregation method and device |
| CN106874389A (en) * | 2017-01-11 | 2017-06-20 | 腾讯科技（深圳）有限公司 | The moving method and device of data |
| US10216695B1 (en) * | 2017-09-21 | 2019-02-26 | Palantir Technologies Inc. | Database system for time series data storage, processing, and analysis |
| CN109558442A (en) * | 2018-11-19 | 2019-04-02 | 中国科学院信息工程研究所 | A kind of real-time assemblage method of data and system |
| CN109710698A (en) * | 2018-12-28 | 2019-05-03 | 北京明朝万达科技股份有限公司 | A kind of data assemblage method, device, electronic equipment and medium |
| CN110377784A (en) * | 2019-07-24 | 2019-10-25 | 广州酷狗计算机科技有限公司 | Sing single update method, device, terminal and storage medium |
| CN112291185A (en) * | 2019-07-24 | 2021-01-29 | 中国移动通信集团贵州有限公司 | Method and device for collecting network data |
| US10657018B1 (en) * | 2019-08-26 | 2020-05-19 | Coupang Corp. | Systems and methods for dynamic aggregation of data and minimization of data loss |
| CN110765204A (en) * | 2019-09-30 | 2020-02-07 | 武汉达梦数据库有限公司 | Method and device for processing incremental synchronous abnormal interrupt condition |
| CN111756794A (en) * | 2020-05-06 | 2020-10-09 | 上海明略人工智能(集团)有限公司 | Data synchronization method and mobile terminal |
| CN111506581A (en) * | 2020-06-17 | 2020-08-07 | 北京北龙超级云计算有限责任公司 | Data aggregation method and server |
| CN112307057A (en) * | 2020-10-27 | 2021-02-02 | 北京健康之家科技有限公司 | Data processing method and device, electronic equipment and computer storage medium |
| CN113192233A (en) * | 2021-04-29 | 2021-07-30 | 北京车和家信息技术有限公司 | Data acquisition method, device, equipment and medium |
| CN113420048A (en) * | 2021-05-19 | 2021-09-21 | 中交公规土木大数据信息技术（北京）有限公司 | Data aggregation method and device, electronic equipment and storage medium |
| CN113535856A (en) * | 2021-07-29 | 2021-10-22 | 上海哔哩哔哩科技有限公司 | Data synchronization method and system |
| CN113791955A (en) * | 2021-09-17 | 2021-12-14 | 济南浪潮数据技术有限公司 | Data aggregation device and method for monitoring system and server |
| CN113886408A (en) * | 2021-09-29 | 2022-01-04 | 北京字节跳动网络技术有限公司 | Data uploading method and device, computer equipment and storage medium |
| CN113656511A (en) * | 2021-10-20 | 2021-11-16 | 天津南大通用数据技术股份有限公司 | Heterogeneous database increment synchronization method and system based on source database non-outage |
| CN114185920A (en) * | 2021-11-17 | 2022-03-15 | 南京星云数字技术有限公司 | Log data processing method and device, computer equipment and storage medium |
| CN114238523A (en) * | 2021-12-17 | 2022-03-25 | 蚂蚁区块链科技(上海)有限公司 | Data synchronization method and device |
| CN114116811A (en) * | 2022-01-29 | 2022-03-01 | 北京优特捷信息技术有限公司 | Log processing method, device, equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115037729A (en) | 2022-09-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108776934B (en) | Distributed data calculation method and device, computer equipment and readable storage medium | |
| CN111190888A (en) | Method and device for managing graph database cluster | |
| CN113190625B (en) | Request processing method, apparatus, electronic device, medium and program product | |
| CN111427701A (en) | Workflow engine system and business processing method | |
| WO2019109519A1 (en) | Service rule management method, apparatus and device, and computer-readable storage medium | |
| CN111044062A (en) | Path planning and recommending method and device | |
| CN114827280B (en) | Request processing method, device, equipment and medium | |
| CN113760522A (en) | Task processing method and device | |
| CN111159207B (en) | Information processing method and device | |
| CN113742376A (en) | Data synchronization method, first server and data synchronization system | |
| CN115037729B (en) | Data aggregation method, device, electronic equipment and computer readable medium | |
| CN112817687A (en) | Data synchronization method and device | |
| CN113760487B (en) | Service processing method and device | |
| CN112905273A (en) | Service calling method and device | |
| CN113778993A (en) | Service data processing method and device | |
| CN113760262A (en) | Task processing method, device, computer system and computer readable storage medium | |
| CN113779412A (en) | Message reach method, node and system based on block chain network | |
| CN113779122A (en) | Method and apparatus for exporting data | |
| CN113779048A (en) | Data processing method and device | |
| CN113741951A (en) | Local packaging method and device | |
| CN111178014A (en) | Method and device for processing business process | |
| CN113448652A (en) | Request processing method and device | |
| CN113127187A (en) | Method and apparatus for cluster scale-up | |
| CN115794353B (en) | Cloud network service quality optimization processing method, device, equipment and storage medium | |
| CN113778657B (en) | Data processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |