CN118227302B - Intermediate state time sequence coordination calculation method, device, equipment and storage medium - Google Patents
Intermediate state time sequence coordination calculation method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN118227302B CN118227302B CN202410658563.3A CN202410658563A CN118227302B CN 118227302 B CN118227302 B CN 118227302B CN 202410658563 A CN202410658563 A CN 202410658563A CN 118227302 B CN118227302 B CN 118227302B
- Authority
- CN
- China
- Prior art keywords
- time sequence
- node
- time
- target
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000004364 calculation method Methods 0.000 title claims abstract description 66
- 230000002776 aggregation Effects 0.000 claims abstract description 85
- 238000004220 aggregation Methods 0.000 claims abstract description 85
- 238000000034 method Methods 0.000 claims abstract description 29
- 238000004590 computer program Methods 0.000 claims description 14
- 230000005540 biological transmission Effects 0.000 claims description 3
- 238000012163 sequencing technique Methods 0.000 claims description 2
- 238000007405 data analysis Methods 0.000 abstract description 2
- 238000004891 communication Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 6
- 238000004422 calculation algorithm Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000006116 polymerization reaction Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application discloses a method, a device, equipment and a storage medium for intermediate state time sequence coordination calculation, which relate to the field of data analysis and comprise the following steps: generating an ordered node set corresponding to the computing nodes through the time sequence scheduling nodes, and determining a time segment corresponding to each computing node in the ordered node set; determining a target time segment corresponding to each piece of time sequence data to be processed, and sending each piece of time sequence data to a target computing node corresponding to the target time segment; merging time sequence data to be processed based on dimension information by a target computing node to obtain a plurality of time sequence aggregation state objects, and respectively sending the plurality of time sequence aggregation state objects to corresponding summarizing nodes based on the dimension information; and merging all time sequence aggregation state objects in the nodes through the aggregation node to obtain a target aggregation result. Therefore, the computing resources can be utilized to the maximum extent, and the performance and stability of the aggregation computing can be improved.
Description
Technical Field
The present invention relates to the field of data analysis, and in particular, to a method, an apparatus, a device, and a storage medium for intermediate state timing coordination calculation.
Background
The time sequence intermediate state is characterized by being an intermediate state data structure with a time axis, dimensions and results, and the data structure can finish the work of merging, time sequence calculation for summation, average value, increment, decrement and the like. The method is characterized in that a plurality of time sequence intermediate state results can be formed by a plurality of complex algorithms such as average value, increment and decrement, and the like, and the plurality of time sequence intermediate state results are combined in a segmented mode. The computation of a single time-series intermediate state must ensure the time-series property of data, and the merging of a plurality of time-series intermediate state objects must also ensure the time-series property. On the basis, the accuracy of calculation can be ensured through the data structure.
However, in the prior art, when performing aggregate index calculation, grouping calculation is performed according to dimensions, and calculation of each dimension must be completed by a single node and thread. And for complex time sequence calculation such as increment, decrement and the like, linear calculation is carried out on a single dimension after data are ordered. Therefore, when data in a certain dimension is too much and data inclination occurs, the calculation performance and stability are greatly reduced.
Disclosure of Invention
In view of the above, the present invention aims to provide a method, a device and a storage medium for intermediate state time sequence coordination calculation, which can be based on innovation of a real-time intelligent technology system, thereby realizing maximum utilization of calculation resources and improving performance and stability of aggregation calculation. The specific scheme is as follows:
In a first aspect, the application discloses a method for calculating intermediate state time sequence coordination, which comprises the following steps:
generating an ordered node set corresponding to the computing nodes through a time sequence scheduling node, and determining a time segment corresponding to each computing node in the ordered node set;
Determining a target time segment corresponding to each piece of time sequence data to be processed, and sending each piece of time sequence data to be processed to a target computing node corresponding to the target time segment;
combining the time sequence data to be processed based on dimension information by the target computing node to obtain a plurality of time sequence aggregation state objects, and respectively transmitting the plurality of time sequence aggregation state objects to corresponding summarizing nodes based on the dimension information;
and merging all time sequence aggregation state objects in the nodes through the summarizing node to obtain a target aggregation result.
Optionally, the generating, by the timing schedule node, the ordered node set corresponding to the computing node includes:
Node information of a computing node is received through a time sequence scheduling node, the computing node is registered through the node information, and an ordered node set corresponding to the computing node is generated based on the node information.
Optionally, the determining the time segment corresponding to each computing node in the ordered node set includes:
marking the computing nodes in the ordered node set based on the node sequence corresponding to the ordered node set and a preset time range so as to determine the time segment which each computing node is responsible for and obtain a first corresponding relation between each computing node and the time segment.
Optionally, the determining a target time segment corresponding to each piece of time sequence data to be processed, so as to send each piece of time sequence data to be processed to a target computing node corresponding to the target time segment, includes:
traversing the time sequence data to be processed through the time sequence scheduling node so as to determine a target time segment corresponding to the time sequence data to be processed according to a time stamp corresponding to the time sequence data;
and determining a target computing node corresponding to the target time segment based on the first corresponding relation, and sending the time sequence data to be processed to the target computing node.
Optionally, the merging, by the target computing node, the time sequence data to be processed based on dimension information to obtain a plurality of time sequence aggregate state objects, and sending the plurality of time sequence aggregate state objects to corresponding summary nodes based on the dimension information respectively, including:
Sequencing the time sequence data to be processed based on time sequence by the target computing node so as to obtain sequenced time sequence data;
Determining dimension information corresponding to the sequenced time sequence data, and merging the time sequence data to be processed based on the dimension information to obtain a plurality of time sequence aggregation state objects;
And respectively sending the time sequence aggregation state objects to corresponding summarizing nodes based on the dimension information.
Optionally, the sending the plurality of time sequence aggregate state objects to corresponding summary nodes based on the dimension information includes:
Carrying out hash calculation on the dimension information to determine a hash value corresponding to the dimension information;
And determining a second corresponding relation between the dimension information and the summarizing nodes through the hash value, so as to respectively send the time sequence aggregation state objects to the corresponding summarizing nodes according to the second corresponding relation.
Optionally, the merging, by the summary node, all the time sequence aggregation state objects in the node to obtain a target aggregation result includes:
Sorting all received time sequence aggregation state objects based on time sequence through the summarizing node, and merging the obtained sorted time sequence aggregation state objects to obtain a target aggregation result; and the time sequence aggregation state objects in the aggregation nodes are time sequence aggregation state objects with the same latitude.
In a second aspect, the present application discloses an intermediate state timing coordination calculating device, including:
The time segment determining module is used for generating an ordered node set corresponding to the computing nodes through the time sequence scheduling nodes and determining time segments corresponding to each computing node in the ordered node set;
the computing node determining module is used for determining a target time segment corresponding to each piece of time sequence data to be processed, so that each piece of time sequence data to be processed is sent to a target computing node corresponding to the target time segment;
The object sending module is used for merging the time sequence data to be processed based on dimension information through the target computing node to obtain a plurality of time sequence aggregation state objects, and sending the plurality of time sequence aggregation state objects to corresponding summarizing nodes based on the dimension information respectively;
and the object merging module is used for merging all time sequence aggregation state objects in the node through the summarizing node so as to obtain a target aggregation result.
In a third aspect, the present application discloses an electronic device, comprising:
a memory for storing a computer program;
And a processor for executing the computer program to implement the intermediate state timing coordination calculation method as described above.
In a fourth aspect, the present application discloses a computer readable storage medium storing a computer program which, when executed by a processor, implements an intermediate state timing coordination calculation method as described above.
According to the method, firstly, an ordered node set corresponding to a computing node is generated through a time sequence scheduling node, time segments corresponding to each computing node in the ordered node set are determined, then a target time segment corresponding to each piece of time sequence data to be processed is determined, each piece of time sequence data to be processed is sent to a target computing node corresponding to the target time segment, the time sequence data to be processed is combined through the target computing node based on dimension information, so that a plurality of time sequence aggregation state objects are obtained, the plurality of time sequence aggregation state objects are respectively sent to corresponding summarizing nodes based on the dimension information, and finally all time sequence aggregation state objects in the nodes are combined through the summarizing nodes, so that a target aggregation result is obtained. Therefore, the method can generate the ordered node set of the computing nodes through the time sequence scheduling node, determine the time segment corresponding to each computing node, and then send each piece of time sequence data to the corresponding target computing node after determining the target time segment corresponding to each piece of time sequence data to be processed, so that the target computing node can combine the time sequence data to be processed in the node based on the dimension information, send a plurality of obtained time sequence aggregation state objects to the corresponding summarizing node based on the dimension information, and then combine all the time sequence aggregation state objects in the node through the summarizing node to obtain the final target aggregation result. In this way, the time sequence coordination processing is carried out on the intermediate state result, the concept of time is introduced into the calculation process except the dimension, so that the data can be uniformly dispersed to each node for calculation, and the accuracy of calculation is ensured. Through the dimension and time double concepts, the computing resources can be utilized to the greatest extent, and the performance and stability of the aggregate computing are improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are required to be used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only embodiments of the present invention, and that other drawings can be obtained according to the provided drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flowchart of an intermediate state timing coordination calculation method disclosed by the application;
FIG. 2 is a timing diagram of an intermediate timing coordination calculation method according to the present application;
FIG. 3 is a schematic diagram of an intermediate timing coordination computing device according to the present disclosure;
Fig. 4 is a block diagram of an electronic device according to the present disclosure.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
In the prior art, when performing aggregate index calculation, grouping calculation is performed according to dimensions, and calculation of each dimension must be completed by a single node and thread. And for complex time sequence calculation such as increment, decrement and the like, linear calculation is carried out on a single dimension after data are ordered. Therefore, when data in a certain dimension is too much and data inclination occurs, the calculation performance and stability are greatly reduced.
In order to overcome the technical problems, the application provides a method, a device, equipment and a storage medium for intermediate state time sequence coordination calculation, which are based on a data structure of a time sequence intermediate state, introduce two concepts of time and dimension to the calculation of data, enable the data to be uniformly distributed on each node for parallel calculation, furthest utilize calculation resources and improve the performance and stability of aggregate calculation.
Referring to fig. 1, the embodiment of the invention discloses a method for calculating intermediate state time sequence coordination, which comprises the following steps:
And S11, generating an ordered node set corresponding to the computing nodes through a time sequence scheduling node, and determining a time segment corresponding to each computing node in the ordered node set.
In this embodiment, an ordered node set corresponding to a computing node needs to be generated by a time sequence scheduling node, specifically, a time sequence scheduling node may be defined, and all computing nodes need to complete information registration with the node after being started, so that node information of the computing nodes may be received by the time sequence scheduling node, after all node information is received, all computing nodes may be registered according to the received node information, and after registration is completed, an ordered node set p= { P1, P2..pn } corresponding to the computing nodes may be generated according to the node information, so that a list storing all computing node information may be maintained by the time sequence scheduling node.
It should be noted that after the ordered node set is generated, the nodes need to be marked sequentially to determine a time segment corresponding to each computing node in the ordered node set, specifically, the computing nodes in the ordered node set may be marked based on the node sequence corresponding to the ordered node set and a preset time range, for example, the nodes are marked sequentially within a time range of data through a specified time window, such as 1 hour, p= { (P1, t 1), (P2, t 2) } (Pn, tn) }, all the nodes are marked from the first hour, node 1 corresponds to 0 point, node 2 corresponds to 1 point, and when the node is marked, the corresponding time is circularly marked from the first node to determine the time segment responsible for each computing node, and determine the first correspondence between each computing node and the time segment.
Step S12, determining a target time segment corresponding to each piece of time sequence data to be processed, and sending each piece of time sequence data to be processed to a target computing node corresponding to the target time segment.
In this embodiment, after marking all the nodes is completed, the time sequence scheduling node may traverse the time sequence data to determine a target time segment corresponding to each piece of time sequence data to be processed, and then send each piece of time sequence data to the corresponding node according to the determined target time segment, specifically, as shown in fig. 2, the time sequence scheduling node may traverse the time sequence data to be processed to determine the target time segment corresponding to the time sequence data according to the timestamp corresponding to the time sequence data, and then send the data to be processed to the corresponding target computing node according to the determined first correspondence between each computing node and the time segment. For example, when the computing node corresponding to the timestamp of the data 1 and the timestamp of the data 2 is determined to be a through scanning, the data 1 and the data 2 are sent to the computing node a.
And step S13, merging the time sequence data to be processed based on dimension information by the target computing node to obtain a plurality of time sequence aggregation state objects, and respectively sending the plurality of time sequence aggregation state objects to corresponding summarizing nodes based on the dimension information.
In this embodiment, each target computing node may combine the time-series data received by itself to obtain a plurality of time-series intermediate states, specifically, as shown in fig. 2, four target computing nodes, namely, a target computing node a, a target computing node b, a target computing node c, and a target computing node d, after all the time-series data are received, may sort the time-series data received by itself according to a time sequence to obtain sorted time-series data, then it is required to determine dimension information corresponding to the sorted time-series data, where the dimension information is a node sending data, the data sent by the same node may be regarded as being at the same latitude, and then the time-series data to be processed according to dimension information are combined to obtain a plurality of time-series aggregate state objects. In this way, each target computing node includes one or more time-ordered time-sequence aggregate objects, and then, according to the determined dimension information, a plurality of time-sequence aggregate objects are sent to the corresponding summary nodes.
It should be noted that, the specific process of sending the plurality of time sequence aggregate objects to the corresponding summary nodes based on the dimension information is as follows: and carrying out hash calculation on the dimension information to determine a hash value corresponding to the dimension information, determining a second corresponding relation between the dimension information and the summarization nodes through the hash value, and respectively transmitting a plurality of time sequence aggregation state objects of each target calculation node to the corresponding summarization nodes according to the second corresponding relation by each target calculation node. As shown in fig. 2, the target computing node a and the target computing node d send their own several sequential aggregate state objects to the same summary node, and the target computing node b and the target computing node c send their own several sequential aggregate state objects to the same summary node.
And step S14, merging all time sequence aggregation state objects in the nodes through the summarizing node to obtain a target aggregation result.
In this embodiment, the summary node may combine all the sequential aggregation state objects in its own node to obtain the target aggregation result, and specifically, the summary node needs to sort all the received sequential aggregation state objects based on time sequence, and then combine the sorted sequential aggregation state objects to obtain the target aggregation result. And after distributing all the time-series intermediate state objects to the corresponding nodes according to dimensions, all that is transmitted at this time is the time-series intermediate state objects rather than the original data. The merging of the time sequence intermediate states of the current node also needs to ensure the sequential merging in the same dimension. As shown in fig. 2, the target polymerization results obtained after the merging are ad and bc, respectively. Therefore, the time sequence intermediate state object can be used for completing the segmentation calculation and the secondary merging operation of complex operators such as summation, average value, increment, decrement and the like, so that the calculation pressure of the nodes is effectively reduced, and the calculation efficiency is improved.
Therefore, the method can generate the ordered node set of the computing nodes through the time sequence scheduling node, determine the time segment corresponding to each computing node, and then send each piece of time sequence data to the corresponding target computing node after determining the target time segment corresponding to each piece of time sequence data to be processed, so that the target computing node can combine the time sequence data to be processed in the node based on the dimension information, send a plurality of obtained time sequence aggregation state objects to the corresponding summarizing node based on the dimension information, and then combine all the time sequence aggregation state objects in the node through the summarizing node to obtain the final target aggregation result. On one hand, the time sequence coordination processing is carried out on the intermediate state result, the concept of time is introduced into the calculation process except the dimension, so that data can be uniformly dispersed to each node for calculation, the calculation accuracy is ensured, the calculation resource can be furthest utilized through the dimension and time double concept, and the performance and stability of the aggregate calculation are improved; on the other hand, the problem that serial calculation is needed for the time sequence data in the same dimension can be effectively solved, calculation resources can be fully utilized through the design of time sequence scheduling, the time sequence data can be calculated in parallel, and the calculation accuracy is guaranteed; on the other hand, the calculation dependency of time sequence data on dimension data can be eliminated, parallel calculation can be performed no matter whether the plurality of pieces of data are in the same dimension or not, and the calculation pressure of a single node when the data are inclined is reduced by maximally utilizing program resources.
Referring to fig. 3, an embodiment of the present invention discloses an intermediate state timing coordination computing device, which includes:
A time segment determining module 11, configured to generate an ordered node set corresponding to a computing node through a time sequence scheduling node, and determine a time segment corresponding to each computing node in the ordered node set;
The computing node determining module 12 is configured to determine a target time segment corresponding to each piece of time-series data to be processed, so as to send each piece of time-series data to a target computing node corresponding to the target time segment;
The object sending module 13 is configured to combine, by the target computing node, the time-series data to be processed based on dimension information, so as to obtain a plurality of time-series aggregate state objects, and send the plurality of time-series aggregate state objects to corresponding aggregation nodes based on the dimension information respectively;
and the object merging module 14 is configured to merge all time sequence aggregation state objects in the node through the aggregation node so as to obtain a target aggregation result.
According to the method, firstly, an ordered node set corresponding to a computing node is generated through a time sequence scheduling node, time segments corresponding to each computing node in the ordered node set are determined, then a target time segment corresponding to each piece of time sequence data to be processed is determined, each piece of time sequence data to be processed is sent to a target computing node corresponding to the target time segment, the time sequence data to be processed is combined through the target computing node based on dimension information, so that a plurality of time sequence aggregation state objects are obtained, the plurality of time sequence aggregation state objects are respectively sent to corresponding summarizing nodes based on the dimension information, and finally all time sequence aggregation state objects in the nodes are combined through the summarizing nodes, so that a target aggregation result is obtained. Therefore, the method can generate the ordered node set of the computing nodes through the time sequence scheduling node, determine the time segment corresponding to each computing node, and then send each piece of time sequence data to the corresponding target computing node after determining the target time segment corresponding to each piece of time sequence data to be processed, so that the target computing node can combine the time sequence data to be processed in the node based on the dimension information, send a plurality of obtained time sequence aggregation state objects to the corresponding summarizing node based on the dimension information, and then combine all the time sequence aggregation state objects in the node through the summarizing node to obtain the final target aggregation result. In this way, the time sequence coordination processing is carried out on the intermediate state result, the concept of time is introduced into the calculation process except the dimension, so that the data can be uniformly dispersed to each node for calculation, and the accuracy of calculation is ensured. Through the dimension and time double concepts, the computing resources can be utilized to the greatest extent, and the performance and stability of the aggregate computing are improved.
In some embodiments, the time segment determining module 11 may specifically include:
And the node set generating unit is used for receiving node information of the computing nodes through the time sequence scheduling nodes, registering the computing nodes through the node information and generating an ordered node set corresponding to the computing nodes based on the node information.
In some embodiments, the time segment determining module 11 may specifically include:
the time segment determining unit is used for marking the computing nodes in the ordered node set based on the node sequence corresponding to the ordered node set and a preset time range so as to determine the time segment which each computing node is responsible for and obtain a first corresponding relation between each computing node and the time segment.
In some embodiments, the computing node determining module 12 may specifically include:
The time segment determining unit is used for traversing the time sequence data to be processed through the time sequence scheduling node so as to determine a target time segment corresponding to the time sequence data to be processed according to a time stamp corresponding to the time sequence data;
And the data sending unit is used for determining a target computing node corresponding to the target time segment based on the first corresponding relation and sending the time sequence data to be processed to the target computing node.
In some embodiments, the object sending module 13 may specifically include:
The data sorting sub-module is used for sorting the time sequence data to be processed based on time sequence through the target computing node so as to obtain time sequence data after sorting;
The data merging sub-module is used for determining dimension information corresponding to the sequenced time sequence data so as to merge the time sequence data to be processed based on the dimension information, so as to obtain a plurality of time sequence aggregate state objects;
and the data transmission sub-module is used for respectively transmitting the plurality of time sequence aggregation state objects to the corresponding summarizing nodes based on the dimension information.
In some embodiments, the data sending sub-module may specifically include:
The hash calculation unit is used for carrying out hash calculation on the dimension information so as to determine a hash value corresponding to the dimension information;
and the data sending unit is used for determining a second corresponding relation between the dimension information and the summarizing nodes through the hash value so as to respectively send the plurality of time sequence aggregation state objects to the corresponding summarizing nodes according to the second corresponding relation.
In some embodiments, the object merging module 14 may specifically include:
The object merging unit is used for sorting all received time sequence aggregation state objects based on time sequence through the summarizing node, and merging the obtained time sequence aggregation state objects after sorting to obtain a target aggregation result; and the time sequence aggregation state objects in the aggregation nodes are time sequence aggregation state objects with the same latitude.
Further, the embodiment of the present application further discloses an electronic device, and fig. 4 is a block diagram of an electronic device 20 according to an exemplary embodiment, where the content of the diagram is not to be considered as any limitation on the scope of use of the present application.
Fig. 4 is a schematic structural diagram of an electronic device 20 according to an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input output interface 25, and a communication bus 26. The memory 22 is configured to store a computer program, which is loaded and executed by the processor 21 to implement relevant steps in the intermediate state timing coordination calculation method disclosed in any of the foregoing embodiments. In addition, the electronic device 20 in the present embodiment may be specifically an electronic computer.
In this embodiment, the power supply 23 is configured to provide an operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and an external device, and the communication protocol in which the communication interface is in compliance is any communication protocol applicable to the technical solution of the present application, which is not specifically limited herein; the input/output interface 25 is used for acquiring external input data or outputting external output data, and the specific interface type thereof may be selected according to the specific application requirement, which is not limited herein.
The memory 22 may be a carrier for storing resources, such as a read-only memory, a random access memory, a magnetic disk, or an optical disk, and the resources stored thereon may include an operating system 221, a computer program 222, and the like, and the storage may be temporary storage or permanent storage.
The operating system 221 is used for managing and controlling various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, netware, unix, linux, etc. The computer program 222 may further include a computer program that can be used to perform other specific tasks in addition to the computer program that can be used to perform the intermediate state timing coordination calculation method performed by the electronic device 20 as disclosed in any of the previous embodiments.
Further, the application also discloses a computer readable storage medium for storing a computer program; the computer program, when executed by the processor, implements the intermediate state time sequence coordination calculation method disclosed in the foregoing. For specific steps of the method, reference may be made to the corresponding contents disclosed in the foregoing embodiments, and no further description is given here.
In this specification, each embodiment is described in a progressive manner, and each embodiment is mainly described in a different point from other embodiments, so that the same or similar parts between the embodiments are referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant points refer to the description of the method section.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative elements and steps are described above generally in terms of functionality in order to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. The software modules may be disposed in Random Access Memory (RAM), memory, read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
Finally, it is further noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The foregoing has outlined rather broadly the more detailed description of the application in order that the detailed description of the application that follows may be better understood, and in order that the present principles and embodiments may be better understood; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in accordance with the ideas of the present application, the present description should not be construed as limiting the present application in view of the above.
Claims (8)
1. An intermediate state timing coordination calculation method is characterized by comprising the following steps:
generating an ordered node set corresponding to the computing nodes through a time sequence scheduling node, and determining a time segment corresponding to each computing node in the ordered node set;
Determining a target time segment corresponding to each piece of time sequence data to be processed, and sending each piece of time sequence data to be processed to a target computing node corresponding to the target time segment;
combining the time sequence data to be processed based on dimension information by the target computing node to obtain a plurality of time sequence aggregation state objects, and respectively transmitting the plurality of time sequence aggregation state objects to corresponding summarizing nodes based on the dimension information;
Combining all time sequence aggregation state objects in the node through the summarizing node to obtain a target aggregation result;
the step of merging the time sequence data to be processed by the target computing node based on dimension information to obtain a plurality of time sequence aggregate state objects, and respectively sending the plurality of time sequence aggregate state objects to corresponding summary nodes based on the dimension information comprises the following steps:
Sequencing the time sequence data to be processed based on time sequence by the target computing node so as to obtain sequenced time sequence data;
Determining dimension information corresponding to the sequenced time sequence data, and merging the time sequence data to be processed based on the dimension information to obtain a plurality of time sequence aggregation state objects;
respectively transmitting the plurality of time sequence aggregation state objects to corresponding summarizing nodes based on the dimension information;
the step of sending the plurality of time sequence aggregate objects to corresponding summary nodes based on the dimension information respectively includes:
Carrying out hash calculation on the dimension information to determine a hash value corresponding to the dimension information;
And determining a second corresponding relation between the dimension information and the summarizing nodes through the hash value, so as to respectively send the time sequence aggregation state objects to the corresponding summarizing nodes according to the second corresponding relation.
2. The intermediate state timing coordination computing method of claim 1, wherein the generating, by the timing scheduling node, an ordered set of nodes corresponding to the computing nodes comprises:
Node information of a computing node is received through a time sequence scheduling node, the computing node is registered through the node information, and an ordered node set corresponding to the computing node is generated based on the node information.
3. The method of claim 1, wherein determining the time segment corresponding to each computing node in the ordered set of nodes comprises:
marking the computing nodes in the ordered node set based on the node sequence corresponding to the ordered node set and a preset time range so as to determine the time segment which each computing node is responsible for and obtain a first corresponding relation between each computing node and the time segment.
4. The method of claim 3, wherein determining a target time segment corresponding to each piece of time-series data to be processed to send each piece of time-series data to a target computing node corresponding to the target time segment comprises:
traversing the time sequence data to be processed through the time sequence scheduling node so as to determine a target time segment corresponding to the time sequence data to be processed according to a time stamp corresponding to the time sequence data;
and determining a target computing node corresponding to the target time segment based on the first corresponding relation, and sending the time sequence data to be processed to the target computing node.
5. The method according to any one of claims 1 to 4, wherein the merging, by the aggregation node, all time-sequential aggregate state objects in a node to obtain a target aggregate result includes:
Sorting all received time sequence aggregation state objects based on time sequence through the summarizing node, and merging the obtained sorted time sequence aggregation state objects to obtain a target aggregation result; and the time sequence aggregation state objects in the aggregation nodes are time sequence aggregation state objects with the same latitude.
6. An intermediate state timing coordination computing device, comprising:
The time segment determining module is used for generating an ordered node set corresponding to the computing nodes through the time sequence scheduling nodes and determining time segments corresponding to each computing node in the ordered node set;
the computing node determining module is used for determining a target time segment corresponding to each piece of time sequence data to be processed, so that each piece of time sequence data to be processed is sent to a target computing node corresponding to the target time segment;
The object sending module is used for merging the time sequence data to be processed based on dimension information through the target computing node to obtain a plurality of time sequence aggregation state objects, and sending the plurality of time sequence aggregation state objects to corresponding summarizing nodes based on the dimension information respectively;
The object merging module is used for merging all time sequence aggregation state objects in the node through the summarizing node so as to obtain a target aggregation result;
wherein, the object sending module includes:
The data sorting sub-module is used for sorting the time sequence data to be processed based on time sequence through the target computing node so as to obtain time sequence data after sorting;
The data merging sub-module is used for determining dimension information corresponding to the sequenced time sequence data so as to merge the time sequence data to be processed based on the dimension information, so as to obtain a plurality of time sequence aggregate state objects;
the data transmission sub-module is used for respectively transmitting the plurality of time sequence aggregation state objects to corresponding summarizing nodes based on the dimension information;
wherein, the data send submodule includes:
The hash calculation unit is used for carrying out hash calculation on the dimension information so as to determine a hash value corresponding to the dimension information;
and the data sending unit is used for determining a second corresponding relation between the dimension information and the summarizing nodes through the hash value so as to respectively send the plurality of time sequence aggregation state objects to the corresponding summarizing nodes according to the second corresponding relation.
7. An electronic device, comprising:
a memory for storing a computer program;
A processor for executing the computer program to implement the intermediate state timing coordination calculation method of any one of claims 1 to 5.
8. A computer readable storage medium for storing a computer program which when executed by a processor implements the intermediate state timing coordination calculation method of any of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410658563.3A CN118227302B (en) | 2024-05-27 | 2024-05-27 | Intermediate state time sequence coordination calculation method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410658563.3A CN118227302B (en) | 2024-05-27 | 2024-05-27 | Intermediate state time sequence coordination calculation method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN118227302A CN118227302A (en) | 2024-06-21 |
| CN118227302B true CN118227302B (en) | 2024-08-06 |
Family
ID=91506506
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410658563.3A Active CN118227302B (en) | 2024-05-27 | 2024-05-27 | Intermediate state time sequence coordination calculation method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118227302B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106202384A (en) * | 2016-07-08 | 2016-12-07 | 清华大学 | A kind of indexing means supporting time series data aggregate function |
| CN108093273A (en) * | 2016-11-22 | 2018-05-29 | 北京国双科技有限公司 | Sort method, the apparatus and system of data |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10244551B2 (en) * | 2017-05-10 | 2019-03-26 | Mitsubishi Electric Research Laboratories, Inc. | Scheduling of delay sensitive convergecast network |
| CN109117275B (en) * | 2018-08-31 | 2024-05-28 | 平安科技（深圳）有限公司 | Account checking method and device based on data slicing, computer equipment and storage medium |
| CN111400008B (en) * | 2020-03-13 | 2023-06-02 | 北京旷视科技有限公司 | Computing resource scheduling method and device and electronic equipment |
| CN116015609A (en) * | 2022-12-12 | 2023-04-25 | 中国工商银行股份有限公司 | Business record storage method, device, computer equipment and storage medium |
| CN116561171B (en) * | 2023-07-10 | 2023-09-15 | 浙江邦盛科技股份有限公司 | Method, device, equipment and medium for processing dual-time-sequence distribution of inclination data |
-
2024
- 2024-05-27 CN CN202410658563.3A patent/CN118227302B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106202384A (en) * | 2016-07-08 | 2016-12-07 | 清华大学 | A kind of indexing means supporting time series data aggregate function |
| CN108093273A (en) * | 2016-11-22 | 2018-05-29 | 北京国双科技有限公司 | Sort method, the apparatus and system of data |
Also Published As
| Publication number | Publication date |
|---|---|
| CN118227302A (en) | 2024-06-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10114682B2 (en) | Method and system for operating a data center by reducing an amount of data to be processed | |
| CN110740103A (en) | Service request processing method and device, computer equipment and storage medium | |
| CN105900064A (en) | Method and apparatus for scheduling data flow task | |
| CN112115133A (en) | Distributed global unique ID generation method and system, storage medium and device | |
| CN107045459A (en) | A kind of O＆M request processing method and device based on ansible | |
| CN111667141A (en) | Pending task case processing method, device, equipment and storage medium | |
| CN116383248A (en) | Data query task acceleration method, device, equipment and storage medium | |
| CN114189818B (en) | Message sending method, device, server and storage medium | |
| CN118227302B (en) | Intermediate state time sequence coordination calculation method, device, equipment and storage medium | |
| CN116820709B (en) | Task chain operation method, device, terminal and computer storage medium | |
| CN112711683A (en) | Data comparison method and device and computer equipment | |
| CN116700929A (en) | Task batch processing method and system based on artificial intelligence | |
| CN111026972A (en) | Subscription data pushing method, device, equipment and storage medium in Internet of things | |
| CN115456188B (en) | Quantum computing task optimization processing method and device and quantum computer | |
| CN114221883B (en) | Message testing method, device, server and storage medium | |
| CN113114612B (en) | Determination method and device for distributed system call chain | |
| CN115086343A (en) | Internet of things data interaction method and system based on artificial intelligence | |
| CN114547384A (en) | Resource object processing method and device and computer equipment | |
| CN111831425B (en) | Data processing method, device and equipment | |
| CN112925831A (en) | Big data mining method and big data mining service system based on cloud computing service | |
| CN113472896B (en) | Public cloud-based data transmission method | |
| CN117555663B (en) | Task scheduling system based on License management | |
| CN116644215B (en) | Data updating method and system of cross-organization structure | |
| CN116361319B (en) | Database query method, device, equipment and storage medium | |
| CN116107724B (en) | AI (advanced technology attachment) acceleration core scheduling management method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |