CN115080577A - Data statistics period switching method, device, equipment and storage medium - Google Patents
Data statistics period switching method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN115080577A CN115080577A CN202210584801.1A CN202210584801A CN115080577A CN 115080577 A CN115080577 A CN 115080577A CN 202210584801 A CN202210584801 A CN 202210584801A CN 115080577 A CN115080577 A CN 115080577A
- Authority
- CN
- China
- Prior art keywords
- request
- switching
- preset
- data processing
- thread
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 45
- 238000012545 processing Methods 0.000 claims abstract description 175
- 238000012544 monitoring process Methods 0.000 claims abstract description 113
- 230000002159 abnormal effect Effects 0.000 claims description 9
- 230000000737 periodic effect Effects 0.000 claims description 9
- 238000007781 pre-processing Methods 0.000 claims description 7
- 230000003139 buffering effect Effects 0.000 claims 1
- 238000010586 diagram Methods 0.000 description 7
- 230000001960 triggered effect Effects 0.000 description 4
- 238000006243 chemical reaction Methods 0.000 description 3
- 238000013500 data storage Methods 0.000 description 3
- 230000005856 abnormality Effects 0.000 description 2
- 238000013139 quantization Methods 0.000 description 2
- 238000011010 flushing procedure Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2379—Updates performed during online database operations; commit processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2462—Approximate or statistical queries
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Probability & Statistics with Applications (AREA)
- Fuzzy Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention relates to the field of big data and discloses a data statistics period switching method, a device, equipment and a storage medium. The method comprises the following steps: when a cycle switching instruction sent by a terminal is monitored, performing thread switching on processing of a data processing request in a preset request queue according to the cycle switching instruction; sequentially warehousing request data corresponding to the data processing request in the switched thread, and monitoring the request state of the switched thread through a preset global counter; acquiring a monitoring result of a request state corresponding to the global counter, and judging whether the global counter meets a preset cycle switching condition or not according to the monitoring result; and if the global counter meets the period switching condition, setting the current time as the switching time of the current period, and marking the input request data by adopting the switching time. The invention realizes the real-time switching of the statistical period in the data statistical process and avoids the data in the current statistical period from being omitted and put in storage.
Description
Technical Field
The present invention relates to the field of big data, and in particular, to a method, an apparatus, a device, and a storage medium for data statistics period switching.
Background
In some statistical scenarios, switching needs to be performed at a time point T controlled by a certain service staff under the condition that a service request is not interrupted, a request received before the time point T is divided into a current statistical period, a request after the time point T is divided into a new statistical period, and data of the current statistical period is generated.
In the industry, the request time stored in the database is generally used for dividing, when the concurrency is large, part of data of which the request time belongs to the current statistical period may not be processed and stored in a warehouse in the statistical process, and the next statistical period is also ignored, so that part of data is missed. Namely, the problem that the data storage is easy to miss due to the switching of the existing data statistics period.
Disclosure of Invention
The invention mainly aims to solve the technical problem that the data storage is easy to miss due to the switching of the conventional data statistics period.
The first aspect of the present invention provides a data statistics period switching method, including: when a cycle switching instruction sent by a terminal is monitored, performing thread switching on processing of a data processing request in a preset request queue according to the cycle switching instruction; sequentially warehousing request data corresponding to the data processing request in the switched thread, and monitoring the request state of the switched thread through a preset global counter; acquiring a monitoring result of a request state corresponding to the global counter, and judging whether the global counter meets a preset period switching condition or not according to the monitoring result; and if the global counter meets the period switching condition, setting the current time as the switching time of the current period, and marking the input request data by adopting the switching time.
Optionally, in a first implementation manner of the first aspect of the present invention, after performing thread switching on processing of a data processing request in a preset request queue according to a cycle switching instruction when the cycle switching instruction sent by the terminal is monitored, the method further includes: acquiring a new data processing request, and judging whether the new data processing request meets a preset preprocessing condition or not; if yes, acquiring request data corresponding to the new data processing request through a preset first thread, and storing the request data corresponding to the new data processing request into a preset temporary data table; and if not, returning the processing exception prompt information corresponding to the new data processing request.
Optionally, in a second implementation manner of the first aspect of the present invention, when a cycle switching instruction sent by a terminal is monitored, performing thread switching on processing of a data processing request in a preset request queue according to the cycle switching instruction includes: when a cycle switching instruction sent by the terminal is monitored, analyzing the cycle switching instruction to obtain a memory address corresponding to a second thread for processing a data processing request in a preset request queue; and searching a corresponding register according to the memory address, and updating a thread pointer and a register pointer in the register so as to perform thread switching on the processing of the data processing request.
Optionally, in a third implementation manner of the first aspect of the present invention, the monitoring the request state of the switched thread by presetting a global counter includes: monitoring each state monitoring factor in the switched thread by presetting a global counter, and counting the count corresponding to each state monitoring factor; generating a global variable corresponding to the thread after the monitoring of the global counter is switched based on counting corresponding to each state monitoring factor; and monitoring the request state of the switched thread by using a global counter according to the global variable.
Optionally, in a fourth implementation manner of the first aspect of the present invention, the different types of state monitoring factors include a first factor, a second factor, and a third factor, each state monitoring factor in the switched thread is monitored by presetting a global counter, and counting counts corresponding to each state monitoring factor includes: monitoring the number of data processing requests in the switched threads by presetting a global counter, and determining the count corresponding to the first factor according to the number of the data processing requests; monitoring whether request data corresponding to the data processing request in the switched thread are successfully put in a storage through a global counter; if the warehousing is successful, adjusting the count corresponding to the second factor; monitoring whether the data processing request in the switched thread is abnormal or not through a global counter; and if the exception is processed, adjusting the count corresponding to the third factor.
Optionally, in a fifth implementation manner of the first aspect of the present invention, determining whether the global counter meets a preset period switching condition according to the monitoring result includes: judging whether the global variable is larger than a preset variable threshold value or not according to the monitoring result; if the global variable is greater than the variable threshold, after presetting the waiting time, judging whether the global variable is greater than the variable threshold again, and determining that the global counter does not meet the preset cycle switching condition until the preset judgment times are reached; and if the global variable is smaller than the variable threshold, determining that the global counter meets the periodic switching condition.
A second aspect of the present invention provides a data statistics period switching apparatus, including: the switching module is used for carrying out thread switching on the processing of the data processing request in the preset request queue according to the cycle switching instruction when the cycle switching instruction sent by the terminal is monitored; the monitoring module is used for sequentially warehousing request data corresponding to the data processing request in the switched threads and monitoring the request state of the switched threads through a preset global counter; the judging module is used for acquiring a monitoring result of a request state corresponding to the global counter and judging whether the global counter meets a preset period switching condition or not according to the monitoring result; and the setting module is used for setting the current time as the switching time of the current period and marking the input request data by adopting the switching time if the global counter meets the period switching condition.
Optionally, in a first implementation manner of the second aspect of the present invention, the data statistics period switching apparatus further includes a temporary storage module, configured to: acquiring a new data processing request, and judging whether the new data processing request meets a preset preprocessing condition or not; if so, acquiring request data corresponding to the new data processing request by presetting a first thread, and storing the request data corresponding to the new data processing request into a preset temporary data table; and if not, returning the processing exception prompt information corresponding to the new data processing request.
Optionally, in a second implementation manner of the second aspect of the present invention, the switching module includes: the analysis unit is used for analyzing the cycle switching instruction when monitoring the cycle switching instruction sent by the terminal to obtain a memory address corresponding to a second thread for processing the data processing request in the preset request queue; and the updating unit is used for searching the corresponding register according to the memory address and updating the thread pointer and the register pointer in the register so as to perform thread switching on the processing of the data processing request.
Optionally, in a third implementation manner of the second aspect of the present invention, the monitoring module includes: the counting unit is used for monitoring each state monitoring factor in the switched thread through a preset global counter and counting the corresponding count of each state monitoring factor; the generating unit is used for generating a global variable corresponding to the thread after the monitoring and switching of the global counter is carried out on the basis of counting corresponding to each state monitoring factor; and the monitoring unit is used for monitoring the request state of the switched thread by using the global counter according to the global variable.
Optionally, in a fourth implementation manner of the second aspect of the present invention, the different types of status monitoring factors include a first factor, a second factor, and a third factor, and the statistics unit is further configured to: monitoring the number of data processing requests in the switched threads by presetting a global counter, and determining the count corresponding to the first factor according to the number of the data processing requests; monitoring whether the request data corresponding to the data processing request in the switched thread is successfully put in a storage through a global counter; if the warehousing is successful, adjusting the count corresponding to the second factor; monitoring whether the data processing request in the switched thread is abnormal or not through a global counter; and if the exception is processed, adjusting the count corresponding to the third factor.
Optionally, in a fifth implementation manner of the second aspect of the present invention, the determining module includes: the judging unit is used for judging whether the global variable is larger than a preset variable threshold value or not according to the monitoring result; the circulating unit is used for judging whether the global variable is greater than the variable threshold again after the preset waiting time length if the global variable is greater than the variable threshold, and determining that the global counter does not meet the preset period switching condition until the preset judging times are reached; and the determining unit is used for determining that the global counter meets the periodic switching condition if the global variable is smaller than the variable threshold.
A third aspect of the present invention provides a data statistics period switching apparatus, including: a memory and at least one processor, the memory having instructions stored therein; at least one processor calls instructions in the memory to cause the data statistics cycle switching device to execute the data statistics cycle switching method.
A fourth aspect of the present invention provides a computer-readable storage medium having stored therein instructions, which when run on a computer, cause the computer to execute the above-mentioned data statistics period switching method.
In the technical scheme provided by the invention, a period switching instruction sent by a terminal is monitored in real time, when the period switching instruction is monitored, a thread switching is carried out on a data processing request, the thread of a conventional data processing request is replaced by another thread after the period switching, so that the requested data is put in storage and the request state is monitored at the same time, whether a global counter meets a period switching condition is determined, the period switching time is used for marking the switching time of the actual period of the requested data after the switching period, the data switched in the period is distinguished, and meanwhile, the omission of the data put in storage in the current statistical period after the statistical period is switched is avoided.
Drawings
FIG. 1 is a diagram of a first embodiment of a data statistics period switching method according to an embodiment of the present invention;
FIG. 2 is a diagram of a second embodiment of a data statistics period switching method according to an embodiment of the present invention;
FIG. 3 is a diagram of a third embodiment of a data statistics period switching method according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of an embodiment of a data statistics period switching apparatus according to the present invention;
FIG. 5 is a schematic diagram of another embodiment of a data statistics period switching apparatus according to an embodiment of the present invention;
fig. 6 is a schematic diagram of an embodiment of a data statistics period switching device in an embodiment of the present invention.
Detailed Description
The embodiment of the invention provides a method, a device, equipment and a storage medium for switching a data statistics period, wherein when a period switching instruction sent by a terminal is monitored, the processing of a data processing request in a preset request queue is switched in a thread mode according to the period switching instruction; sequentially warehousing request data corresponding to the data processing request in the switched thread, and monitoring the request state of the switched thread through a preset global counter; acquiring a monitoring result of a request state corresponding to the global counter, and judging whether the global counter meets a preset period switching condition or not according to the monitoring result; and if the global counter meets the period switching condition, setting the current time as the switching time of the current period, and marking the input request data by adopting the switching time. The invention realizes the real-time switching of the statistical period in the data statistical process and avoids the data in the current statistical period from being omitted and put in storage.
The terms "first," "second," "third," "fourth," and the like in the description and in the claims, as well as in the drawings, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the data so used may be interchanged under appropriate circumstances such that the embodiments described herein may be practiced otherwise than as specifically illustrated or described herein. Furthermore, the terms "comprises," "comprising," or "having," and any variations thereof, are intended to cover non-exclusive inclusions, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
For convenience of understanding, a detailed flow of the embodiment of the present invention is described below, and referring to fig. 1, a first embodiment of the data statistics period switching method in the embodiment of the present invention includes:
101. when a cycle switching instruction sent by a terminal is monitored, performing thread switching on processing of a data processing request in a preset request queue according to the cycle switching instruction;
it is to be understood that the execution subject of the present invention may be a data statistics period switching device, and may also be a terminal or a server, which is not limited herein. The embodiment of the present invention is described by taking a server as an execution subject.
In this embodiment, a periodic switching instruction sent by the terminal is monitored in real time, where the periodic switching instruction is triggered by a periodic switching switch, and the switch is set to be on when the periodic switching switch is triggered, otherwise, the switch is set to be off. Under the condition of high concurrent data processing requests, all uploaded data processing requests cannot be processed in real time due to computer computing power, the data processing requests need to be sequentially written into a request queue according to the sequence of sending the data processing requests, reading and processing are carried out according to the sequence written in the queue, and corresponding request data are stored in a warehouse.
When the triggering of the cycle switching instruction is monitored, in order to prevent the data processing requests in the request queue from being missed and forgetting to be processed, the unprocessed data processing requests in the request queue can be switched to a new thread for processing; and the new data processing request received after the trigger period switching instruction is processed by adopting the original thread, and the two are processed in parallel without influencing each other.
102. Sequentially warehousing request data corresponding to the data processing request in the switched thread, and monitoring the request state of the switched thread through a preset global counter;
in this embodiment, after the cycle switching instruction is triggered, for a data processing request that needs to be processed in the current cycle, a corresponding request data is put into a database, and the request database is stored in a preset database for subsequent call. The total number of the data processing requests is counted and classified into the request states of the corresponding types, when the data processing requests enter the warehousing process, the count is classified into the request states of the corresponding types, when the data processing requests are completed, the count is classified into the request states of the corresponding types, when the data processing requests are abnormal, the count is classified into the request states of the corresponding types, and the monitoring of the request states of the multiple types is completed.
103. Acquiring a monitoring result of a request state corresponding to the global counter, and judging whether the global counter meets a preset period switching condition or not according to the monitoring result;
in this embodiment, the monitoring result of the request state corresponding to the global counter is obtained, that is, in order to obtain the global variable, the monitoring result is measured by the global variable to determine whether the current global counter meets a preset cycle switching condition, where the cycle switching condition is used to determine whether processing of the data processing request in the current cycle is completed, and both the completion of warehousing of the request data and the request exception of the data processing request are regarded as the completion of processing of the data processing request. The periodic switching condition is that the global variable is 0, and the data processing request is completely processed.
104. And if the global counter meets the period switching condition, setting the current time as the switching time of the current period, and marking the input request data by adopting the switching time.
In this embodiment, it is determined that all processing of the data processing request is completed until the global counter satisfies the cycle switching condition, at this time, the current processing completion time may be set as the switching time of the current cycle, and stored in the database, and a mark is performed in each piece of request data obtained by processing, so as to distinguish the request data obtained by requesting in each cycle, so as to enable a subsequent service requirement to be called. Wherein, the switch and the global counter can introduce middleware which can be used for saving global variables such as Redis, ZooKeeper and the like.
In the embodiment of the invention, a period switching instruction sent by a terminal is monitored in real time, when the instruction is monitored, the thread switching is carried out on a data processing request, the conventional thread of the data processing request is replaced by another thread after the period switching, so that the requested data is put in storage and the request state is monitored at the same time, whether a global counter meets a period switching condition is determined, the period switching condition is used for marking the switching time of the actual period of the requested data after the switching period, the data switched in the period is distinguished, and meanwhile, the omission of the storage of the data in the current statistical period after the statistical period is switched is avoided.
Referring to fig. 2, a second embodiment of the data statistics period switching method according to the embodiment of the present invention includes:
201. when a cycle switching instruction sent by the terminal is monitored, analyzing the cycle switching instruction to obtain a memory address corresponding to a second thread for processing a data processing request in a preset request queue;
202. searching a corresponding register according to the memory address, and updating a thread pointer and a register pointer in the register so as to perform thread switching on the processing of the data processing request;
in this embodiment, a process cache frame such as RegCache is used to set multiple thread execution sites inside a Central Processing Unit (CPU), and a RegCache line switching instruction (i.e., a cycle switching instruction) is added to implement conversion between different threads, so that an original thread of a data Processing request is switched to a new second thread.
In this embodiment, the cycle switching instruction may be a ctx _ sw ra instruction, where ctx _ sw is an operation code, ra is an operand, and ra is a memory address corresponding to a second thread pointing to a data processing request in the request queue. Aiming at the original processing thread, the thread pointer is Cur _ ctx _ ptr, when a cycle switching instruction is executed, the register rx adopted by the second thread after conversion needs to be pointed at first, the position of the register rx is in RegCache [ Cur _ ctx _ ptr ]. Reg [ rx ], and X exists in the register where the second thread is located, so that the RegCache [ X ]. tag ═ ra is directly switched to the register where the RegCache [ X ]. tag ═ ra, and the fast conversion among different threads can be realized.
Specifically, the method may execute Cur _ ctx _ ptr ═ X to update the initial thread pointer Cur _ ctx _ ptr in the register; PC _ ptr +1 may be executed to update the register pointer.
203. Acquiring a new data processing request, and judging whether the new data processing request meets a preset preprocessing condition or not;
204. if so, acquiring request data corresponding to the new data processing request by presetting a first thread, and storing the request data corresponding to the new data processing request into a preset temporary data table;
205. if not, returning the processing exception prompt information corresponding to the new data processing request;
in this embodiment, after the cycle switching instruction is triggered, the data processing request previously stored in the request queue is processed by the second thread, and the new data processing request is pre-stored in the temporary table and normally processed by the original first thread, where when the new data processing request is obtained, it is further required to perform preprocessing condition screening on the new data processing request, for example, when it is determined that the current sending speed exceeds a preset speed threshold when the new data processing request is sent, and when the new data processing request is sent, the new data processing request is taken as an abnormal data processing request and returns processing exception prompt information, and preferably, the speed threshold may be 200 per second, so as to prevent network attack or a list-flushing situation.
206. Monitoring the request state of the switched thread by presetting a global counter;
207. acquiring a monitoring result of a request state corresponding to the global counter, and judging whether the global counter meets a preset period switching condition or not according to the monitoring result;
208. and if the global counter meets the period switching condition, setting the current time as the switching time of the current period, and marking the warehoused request data by adopting the switching time.
In the embodiment of the invention, after the statistical period is switched, the original first thread is still adopted to put the request data into a database for processing a new data received after the current statistical period, wherein the request data can be temporarily stored by adopting the temporary data table, and the request data in the temporary data table is put into the database after the subsequent storage of the request data in the current statistical period is finished, so that the request data in different statistical periods can be distinguished.
Referring to fig. 3, a third embodiment of the data statistics period switching method according to the embodiment of the present invention includes:
301. when a cycle switching instruction sent by a terminal is monitored, performing thread switching on processing of a data processing request in a preset request queue according to the cycle switching instruction;
302. sequentially warehousing request data corresponding to the data processing request in the switched thread, monitoring each state monitoring factor in the switched thread through a preset global counter, and counting the count corresponding to each state monitoring factor;
303. generating a global variable corresponding to the thread after the monitoring of the global counter is switched based on counting corresponding to each state monitoring factor;
304. according to the global variable, a global counter is used for monitoring the request state of the switched thread;
in this embodiment, a plurality of different types of state monitoring factors are preset to monitor each processing node in the whole logical thread processing process for the data processing request, measure the processing condition of the data processing request in the second thread according to the count number of each state monitoring factor, and synthesize each count number to measure the request state of the second thread by a global variable.
Specifically, the total number of the data processing requests is classified into a first factor, a count of completion of data storage requested by the data processing requests is classified into a second factor, and a count of occurrence of an abnormality of the data processing requests is classified into a third factor, where the count is obtained by a formula: and the first factor count, the second factor count and the third factor count are used as global variables corresponding to the switched threads monitored by the global counter. The monitoring and counting process for each type of state monitoring factor may include the following steps:
1) monitoring the number of data processing requests in the switched threads by presetting a global counter, and determining the count corresponding to the first factor according to the number of the data processing requests;
2) monitoring whether the request data corresponding to the data processing request in the switched thread is successfully put in a storage through a global counter;
3) if the warehousing is successful, adjusting the count corresponding to the state monitoring factor of the second factor;
4) monitoring whether the data processing request in the switched thread is abnormal or not through a global counter;
5) and if the abnormal processing is carried out, adjusting the count corresponding to the state monitoring factor of the third factor.
305. Acquiring a monitoring result of a request state corresponding to the global counter, and judging whether the global variable is greater than a preset variable threshold value according to the monitoring result;
306. if the global variable is greater than the variable threshold, after presetting the waiting time, judging whether the global variable is greater than the variable threshold again, and determining that the global counter does not meet the preset cycle switching condition until the preset judgment times are reached;
307. if the global variable is smaller than the variable threshold, determining that the global counter meets the periodic switching condition;
in this embodiment, a global variable in the monitoring result is extracted, and then it is determined whether the global variable is greater than a preset variable threshold, where the variable threshold is set according to a service requirement, and is preferably set to 0, that is, it indicates that all data processing requests are completely processed, and in order to ensure that the global variable is 0 when the data processing requests are completely processed due to an abnormality, a loop of preset determination times is performed, that is, the loop determines whether the global variable is greater than the variable threshold for multiple times, a determination interval for each time is preset as a waiting time, and preferably, the waiting time may be set to 3 seconds, and the determination times may be set to 3 times.
308. And if the global counter meets the period switching condition, setting the current time as the switching time of the current period, and marking the input request data by adopting the switching time.
In the embodiment of the invention, the request state of the data processing request in the statistical period is monitored by the global counter, the quantization is carried out by the state monitoring factor, and whether the global counter meets the period switching condition or not is measured by the global variable, namely the statistical period of the data processing request is accurately switched by the global counter.
The above describes the data statistics period switching method in the embodiment of the present invention, and the following describes the data statistics period switching device in the embodiment of the present invention, with reference to fig. 4, an embodiment of the data statistics period switching device in the embodiment of the present invention includes:
the switching module 401 is configured to, when a cycle switching instruction sent by the terminal is monitored, perform thread switching on processing of a data processing request in a preset request queue according to the cycle switching instruction;
a monitoring module 402, configured to store request data corresponding to the data processing request in sequence in the switched thread, and perform request state monitoring on the switched thread through a preset global counter;
the judging module 403 is configured to obtain a monitoring result of the request state corresponding to the global counter, and determine whether the global counter satisfies a preset cycle switching condition according to the monitoring result;
a setting module 404, configured to set the current time as the switching time of the current period if the global counter meets the period switching condition, and mark the request data after being put into the database with the switching time.
In the embodiment of the invention, a period switching instruction sent by a terminal is monitored in real time, when the instruction is monitored, the thread switching is carried out on a data processing request, the conventional thread of the data processing request is replaced by another thread after the period switching, so that the requested data is put in storage and the request state is monitored at the same time, whether a global counter meets a period switching condition is determined, the period switching condition is used for marking the switching time of the actual period of the requested data after the switching period, the data switched in the period is distinguished, and meanwhile, the omission of the storage of the data in the current statistical period after the statistical period is switched is avoided.
Referring to fig. 5, another embodiment of the data statistics period switching apparatus according to the embodiment of the present invention includes:
the switching module 401 is configured to, when a cycle switching instruction sent by the terminal is monitored, perform thread switching on processing of a data processing request in a preset request queue according to the cycle switching instruction;
a monitoring module 402, configured to store request data corresponding to the data processing request in sequence in the switched thread, and perform request state monitoring on the switched thread through a preset global counter;
the judging module 403 is configured to obtain a monitoring result of the request state corresponding to the global counter, and determine whether the global counter satisfies a preset cycle switching condition according to the monitoring result;
a setting module 404, configured to set the current time as the switching time of the current period if the global counter meets the period switching condition, and mark the request data after being put into the database with the switching time.
Specifically, the data statistics cycle switching apparatus further includes a temporary storage module 405, configured to:
acquiring a new data processing request, and judging whether the new data processing request meets a preset preprocessing condition or not;
if so, acquiring request data corresponding to the new data processing request by presetting a first thread, and storing the request data corresponding to the new data processing request into a preset temporary data table;
and if not, returning the processing exception prompt information corresponding to the new data processing request.
Specifically, the switching module 401 includes:
the analyzing unit 4011 is configured to analyze the cycle switching instruction when the cycle switching instruction sent by the terminal is monitored, and obtain a memory address corresponding to a second thread that processes a data processing request in the preset request queue;
the updating unit 4012 is configured to search a corresponding register according to the memory address, and update the thread pointer and the register pointer in the register, so as to perform thread switching on processing of the data processing request.
Specifically, the monitoring module 402 includes:
a counting unit 4021, configured to monitor each state monitoring factor in the switched thread by presetting a global counter, and count the count corresponding to each state monitoring factor;
a generating unit 4022, configured to generate a global variable corresponding to the thread after the monitoring and switching of the global counter based on counting corresponding to each state monitoring factor;
the monitoring unit 4023 is configured to perform request state monitoring on the switched threads by using the global counter according to the global variable.
Specifically, the different types of status monitoring factors include a first factor, a second factor, and a third factor, and the statistical unit 4021 is further configured to:
monitoring the number of data processing requests in the switched threads by presetting a global counter, and determining the count corresponding to the first factor according to the number of the data processing requests;
monitoring whether request data corresponding to the data processing request in the switched thread are successfully put in a storage through a global counter;
if the warehousing is successful, adjusting the count corresponding to the second factor;
monitoring whether the data processing request in the switched thread is abnormal or not through a global counter;
and if the exception is processed, adjusting the count corresponding to the third factor.
Specifically, the determining module 403 includes:
a determination unit 4031, configured to determine whether the global variable is greater than a preset variable threshold according to the monitoring result;
a loop unit 4032, configured to, if the global variable is greater than the variable threshold, re-determine whether the global variable is greater than the variable threshold after a preset wait time, and determine that the global counter does not satisfy a preset period switching condition until a preset determination number is reached;
a determining unit 4033, configured to determine that the global counter satisfies the cycle switching condition if the global variable is smaller than the variable threshold.
In the embodiment of the invention, after the statistical period is switched, the original first thread is still adopted to put the request data into a warehouse for the new data processing request received after the current statistical period, wherein a temporary data table can be adopted to temporarily store the request data, and after the subsequent warehousing of the request data of the current statistical period is finished, the request data in the temporary data table is put into the warehouse, so that the request data of different statistical periods are distinguished; in addition, the request state of the data processing request in the statistical period is monitored through a global counter, quantization is carried out through a state monitoring factor, and whether the global counter meets the period switching condition or not is measured through a global variable, namely the statistical period of the data processing request is accurately switched through the global counter.
Fig. 4 and fig. 5 describe the data statistics period switching device in the embodiment of the present invention in detail from the perspective of the modular functional entity, and the data statistics period switching device in the embodiment of the present invention is described in detail from the perspective of hardware processing.
Fig. 6 is a schematic structural diagram of a data statistics cycle switching apparatus 600 according to an embodiment of the present invention, where the data statistics cycle switching apparatus 600 may generate relatively large differences due to different configurations or performances, and may include one or more processors (CPUs) 610 (e.g., one or more processors) and a memory 620, and one or more storage media 630 (e.g., one or more mass storage devices) storing applications 633 or data 632. Memory 620 and storage medium 630 may be, among other things, transitory or persistent storage. The program stored in the storage medium 630 may include one or more modules (not shown), each of which may include a series of instruction operations for the data statistics cycle switching apparatus 600. Still further, the processor 610 may be configured to communicate with the storage medium 630 and execute a series of instruction operations in the storage medium 630 on the data statistics cycle switching apparatus 600.
The statistics cycle switching apparatus 600 may also include one or more power supplies 640, one or more wired or wireless network interfaces 650, one or more input-output interfaces 660, and/or one or more operating systems 631, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, and the like. Those skilled in the art will appreciate that the configuration of the statistics cycle switching device shown in fig. 6 does not constitute a limitation of the statistics cycle switching device, and may include more or fewer components than shown, or some components may be combined, or a different arrangement of components.
The invention further provides a data statistics period switching device, where the computer device includes a memory and a processor, where the memory stores computer readable instructions, and the computer readable instructions, when executed by the processor, cause the processor to execute the steps of the data statistics period switching method in the foregoing embodiments.
The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium, and may also be a volatile computer-readable storage medium, having stored therein instructions, which, when run on a computer, cause the computer to perform the steps of the data statistics cycle switching method.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a read-only memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
The above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. A data statistics cycle switching method is characterized by comprising the following steps:
when a cycle switching instruction sent by a terminal is monitored, carrying out thread switching on the processing of a data processing request in a preset request queue according to the cycle switching instruction;
sequentially warehousing request data corresponding to the data processing request in the switched thread, and monitoring the request state of the switched thread through a preset global counter;
acquiring a monitoring result of a request state corresponding to the global counter, and judging whether the global counter meets a preset period switching condition or not according to the monitoring result;
and if the global counter meets the period switching condition, setting the current time as the switching time of the current period, and marking the input request data by adopting the switching time.
2. The method according to claim 1, wherein after the thread switching is performed on the processing of the data processing request in the preset request queue according to the cycle switching instruction when the cycle switching instruction sent by the terminal is monitored, the method further comprises:
acquiring a new data processing request, and judging whether the new data processing request meets a preset preprocessing condition or not;
if so, acquiring request data corresponding to the new data processing request through a preset first thread, and storing the request data corresponding to the new data processing request into a preset temporary data table;
and if not, returning the processing exception prompt information corresponding to the new data processing request.
3. The method according to claim 1, wherein the performing thread switching on processing of a data processing request in a preset request queue according to the cycle switching instruction when the cycle switching instruction sent by the terminal is monitored comprises:
when a cycle switching instruction sent by a terminal is monitored, analyzing the cycle switching instruction to obtain a memory address corresponding to a second thread for processing a data processing request in a preset request queue;
and searching a corresponding register according to the memory address, and updating a thread pointer and a register pointer in the register so as to perform thread switching on the processing of the data processing request.
4. The method according to claim 1, wherein the monitoring the request status of the switched thread by a preset global counter comprises:
monitoring each state monitoring factor in the switched thread through a preset global counter, and counting the count corresponding to each state monitoring factor;
generating a global variable corresponding to the thread after the switching by the global counter based on counting the count corresponding to each state monitoring factor;
and monitoring the request state of the switched thread by using the global counter according to the global variable.
5. The method according to claim 4, wherein the different types of state monitoring factors include a first factor, a second factor, and a third factor, and the monitoring each state monitoring factor in the switched thread by presetting a global counter and counting the count corresponding to each state monitoring factor includes:
monitoring the number of data processing requests in the switched thread through a preset global counter, and determining the count corresponding to the first factor according to the number of the data processing requests;
monitoring whether the request data corresponding to the data processing request in the switched thread is successfully put in a storage through the global counter;
if the warehousing is successful, adjusting the count corresponding to the second factor;
monitoring whether the data processing request in the switched thread is abnormal or not through the global counter;
and if the processing is abnormal, adjusting the count corresponding to the third factor.
6. The method according to claim 4, wherein the determining whether the global counter satisfies a preset period switching condition according to the monitoring result comprises:
judging whether the global variable is larger than a preset variable threshold value or not according to the monitoring result;
if the global variable is greater than the variable threshold, after presetting the waiting time, judging whether the global variable is greater than the variable threshold again, and determining that the global counter does not meet the preset cycle switching condition until the preset judgment times are reached;
and if the global variable is smaller than the variable threshold, determining that the global counter meets the periodic switching condition.
7. A data statistics period switching apparatus, characterized in that the data statistics period switching apparatus comprises:
the switching module is used for carrying out thread switching on the processing of the data processing request in the preset request queue according to the cycle switching instruction when the cycle switching instruction sent by the terminal is monitored;
the monitoring module is used for sequentially warehousing request data corresponding to the data processing request in the switched threads and monitoring the request state of the switched threads through a preset global counter;
the judging module is used for acquiring a monitoring result of a request state corresponding to the global counter and judging whether the global counter meets a preset cycle switching condition or not according to the monitoring result;
and the setting module is used for setting the current time as the switching time of the current period and marking the input request data by adopting the switching time if the global counter meets the period switching condition.
8. The apparatus according to claim 7, further comprising a buffering module configured to:
acquiring a new data processing request, and judging whether the new data processing request meets a preset preprocessing condition or not;
if so, acquiring request data corresponding to the new data processing request through a preset first thread, and storing the request data corresponding to the new data processing request into a preset temporary data table;
and if not, returning the processing exception prompt information corresponding to the new data processing request.
9. A data statistics period switching apparatus, characterized in that the data statistics period switching apparatus comprises: a memory and at least one processor, the memory having instructions stored therein;
the at least one processor invoking the instructions in the memory to cause the datacom period switching apparatus to perform the steps of the datacom period switching method of any one of claims 1-6.
10. A computer-readable storage medium having instructions stored thereon, wherein the instructions, when executed by a processor, implement the steps of the data statistics cycle switching method according to any of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210584801.1A CN115080577B (en) | 2022-05-27 | 2022-05-27 | Data statistics period switching method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210584801.1A CN115080577B (en) | 2022-05-27 | 2022-05-27 | Data statistics period switching method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115080577A true CN115080577A (en) | 2022-09-20 |
| CN115080577B CN115080577B (en) | 2024-08-27 |
Family
ID=83250184
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210584801.1A Active CN115080577B (en) | 2022-05-27 | 2022-05-27 | Data statistics period switching method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115080577B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO1999021089A1 (en) * | 1997-10-23 | 1999-04-29 | International Business Machines Corporation | Method and apparatus for altering thread priorities in a multithreaded processor |
| CN107908518A (en) * | 2017-06-29 | 2018-04-13 | 平安科技（深圳）有限公司 | Database performance monitoring method, device, storage medium and computer equipment |
| CN109194500A (en) * | 2018-08-08 | 2019-01-11 | 新华三技术有限公司 | Forwarding table state switching method, device and communication equipment |
| CN112631858A (en) * | 2020-12-18 | 2021-04-09 | 苏州浪潮智能科技有限公司 | Thread monitoring method, system, device and medium |
-
2022
- 2022-05-27 CN CN202210584801.1A patent/CN115080577B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO1999021089A1 (en) * | 1997-10-23 | 1999-04-29 | International Business Machines Corporation | Method and apparatus for altering thread priorities in a multithreaded processor |
| CN107908518A (en) * | 2017-06-29 | 2018-04-13 | 平安科技（深圳）有限公司 | Database performance monitoring method, device, storage medium and computer equipment |
| CN109194500A (en) * | 2018-08-08 | 2019-01-11 | 新华三技术有限公司 | Forwarding table state switching method, device and communication equipment |
| CN112631858A (en) * | 2020-12-18 | 2021-04-09 | 苏州浪潮智能科技有限公司 | Thread monitoring method, system, device and medium |
Non-Patent Citations (1)
| Title |
|---|
| 姚兰;桂勋;: "基于多线程的气象传感器数字网络化系统", 仪表技术与传感器, no. 09, 15 September 2008 (2008-09-15) * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115080577B (en) | 2024-08-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11157061B2 (en) | Processor management via thread status | |
| US4905171A (en) | Workstation controller performance monitor | |
| US7379953B2 (en) | Systems and methods for resource-adaptive workload management | |
| US10691571B2 (en) | Obtaining application performance data for different performance events via a unified channel | |
| US8132170B2 (en) | Call stack sampling in a data processing system | |
| US8286134B2 (en) | Call stack sampling for a multi-processor system | |
| US20100017583A1 (en) | Call Stack Sampling for a Multi-Processor System | |
| US5872913A (en) | System and method for low overhead, high precision performance measurements using state transistions | |
| CN108508874B (en) | Method and device for monitoring equipment fault | |
| US11132220B2 (en) | Process scheduling | |
| EP2239664A2 (en) | Context switch sampling | |
| CN110119307B (en) | Data processing request processing method and device, storage medium and electronic device | |
| US20110138146A1 (en) | Kernel subsystem for handling performance counters and events | |
| CN115756812A (en) | Resource adjusting method and device and storage medium | |
| CN114546590A (en) | Java virtual machine heap memory set object monitoring method and memory overflow analysis method | |
| CN114416849A (en) | Data processing method and device, electronic equipment and storage medium | |
| Erickson et al. | Response time bounds for G-EDF without intra-task precedence constraints | |
| CN115080577A (en) | Data statistics period switching method, device, equipment and storage medium | |
| JP2009037369A (en) | Resource assignment method to database server | |
| CN112685634A (en) | Data query method and device, electronic equipment and storage medium | |
| CN111124678A (en) | Memory scheduling processing method and device | |
| CN111159009A (en) | Pressure testing method and device for log service system | |
| CN115438056A (en) | Data acquisition method, device, equipment and storage medium | |
| CN116108065A (en) | Active time sequence data management method and system | |
| JP4453823B2 (en) | Performance bottleneck analysis system and performance bottleneck analysis method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |