CN118193519A - Data processing method and device - Google Patents
Data processing method and device Download PDFInfo
- Publication number
- CN118193519A CN118193519A CN202211610122.3A CN202211610122A CN118193519A CN 118193519 A CN118193519 A CN 118193519A CN 202211610122 A CN202211610122 A CN 202211610122A CN 118193519 A CN118193519 A CN 118193519A
- Authority
- CN
- China
- Prior art keywords
- data
- service identifier
- data table
- service
- valid
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title abstract description 19
- 238000011010 flushing procedure Methods 0.000 claims abstract description 27
- 238000000034 method Methods 0.000 claims abstract description 22
- 230000004044 response Effects 0.000 claims abstract description 15
- 238000012545 processing Methods 0.000 claims description 24
- 238000012216 screening Methods 0.000 claims description 14
- 238000011144 upstream manufacturing Methods 0.000 claims description 13
- 238000004590 computer program Methods 0.000 claims description 8
- 238000010586 diagram Methods 0.000 description 17
- 230000000694 effects Effects 0.000 description 11
- 238000004080 punching Methods 0.000 description 11
- 230000006870 function Effects 0.000 description 9
- 238000004891 communication Methods 0.000 description 7
- 238000009825 accumulation Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 238000012217 deletion Methods 0.000 description 4
- 230000037430 deletion Effects 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 239000000835 fiber Substances 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 238000010276 construction Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a data processing method and device, and relates to the technical field of computers. One embodiment of the method comprises the steps of obtaining a data table and new data, marking the new data as valid, and recording the data to the data table; judging whether the service identifier corresponding to the data is larger than a cursor value or not; if the service identifier is larger than the cursor value, updating the cursor value as the service identifier, and accumulating all effective data in the data table; if the service identifier is not greater than the cursor value, judging whether to flush the effective data corresponding to the service identifier in the data table; responsive to not flushing the valid data corresponding to the service identifier, accumulating all the data marked as valid in the data table; or in response to the effective data corresponding to the service identifier, accumulating the effective data corresponding to the service identifier after the service identifier is punched and the effective data corresponding to other service identifiers. Therefore, the embodiment of the invention can solve the technical problem that the reset data cannot be accurately and efficiently identified.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a data processing method and apparatus.
Background
At present, the application scenes of orderly storing data into a data table and calculating the accumulation amount are quite common, so that convenient and quick data processing service is provided for users, physical storage resources can be greatly saved, and the data storage cost of the users is reduced.
In the process of implementing the present invention, the inventor finds that at least the following problems exist in the prior art:
When enjoying the convenient and efficient instant data processing function, the situation that the acquired data is wrong and the data processing is wrong is common, and especially in the scene of large data volume and high concurrency, if a certain node of the data table is required to reprocess part of related data, more serious data disorder can be caused. In addition, although the existing data table provides a basic data resetting processing method, the method can only conduct data correction once on data in a specified range, and the processing range is small; if the data records at a plurality of positions of the data table are wrong, the user is required to perform the same operation for a plurality of times, and the processing efficiency is low.
Disclosure of Invention
In view of the above, embodiments of the present invention provide a data processing method and apparatus, which can solve the existing technical problem that the reset data cannot be accurately and efficiently identified.
To achieve the above object, according to an aspect of the embodiments of the present invention, there is provided a data processing method, including acquiring a data table and new data, marking a new data state as valid, and recording the valid data to the data table; judging whether the service identifier corresponding to the effective data is larger than a cursor value or not; if the service identifier is larger than the cursor value, updating the cursor value into the service identifier, and accumulating all effective data in a data table; if the service identifier is not greater than the running index value, judging whether to perform positive flushing on the effective data corresponding to the service identifier in the data table; accumulating all data marked as valid in the data table in response to not flushing the valid data corresponding to the service identifier; or in response to the effective data corresponding to the service identifier after the service identifier is punched, accumulating the effective data corresponding to the service identifier after the service identifier is punched and the effective data corresponding to other service identifiers.
Optionally, obtaining the data table and the new data includes:
acquiring new data, and generating a corresponding service identifier according to upstream and downstream service attributes in a service type label of the new data, wherein the service identifier value of an upstream service is smaller than the service identifier value of a corresponding downstream service; generating a sub-data table corresponding to the service identifier for the data with the same service identifier; and merging all the sub data tables according to the sequence of the service identification to generate a data table, and initializing the cursor value of the data table.
Optionally, the method comprises:
responding to the reset of the cursor value, obtaining the cursor value after the reset, and replacing the cursor value in the data table; screening according to the data table to obtain data with service identifiers larger than the replaced post-cursor value and marked as valid, updating the marks of the data as invalid, and logically deleting the data updated as invalid; all data marked as valid in the data table are accumulated.
Optionally, determining whether to perform positive flushing on the valid data corresponding to the service identifier according to the data table includes:
Screening to obtain data corresponding to the service identifier in the data table, and determining that the data mark recorded last time except the new data is valid, and then flushing the valid data corresponding to the service identifier.
Optionally, flushing the valid data corresponding to the service identifier includes:
And acquiring effective data corresponding to the service identifier in the data table, screening to obtain effective data which is recorded last time except the new data in the effective data and is used as punched positive data, updating the mark of the punched positive data into invalid, and logically deleting the punched positive data which is updated into invalid.
Optionally, the method comprises:
And receiving a data locking request, starting a locking program, locking the valid data corresponding to all the current service identifiers, and updating the running index value to be an initial value.
Optionally, the method comprises:
and acquiring the number of the data which has the same service identifier and is marked as effective in the data table, and if the number is determined to be larger than the preset number threshold corresponding to the service identifier, generating and outputting false prompt information of the data table.
In addition, the invention also provides a data processing device, which comprises an acquisition module, a data processing module and a data processing module, wherein the acquisition module is used for acquiring a data table and new data, marking the new data state as effective, and recording the effective data to the data table; the identification module is used for judging whether the service identifier corresponding to the effective data is larger than a cursor value, if the service identifier is larger than the cursor value, updating the cursor value into the service identifier, and accumulating all the effective data in a data table; if the service identifier is not greater than the running index value, judging whether to perform positive flushing on the effective data corresponding to the service identifier in the data table; accumulating all data marked as valid in the data table in response to not flushing the valid data corresponding to the service identifier; or in response to the effective data corresponding to the service identifier after the service identifier is punched, accumulating the effective data corresponding to the service identifier after the service identifier is punched and the effective data corresponding to other service identifiers.
One embodiment of the above invention has the following advantages or benefits: according to the invention, the data service type labels are ordered according to the time attribute included in the data service type labels, namely the upstream and downstream relation corresponding to the data service type labels, the ordering result is used as the service identification corresponding to the service type labels, a data table is generated for the data corresponding to each service identification, the data tables are combined according to the sequence of the service identifications to generate the data table, and the cursor value of the data table is initialized to zero, so that the data are distributed in the data table according to the upstream and downstream service relation of the data, the effect of orderly acquiring and recording the data in the data table is achieved, and convenience is brought to the operation of orderly positioning the time node with error data record and logically deleting the error data and re-acquiring the correct data; in addition, through updating the running index value and judging the data alignment in the data table along with the new data service identifier, only one unlocked effective data is accumulated in the data table for each service identifier in the range before the data locking request is received, the effective data is the data which is acquired in the data table for each service identifier at the latest time, the timeliness of the effective data is ensured, the effect that the data table automatically performs the data alignment for a plurality of times after a user only issues a data table resetting instruction is achieved, and the processing efficiency of the data table is greatly improved; meanwhile, by receiving a data locking request, locking is carried out on valid data corresponding to all current service identifiers of a data table, so that the effect of protecting correct data in the data table is achieved, the locked data is counted into an accumulation result of the data table, so that the locked data is not punched by new data, and the running index value is updated to be zero, so that a new line is needed to be additionally started in the data table to record new data received subsequently, the effect that each service identifier corresponds to a plurality of valid data in the data table is achieved, the application range of the data table is expanded, the function of data punching processing and the function of conventional accumulated data storage are included, and the application scene of a user using the data table is increased; in addition, the invention achieves the effect of monitoring whether the data quantity recorded by the data table is correct in real time by acquiring the quantity of the data which has the same service identifier and is marked as valid in the data table and comparing the quantity with the preset quantity threshold corresponding to the service identifier, if not, generating false prompt information of the data table and outputting the false prompt information in time, thereby avoiding the situation that the data is judged to be false after the data table has completed all data records, realizing the effect of stopping the damage of the data table recorded with false in time, further improving the working efficiency and improving the user experience.
Further effects of the above-described non-conventional alternatives are described below in connection with the embodiments.
Drawings
The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:
FIG. 1 is a schematic diagram of the main flow of a data processing method according to a first embodiment of the present invention;
FIG. 2 is a schematic diagram of data table 1 according to an embodiment of the invention;
FIG. 3 is a schematic diagram of data table 2 according to an embodiment of the invention;
FIG. 4 is a schematic diagram of the main flow of a data processing method according to a second embodiment of the present invention;
FIG. 5 is a schematic diagram of the main flow of a data processing method according to a third embodiment of the present invention;
FIG. 6 is a schematic diagram of the main modules of a data processing apparatus according to an embodiment of the present invention;
FIG. 7 is an exemplary system architecture diagram in which embodiments of the present invention may be applied;
fig. 8 is a schematic diagram of a computer system suitable for use in implementing an embodiment of the invention.
Detailed Description
Exemplary embodiments of the present invention will now be described with reference to the accompanying drawings, in which various details of the embodiments of the present invention are included to facilitate understanding, and are to be considered merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
Fig. 1 is a schematic diagram of main flow of a data processing method according to a first embodiment of the present invention, and as shown in fig. 1, the data processing method includes:
Step S101, a data table and new data are obtained, the new data state is marked as valid, and the valid data is recorded into the data table.
In the embodiment, the latest acquired data are considered as valid data, and the latest acquired valid data are recorded into the data table, so that the effect that any latest acquired data is not missed by the data table is achieved, and the timeliness of the data marked as valid in the data table is ensured.
In some embodiments, when the data table and the new data are acquired, the new data is acquired first, and a corresponding service identifier is generated according to the upstream and downstream service attributes in the service type tag of the new data, wherein the service identifier value of the upstream service is smaller than the service identifier value of the corresponding downstream service; generating a sub-data table corresponding to the service identifier for the data with the same service identifier; and merging all the sub data tables according to the sequence of the service identification to generate a data table, and initializing the cursor value of the data table. For example, if new data is acquired and the service type label is obtained, the following steps are: the upstream industry service a indicates that the time information of the service type tag is: the data belongs to the first service A in the upstream industry, so that the corresponding service identifier is 1, and the data is recorded to the corresponding row in the 1 st column of the data table; if new data is acquired and the service type label is obtained, the method comprises the following steps: the downstream industry service G indicates that the time information of the service type tag is: the data belongs to the seventh service G in the downstream industry, so the corresponding service identifier is 7, the data is recorded to the corresponding row in the 7 th column of the data table, and the running index value of the data table is initialized to be 0. The received business data are ordered according to the upstream and downstream industry sequence, so that the ordering of the data in the data table can be ensured, and the user can intuitively and clearly browse the data or determine the conventional operations such as data resetting nodes.
In some embodiments, after the user determines that some data in the data table is wrong, the user may respond to the reset of the cursor value to obtain the reset cursor value, and replace the cursor value in the data table; screening according to a data table to obtain data with service identifiers larger than the replaced post-cursor value and marked as valid, updating the marks of the data as invalid, and logically deleting the data updated as invalid; all data marked as valid in the data table are accumulated. For example, as shown in fig. 2, when the running index value is reset from 5 to 2 in the 2 nd line, the data recorded in the data table (3), the data table (4) and the data table (5) in the 1 st line are marked as invalid data, the invalid data is logically deleted, the accumulated value of the data table is updated to be the accumulated value of all the data marked as valid in the data table, namely, the accumulated value of the data table before the running index value is reset is subtracted by the accumulated value of the invalid data, namely, 18-1-2-3=12. Wherein logical deletion is defined as not deleting the corresponding data record, only marking the data so that it does not account for the data table accumulation calculation. The operation of resetting the running index value achieves the effect of logically deleting the error data in the data table, ensures the correctness of the accumulated value in the data table, and in addition, the logic deleting the data to keep the record of the data, thereby bringing convenience to the consulting and rechecking of the data table.
Step S102, judging whether the service identifier corresponding to the effective data is larger than a cursor value. If yes, go to step S103, otherwise go to step S104.
In the embodiment, whether the cursor value of the data table is updated is judged by comparing the service identifier of the effective data with the cursor value of the data table. The cursor value of the data table indicates the position information of the last valid data recorded in the data table, so that when new data is acquired, the effect of quickly and preliminarily judging whether the service identifier corresponding to the new data is recorded with unlocked valid data can be achieved, and the fact that at most one unlocked valid data is recorded in the data table for each service identifier is ensured.
Step S103, if the service identifier is larger than the cursor value, updating the cursor value into the service identifier, and accumulating all effective data in a data table.
For example, as shown in fig. 2, the valid data recorded in line 3 is 2, the corresponding service identifier is 3, and is greater than the current cursor value 2 in the data table, the cursor value is updated to 3, and the data accumulated value is added with 2 correspondingly, so as to obtain 12+2=14; and the effective data recorded in the 4 th row is 1, the corresponding service identifier is 4, and the service identifier is greater than the cursor value 3 of the current data table, the cursor value is updated to be 4, and the data accumulated value is correspondingly added with 1, so that 14+1=15 is obtained. And through updating the cursor value, the updating of the service identification position corresponding to the latest recorded and unlocked effective data in the data table is intuitively embodied.
Step S104, if the service identifier is not greater than the free label value, judging whether to flush the effective data corresponding to the service identifier in the data table, if yes, executing step S106, otherwise, executing step S105.
In some embodiments, determining whether to flush valid data corresponding to the service identifier according to the data table includes: screening to obtain data corresponding to the service identifier in a data table, judging whether the data which is recorded in the data, except for new data, is effective data or not, and if so, correcting the effective data corresponding to the service identifier; and if not, not correcting the effective data corresponding to the service identifier.
As shown in fig. 2, the valid data recorded in line 5 is 6, and the corresponding service identifier is 3, which is smaller than the current cursor value 4, so that the cursor value is not updated, the data corresponding to the service identifier 3 is found in the data table, and besides the current line of the valid data 6, the valid data 2 recorded last time also exists, so that the valid data corresponding to the service identifier 3 in the data table is subjected to positive punching; as shown in fig. 3, the valid data recorded in line 4 is 1, and the corresponding service identifier 4 is smaller than the current cursor value 5, so that the cursor value is not updated, the data corresponding to the service identifier 4 is searched in the data table, and other valid data recorded last time is not present except that the valid data 1 is currently in the current process, so that the valid data corresponding to the service identifier 4 in the data table is not corrected. And judging whether to perform positive flushing processing on the effective data corresponding to the specified service identifier according to the running index value and the data table, so that each service identifier in the data table only corresponds to one unlocked effective data, the phenomenon of disorder such as repeated data recording caused by the positive flushing operation of the data is avoided, and the accuracy of the data table is improved.
And step S105, in response to not flushing the valid data corresponding to the service identification, accumulating all the data marked as valid in the data table.
As shown in fig. 3, the valid data recorded in row 4 is 1, and the corresponding service identifier 4 is determined not to be positive by punching according to the rule of punching, so that the valid data recorded in row 4 is accumulated to be 1 to the accumulated value of the data table, resulting in 13+1=14. The step plays a role in updating the accumulated result of the data table when the data flushing is not performed, and ensures the correctness of the accumulated value of the data table.
And step S106, in response to the effective data corresponding to the service identifier, the effective data corresponding to the service identifier is subjected to positive punching, and the effective data corresponding to the service identifier after positive punching and the effective data corresponding to other service identifiers are accumulated.
In some embodiments, the flushing the valid data corresponding to the service identifier includes: and acquiring effective data corresponding to the service identifier in a data table, screening to obtain effective data which is recorded last time except new data in the effective data and is used as punched positive data, updating the mark of the punched positive data to be invalid, and logically deleting the punched positive data updated to be invalid. For example, as shown in fig. 2, the valid data recorded in the 5 th row is 6, the corresponding service identifier 3 is determined to be the valid data corresponding to the latest record of the service identifier 3 in the data table according to the regular punching, namely, the valid data 2 recorded in the third row is punched, so that the valid data 2 recorded in the 3 rd row is marked as invalid data 2, the invalid data 2 is logically deleted, and the valid data recorded in the 5 th row is accumulated from 6 to the accumulated value of the data table, and 15-2+6=19 is obtained. The step is completed to perform positive flushing processing on the effective data corresponding to the service identifier in the data table, so that the same service identifier in the data table corresponds to at most one unlocked effective data.
In some embodiments, to achieve the protection effect on the correct data recorded in the data table, it is also possible to: and receiving a data locking request, starting a locking program, locking the valid data corresponding to all the current service identifiers, and updating the running index value to be an initial value. For example, as shown in fig. 3, a lock data request is received in line 5, locking is performed on all valid data in lines 1-4, accumulated values 14 to corresponding positions of line 5 of all valid data in lines 1-4 are recorded, and the cursor value is updated to be an initial value of 0; the valid data 5 recorded in the 6 th row is 1, if the valid data corresponding to the service identifier 1 in the data table is queried, the valid data 5 recorded in the current row and the valid data 5 recorded in the 1 st row are obtained, but because the valid data 5 recorded in the 1 st row is locked, the valid data corresponding to the service identifier 1 is not subjected to positive flushing processing, only the newly obtained valid data is accumulated in the 6 th row, and the accumulated value updated by the data table is 14+5=19. In this step, the locking mechanism is set up to avoid erroneous deletion of correct data in the data table, thereby further improving the working efficiency of the data table.
In some embodiments, to monitor the accuracy of the data table in real time, it is also possible to: and acquiring the number of the data which have the same service identifier and are marked as valid in the data table, and if the number is determined to be larger than the preset number threshold corresponding to the service identifier, generating and outputting false prompt information of the data table. For example, as shown in fig. 3, after the valid data 5 is recorded in the 6 th row, the number of valid data corresponding to the service identifier 1 in the data table is 2, if the preset number threshold of the service identifier 1 is 1, it indicates that the data table only corresponds to 1 valid data in the case of correctly recording data, and at this time, the number of valid data corresponding to the service identifier 1 in the data table exceeds the preset threshold 1, it indicates that the situation of disordered data recording occurs, and error prompt information is generated and output in the data table.
Fig. 4 is a schematic diagram of the main flow of a data processing method according to a second embodiment of the present invention, the data processing method including:
Step S401, obtaining new data, obtaining time attribute of service type label, generating service identification corresponding to the service type label according to the time attribute.
Preferably, the values of the service identifiers in the data table are all positive integers.
Step S402, according to the different service identifiers, the generated sub data tables are combined according to the size sequence of the service identifiers to generate a data table; initializing a cursor value of 0 in the data table.
Step S403, marking the new data state as valid, and recording the valid data to a data table.
Step S404, judging whether the service identifier corresponding to the effective data is larger than a cursor value. If yes, go to step S405, otherwise go to step S406.
And step S405, updating the cursor value to the service identifier, and accumulating all effective data in the data table.
Preferably, if the service identifier corresponding to the valid data is smaller than or equal to the running index value, the current running index value is maintained.
Step S406, judging whether to perform positive flushing on the effective data corresponding to the service identifier in the data table. If yes, go to step S408, otherwise go to step S407.
In step S407, all the data marked as valid in the data table are accumulated.
Step S408, the effective data corresponding to the service identifier is processed by positive punching, and the effective data corresponding to the service identifier and the effective data corresponding to other service identifiers after the positive punching are accumulated.
Step S409, judging whether a cursor value resetting request is received. If yes, go to step S410, otherwise go to step S411.
Step S410, resetting the cursor value, screening to obtain effective data with service identifiers larger than the cursor value in the data table, updating the marks of the data to be invalid, and logically deleting the data updated to be invalid; all valid data in the data table is accumulated.
Preferably, logical deletion is defined as not deleting the corresponding data record, only marking the data as invalid so that it does not account for the data table accumulation calculation.
In step S411, it is determined whether there is new data to be acquired. If yes, go to step S403, otherwise go to step S412.
Step S412, accumulating the locked and unlocked valid data in the data table; outputting the data table and the accumulated value of the data table.
Fig. 5 is a schematic diagram of the main flow of a data processing method according to a third embodiment of the present invention, the data processing method including:
Step S501, obtaining new data, obtaining the time attribute of the service type label, and generating the service identification corresponding to the service type label according to the time attribute.
Preferably, the values of the service identifiers in the data table are all positive integers.
Step S502, according to the different service identifiers, the generated sub data tables are combined according to the size sequence of the service identifiers to generate a data table; initializing a cursor value of 0 in the data table.
In step S503, the new data status is marked as valid, and the valid data is recorded into the data table.
Step S504, judging whether the service identifier corresponding to the effective data is larger than a cursor value, if so, executing step S505, otherwise, executing step S506.
And step S505, updating the cursor value to the service identifier, and accumulating all valid data in the data table.
Preferably, if the service identifier corresponding to the valid data is smaller than or equal to the running index value, the current running index value is maintained.
Step S506, judging whether to flush the effective data corresponding to the service identifier in the data table, if yes, executing step S508, otherwise, executing step S507.
In step S507, all data marked as valid in the data table are accumulated.
And step S508, the effective data corresponding to the service identifier is subjected to positive punching, and the effective data corresponding to the service identifier and the effective data corresponding to other service identifiers after positive punching are accumulated.
Step S509, judging whether a cursor value resetting request is received, if yes, executing step S510, and if not, executing step S511.
Step S510, resetting the cursor value, screening to obtain effective data with service identifiers larger than the cursor value in the data table, updating the marks of the data to be invalid, and logically deleting the data updated to be invalid; all valid data in the data table is accumulated.
Preferably, logical deletion is defined as not deleting the corresponding data record, only marking the data as invalid so that it does not account for the data table accumulation calculation.
Step S511, determining whether a lock data request is received, if yes, executing step S512, and if not, executing step S513.
Step S512, obtaining time information, accumulating effective data corresponding to all service identifiers in a data table, and recording the accumulated result and the time information to the data table; locking all valid data in the data table; updating the cursor value in the data table to be 0.
Preferably, no data flushing is performed on valid data that has been locked in the data table.
Step S513, determining whether new data still exists to be acquired, if yes, executing step S503, and if not, executing step S514.
Step S514, accumulating the locked and unlocked valid data in the data table; outputting the data table and the accumulated value of the data table.
Fig. 6 is a schematic diagram of main modules of a data processing apparatus according to an embodiment of the present invention, and as shown in fig. 6, the data processing apparatus 600 includes an acquisition module 601 and an identification module 602. The acquiring module 601 acquires a data table and new data, marks the new data state as valid, and records the valid data to the data table; the identification module 602 judges whether the service identifier corresponding to the valid data is greater than a cursor value, if the service identifier is greater than the cursor value, the cursor value is updated to the service identifier, and all valid data in a data table are accumulated; if the service identifier is not greater than the running index value, judging whether to perform positive flushing on the effective data corresponding to the service identifier in the data table; accumulating all data marked as valid in the data table in response to not flushing the valid data corresponding to the service identifier; or in response to the effective data corresponding to the service identifier after the service identifier is punched, accumulating the effective data corresponding to the service identifier after the service identifier is punched and the effective data corresponding to other service identifiers.
In some embodiments, the identification module 602 is further configured to: acquiring new data, and generating a corresponding service identifier according to upstream and downstream service attributes in a service type label of the new data, wherein the service identifier value of an upstream service is smaller than the service identifier value of a corresponding downstream service; generating a sub-data table corresponding to the service identifier for the data with the same service identifier; and merging all the sub data tables according to the sequence of the service identification to generate a data table, and initializing the cursor value of the data table.
In some embodiments, the identification module 602 is further configured to: responding to the reset of the cursor value, obtaining the cursor value after the reset, and replacing the cursor value in the data table; screening according to the data table to obtain data with service identifiers larger than the replaced post-cursor value and marked as valid, updating the marks of the data as invalid, and logically deleting the data updated as invalid; all data marked as valid in the data table are accumulated.
In some embodiments, the identification module 602 is further configured to: screening to obtain data corresponding to the service identifier in the data table, and determining that the data mark recorded last time except the new data is valid, and then flushing the valid data corresponding to the service identifier.
In some embodiments, the identification module 602 is further configured to: and acquiring effective data corresponding to the service identifier in the data table, screening to obtain effective data which is recorded last time except the new data in the effective data and is used as punched positive data, updating the mark of the punched positive data into invalid, and logically deleting the punched positive data which is updated into invalid.
In some embodiments, the identification module 602 is further configured to: and receiving a data locking request, starting a locking program, locking the valid data corresponding to all the current service identifiers, and updating the running index value to be an initial value.
In some embodiments, the identification module 602 is further configured to: and acquiring the number of the data which has the same service identifier and is marked as effective in the data table, and if the number is determined to be larger than the preset number threshold corresponding to the service identifier, generating and outputting false prompt information of the data table.
In the present invention, the data processing method and the data processing apparatus have a corresponding relationship in terms of implementation, and therefore, the description of the repeated contents is omitted.
Fig. 7 illustrates an exemplary system architecture 700 to which a data processing method or data processing apparatus of an embodiment of the present invention may be applied.
As shown in fig. 7, a system architecture 700 may include terminal devices 701, 702, 703, a network 704, and a server 705. The network 704 is the medium used to provide communication links between the terminal devices 701, 702, 703 and the server 705. The network 704 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.
A user may interact with the server 705 via the network 704 using the terminal devices 701, 702, 703 to receive or send messages or the like. Various communication client applications can be installed on the terminal devices 701, 702, 703.
The terminal devices 701, 702, 703 may be various electronic devices having a page display screen and supporting web browsing, including but not limited to smartphones, tablets, laptop and desktop computers, and the like.
The server 705 may be a server providing various services, such as a background management server (by way of example only) providing support for users with the terminal devices 701, 702, 703. The background management server may analyze and process the received data such as the product information query request, and feedback the processing result (e.g., the target push information, the product information—only an example) to the terminal device.
It should be noted that, the data processing method provided by the embodiment of the present invention is generally executed by the server 705, and accordingly, the computing device is generally disposed in the server 705.
It should be understood that the number of terminal devices, networks and servers in fig. 7 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
Referring now to FIG. 8, there is illustrated a schematic diagram of a computer system 800 suitable for use in implementing an embodiment of the present invention. The terminal device shown in fig. 8 is only an example, and should not impose any limitation on the functions and the scope of use of the embodiment of the present invention.
As shown in fig. 8, the computer system 800 includes a Central Processing Unit (CPU) 801 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 802 or a program loaded from a storage section 808 into a Random Access Memory (RAM) 803. In the RAM803, various programs and data required for the operation of the computer system 800 are also stored. The CPU801, ROM802, and RAM803 are connected to each other by a bus 804. An input/output (I/O) interface 805 is also connected to the bus 804.
The following components are connected to the I/O interface 805: an input section 805 including a keyboard, a mouse, and the like; an output portion 806 including a Cathode Ray Tube (CRT), a liquid crystal data processor (LCD), and the like, and a speaker, and the like; a storage section 808 including a hard disk or the like; and a communication section 809 including a network interface card such as a LAN card, a modem, or the like. The communication section 809 performs communication processing via a network such as the internet. The drive 810 is also connected to the I/O interface 805 as needed. A removable medium 811 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 810 as needed so that a computer program read out therefrom is mounted into the storage section 808 as needed.
In particular, according to embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication section 809, and/or installed from the removable media 811. The above-described functions defined in the system of the present invention are performed when the computer program is executed by a Central Processing Unit (CPU) 801.
The computer readable medium shown in the present invention may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules involved in the embodiments of the present invention may be implemented in software or in hardware. The described modules may also be provided in a processor, for example, as: a processor includes an acquisition module and an identification module. The names of these modules do not constitute a limitation on the module itself in some cases.
As another aspect, the present invention also provides a computer-readable medium that may be contained in the apparatus described in the above embodiments; or may be present alone without being fitted into the device. The computer readable medium carries one or more programs which, when executed by the apparatus, cause the apparatus to include acquiring a data table and new data, marking the new data status as valid, and recording the valid data to the data table; judging whether the service identifier corresponding to the effective data is larger than a cursor value or not; if the service identifier is larger than the cursor value, updating the cursor value into the service identifier, and accumulating all effective data in a data table; if the service identifier is not greater than the running index value, judging whether to perform positive flushing on the effective data corresponding to the service identifier in the data table; accumulating all data marked as valid in the data table in response to not flushing the valid data corresponding to the service identifier; or in response to the effective data corresponding to the service identifier after the service identifier is punched, accumulating the effective data corresponding to the service identifier after the service identifier is punched and the effective data corresponding to other service identifiers.
According to the technical scheme provided by the embodiment of the invention, the technical problem that the reset data cannot be accurately and efficiently identified in the prior art can be solved.
The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives can occur depending upon design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.
Claims (10)
1. A method of data processing, comprising:
Acquiring a data table and new data, marking the new data state as valid, and recording the valid data to the data table;
judging whether the service identifier corresponding to the effective data is larger than a cursor value or not;
If the service identifier is larger than the cursor value, updating the cursor value into the service identifier, and accumulating all effective data in the data table;
If the service identifier is not greater than the running index value, judging whether to perform positive flushing on the effective data corresponding to the service identifier in the data table; accumulating all data marked as valid in the data table in response to not flushing the valid data corresponding to the service identifier; or in response to the effective data corresponding to the service identifier after the service identifier is punched, accumulating the effective data corresponding to the service identifier after the service identifier is punched and the effective data corresponding to other service identifiers.
2. The method of claim 1, wherein obtaining the data table and the new data comprises:
Acquiring new data, and generating a corresponding service identifier according to upstream and downstream service attributes in a service type label of the new data, wherein the service identifier value of an upstream service is smaller than the service identifier value of a corresponding downstream service;
generating a sub-data table corresponding to the service identifier for the data with the same service identifier;
and merging all the sub data tables according to the sequence of the service identification to generate a data table, and initializing the cursor value of the data table.
3. The method according to claim 1, characterized in that it comprises:
Responding to the reset of the cursor value, acquiring the reset cursor value, and replacing the cursor value in the data table;
screening according to the data table to obtain data with service identifiers larger than the replaced post-cursor value and marked as valid, updating the marks of the data as invalid, and logically deleting the data updated as invalid;
and accumulating all the data marked as valid in the data table.
4. The method of claim 1, wherein determining whether to flush valid data corresponding to the service identifier according to the data table comprises:
And screening to obtain data corresponding to the service identifier in the data table, determining that the data which is recorded last time except for new data is marked as valid, and correcting the valid data corresponding to the service identifier.
5. The method of claim 1, wherein the flushing the valid data corresponding to the service identifier comprises:
And acquiring effective data corresponding to the service identifier in the data table, screening to obtain effective data which is recorded last time except the new data in the effective data and is used as punched positive data, updating the mark of the punched positive data into invalid, and logically deleting the punched positive data which is updated into invalid.
6. The method according to claim 5, comprising:
And receiving a data locking request, starting a locking program, locking the valid data corresponding to all the current service identifiers, and updating the running index value to be an initial value.
7. The method as recited in claim 1, further comprising:
and acquiring the number of the data which has the same service identifier and is marked as effective in the data table, and if the number is determined to be larger than the preset number threshold corresponding to the service identifier, generating and outputting false prompt information of the data table.
8. A page display processing apparatus, comprising:
The acquisition module is used for acquiring a data table and new data, marking the new data state as effective, and recording the effective data to the data table;
The identification module is used for judging whether the service identifier corresponding to the effective data is larger than a cursor value or not; if the service identifier is larger than the cursor value, updating the cursor value into the service identifier, and accumulating all effective data in a data table; if the service identifier is not greater than the running index value, judging whether to perform positive flushing on the effective data corresponding to the service identifier in the data table; accumulating all data marked as valid in the data table in response to not flushing the valid data corresponding to the service identifier; or in response to the effective data corresponding to the service identifier after the service identifier is punched, accumulating the effective data corresponding to the service identifier after the service identifier is punched and the effective data corresponding to other service identifiers.
9. An electronic device, comprising:
one or more processors;
storage means for storing one or more programs,
When executed by the one or more processors, causes the one or more processors to implement the method of any of claims 1-7.
10. A computer readable medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the method according to any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211610122.3A CN118193519A (en) | 2022-12-14 | 2022-12-14 | Data processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211610122.3A CN118193519A (en) | 2022-12-14 | 2022-12-14 | Data processing method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN118193519A true CN118193519A (en) | 2024-06-14 |
Family
ID=91404913
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211610122.3A Pending CN118193519A (en) | 2022-12-14 | 2022-12-14 | Data processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118193519A (en) |
-
2022
- 2022-12-14 CN CN202211610122.3A patent/CN118193519A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP6217644B2 (en) | Rule distribution server, event processing system, method and program | |
| CN107247798B (en) | Method and device for constructing search word bank | |
| US11599425B2 (en) | Method, electronic device and computer program product for storage management | |
| US10205813B2 (en) | Method and system for detecting abnormal contact information and server | |
| CN112118352B (en) | Method and device for processing notification trigger message, electronic equipment and computer readable medium | |
| CN110866001A (en) | Method and device for determining order to be processed | |
| CN113761565B (en) | Data desensitization method and device | |
| CN112433757A (en) | Method and device for determining interface calling relationship | |
| CN113590543A (en) | Method, apparatus and computer program product for information processing | |
| CN113220705A (en) | Slow query identification method and device | |
| CN109214846B (en) | Information storage method and device | |
| CN118193519A (en) | Data processing method and device | |
| CN112887426B (en) | Information stream pushing method and device, electronic equipment and storage medium | |
| CN111813765B (en) | Method, device, electronic equipment and computer readable medium for processing abnormal data | |
| CN109087097B (en) | Method and device for updating same identifier of chain code | |
| CN114064803A (en) | Data synchronization method and device | |
| CN111950232B (en) | Method and device for automatically switching number segments | |
| CN113760695A (en) | Method and device for positioning problem code | |
| CN113760600A (en) | Database backup method, database restoration method and related device | |
| CN113742376A (en) | Data synchronization method, first server and data synchronization system | |
| CN111431764A (en) | Node determination method, device, system and medium | |
| CN112711572A (en) | Online capacity expansion method and device suitable for sub-warehouse and sub-meter | |
| CN111179097B (en) | Method, device, electronic equipment and storage medium for modifying warranty | |
| CN113726885B (en) | Flow quota adjusting method and device | |
| CN111290927A (en) | Data monitoring method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination |