CN112507010B - Service data processing method and device, computer equipment and storage medium - Google Patents
Service data processing method and device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN112507010B CN112507010B CN202011466073.1A CN202011466073A CN112507010B CN 112507010 B CN112507010 B CN 112507010B CN 202011466073 A CN202011466073 A CN 202011466073A CN 112507010 B CN112507010 B CN 112507010B
- Authority
- CN
- China
- Prior art keywords
- service data
- data
- service
- state value
- work task
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/252—Integrating or interfacing systems involving database management systems between a Database Management System and a front-end application
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24553—Query execution of query operations
- G06F16/24554—Unary operations; Data partitioning operations
- G06F16/24556—Aggregation; Duplicate elimination
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2471—Distributed queries
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Probability & Statistics with Applications (AREA)
- Mathematical Physics (AREA)
- Fuzzy Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application relates to a business data processing method, a business data processing device, a computer device and a storage medium. The method comprises the following steps: acquiring service data, a work task schedule table and a work task configuration table; the work task schedule includes serialized work tasks; the work task configuration table records the corresponding relation between the work tasks and the service data state values of the service data. Initializing and generating a service data state index table according to the work task schedule; the service data state index table includes a current service data state value of the service data. And executing the working task corresponding to the current business data state value so as to perform corresponding business processing on the business data. And when the execution of the work task is finished, updating the current business data state value. And when the updated current business data state value can be migrated, returning to execute the work task corresponding to the current business data state value so as to perform the step of corresponding business processing on the business data to continue executing. By adopting the method, the service processing efficiency can be improved.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a method and an apparatus for processing service data, a computer device, and a storage medium.
Background
With the development of computer technology, big data technology appears, which refers to a data set that cannot be captured, managed and processed by a conventional software tool within a certain time range, and is a massive, high-growth rate and diversified information asset that needs a new processing mode to have stronger decision-making power, insight discovery power and process optimization capability. At present, people's life increasingly depends on intelligent service systems, each service system generates new service data at every moment, the service data volume is increasingly large, the service data storage is increasingly complicated, the difficulty of processing the service data is high, and therefore the service processing efficiency of the service data is low.
Disclosure of Invention
In view of the foregoing, it is necessary to provide a service data processing method, a device, a computer device, and a storage medium, which can improve service processing efficiency.
A method for processing service data, the method comprising:
acquiring service data, a work task schedule and a work task configuration table; the work task schedule comprises serialized work tasks; the corresponding relation between the work task and the service data state value of the service data is recorded in the work task configuration table;
initializing and generating a service data state index table according to the work task schedule; the service data state index table comprises the current service data state value of the service data;
executing a working task corresponding to the current business data state value to perform corresponding business processing on the business data;
when the execution of the work task is finished, updating the current business data state value;
and when the updated current business data state value can be migrated, returning to execute the work task corresponding to the current business data state value so as to perform the step of corresponding business processing on the business data to continue execution.
In one embodiment, the acquiring the service data includes:
acquiring service data of a text type;
formatting the text type service data according to a formatting function to obtain preset storage type service data;
and converting the service data of the preset storage type according to a conversion function to obtain service data resolvable by the work task.
In one embodiment, after the executing the work task corresponding to the current service data state value to perform corresponding service processing on the service data, the method further includes:
generating a service processing result corresponding to the service data;
determining the importance level of the service data;
and storing the service processing result to a storage device corresponding to the importance level of the service data.
In one embodiment, the method further comprises:
receiving a data query request sent by a user terminal through a network service; the data query request carries a data identifier of the service data;
responding to the data query request, and querying a service processing result corresponding to the data identifier;
and sending the service processing result obtained by the query to the user terminal.
In one embodiment, the receiving a data query request sent by a user terminal through a network service includes:
determining a request type and a request mode corresponding to a data query request sent by a user terminal through network service;
searching a target data query interface matched with the request type and the request mode from the candidate data query interfaces according to a preset identification field of the candidate data query interface;
and receiving a data query request sent by a user terminal through a network service through the target data query interface.
In one embodiment, the method further comprises:
determining result summarizing conditions;
summarizing the service processing results meeting the result summarizing conditions to obtain a corresponding summarized report;
and broadcasting the summarized report to a report receiving terminal.
In one embodiment, the executing a work task corresponding to the current service data state value to perform corresponding service processing on the service data includes:
executing a working task corresponding to the current business data state value to identify and process different types of business data through an aggregation operation model, and obtaining business data conforming to a preset field format;
and carrying out aggregation operation processing on the service data which accords with the preset field format.
A traffic data processing apparatus, the apparatus comprising:
the acquisition module is used for acquiring service data, a work task schedule and a work task configuration table; the work task schedule comprises serialized work tasks; the corresponding relation between the work task and the service data state value of the service data is recorded in the work task configuration table;
the generating module is used for initializing and generating a service data state index table according to the work task schedule; the service data state index table comprises the current service data state value of the service data;
the processing module is used for executing a working task corresponding to the current business data state value so as to perform corresponding business processing on the business data;
the updating module is used for updating the current service data state value when the execution of the work task is finished; and when the updated current business data state value can be migrated, informing the processing module to execute a work task corresponding to the current business data state value so as to perform a corresponding business processing step on the business data to continue execution.
A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the following steps when executing the computer program:
acquiring service data, a work task schedule and a work task configuration table; the work task schedule comprises serialized work tasks; the corresponding relation between the work task and the service data state value of the service data is recorded in the work task configuration table;
initializing to generate a service data state index table according to the work task schedule; the service data state index table comprises the current service data state value of the service data;
executing a working task corresponding to the current business data state value to perform corresponding business processing on the business data;
when the execution of the work task is finished, updating the current business data state value;
and when the updated current business data state value can be migrated, returning to execute the work task corresponding to the current business data state value so as to perform the step of corresponding business processing on the business data to continue execution.
A computer-readable storage medium, on which a computer program is stored which, when executed by a processor, carries out the steps of:
acquiring service data, a work task schedule and a work task configuration table; the work task schedule comprises serialized work tasks; the corresponding relation between the work task and the service data state value of the service data is recorded in the work task configuration table;
initializing and generating a service data state index table according to the work task schedule; the service data state index table comprises the current service data state value of the service data;
executing a work task corresponding to the current business data state value to perform corresponding business processing on the business data;
when the execution of the work task is finished, updating the current business data state value;
and when the updated current business data state value can be migrated, returning to the step of executing the work task corresponding to the current business data state value so as to perform the step of corresponding business processing on the business data to continue executing.
The business data processing method, the device, the computer equipment and the storage medium acquire the business data, the work task schedule and the work task configuration table; the work task schedule comprises serialized work tasks; the corresponding relation between the work task and the service data state value of the service data is recorded in the work task configuration table. Initializing and generating a service data state index table according to the work task schedule; the service data state index table includes a current service data state value of the service data. And executing the working task corresponding to the current business data state value so as to perform corresponding business processing on the business data. And when the execution of the work task is finished, updating the current service data state value. And when the updated current business data state value can be migrated, returning to execute the work task corresponding to the current business data state value so as to perform the step of corresponding business processing on the business data to continue executing. Therefore, the work task schedule is pre-configured, and the service data state index table automatically controls the serialized work tasks in the work task schedule to be sequentially executed, so that the corresponding service processing is automatically performed on the service data, the difficulty of performing the service processing on the service data is reduced, and the service processing efficiency of the service data is improved.
Drawings
Fig. 1 is an application scenario diagram of a service data processing method in an embodiment;
fig. 2 is a schematic flowchart of a service data processing method in an embodiment;
FIG. 3 is a network topology diagram of a method for service data processing in one embodiment;
FIG. 4 is a system architecture diagram of a business data processing method in one embodiment;
FIG. 5 is a block diagram of a business data processing apparatus in one embodiment;
fig. 6 is a block diagram of a service data processing apparatus in another embodiment;
FIG. 7 is a diagram of the internal structure of a computer device in one embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
The service data processing method provided by the application can be applied to the application environment shown in fig. 1. The application environment includes a terminal 102 and a server 104. The terminal 102 and the server 104 communicate via a network. The terminal 102 may specifically include a desktop terminal or a mobile terminal. The mobile terminal may specifically include at least one of a mobile phone, a tablet computer, a notebook computer, and the like. The server 104 may be implemented as a stand-alone server or as a server cluster comprised of multiple servers. Those skilled in the art will understand that the application environment shown in fig. 1 is only a part of the scenario related to the present application, and does not constitute a limitation to the application environment of the present application.
The server 104 acquires service data, a work task schedule and a work task configuration table from the terminal 102; the work task schedule comprises serialized work tasks; the corresponding relation between the work task and the service data state value of the service data is recorded in the work task configuration table. The server 104 initializes and generates a service data state index table according to the work task schedule; the service data state index table includes a current service data state value of the service data. The server 104 executes a work task corresponding to the current service data state value to perform corresponding service processing on the service data. When the execution of the work task ends, the server 104 updates the current business data state value. When the updated current service data state value is migratable, the server 104 returns to execute the work task corresponding to the current service data state value, so as to perform the step of performing corresponding service processing on the service data to continue execution.
In an embodiment, as shown in fig. 2, a service data processing method is provided, which is described by taking the application of the method to the server 104 in fig. 1 as an example, and includes the following steps:
s202, acquiring service data, a work task schedule table and a work task configuration table; the work task schedule comprises serialized work tasks; the corresponding relation between the work task and the service data state value of the service data is recorded in the work task configuration table.
The work task refers to performing corresponding business processing on the business data, and for example, the work task may specifically include at least one of an aggregation operation processing task, a statistics task, a backup task, a migration task, and the like. The serialized work tasks refer to the work tasks in the work task schedule being ordered in sequence according to the execution order of the work tasks. The service data state value is an enumeration value of the service data state and is used for caching and recording the current workflow state of the service data.
Specifically, the server may obtain business data generated by the application. The user can configure the work task schedule and the work task configuration table in a user-defined mode, and the server can obtain the work task schedule and the work task configuration table configured by the user in the user-defined mode.
For example, the service data status value is denoted by ST, ST =0 denotes that the current service data has no corresponding work task, ST =1 denotes that the current service data is performing a statistical task, ST =2 denotes that the statistical task of the current service data has been completed, ST =3 denotes that the current service data is performing a backup task, ST =4 denotes that the backup task of the current service data has been completed, and ST =5 denotes that the current service data is performing a migration task.
S204, initializing to generate a business data state index table according to the work task schedule; the service data state index table includes a current service data state value of the service data.
Specifically, the server may start the work task schedule, and use the service data state value corresponding to the first work task in the work tasks serialized in the work task schedule as the current service data state value. The server can initialize and generate a business data state index table according to the current business data state value.
And S206, executing the working task corresponding to the current service data state value so as to perform corresponding service processing on the service data.
Specifically, the server may determine the current service data state value from the service data state index table, and determine a work task corresponding to the current service data state value. The server can execute the work task corresponding to the current service data state value so as to perform corresponding service processing on the service data.
In one embodiment, the server may manage the work tasks through the file system, for example, periodically execute the corresponding work tasks through a crontab file, monitor the execution of the work tasks through a screen file, and implement atomicity of the business process by creating a state file (e.g., x.stop) under the execution directory.
And S208, when the execution of the work task is finished, updating the current service data state value.
Specifically, the server may monitor the execution state of the work task in real time for each work task. When the execution of the work task is finished, the server can automatically update the current service data state value in the service data state index table.
For example, when the current service data state value is ST =1, the server may perform a work task corresponding to ST = 1. When the execution of the work task corresponding to ST =1 is finished, the server may update the current traffic data state value ST =1 to ST =2, that is, ST =2 is a new current traffic data state value.
And S210, when the updated current business data state value can be migrated, returning to execute the work task corresponding to the current business data state value, so as to perform the step of corresponding business processing on the business data to continue execution.
The current service data state value can be migrated, which means that a service data state value larger than the current service data state value exists in the work task schedule. It will be appreciated that the traffic data state value is a transition from a low state to a high state. For example, if the current traffic data state value is ST =1, and a traffic data state value ST =2 larger than the current traffic data state value ST =1 exists in the work task plan table, it indicates that the current traffic data state value ST =1 may be migrated to ST =2.
Specifically, the server may obtain the current business data state value and the largest business data state value in the work tasks serialized in the work task schedule. The server can compare the current business data state value with the largest business data state value in the work tasks serialized in the work task schedule. When the current business data state value is smaller than the largest business data state value in the work tasks serialized in the work task schedule, the updated current business data state value is shown to be transferable, and at the moment, the server can return to execute the work task corresponding to the current business data state value so as to perform the step of corresponding business processing on the business data to continue execution.
In the business data processing method, business data, a work task schedule and a work task configuration table are obtained; the work task schedule comprises serialized work tasks; the corresponding relation between the work task and the service data state value of the service data is recorded in the work task configuration table. Initializing and generating a service data state index table according to the work task schedule; the service data state index table includes a current service data state value of the service data. And executing the working task corresponding to the current business data state value so as to perform corresponding business processing on the business data. And when the execution of the work task is finished, updating the current business data state value. And when the updated current business data state value can be migrated, returning to execute the work task corresponding to the current business data state value so as to perform the step of corresponding business processing on the business data to continue executing. Therefore, the work task schedule is pre-configured, and the service data state index table automatically controls the serialized work tasks in the work task schedule to be sequentially executed, so that the corresponding service processing is automatically performed on the service data, the difficulty of performing the service processing on the service data is reduced, and the service processing efficiency of the service data is improved.
In an embodiment, the step of acquiring the service data in step S202 specifically includes: acquiring text type service data; formatting the text type service data according to a formatting function to obtain preset storage type service data; and converting the preset storage type service data according to the conversion function to obtain service data resolvable by the work task.
Specifically, the application device may generate a log file, and the server may acquire text-type service data from the log file. The user can customize the preset storage type of the service data through the server. The server can format the service data of the text type according to the format function format to obtain the service data of the preset storage type. The server can convert the preset storage type service data according to the conversion function convert to obtain service data resolvable by the work task.
In the above embodiment, the text-type service data is formatted through the formatting function, so that the text-type service data is converted into the preset storage-type service data, and the preset storage-type service data is converted into the service data that can be resolved by the job task according to the conversion function, thereby facilitating the service processing of the service data and further improving the service data processing efficiency.
In an embodiment, after step S206, that is, after the step of executing the work task corresponding to the current service data state value to perform corresponding service processing on the service data, the service data processing method specifically further includes: generating a service processing result corresponding to the service data; determining the importance level of the service data; and storing the service processing result to a storage device corresponding to the importance level of the service data.
Specifically, after performing corresponding service processing on the service data, the server may generate a service processing result corresponding to the service data. Different service data can correspond to corresponding importance levels, some service data have high importance levels, and some service data have low importance levels. The server can determine the importance level of the service data and store the service processing result to the storage device corresponding to the importance level of the service data.
In the embodiment, the service processing result is stored in the storage device corresponding to the important level of the service data, so that the safety of the service processing result is ensured, and the reasonable utilization of the storage resource energy is facilitated.
In an embodiment, the method for processing service data specifically further includes: receiving a data query request sent by a user terminal through network service; the data query request carries a data identifier of the service data; responding to the data query request, and querying a service processing result corresponding to the data identifier; and sending the service processing result obtained by the query to the user terminal.
The data identifier of the service data is a character string for uniquely identifying the service data. The data identification includes at least one of a number, a letter, and a special character. The special characters may include at least one of underlines, number keys, percentiles, and the like.
Specifically, the user terminal may generate a data query request and transmit the data query request to the server through the web service. The server can receive a data query request sent by the user terminal through network service; the data query request carries the data identifier of the service data. The server can respond to the data query request and query the business processing result corresponding to the data identification. The server can send the service processing result obtained by the query to the user terminal.
In the embodiment, the business processing result corresponding to the data identifier is queried in response to the data query request, so that the query efficiency of the business processing result is improved.
In one embodiment, the step of receiving a data query request sent by a user terminal through a network service specifically includes: determining a request type and a request mode corresponding to a data query request sent by a user terminal through network service; searching a target data query interface matched with the request type and the request mode from the candidate data query interfaces according to the identification field of the preset candidate data query interface; and receiving a data query request sent by the user terminal through the network service through the target data query interface.
The request type is an operation type for the service data, and for example, the request type may specifically include any one of a GET method request type, a PUT method request type, a DELETE method request type, and a POST method request type of an HTTP (HyperText Transfer Protocol) Protocol. The identification field is an identification field of the data query interface for uniquely identifying the data query interface.
Specifically, the data query requests sent by the user terminal through the network service all correspond to request types and request modes. The server can determine the request type and the request mode corresponding to the data query request sent by the user terminal through the network service. The server is provided with a preset candidate data query interface, the candidate data query interface is defined with an identification field, and the identification field is associated with the request type and the request mode of the network service. The server can search a target data query interface matched with the request type and the request mode from the candidate data query interfaces according to the preset identification fields of the candidate data query interfaces, and receive a data query request sent by the user terminal through the network service through the target data query interface.
In one embodiment, the identification field of the data query interface may be provided in a prefix portion of the data query interface, in a suffix portion of the data query interface, or in another portion of the data query interface.
In one embodiment, the identification field may specifically include GET reading data, restricting the GET method using the HTTP protocol; update: updating and modifying data, and limiting the PUT method using the HTTP protocol; insert: adding insertion data and limiting the POST method using HTTP; DELETE, DELETE the data already backed up, limit using the DELETE method of HTTP agreement; upsert: if yes, the method is increased, and the PUT method using the HTTP protocol is limited; an execute step of submitting a task execution plan and limiting the use of a POST method of an HTTP protocol; report, acquiring the statistical result of the data table, and limiting the GET method using HTTP protocol; and lock, namely locking the specified data table and limiting the POST method using the HTTP protocol.
In the above embodiment, the target data query interface matched with the request type and the request mode is searched from the candidate data query interfaces through the preset identification field of the candidate data query interface, so that the query efficiency can be improved.
In an embodiment, the method for processing service data specifically further includes: determining result summarizing conditions; summarizing the service processing results meeting the result summarizing conditions to obtain a corresponding summarizing report; and broadcasting the summarized report to a report receiving terminal.
And the result summarizing condition is a condition for summarizing the business processing result.
Specifically, the user may set a result summarizing condition, and the server may acquire the result summarizing condition set by the user. The server can judge whether the service processing result meets the result summarizing condition or not, and summarize the service processing result meeting the result summarizing condition to obtain a corresponding summarized report. The server can broadcast the summarized report to the report receiving terminal.
In the embodiment, the result summarizing conditions are determined, the service processing results meeting the result summarizing conditions are automatically summarized, the corresponding summary report is obtained, and the processing efficiency of the service processing results is improved.
In an embodiment, step S206, that is, a step of executing a work task corresponding to the current service data state value to perform corresponding service processing on the service data specifically includes: executing a working task corresponding to the current business data state value to identify and process different types of business data through an aggregation operation model, and obtaining business data conforming to a preset field format; and performing aggregation operation processing on the service data conforming to the preset field format.
Specifically, the server may train the aggregation operation model through the sample service data, so as to obtain the trained aggregation operation model. The aggregation operation model is used for identifying the service data in the preset field format. The server can execute a working task corresponding to the current business data state value so as to identify and process different types of business data through the aggregation operation model and obtain the business data conforming to the preset field format. The server can carry out aggregation operation processing on the service data which accords with the preset field format.
In one embodiment, the field format of the date satisfies the preset field format, and the aggregation operation model may perform aggregation operation processing on the service data according to the date. For example, the aggregation calculation processing is performed on the traffic data with the date of 2020, 8 months and 8 days, for example, the traffic data such as the vehicle speed data, the traffic light data in front of the vehicle, and the distance data of the vehicle from the obstacle.
In the embodiment, the aggregation operation model is used for identifying and processing the different types of service data to obtain the service data conforming to the preset field format, and the aggregation operation processing is performed on the service data conforming to the preset field format to realize the same processing on the different types of service data, so that the service data processing efficiency is further improved.
In one embodiment, as shown in FIG. 3, the application services of the global device may generate a large amount of text-type business data and store the business data to the original database. The server can obtain the original text type service data from the original database or data lake, and perform corresponding service processing on the service data, and the user can search the service processing result generated after the service processing from the server. The type of the database comprises at least one of MySQL, SQLite, postgressSQL, qracle, tiDB, mongoDB and the like.
In one embodiment, as shown in fig. 4, a user may pre-configure a job task configuration table, a service data status index table, and a job task schedule table through a configuration terminal. The original database stores text type service data, and the server can process the text type service data through a formatting function and a conversion function. The server can automatically control and sequentially execute the work tasks in the work task schedule according to the service data state index table so as to perform corresponding service processing on the service data, such as statistical processing, aggregation operation processing, backup processing or migration processing. The user can inquire the service processing result and control the service data processing process through the network service. The server can also automatically generate a summary report form from the service processing result meeting the summary condition according to the preset summary condition, and broadcast the summary report form to the related users.
In one embodiment, the user can update the work task schedule and the work task configuration table through the network service without the need of a remote login server for reconfiguration, and the efficiency of business data processing is further improved.
It should be understood that although the various steps of fig. 2 are shown in order, the steps are not necessarily performed in order. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in fig. 2 may include multiple sub-steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of performing the sub-steps or stages is not necessarily sequential, but may be performed alternately or alternatively with other steps or at least some of the sub-steps or stages of other steps.
In one embodiment, as shown in fig. 5, there is provided a service data processing apparatus 500, including: an obtaining module 501, a generating module 502, a processing module 503 and an updating module 504, wherein:
an obtaining module 501, configured to obtain service data, a work task schedule table, and a work task configuration table; the work task schedule comprises serialized work tasks; the corresponding relation between the work task and the service data state value of the service data is recorded in the work task configuration table.
A generating module 502, configured to generate a service data state index table through initialization according to the work task schedule; the service data state index table includes a current service data state value of the service data.
The processing module 503 is configured to execute a work task corresponding to the current service data state value, so as to perform corresponding service processing on the service data.
An updating module 504, configured to update a current service data state value when execution of a work task is finished; and when the updated current business data state value can be migrated, informing the processing module to execute a work task corresponding to the current business data state value so as to perform the step of corresponding business processing on the business data to continue execution.
In one embodiment, the obtaining module 501 is further configured to obtain text-type service data; formatting the text type service data according to a formatting function to obtain preset storage type service data; and converting the service data of the preset storage type according to the conversion function to obtain service data resolvable by the work task.
In an embodiment, the processing module 503 is further configured to execute a work task corresponding to the current service data state value, so as to perform recognition processing on different types of service data through an aggregation operation model, and obtain service data conforming to a preset field format; and performing aggregation operation processing on the service data conforming to the preset field format.
Referring to fig. 6, in one embodiment, the service data processing apparatus 500 further includes: a storage module 505, a query module 506, and a summarization module 507, wherein:
a storage module 505, configured to generate a service processing result corresponding to the service data; determining the importance level of the service data; and storing the service processing result to a storage device corresponding to the importance level of the service data.
A query module 506, configured to receive a data query request sent by a user terminal through a network service; the data query request carries a data identifier of the service data; responding to the data query request, and querying a service processing result corresponding to the data identifier; and sending the service processing result obtained by the query to the user terminal.
A summarizing module 507, configured to determine a result summarizing condition; summarizing the service processing results meeting the result summarizing conditions to obtain a corresponding summarizing report; and broadcasting the summarized report to a report receiving terminal.
In one embodiment, the query module 506 is further configured to determine a request type and a request manner corresponding to a data query request sent by a user terminal through a network service; searching a target data query interface matched with the request type and the request mode from the candidate data query interfaces according to the identification field of the preset candidate data query interface; and receiving a data query request sent by the user terminal through the network service through the target data query interface.
The business data processing device acquires business data, a work task schedule table and a work task configuration table; the work task schedule comprises serialized work tasks; the corresponding relation between the work task and the service data state value of the service data is recorded in the work task configuration table. Initializing to generate a business data state index table according to the work task schedule; the service data state index table includes a current service data state value of the service data. And executing the working task corresponding to the current business data state value so as to perform corresponding business processing on the business data. And when the execution of the work task is finished, updating the current business data state value. And when the updated current business data state value can be migrated, returning to execute the work task corresponding to the current business data state value so as to perform the step of corresponding business processing on the business data to continue executing. Therefore, the work task schedule is pre-configured, and the service data state index table automatically controls the serialized work tasks in the work task schedule to be sequentially executed, so that the corresponding service processing is automatically performed on the service data, the difficulty of performing the service processing on the service data is reduced, and the service processing efficiency of the service data is improved.
For specific limitations of the service data processing apparatus, reference may be made to the above limitations on the service data processing method, which is not described herein again. The modules in the business data processing device can be wholly or partially implemented by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, a computer device is provided, which may be the server 104 in fig. 1 described above, and its internal structure diagram may be as shown in fig. 7. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operating system and the computer program to run on the non-volatile storage medium. The database of the computer device is used for storing the business data processing data. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a business data processing method.
Those skilled in the art will appreciate that the architecture shown in fig. 7 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, a computer device is provided, comprising a memory having a computer program stored therein and a processor that when executing the computer program performs the steps of:
acquiring service data, a work task schedule and a work task configuration table; the work task schedule comprises serialized work tasks; the corresponding relation between the work task and the service data state value of the service data is recorded in the work task configuration table;
initializing and generating a service data state index table according to the work task schedule; the service data state index table comprises the current service data state value of the service data;
executing a working task corresponding to the current business data state value to perform corresponding business processing on the business data;
when the execution of the work task is finished, updating the state value of the current service data;
and when the updated current business data state value can be migrated, returning to execute the work task corresponding to the current business data state value so as to perform the step of corresponding business processing on the business data to continue executing.
In one embodiment, the processor, when executing the computer program, further performs the steps of:
acquiring text type service data;
formatting the text type service data according to a formatting function to obtain preset storage type service data;
and converting the preset storage type service data according to the conversion function to obtain service data resolvable by the work task.
In one embodiment, the processor, when executing the computer program, further performs the steps of:
generating a service processing result corresponding to the service data;
determining the importance level of the service data;
and storing the service processing result to a storage device corresponding to the importance level of the service data.
In one embodiment, the processor, when executing the computer program, further performs the steps of:
receiving a data query request sent by a user terminal through network service; the data query request carries a data identifier of the service data;
responding to the data query request, and querying a service processing result corresponding to the data identifier;
and sending the service processing result obtained by the query to the user terminal.
In one embodiment, the processor, when executing the computer program, further performs the steps of:
determining a request type and a request mode corresponding to a data query request sent by a user terminal through network service;
searching a target data query interface matched with the request type and the request mode from the candidate data query interfaces according to the identification field of the preset candidate data query interface;
and receiving a data query request sent by the user terminal through the network service through the target data query interface.
In one embodiment, the processor, when executing the computer program, further performs the steps of:
determining result summarizing conditions;
summarizing the service processing results meeting the result summarizing conditions to obtain a corresponding summarizing report;
and broadcasting the summarized report to a report receiving terminal.
In one embodiment, the processor, when executing the computer program, further performs the steps of:
executing a working task corresponding to the current business data state value to identify and process different types of business data through an aggregation operation model, and obtaining business data conforming to a preset field format;
and performing aggregation operation processing on the service data which accords with the preset field format.
In one embodiment, a computer-readable storage medium is provided, having a computer program stored thereon, which when executed by a processor, performs the steps of:
acquiring service data, a work task schedule and a work task configuration table; the work task schedule comprises serialized work tasks; the corresponding relation between the work tasks and the service data state values of the service data is recorded in the work task configuration table;
initializing and generating a service data state index table according to the work task schedule; the service data state index table comprises the current service data state value of the service data;
executing a working task corresponding to the current business data state value to perform corresponding business processing on the business data;
when the execution of the work task is finished, updating the state value of the current service data;
and when the updated current business data state value can be migrated, returning to execute the work task corresponding to the current business data state value so as to perform the step of corresponding business processing on the business data to continue executing.
In one embodiment, the computer program when executed by the processor further performs the steps of:
acquiring text type service data;
formatting the service data of the text type according to a formatting function to obtain service data of a preset storage type;
and converting the service data of the preset storage type according to the conversion function to obtain service data resolvable by the work task.
In one embodiment, the computer program when executed by the processor further performs the steps of:
generating a service processing result corresponding to the service data;
determining the importance level of the service data;
and storing the service processing result to a storage device corresponding to the importance level of the service data.
In one embodiment, the computer program when executed by the processor further performs the steps of:
receiving a data query request sent by a user terminal through network service; the data query request carries a data identifier of the service data;
responding to the data query request, and querying a service processing result corresponding to the data identifier;
and sending the service processing result obtained by the query to the user terminal.
In one embodiment, the computer program when executed by the processor further performs the steps of:
determining a request type and a request mode corresponding to a data query request sent by a user terminal through network service;
searching a target data query interface matched with the request type and the request mode from the candidate data query interfaces according to the identification field of the preset candidate data query interface;
and receiving a data query request sent by the user terminal through the network service through the target data query interface.
In one embodiment, the computer program when executed by the processor further performs the steps of:
determining result summarizing conditions;
summarizing the service processing results meeting the result summarizing conditions to obtain a corresponding summarizing report;
and broadcasting the summarized report to a report receiving terminal.
In one embodiment, the computer program when executed by the processor further performs the steps of:
executing a working task corresponding to the current business data state value to identify and process different types of business data through an aggregation operation model, and obtaining business data conforming to a preset field format;
and performing aggregation operation processing on the service data conforming to the preset field format.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above may be implemented by hardware instructions of a computer program, which may be stored in a non-volatile computer-readable storage medium, and when executed, may include the processes of the embodiments of the methods described above. Any reference to memory, storage, database or other medium used in the embodiments provided herein can include non-volatile and/or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), rambus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, and these are all within the scope of protection of the present application. Therefore, the protection scope of the present patent application shall be subject to the appended claims.
Claims (14)
1. A method for processing service data, the method comprising:
acquiring service data, a work task schedule and a work task configuration table; the work task schedule comprises serialized work tasks; the corresponding relation between the work task and the service data state value of the service data is recorded in the work task configuration table; the serialized work tasks refer to the work tasks in the work task schedule which are sequentially sequenced according to the execution sequence of the work tasks; the service data state value is used for caching and recording the current workflow state of the service data; the traffic data state value is a transition from a low state to a high state;
initializing and generating a service data state index table according to the work task schedule; the service data state index table comprises the current service data state value of the service data;
executing a working task corresponding to the current business data state value to perform corresponding business processing on the business data;
after corresponding business processing is carried out on the business data, a business processing result corresponding to the business data is generated;
when the execution of the work task is finished, updating the current business data state value;
when the updated current business data state value can be migrated, returning to the step of executing the work task corresponding to the current business data state value so as to perform the step of corresponding business processing on the business data to continue execution; the current service data state value can be migrated, which means that a service data state value larger than the current service data state value exists in the work task schedule;
receiving a data query request sent by a user terminal through a network service; the data query request carries a data identifier of the service data;
responding to the data query request, and querying a service processing result corresponding to the data identifier; and sending the service processing result obtained by the query to the user terminal.
2. The method of claim 1, wherein the obtaining the service data comprises:
acquiring service data of a text type;
formatting the service data of the text type according to a formatting function to obtain service data of a preset storage type;
and according to a conversion function, converting the service data of the preset storage type to obtain service data which can be analyzed by the work task.
3. The method according to claim 1, wherein after the corresponding service processing is performed on the service data and a service processing result corresponding to the service data is generated, the method further comprises:
determining the importance level of the service data;
and storing the service processing result to a storage device corresponding to the importance level of the service data.
4. The method of claim 1, wherein the receiving a data query request sent by a user terminal through a network service comprises:
determining a request type and a request mode corresponding to a data query request sent by a user terminal through network service;
searching a target data query interface matched with the request type and the request mode from the candidate data query interfaces according to a preset identification field of the candidate data query interface;
and receiving a data query request sent by a user terminal through a network service through the target data query interface.
5. The method of claim 3, further comprising:
determining result summarizing conditions;
summarizing the service processing results meeting the result summarizing conditions to obtain a corresponding summarized report;
and broadcasting the summarized report to a report receiving terminal.
6. The method according to any one of claims 1 to 5, wherein the executing the work task corresponding to the current business data state value to perform corresponding business processing on the business data comprises:
executing a working task corresponding to the current business data state value to identify and process different types of business data through an aggregation operation model, and obtaining business data conforming to a preset field format;
and carrying out aggregation operation processing on the service data which accords with the preset field format.
7. A service data processing apparatus, characterized in that the apparatus comprises:
the acquisition module is used for acquiring business data, a work task schedule and a work task configuration table; the work task schedule comprises serialized work tasks; the corresponding relation between the work task and the service data state value of the service data is recorded in the work task configuration table; the serialized work tasks refer to the work tasks in the work task schedule which are sequentially ordered according to the execution sequence of the work tasks; the service data state value is used for caching and recording the current workflow state of the service data; the traffic data state value is a transition from a low state to a high state;
the generating module is used for initializing and generating a business data state index table according to the work task schedule table; the service data state index table comprises the current service data state value of the service data;
the processing module is used for executing a work task corresponding to the current business data state value so as to perform corresponding business processing on the business data;
the storage module is used for generating a service processing result corresponding to the service data after the corresponding service processing is carried out on the service data;
the updating module is used for updating the current service data state value when the execution of the work task is finished; when the updated current business data state value can be migrated, informing the processing module to execute a work task corresponding to the current business data state value so as to perform a corresponding business processing step on the business data to continue execution; wherein, the current service data state value can be migrated, which means that a service data state value larger than the current service data state value exists in the work task schedule;
the query module is used for receiving a data query request sent by a user terminal through network service; the data query request carries a data identifier of the service data; responding to the data query request, and querying a service processing result corresponding to the data identifier; and sending the service processing result obtained by the query to the user terminal.
8. The apparatus of claim 7, wherein the obtaining module is further configured to obtain service data of text type; formatting the text type service data according to a formatting function to obtain preset storage type service data; and converting the service data of the preset storage type according to the conversion function to obtain service data resolvable by the work task.
9. The apparatus of claim 7, wherein the service data processing apparatus further comprises a storage module;
the storage module is used for determining the importance level of the service data; and storing the service processing result to a storage device corresponding to the importance level of the service data.
10. The apparatus of claim 7, wherein the query module is further configured to determine a request type and a request manner corresponding to a data query request sent by a user terminal through a network service; searching a target data query interface matched with the request type and the request mode from the candidate data query interfaces according to the identification field of the preset candidate data query interface; and receiving a data query request sent by the user terminal through the network service through the target data query interface.
11. The apparatus of claim 9, wherein the service data processing apparatus further comprises an aggregation module; the summarizing module is used for determining result summarizing conditions; summarizing the service processing results meeting the result summarizing condition to obtain a corresponding summarizing report; and broadcasting the summarized report to a report receiving terminal.
12. The device according to any one of claims 7 to 11, wherein the processing module is further configured to execute a work task corresponding to a current service data state value, so as to perform recognition processing on different types of service data through an aggregation operation model, and obtain service data conforming to a preset field format; and performing aggregation operation processing on the service data conforming to the preset field format.
13. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the steps of the method of any of claims 1 to 6 are implemented by the processor when executing the computer program.
14. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011466073.1A CN112507010B (en) | 2020-12-14 | 2020-12-14 | Service data processing method and device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011466073.1A CN112507010B (en) | 2020-12-14 | 2020-12-14 | Service data processing method and device, computer equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112507010A CN112507010A (en) | 2021-03-16 |
| CN112507010B true CN112507010B (en) | 2022-10-28 |
Family
ID=74972895
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011466073.1A Active CN112507010B (en) | 2020-12-14 | 2020-12-14 | Service data processing method and device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112507010B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113268500B (en) * | 2021-06-08 | 2023-06-02 | 中国联合网络通信集团有限公司 | Service processing method and device and electronic equipment |
| CN117851478A (en) * | 2024-03-07 | 2024-04-09 | 成方金融科技有限公司 | Data statistics method, device, electronic equipment and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107665234A (en) * | 2017-07-25 | 2018-02-06 | 平安科技（深圳）有限公司 | Method for processing business, device, server and storage medium |
| CN109144701A (en) * | 2018-09-05 | 2019-01-04 | 郑州云海信息技术有限公司 | A kind of task flow management method, device, equipment and system |
| CN110597858A (en) * | 2019-08-30 | 2019-12-20 | 深圳壹账通智能科技有限公司 | Task data processing method and device, computer equipment and storage medium |
| CN111783108A (en) * | 2019-04-04 | 2020-10-16 | 华控清交信息科技(北京)有限公司 | Data query and calculation method and system, and storage medium |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9460171B2 (en) * | 2013-11-08 | 2016-10-04 | International Business Machines Corporation | Processing data in data migration |
| US20170192795A1 (en) * | 2015-03-23 | 2017-07-06 | Middleware, Inc. | Universal Computing Element (UCE) and the Mode of Computing with the Use of UCE |
| US20180005629A1 (en) * | 2016-06-30 | 2018-01-04 | Microsoft Technology Licensing, Llc | Policy authoring for task state tracking during dialogue |
| US20180246886A1 (en) * | 2017-02-27 | 2018-08-30 | OSF Global Services Inc. | Data migration for platform integration |
-
2020
- 2020-12-14 CN CN202011466073.1A patent/CN112507010B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107665234A (en) * | 2017-07-25 | 2018-02-06 | 平安科技（深圳）有限公司 | Method for processing business, device, server and storage medium |
| CN109144701A (en) * | 2018-09-05 | 2019-01-04 | 郑州云海信息技术有限公司 | A kind of task flow management method, device, equipment and system |
| CN111783108A (en) * | 2019-04-04 | 2020-10-16 | 华控清交信息科技(北京)有限公司 | Data query and calculation method and system, and storage medium |
| CN110597858A (en) * | 2019-08-30 | 2019-12-20 | 深圳壹账通智能科技有限公司 | Task data processing method and device, computer equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112507010A (en) | 2021-03-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109408746B (en) | Image information query method, image information query device, computer equipment and storage medium | |
| CN107220142B (en) | Method and device for executing data recovery operation | |
| CN109768623B (en) | Monitoring method and device of power system, computer equipment and storage medium | |
| CN103023875B (en) | A kind of account management system and method | |
| CN110620812B (en) | Interactive information pushing method and device, computer equipment and storage medium | |
| CN110647438A (en) | Event monitoring method and device, computer equipment and storage medium | |
| CN111310427A (en) | Service data configuration processing method and device, computer equipment and storage medium | |
| CN112507010B (en) | Service data processing method and device, computer equipment and storage medium | |
| CN109389299B (en) | Workflow process deployment method, workflow process deployment device, computer equipment and storage medium | |
| CN110908778B (en) | Task deployment method, system and storage medium | |
| CN111027295A (en) | Data processing method, data processing device, computer equipment and storage medium | |
| CN112052247A (en) | Index updating system, method and device of search engine, electronic equipment and storage medium | |
| CN111737227B (en) | Data modification method and system | |
| CN110716989A (en) | Dimension data processing method and device, computer equipment and storage medium | |
| CN111767297B (en) | Big data processing method, device, equipment and medium | |
| CN112527504A (en) | Multi-tenant resource quota management method and device, and computer equipment | |
| CN108595280B (en) | Interface adaptation method and device, computer equipment and storage medium | |
| CN110659297A (en) | Data processing method, data processing device, computer equipment and storage medium | |
| CN111222153B (en) | Application program authority management method and device and storage medium | |
| CN111177121A (en) | Order data feedback method and device, computer equipment and storage medium | |
| CN111177481B (en) | User identifier mapping method and device | |
| CN113407560B (en) | Update message processing method, data synchronization method and configuration information configuration method | |
| CN109218131B (en) | Network monitoring method and device, computer equipment and storage medium | |
| CN111209061A (en) | Method and device for filling in user information, computer equipment and storage medium | |
| CN113065887A (en) | Resource processing method, resource processing device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP03 | Change of name, title or address | ||
| CP03 | Change of name, title or address |
Address after: Floor 25, Block A, Zhongzhou Binhai Commercial Center Phase II, No. 9285, Binhe Boulevard, Shangsha Community, Shatou Street, Futian District, Shenzhen, Guangdong 518000 Patentee after: Shenzhen Youjia Innovation Technology Co.,Ltd. Address before: 518051 1101, west block, Skyworth semiconductor design building, 18 Gaoxin South 4th Road, Gaoxin community, Yuehai street, Nanshan District, Shenzhen City, Guangdong Province Patentee before: SHENZHEN MINIEYE INNOVATION TECHNOLOGY Co.,Ltd. |