Detailed Description
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application. This application is capable of implementation in many different ways than those herein set forth and of similar import by those skilled in the art without departing from the spirit of this application and is therefore not limited to the specific implementations disclosed below.
The terminology used in the description of the one or more embodiments is for the purpose of describing the particular embodiments only and is not intended to be limiting of the description of the one or more embodiments. As used in one or more embodiments of the present specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used in one or more embodiments of the present specification refers to and encompasses any and all possible combinations of one or more of the associated listed items.
It will be understood that, although the terms first, second, etc. may be used herein in one or more embodiments to describe various information, these information should not be limited by these terms. These terms are only used to distinguish one type of information from another. For example, a first can also be referred to as a second and, similarly, a second can also be referred to as a first without departing from the scope of one or more embodiments of the present description. The word "if" as used herein may be interpreted as "at … …" or "when … …" or "in response to a determination", depending on the context.
First, the noun terms to which one or more embodiments of the present invention relate are explained.
Timing tasks: a certain program task is executed periodically.
In this specification, a data processing method and apparatus, a computing device, and a storage medium are provided, and detailed descriptions are individually made in the following embodiments.
Fig. 1 is a block diagram illustrating a configuration of a computing device 100 according to an embodiment of the present specification. The components of the computing device 100 include, but are not limited to, memory 110 and processor 120. The processor 120 is coupled to the memory 110 via the bus 130, and the database 150 is used for storing the batch numbers and the detail data corresponding to each batch number.
Computing device 100 also includes access device 140, access device 140 enabling computing device 100 to communicate via one or more networks 160. Examples of such networks include the Public Switched Telephone Network (PSTN), a Local Area Network (LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), or a combination of communication networks such as the internet. Access device 140 may include one or more of any type of network interface (e.g., a Network Interface Card (NIC)) whether wired or wireless, such as an IEEE802.11 Wireless Local Area Network (WLAN) wireless interface, a worldwide interoperability for microwave access (Wi-MAX) interface, an ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a bluetooth interface, a Near Field Communication (NFC) interface, and so forth.
In one embodiment of the present description, the other components of the computing device 100 described above and not shown in FIG. 1 may also be connected to each other, such as by a bus. It should be understood that the block diagram of the computing device architecture shown in FIG. 1 is for purposes of example only and is not limiting as to the scope of the description. Those skilled in the art may add or replace other components as desired.
Computing device 100 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., tablet, personal digital assistant, laptop, notebook, netbook, etc.), a mobile phone (e.g., smartphone), a wearable computing device (e.g., smartwatch, smartglasses, etc.), or other type of mobile device, or a stationary computing device such as a desktop computer or PC. Computing device 100 may also be a mobile or stationary server.
Wherein the processor 120 may perform the steps of the method shown in fig. 2. Fig. 2 is a schematic flow chart diagram illustrating a data processing method according to an embodiment of the present specification, including step 202 to step 208.
Step 202: and acquiring at least one detail datum corresponding to at least one batch number.
In one or more embodiments of the present description, the batch number is a set of numbers or letters plus a number used to identify a "batch" and is typically used to identify a product and facilitate traceability. The detail data is specific data under a certain batch number.
The operation of the service system is used to specifically describe the batch number and the detail data, for example, the service system a generates some abnormal data at intervals in the operation process, the service system a combs the generated abnormal data at intervals of ten minutes or twenty minutes, and the like, at this time, a batch number is generated at intervals of ten minutes or twenty minutes, the batch number may be a combination of numbers, characters or numbers plus characters generated according to a preset rule, and the detail of all the abnormal data generated in the ten minutes or twenty minutes below the batch number, that is, the detail data.
Step 204: adding a state lock to the at least one detail datum, and analyzing and processing the at least one detail datum.
Step 206: and releasing the state lock of the at least one detail datum after the analysis processing.
In one or more embodiments of the present specification, before performing analysis processing on each piece of detail data, a state lock is added to each piece of detail data, and after completing the analysis processing on each piece of detail data, the state lock is released for the piece of detail data, so that the processing state of each piece of detail data can be well distinguished, and preparation is made for subsequent execution.
For example, before the at least one piece of detail data is analyzed, a state lock is added to each piece of detail data, the state lock is I, after the analysis processing of each piece of detail data is completed, the state lock of the at least one piece of detail data after the analysis processing is released, the released state lock is F, when the subsequent processing is performed, whether the corresponding piece of detail data is processed or not can be known by checking the state of the state lock, and the working efficiency is greatly improved.
In one or more embodiments of the present specification, if the detail data is abnormal data, performing analysis processing on the detail data includes, but is not limited to, checking the detail data, performing correction processing on the detail data with an abnormality, and the like.
For example, the abnormal detail data is an account that is blocked by a fraud possibility, and the analyzing the detail data may be to check the fraud possibility account, determine that the account is a legal account, and unfreeze the account.
Step 208: acquiring each batch number based on preset conditions, judging whether the state lock of at least one piece of detail data corresponding to each batch number is released completely,
if yes, the service state of at least one piece of detail data corresponding to each batch number is updated, and if not, the step 206 is continued.
In one or more embodiments of the present specification, the obtaining each of the lot numbers based on the preset condition includes:
each of the lot numbers is acquired based on an acquisition time set in advance for each of the lot numbers.
In one or more embodiments of the present specification, each of the batch numbers is obtained according to a preset obtaining time, so that the detail data corresponding to the obtained batch number is analyzed and processed as far as possible, and resource waste is avoided.
For example, if 200 lot numbers are acquired at a time for processing, the service status of the lot numbers is updated, but the detailed data corresponding to 100 lot numbers is slower in analysis and processing and is not completed in analysis and processing due to various reasons, so that only 100 lot numbers are actually completed when one lot number is acquired.
In the data processing method, an estimated time for processing the batch number is given according to the historical processing time of the batch number of the same type. For example, if the average time of historical processing for a batch number is 30s, the acquisition time of 30s may be set for the batch number, and the batch number is processed only after the acquisition time of 30 s. The first 29s is not used for obtaining the batch, so that the working efficiency is greatly improved.
In one or more embodiments of the present description, when performing batch processing on a service system, the data processing method decouples detailed data and a batch processing task, avoids resource preemption, and performs batch acquisition and processing by setting acquisition time during the batch processing, so as to avoid resource waste and greatly improve working efficiency.
Referring to fig. 3, one or more embodiments of the present disclosure provide processing steps of detail data in a data processing method, including steps 302 to 312.
Step 302: and acquiring at least one detail datum corresponding to at least one batch number.
Step 304: adding a state lock to the at least one detail datum, and analyzing and processing the at least one detail datum.
Step 306: and releasing the state lock of the at least one detail datum after the analysis processing.
Step 308: and acquiring each batch number based on preset conditions.
Step 310: and judging whether the state locks of the at least one piece of detail data corresponding to each batch number are all released, if so, executing step 312, and if not, executing step 306.
Step 312: and updating the service state of at least one piece of detail data corresponding to each batch number.
In practical application, in order to decouple the processing of the detail data and the batch, the detail data and the batch can be processed by using two parallel links, but the batch processing is performed on the basis of the analysis and processing of the detail data, so the analysis and processing of the detail data are performed in the initial processing, and the batch is processed in the analysis and processing process of the detail data, so the condition of batch processing does not need to be concerned in the analysis and processing process of the detail data, so that the processing of the detail data is more single, the complexity of batch processing and detail data processing is reduced, and the implementation is more flexible.
Assuming that there are 1000 lot numbers at present, each lot number corresponds to 1000 pieces of detail data, the data processing method provided by one or more embodiments of the present specification can achieve the following technical effects: firstly, the number of sql is saved, the database access is reduced, and the original scheme is as follows: the sql number (1000 times of the analysis processing of the detail data +1000 times of the judgment of whether the analysis processing of other detail data is completed) is 1000 times of the processing of the lot number; the technical scheme of the application is as follows: the data processing method provided by one or more embodiments of the present disclosure can save the number of sql of 1000 × 1000; secondly, the problem of lock robbing cannot be caused, under the original scheme, the batch number is locked after the execution of the batch number is finished, the batch processing condition is judged, the problem of lock robbing is caused, and under the existing scheme, the problem of lock robbing cannot be caused; thirdly, the detailed data and batch processing are simpler and more flexible to realize.
Referring to fig. 4, one or more embodiments of the present disclosure provide processing steps of detail data in a data processing method, including steps 402 to 412.
Step 402: receiving a batch acquisition message, wherein the batch acquisition message comprises at least one batch number and the number of at least one piece of detail data corresponding to each batch number.
Step 404: and acquiring at least one detail datum corresponding to at least one batch number.
Step 406: judging whether the quantity of the at least one piece of detail data corresponding to each acquired batch number is the same as the quantity of the detail data corresponding to each batch number in the batch acquisition message,
if yes, go to step 408, otherwise go to step 404.
In one or more embodiments of the present specification, if the quantity of the detail data corresponding to a certain acquired batch number is the same as the quantity of the detail data corresponding to the batch number in the batch acquisition message, a state lock may be added to the acquired detail data, and the detail data may be analyzed; if the number of the detail data corresponding to a certain batch number is different from the number of the detail data corresponding to the batch number in the batch acquisition message, the detail data corresponding to the batch number is acquired again, and then the judgment is continued, so that incomplete data processing caused by incomplete acquired detail data is avoided.
Step 408: adding a state lock to the at least one detail datum, and analyzing and processing the at least one detail datum.
Step 410: and releasing the state lock of the at least one detail datum after the service processing.
Step 412: acquiring each batch number based on preset conditions, judging whether the state lock of at least one piece of detail data corresponding to each batch number is released completely,
if yes, go to step 414, otherwise, go to step 410.
Step 414: and updating the service state of at least one piece of detail data corresponding to each batch number.
In one or more embodiments of the present description, before processing batches and detail data of a service system, the data processing method performs query and determination according to the number of the detail data corresponding to each batch number in a received batch acquisition message, so as to ensure that the number of the acquired detail data is the same as the number of the detail data in the batch acquisition message, and ensure the integrity of data processing.
Referring to fig. 5, one or more embodiments of the present specification provide a data processing apparatus including:
an obtaining module 502 configured to obtain at least one detail datum corresponding to at least one batch number;
an analysis module 504 configured to add a state lock to the at least one detail datum and perform analysis processing on the at least one detail datum;
a releasing module 506 configured to release a state lock of the at least one detail datum after the analysis processing;
a first determining module 508, configured to obtain each batch number based on a preset condition, and determine whether the state locks of at least one detail datum corresponding to each batch number are all released,
and if so, updating the service state of at least one piece of detail data corresponding to each batch number, and if not, continuing to execute the release module.
Optionally, the apparatus further comprises:
the receiving module is configured to receive a batch acquisition message, wherein the batch acquisition message includes at least one batch number and the number of at least one piece of detail data corresponding to each batch number.
Optionally, the apparatus further comprises:
a second judging module configured to judge whether the number of the acquired at least one piece of detail data corresponding to each of the batch numbers is the same as the number of the detail data corresponding to each of the batch numbers in the batch acquisition message,
if yes, the analysis module is executed, and if not, the acquisition module is executed.
Optionally, the first judging module is further configured to acquire each of the batch numbers based on an acquisition time set for each of the batch numbers in advance.
In one or more embodiments of the present description, when the data processing apparatus performs batch processing on a service system, the detailed data and the batch processing task are decoupled, resource preemption is avoided, and when the batch processing is performed, the batch acquisition and processing are performed by setting the acquisition time, so that resource waste can be avoided, and the working efficiency is greatly improved.
An embodiment of the present application further provides a computer readable storage medium, which stores computer instructions, and the instructions, when executed by a processor, implement the steps of the data processing method as described above.
The above is an illustrative scheme of a computer-readable storage medium of the present embodiment. It should be noted that the technical solution of the storage medium belongs to the same concept as the technical solution of the data processing method, and details that are not described in detail in the technical solution of the storage medium can be referred to the description of the technical solution of the data processing method.
The foregoing description has been directed to specific embodiments of this disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.
The computer instructions comprise computer program code which may be in the form of source code, object code, an executable file or some intermediate form, or the like. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, and the like. It should be noted that the computer readable medium may contain content that is subject to appropriate increase or decrease as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable media does not include electrical carrier signals and telecommunications signals as is required by legislation and patent practice.
It should be noted that, for the sake of simplicity, the above-mentioned method embodiments are described as a series of acts or combinations, but those skilled in the art should understand that the present application is not limited by the described order of acts, as some steps may be performed in other orders or simultaneously according to the present application. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required in this application.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
The preferred embodiments of the present application disclosed above are intended only to aid in the explanation of the application. Alternative embodiments are not exhaustive and do not limit the invention to the precise embodiments described. Obviously, many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles of the application and the practical application, to thereby enable others skilled in the art to best understand and utilize the application. The application is limited only by the claims and their full scope and equivalents.