CN111857707A

CN111857707A - List data processing method, device, equipment and storage medium

Info

Publication number: CN111857707A
Application number: CN202010711832.XA
Authority: CN
Inventors: 袁俊磊
Original assignee: Ping An Property and Casualty Insurance Company of China Ltd
Current assignee: Ping An Property and Casualty Insurance Company of China Ltd
Priority date: 2020-07-22
Filing date: 2020-07-22
Publication date: 2020-10-30

Abstract

The invention relates to the technical field of big data, and discloses a method, a device, equipment and a storage medium for processing business list data, which are used for processing the business list data by splicing a plurality of independent function functions and reducing the redundancy of codes. The list data processing method comprises the following steps: when a request for creating a list synchronization task sent by a terminal is received, code splicing is carried out on a target function set to obtain a list synchronization task; when receiving a request for executing a list synchronization task sent by a terminal, acquiring list data to be processed through a source list value-taking function; carrying out duplicate removal processing on the list data to be processed through a duplicate removal function, and synchronously processing the duplicate removed list data to obtain first list data; performing blacklist verification on the first list data through a verification function to obtain second list data, wherein the second list data comprises a target number of a filtered blacklist; and synchronizing the second list data through the target list processing function, and carrying out voice call on the target number.

Description

List data processing method, device, equipment and storage medium

Technical Field

The invention relates to the field of difference synchronization of big data technology, in particular to a method, a device, equipment and a storage medium for processing business form data.

Background

The kettle button is a tool for data extraction, conversion and loading, can run on different operating systems, and is efficient and stable in data extraction. The keylet manages data from different databases by providing a graphical user environment and then outputs the data in a specified format. There are two script files in the button, namely transformation and task job, wherein the transformation is used for indicating the basic transformation of data, and the job is used for indicating the control of the whole workflow.

At present, a scheduling system receives a list from an upstream through an interface or a data migration tool, when the list reaches the scheduling system, a list synchronization table is written first, and then the list synchronization table is synchronized to the list table through a job, a plurality of jobs which run independently at the same time exist at present, different types of list synchronization data are processed respectively, but core code processing logics corresponding to each job are basically consistent, table structures are basically the same, and when a job is newly added, codes with related functions need to be rewritten, so that redundant codes are excessive.

Disclosure of Invention

The invention mainly aims to solve the technical problem of code redundancy when a plurality of list synchronization tasks are executed through a jobin the conventional list synchronization strategy.

In order to achieve the above object, a first aspect of the present invention provides a method for processing business form data, including: when a list creation synchronous task request sent by a terminal is received, a target function set is obtained according to the list creation synchronous task request, codes of the target function set are spliced, and a list synchronous task is obtained, wherein the target function set comprises a source table value function, a duplicate removal function, a verification function and a target table processing function; when a request for executing the list synchronization task sent by the terminal is received, the list data to be processed is obtained from an initial list data table through the source table value-taking function; carrying out duplicate removal processing on the to-be-processed list data through the duplicate removal function, and synchronizing the duplicate removed list data into a first target list data table to obtain first list data; performing blacklist verification on the first list data through the verification function to obtain second list data, wherein the second list data comprises target numbers of filtered blacklists; and synchronizing the second list data to a second target list data table through the target list processing function, and carrying out voice call on the target number of the filtered blacklist.

Optionally, in a first implementation manner of the first aspect of the present invention, when receiving a list creation synchronization task request sent by a terminal, obtaining a target function set according to the list creation synchronization task request, and performing code splicing on the target function set to obtain a list synchronization task, where the target function set includes a source table value function, a deduplication function, a check function, and a target table processing function, and includes: when a list creation synchronization task request sent by a terminal is received, analyzing the list creation synchronization task request to obtain a task type; inquiring corresponding target configuration information according to the task type; reading a corresponding target function set according to the target configuration information, and performing code encapsulation on the target function set to obtain a target code file, wherein the target function set comprises a source table value function, a duplicate removal function, a verification function and a target table processing function; and setting a task entry address for the target code file, and writing the task entry address into a preset task list to obtain a list synchronization task.

Optionally, in a second implementation manner of the first aspect of the present invention, when receiving the request for executing the list synchronization task sent by the terminal, acquiring the to-be-processed list data from an initial list data table through the source table value taking function includes: when a request for executing the list synchronization task sent by the terminal is received, generating a first query statement for a first preset list type and an initial list data table according to a structured query language grammar rule; and setting the first query statement as a parameter value corresponding to the source table value-taking function, and executing the source table value-taking function based on the parameter value to obtain the to-be-processed list data, wherein the to-be-processed list data comprises a to-be-screened number.

Optionally, in a third implementation manner of the first aspect of the present invention, the performing deduplication processing on the to-be-processed list data by using the deduplication function, and synchronizing the deduplicated list data into a first target list data table to obtain the first list data includes: extracting the numbers to be screened from the list data to be processed; judging whether the number to be screened exists in a first target name list data table or not through the duplication removal function; if the number to be screened does not exist in the first target name list data table, keeping the name list data corresponding to the number to be screened; if the number to be screened exists in the first target name list data table, reading the latest synchronous date corresponding to the number to be screened from the first target name list data table; acquiring a current date, and performing difference operation on the current date and the latest synchronous date to obtain difference duration; judging whether the difference time length is less than a preset time length or not; if the difference duration is less than the preset duration, deleting the duplicate menu data corresponding to the number to be screened; if the difference duration is greater than or equal to a preset duration, keeping the list data corresponding to the number to be screened; and packaging the removed list data according to a preset data format, and updating the packaged list data into the first target list data table to obtain first list data.

Optionally, in a fourth implementation manner of the first aspect of the present invention, the performing blacklist check on the first list data through the check function to obtain second list data, where the second list data includes a target number of a filtered blacklist, includes: reading a number to be verified from the first list data, and judging whether the number to be verified exists in a preset blacklist or not through the verification function; if the number to be verified does not exist in a preset blacklist, reserving the list data corresponding to the number to be verified; if the number to be verified exists in a preset blacklist, deleting the list data corresponding to the number to be verified; and when all the numbers to be verified in the first list data are verified, setting the list data of the filtered blacklist as second list data, wherein the second list data comprises the target numbers of the filtered blacklist.

Optionally, in a fifth implementation manner of the first aspect of the present invention, when receiving a list creation synchronization task request sent by a terminal, obtaining a target function set according to the list creation synchronization task request, and performing code splicing on the target function set to obtain a list synchronization task, where the target function set includes a source table value taking function, a deduplication function, a check function, and a target table processing function, the list data processing method further includes: circularly extracting a plurality of preset codes from a plurality of developed source codes according to a preset service type, and preprocessing the plurality of preset codes, wherein the plurality of preset codes are used for indicating code blocks for realizing a plurality of synchronous data processing according to the preset service type; carrying out syntax analysis on the preprocessed plurality of preset codes to obtain a plurality of abstract syntax trees; extracting any two abstract syntax trees from the plurality of abstract syntax trees, and calculating cosine distances based on vocabularies in the any two abstract syntax trees to obtain similar preset codes; performing the same category identification on the similar preset codes based on a preset rule to obtain a plurality of category identification codes; screening and de-duplicating the identified preset codes according to the plurality of category identification codes to obtain code blocks to be encapsulated corresponding to each category; and performing function encapsulation on the code block to be encapsulated corresponding to each category according to the preset service type, and adding corresponding preset parameters to the encapsulated functions to obtain a target function set.

Optionally, in a sixth implementation manner of the first aspect of the present invention, after the synchronizing, by the target table processing function, the second list data to a second target list data table and making a voice call to a target number of the filtered blacklist, the list data processing method further includes: when a list pushing request sent by the terminal is received, setting a second query statement based on a data list to be pushed and a second preset list type; executing the second query statement through the source table value-taking function to obtain list data to be pushed; and acquiring identification information of a target terminal through the target table processing function, and pushing the list data to be pushed to the target terminal according to the identification information.

A second aspect of the present invention provides a business form data processing apparatus, including: the system comprises a creating module, a list synchronizing task processing module and a list synchronizing task processing module, wherein the creating module is used for acquiring a target function set according to a list synchronizing task creating request sent by a terminal and performing code splicing on the target function set to obtain a list synchronizing task, and the target function set comprises a source list value-taking function, a duplicate-removing function, a verification function and a target list processing function; the acquisition module is used for acquiring the to-be-processed list data from an initial list data table through the source table value-taking function when receiving the list synchronization task execution request sent by the terminal; the duplication removing module is used for carrying out duplication removing processing on the to-be-processed list data through the duplication removing function and synchronizing the duplicated list data into a first target list data table to obtain first list data; the verification module is used for performing blacklist verification on the first list data through the verification function to obtain second list data, and the second list data comprises target numbers of filtered blacklists; and the calling module is used for synchronizing the second list data into a second target list data table through the target list processing function and carrying out voice calling on the target number of the filtered blacklist.

Optionally, in a first implementation manner of the second aspect of the present invention, the creating module is specifically configured to: when a list creation synchronization task request sent by a terminal is received, analyzing the list creation synchronization task request to obtain a task type; inquiring corresponding target configuration information according to the task type; reading a corresponding target function set according to the target configuration information, and performing code encapsulation on the target function set to obtain a target code file, wherein the target function set comprises a source table value function, a duplicate removal function, a verification function and a target table processing function; and setting a task entry address for the target code file, and writing the task entry address into a preset task list to obtain a list synchronization task.

Optionally, in a second implementation manner of the second aspect of the present invention, the obtaining module is specifically configured to: when a request for executing the list synchronization task sent by the terminal is received, generating a first query statement for a first preset list type and an initial list data table according to a structured query language grammar rule; and setting the first query statement as a parameter value corresponding to the source table value-taking function, and executing the source table value-taking function based on the parameter value to obtain the to-be-processed list data, wherein the to-be-processed list data comprises a to-be-screened number.

Optionally, in a third implementation manner of the second aspect of the present invention, the deduplication module is specifically configured to: extracting the numbers to be screened from the list data to be processed; judging whether the number to be screened exists in a first target name list data table or not through the duplication removal function; if the number to be screened does not exist in the first target name list data table, keeping the name list data corresponding to the number to be screened; if the number to be screened exists in the first target name list data table, reading the latest synchronous date corresponding to the number to be screened from the first target name list data table; acquiring a current date, and performing difference operation on the current date and the latest synchronous date to obtain difference duration; judging whether the difference time length is less than a preset time length or not; if the difference duration is less than the preset duration, deleting the duplicate menu data corresponding to the number to be screened; if the difference duration is greater than or equal to a preset duration, keeping the list data corresponding to the number to be screened; and packaging the removed list data according to a preset data format, and updating the packaged list data into the first target list data table to obtain first list data.

Optionally, in a fourth implementation manner of the second aspect of the present invention, the verification module is specifically configured to: reading a number to be verified from the first list data, and judging whether the number to be verified exists in a preset blacklist or not through the verification function; if the number to be verified does not exist in a preset blacklist, reserving the list data corresponding to the number to be verified; if the number to be verified exists in a preset blacklist, deleting the list data corresponding to the number to be verified; and when all the numbers to be verified in the first list data are verified, setting the list data of the filtered blacklist as second list data, wherein the second list data comprises the target numbers of the filtered blacklist.

Optionally, in a fifth implementation manner of the second aspect of the present invention, the list data processing apparatus further includes: the system comprises a preprocessing module, a data processing module and a data processing module, wherein the preprocessing module is used for circularly extracting a plurality of preset codes from a plurality of developed source codes according to preset service types and preprocessing the preset codes, and the preset codes are used for indicating code blocks for realizing a plurality of synchronous data processing according to the preset service types; the analysis module is used for carrying out syntax analysis on the preprocessed multiple preset codes to obtain multiple abstract syntax trees; the calculation module is used for extracting any two abstract syntax trees from the plurality of abstract syntax trees and calculating cosine distances based on vocabularies in the any two abstract syntax trees to obtain similar preset codes; the identification module is used for carrying out same category identification on the similar preset codes based on a preset rule to obtain a plurality of category identification codes; the screening module is used for screening and de-duplicating the identified preset codes according to the plurality of category identification codes to obtain the code blocks to be encapsulated corresponding to each category; and the encapsulation module is used for carrying out function encapsulation on the code block to be encapsulated corresponding to each category according to the preset service type, and adding corresponding preset parameters to the encapsulated functions to obtain a target function set.

Optionally, in a sixth implementation manner of the second aspect of the present invention, the list data processing apparatus further includes: the setting module is used for setting a second query statement based on a data table to be pushed and a second preset list type when receiving a list pushing request sent by the terminal; the execution module is used for executing the second query statement through the source table value-taking function to obtain the list data to be pushed; and the pushing module is used for acquiring the identification information of the target terminal through the target table processing function and pushing the list data to be pushed to the target terminal according to the identification information.

A third aspect of the present invention provides a business form data processing apparatus comprising: a memory having instructions stored therein and at least one processor, the memory and the at least one processor interconnected by a line; the at least one processor invokes the instructions in the memory to cause the roster data processing apparatus to perform the above-described roster data processing method.

A fourth aspect of the present invention provides a computer-readable storage medium having stored therein instructions, which, when run on a computer, cause the computer to execute the above-described roster data processing method.

In the technical scheme provided by the invention, when a list creation synchronous task request sent by a terminal is received, a target function set is obtained according to the list creation synchronous task request, codes are spliced on the target function set to obtain a list synchronous task, and the target function set comprises a source table value function, a duplicate removal function, a verification function and a target table processing function; when a request for executing the list synchronization task sent by the terminal is received, the list data to be processed is obtained from an initial list data table through the source table value-taking function; carrying out duplicate removal processing on the to-be-processed list data through the duplicate removal function, and synchronizing the duplicate removed list data into a first target list data table to obtain first list data; performing blacklist verification on the first list data through the verification function to obtain second list data, wherein the second list data comprises target numbers of filtered blacklists; and synchronizing the second list data to a second target list data table through the target list processing function, and carrying out voice call on the target number of the filtered blacklist. In the embodiment of the invention, the decoupling processing is carried out on the code logic of the existing synchronous task, and the decoupling processing is divided into the independent function functions of the data stream processing examples, so that for different list processing tasks, different independent function functions can be called according to the service logic, the trouble of adding new services or modifying code logic rewrite jobs is avoided, the redundancy of codes is reduced, and the reuse rate of the codes is improved.

Drawings

FIG. 1 is a schematic diagram of an embodiment of a method for processing list data according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of another embodiment of a method for processing list data according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of an embodiment of a device for processing list data according to an embodiment of the present invention;

FIG. 4 is a schematic diagram of another embodiment of a device for processing list data according to an embodiment of the present invention;

fig. 5 is a schematic diagram of an embodiment of a device for processing the list data according to an embodiment of the present invention.

Detailed Description

Embodiments of the present invention provide a method, an apparatus, a device, and a storage medium for processing business list data, which are used to perform decoupling processing on code logic of an existing synchronization task, and divide the code logic into independent function functions of data stream processing examples, and for different list processing tasks, different independent function functions can be called according to business logic, thereby avoiding the trouble of adding new business or modifying code logic rewrite jobs, reducing code redundancy, and improving code reuse rate.

The terms "first," "second," "third," "fourth," and the like in the description and in the claims, as well as in the drawings, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the data so used may be interchanged under appropriate circumstances such that the embodiments described herein may be practiced otherwise than as specifically illustrated or described herein. Furthermore, the terms "comprises," "comprising," or "having," and any variations thereof, are intended to cover non-exclusive inclusions, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

For convenience of understanding, a specific flow of an embodiment of the present invention is described below, and referring to fig. 1, an embodiment of a method for processing list data according to an embodiment of the present invention includes:

101. when a list creation synchronous task request sent by a terminal is received, a target function set is obtained according to the list creation synchronous task request, codes of the target function set are spliced, and a list synchronous task is obtained, wherein the target function set comprises a source table value function, a duplicate removal function, a verification function and a target table processing function.

The target function set is a code block with different service functions and is responsible for service logic processing of various types, so that the target function set can be combined and called according to actual service requirements. The source table value function is used for extracting data according to preset conditions, and the duplication removing function is used for deleting repeated data from the extracted data to obtain data to be verified; the checking function is used for checking the data to be checked according to a preset rule; the target table processing function is used for processing the data passing the verification.

Further, the server acquires the type of the synchronous task according to the request for creating the list synchronous task; and the server acquires the target function set based on the type of the synchronization task, performs code splicing on the target function set, and creates a list synchronization task.

It is to be understood that the executing subject of the present invention may be a business form data processing device, and may also be a terminal or a server, which is not limited herein. The embodiment of the present invention is described by taking a server as an execution subject.

102. And when receiving an execution list synchronization task sent by a terminal, acquiring list data to be processed from the initial list data table through a source list value-taking function.

Each business logic in the developed source code is used as a function through modular processing, and all the function functions are called and connected in series through terminal page operation to form a complete workflow. Therefore, one record in the initial list data table is a workflow, and the workflow is a task formed by combining a plurality of function functions. The initial list data table is a source table of a name list type, and refers to a data table placed before list data is unprocessed, and data in the initial list data table come from different upstream systems. Therefore, when the timed execution of the list synchronization task is detected, the server calls the source table value function to obtain the output value of the source table value function, and the output value of the source table value function is set as the list data to be processed.

103. And performing duplicate removal processing on the list data to be processed through a duplicate removal function, and synchronizing the duplicate removed list data to a first target list data table to obtain first list data.

The duplication elimination processing refers to removing repeated data from the list data to be processed, and the processing rule corresponding to the duplication elimination function is to perform processing according to a preset duplication elimination number within a preset time length, for example, the duplication elimination function performs duplication elimination on the list data to be processed by using preset duplication elimination logic, the list data to be processed includes numbers to be screened, wherein the preset duplication elimination logic refers to Y duplication elimination logic in X days, that is, the numbers to be screened are reserved for Y times at most in X days, for example, 1 time in 7 days, that is, the numbers to be screened can be reserved for 1 time in 7 days, and the numbers to be screened include mobile numbers and landline numbers.

Further, the server synchronously stores the duplicate-removed list data in the first target list data table to obtain first list data, and each duplicate-removed list data is written into the first target list data table in the process of synchronizing the duplicate-removed list data, and the date and the times of data writing are recorded. For example, list synchronization is performed on the deduplicated list data A in 2019-12-12, the deduplicated list data A is reserved in the first target name list data table _ data, and meanwhile, the date and the number of data writing are also recorded in the table _ data, namely [ { "count":1, "savDate": 2019-12-12 "}.

104. And carrying out blacklist verification on the first list data through a verification function to obtain second list data, wherein the second list data comprises the target number of the filtered blacklist.

And the blacklist check is used for deleting the list data which can not be called out from the first list data. It should be noted that, in the process of list data synchronization, after the server stores the target number into the first target list data table, a verification function is also required to be invoked to verify whether the number to be screened can be dialed, the principle of the verification is that the server performs one-by-one verification processing on the numbers to be screened in the first list data, that is, the server queries to obtain a blacklist corresponding to the first list data and retains or deletes the first list data based on the blacklist, for example, if the server verifies that the number a to be screened in the first list data is stored in the blacklist, the duplicate list data corresponding to the number a to be screened in the first list data is deleted; and if the server verifies that the number B to be screened in the first list data does not exist in the blacklist, reserving the list data corresponding to the number B to be screened in the first list data. And after the numbers to be screened are completely verified, second list data are obtained, the second list data comprise target numbers of the filtered blacklist, and the server writes the second list data into the memory database, so that the data query efficiency is improved.

105. And synchronizing the second list data to a second target list data table through the target table processing function, and carrying out voice call on the second list data in the second target list data table.

Furthermore, after the second list data is synchronized to the second target list data table through the target table processing function, the synchronization state is updated for the relevant fields of the initial list data table and the first list data table, so that the list data cannot be repeatedly screened when the synchronization is triggered next time. The server then makes a voice call to the target number of the filtered blacklist in the second target list data table, counts the number of calls, and records call detail information including call time, call object, and call result, e.g., 2020-3-112: 36:45, target number a, call failure.

The server obtains the target function set by adopting modular processing, the code blocks of each function in the target function set are independent and are not influenced by code logic optimization and iteration in other function functions, the server is not only applied to list synchronization tasks, but also other synchronization processes can use the modular target function set to customize, and code optimization and code redundancy removal are realized.

In the embodiment of the invention, the decoupling processing is carried out on the code logic of the existing synchronous task, and the decoupling processing is divided into the independent function functions of the data stream processing examples, so that for different list processing tasks, different independent function functions can be called according to the service logic, the trouble of adding new services or modifying code logic rewrite jobs is avoided, the redundancy of codes is reduced, and the reuse rate of the codes is improved.

Referring to fig. 2, another embodiment of the method for processing the list data according to the embodiment of the present invention includes:

201. and circularly extracting a plurality of preset codes from a plurality of developed source codes according to a preset service type, and preprocessing the plurality of preset codes, wherein the plurality of preset codes are used for indicating code blocks for realizing a plurality of synchronous data processing according to the preset service type.

The preprocessing comprises deleting the text contents such as release information and operators in a plurality of preset codes so as to obtain the effective contents for searching the repetition. It should be noted that a plurality of developed source codes simultaneously develop a plurality of synchronization tasks according to preset service types, for example, types of lists, and the same code block exists between each synchronization task except for differences of individual fields, which results in code redundancy.

202. And carrying out syntax analysis on the plurality of preprocessed preset codes to obtain a plurality of abstract syntax trees.

The server may perform syntax parsing on the preprocessed multiple preset codes through a code parsing tool, for example, the syntax parser antrl, the abstract syntax tree parser astparser, and the syntax and lexical parser javacc perform syntax parsing on the file codes to obtain multiple abstract syntax trees.

203. And extracting any two abstract syntax trees from the plurality of abstract syntax trees, and calculating cosine distances based on vocabularies in any two abstract syntax trees to obtain similar preset codes.

The vocabulary in each leaf node in the abstract syntax tree comprises variables, the abstract syntax tree is mapped into a space vector according to the vocabulary, the code similarity is calculated based on the cosine distance of the space vector, the similar leaf nodes are determined based on the code similarity, the similarity of any two abstract syntax trees is finally determined, and the similar preset codes are obtained.

204. And performing the same category identification on the similar preset codes based on preset rules to obtain a plurality of category identification codes.

The preset rule is that corresponding code annotations are added to similar preset codes, the same preset identifiers are used in the code annotations to represent the similar preset codes, the preset identifiers comprise numbers, letters, underlines and other characters, and for example, the server uses label _1, label _2 and label _3 as a plurality of category identification codes.

205. And screening and de-duplicating the identified preset codes according to the plurality of category identification codes to obtain the code blocks to be encapsulated corresponding to each category.

Specifically, the server acquires each repeated identified preset code one by one according to the category identification code, further acquires the position information of each identified preset code, and performs deduplication processing on the repeated identified preset code according to the position information to obtain the code block to be encapsulated corresponding to each category.

206. And performing function encapsulation on the code block to be encapsulated corresponding to each category according to a preset service type, and adding corresponding preset parameters to the encapsulated functions to obtain a target function set.

Furthermore, the server debugs any function in the target function set, so that the accuracy of a plurality of parameters and code logic in the target function set is ensured. The multiple repeated code blocks are also multiple existing synchronous tasks and comprise universal code blocks, redundant information and preset parameters in the universal code blocks are packaged into functions together, a target function set is constructed simultaneously, the code blocks among the functional functions in the target function set are independent and are not influenced by code logic optimization and iteration in other functional functions, and code optimization redundancy removal is achieved.

207. When a list creation synchronous task request sent by a terminal is received, a target function set is obtained according to the list creation synchronous task request, codes of the target function set are spliced, and a list synchronous task is obtained, wherein the target function set comprises a source table value function, a duplicate removal function, a verification function and a target table processing function.

The source table value function is used for extracting data according to preset conditions; the duplicate removal function is used for deleting repeated data in the extracted data to obtain data to be verified; the verification function is used for performing blacklist verification on the data to be verified; the target table processing function is used for processing the data passing the verification.

Specifically, when a request for creating a list synchronization task sent by a terminal is received, the server analyzes the request for creating the list synchronization task to obtain a task type, for example, the task type is 001 for creating the list synchronization task, and 002 for pushing the list data task, which is not limited herein; the server inquires corresponding target configuration information according to the task type, wherein the target configuration information is used for indicating different target function sets, and the different target function sets comprise different function functions; the server reads a corresponding target function set according to the target configuration information, and performs code encapsulation on the target function set to obtain a target code file, wherein the target function set comprises a source table value function, a duplicate removal function, a verification function and a target table processing function; the server stores the target code file into a preset file directory to obtain a corresponding storage path and a corresponding file name; the server sets a task entry address based on the storage path and the file name, and writes the task entry address into a preset task list to obtain a list synchronization task, wherein the task entry address can be http://127.0.0.1/task/data _ sync _01.php, or can be task/data _ sync _01.php, and is not limited herein.

208. And when receiving a request for executing the list synchronization task sent by the terminal, acquiring the list data to be processed from the initial list data table through the source table value-taking function.

Specifically, firstly, when receiving a request for executing a list synchronization task sent by a terminal, a server generates a first query statement for a first preset list type and an initial list data table according to a structured query language grammar rule, wherein the first preset list type is used for indicating the type of list data in the initial list data table, and further, the server scans the initial list data table to obtain a query result; the server judges whether the query result is a null value; if the query result is a null value, the server determines that the list data to be processed does not exist in the initial list data table; and if the query result is not a null value, the server determines that the list data to be processed exists in the initial list data table.

Then, the server sets the first query statement as a parameter value corresponding to a source table value taking function, and executes the source table value taking function based on the parameter value to obtain list data to be processed, wherein the list data to be processed comprises a target number, and the parameter transmission mode comprises that the server calls the source table value taking function, takes a first target parameter value as a parameter value of the source table value taking function, sets a return value of the source table value taking function as the list data to be processed, further, the server obtains a preset file address and a function name corresponding to the source table value taking function, and sets a request uniform resource locator according to the first target parameter value, the preset file address and the function name; and the server sends a request uniform resource locator to the source list value-taking function to obtain response data, and the response data is set as list data to be processed.

Optionally, when receiving a request of a synchronous list sent by the terminal, the server receives the request of processing the shortlist data sent by the terminal before generating a first query statement by the server according to the first preset list type and the initial list data table according to the structured query language grammar rule; the server analyzes the request for processing the list data to obtain request parameters, wherein the request parameters comprise an initial list data table, an operation type and the list data to be processed; the server sets an initial list data table, an operation type and list data to be processed according to a preset parameter format to obtain a request parameter value, and calls a preset interface according to a request function name and the request parameter value; the server performs data processing on the list data to be processed in the initial list data table according to the operation type, where the data processing includes adding data, deleting data, and updating data, and further, the adding data, the deleting data, and the updating data are states of modifying the list data to be processed, and the states of the list data to be processed include an unprocessed state, a processing success state, and a processing failure state, and may be respectively identified by 0, 1, and 2, and the specific details are not limited herein.

209. And performing duplicate removal processing on the list data to be processed through a duplicate removal function, and synchronizing the duplicate removed list data to a first target list data table to obtain first list data.

The deduplication processing refers to removing repeated data from the list data to be processed according to a preset deduplication number within a preset time length, where the preset deduplication number is 1 day, for example, 7 days and 1 deduplication logic. Specifically, the server extracts the numbers to be screened from the list data to be processed; judging whether the number to be screened exists in a first target name list data table or not through a duplicate function; if the number to be screened does not exist in the first target name list data table, the server reserves the name list data corresponding to the number to be screened; if the number to be screened exists in the first target name list data table, the server reads the latest synchronous date corresponding to the number to be screened from the first target name list data table; the server acquires the current date, and performs difference operation on the current date and the latest synchronous date to obtain difference duration; the server judges whether the difference time length is less than a preset time length; and if the difference duration is less than the preset duration, the server deletes duplicate business form data corresponding to the numbers to be screened, and further, in the process of synchronizing the duplicate name list data, each duplicate number to be screened is written into the first target business form data table, and the written date and times are recorded. For example, the target number 11 in the to-be-processed list data a is subjected to list synchronization in 2019-12-12, and then the corresponding record in the first target name list data table is [ { "count":1, "saveDate": 2019-12-12 "}; on the same day, the to-be-processed list data B comprises the same target number 11, when list synchronization is carried out again, the number is searched from the first target list data table to know that the latest synchronization date is within 7 days, the to-be-processed list data B in the initial list data table is marked as duplication removal, and the number base is updated to [ { "count":2, "saveDate": 2019-12-12"} ]; if the difference time length is greater than or equal to the preset time length, the server keeps the list data corresponding to the numbers to be screened; the server packages the deduplicated list data according to a preset data format, and updates the packaged list data into a first target name list data table to obtain first list data, for example, 2019-12-20, the list data C to be processed has the same target number 11 and is synchronized again, if the latest synchronization date of the number to be screened in the first target name list data table exceeds 7 days, the list data C to be processed is not deduplicated, and the first target name list data table is updated to { "count":2, "savdate": 2019-12-12"}, {" count ":1," savdate ": 2019-12-20" }.

Optionally, the server may further write the list data to be processed into a preset role table and/or a preset resource table according to the deduplication function and the preset list type, and mark the list data after deduplication, that is, identify whether the list is deduplication-free through a field sign _ status, where non-deduplication is 0, and after deduplication is performed, the field in the initial list data table is updated to 1.

210. And carrying out blacklist verification on the first list data through a verification function to obtain second list data, wherein the second list data comprises the target number of the filtered blacklist.

Specifically, the server reads the number to be verified from the first list data, and judges whether the number to be verified exists in the preset blacklist data through a verification function; if the number to be verified does not exist in the preset blacklist data, the server reserves the list data corresponding to the number to be verified; if the target number in the first list data exists in the preset blacklist data, the server deletes the list data corresponding to the number to be verified to obtain second list data; and when the server completes verification of all the numbers to be verified in the first list data, setting the list data of the filtered blacklist as second list data, wherein the second list data comprises the target numbers of the filtered blacklist.

211. And synchronizing the second list data to a second target list data table through a target list processing function, and carrying out voice call on the target number of the filtered blacklist.

Furthermore, after the server successfully synchronizes the second list data through the target list processing function, the related fields of the initial list data table and the first target list data table are synchronously updated, so that the list data cannot be repeatedly screened when the list synchronization task is executed next time. After the server performs voice call processing on the target number of the filtered blacklist in the second target list data table, the server also needs to update the call times of the successfully called list data and perform retry call operation on the unsuccessfully called list data.

Optionally, when a list pushing request sent by the terminal is received, the server sets a second query statement based on the data table to be pushed and a second preset list type; the server executes a second query statement through the source table value function to obtain list data to be pushed; the server acquires the identification information of the target terminal through the target table processing function and pushes the list data to be pushed to the target terminal according to the identification information.

With reference to fig. 3, the above describes a method for processing the list data in the embodiment of the present invention, and a device for processing the list data in the embodiment of the present invention is described below, where an embodiment of the device for processing the list data in the embodiment of the present invention includes:

the creating module 301 is configured to, when receiving a list creating synchronization task request sent by a terminal, obtain a target function set according to the list creating synchronization task request, and perform code splicing on the target function set to obtain a list synchronization task, where the target function set includes a source table value function, a duplicate removal function, a verification function, and a target table processing function;

an obtaining module 302, configured to obtain to-be-processed list data from an initial list data table through a source table value-taking function when receiving a request for executing a list synchronization task sent by a terminal;

the duplication elimination module 303 is configured to perform duplication elimination on the to-be-processed list data through a duplication elimination function, and synchronize the duplication eliminated list data into the first target list data table to obtain first list data;

the verification module 304 is configured to perform blacklist verification on the first list data through a verification function to obtain second list data, where the second list data includes a target number of a filtered blacklist;

and the calling module 305 is configured to synchronize the second list data to the second target list data table through the target list processing function, and perform a voice call to the target number of the filtered blacklist.

Referring to fig. 4, another embodiment of the apparatus for processing the list data according to the embodiment of the present invention includes:

Optionally, the creating module 301 may be further specifically configured to:

when a list creation synchronization task request sent by a terminal is received, analyzing the list creation synchronization task request to obtain a task type;

inquiring corresponding target configuration information according to the task type;

reading a corresponding target function set according to the target configuration information, and performing code encapsulation on the target function set to obtain a target code file, wherein the target function set comprises a source table value function, a duplicate removal function, a verification function and a target table processing function;

and setting a task entry address for the target code file, and writing the task entry address into a preset task list to obtain a list synchronization task.

Optionally, the obtaining module 302 may be further specifically configured to:

when a list synchronization task execution request sent by a terminal is received, generating a first query statement for a first preset list type and an initial list data table according to a structured query language grammar rule;

and setting the first query statement as a parameter value corresponding to the source table value-taking function, and executing the source table value-taking function based on the parameter value to obtain the to-be-processed list data, wherein the to-be-processed list data comprises the to-be-screened numbers.

Optionally, the duplicate removal module 303 may be further specifically configured to:

extracting numbers to be screened from the list data to be processed;

judging whether the number to be screened exists in a first target name list data table or not through a duplicate function;

if the number to be screened does not exist in the first target name list data table, keeping the name list data corresponding to the number to be screened;

if the number to be screened exists in the first target name list data table, reading the latest synchronous date corresponding to the number to be screened from the first target name list data table;

acquiring a current date, and performing difference operation on the current date and the latest synchronous date to obtain difference duration;

judging whether the difference time length is less than a preset time length or not;

if the difference duration is less than the preset duration, deleting the duplicate menu data corresponding to the number to be screened;

if the difference time length is greater than or equal to the preset time length, keeping the list data corresponding to the numbers to be screened;

and packaging the removed list data according to a preset data format, and updating the packaged list data into a first target list data table to obtain first list data.

Optionally, the checking module 304 may be further specifically configured to:

reading a number to be verified from the first list data, and judging whether the number to be verified exists in a preset blacklist or not through a verification function;

if the number to be verified does not exist in the preset blacklist, reserving the list data corresponding to the number to be verified;

if the number to be verified exists in the preset blacklist, deleting the list data corresponding to the number to be verified;

and when all the numbers to be verified in the first list data are verified, setting the list data of the filtered blacklist as second list data, wherein the second list data comprises the target numbers of the filtered blacklist.

Optionally, the list data processing apparatus further includes:

the preprocessing module 306 is configured to cyclically extract a plurality of preset codes from a plurality of developed source codes according to preset service types, and preprocess the plurality of preset codes, where the plurality of preset codes are used to indicate code blocks for implementing a plurality of synchronous data processing according to the preset service types;

the parsing module 307 is configured to perform syntax parsing on the plurality of preprocessed preset codes to obtain a plurality of abstract syntax trees;

the calculating module 308 is configured to extract any two abstract syntax trees from the plurality of abstract syntax trees, and calculate a cosine distance based on vocabularies in any two abstract syntax trees to obtain similar preset codes;

the identification module 309 is configured to perform the same category identification on similar preset codes based on preset rules to obtain a plurality of category identification codes;

the screening module 310 is configured to screen and deduplicate the identified preset codes according to the multiple category identification codes to obtain a to-be-encapsulated code block corresponding to each category;

and the encapsulating module 311 is configured to perform function encapsulation on the code block to be encapsulated corresponding to each category according to a preset service type, and add respective corresponding preset parameters to the encapsulated function to obtain a target function set.

Optionally, the list data processing apparatus further includes:

a setting module 312, configured to set a second query statement based on the data table to be pushed and a second preset list type when receiving a push list request sent by the terminal;

the execution module 313 is configured to execute the second query statement through the source table value taking function to obtain the list data to be pushed;

the pushing module 314 is configured to obtain identification information of the target terminal through the target table processing function, and push the list data to be pushed to the target terminal according to the identification information.

Fig. 3 and fig. 4 describe the list data processing apparatus in the embodiment of the present invention in detail from the perspective of the modular functional entity, and the list data processing apparatus in the embodiment of the present invention is described in detail from the perspective of hardware processing.

Fig. 5 is a schematic structural diagram of a business form data processing apparatus 500 according to an embodiment of the present invention, which may have a relatively large difference due to different configurations or performances, and may include one or more processors (CPUs) 510 (e.g., one or more processors) and a memory 520, and one or more storage media 530 (e.g., one or more mass storage devices) for storing applications 533 or data 532. Memory 520 and storage media 530 may be, among other things, transient or persistent storage. The program stored on the storage medium 530 may include one or more modules (not shown), each of which may include a series of instruction operations for the roster data processing apparatus 500. Still further, the processor 510 may be configured to communicate with the storage medium 530 to execute a series of instruction operations in the storage medium 530 on the roster data processing apparatus 500.

The list data processing apparatus 500 may also include one or more power supplies 540, one or more wired or wireless network interfaces 550, one or more input-output interfaces 560, and/or one or more operating systems 531, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, and the like. Those skilled in the art will appreciate that the roster data processing facility configuration depicted in FIG. 5 does not constitute a limitation of the roster data processing facility and may include more or less components than those illustrated, or some components in combination, or a different arrangement of components.

The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium, and which may also be a volatile computer-readable storage medium, having stored therein instructions, which, when run on a computer, cause the computer to perform the steps of the roster data processing method.

It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a read-only memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.

The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims

1. A method for processing data of a list is characterized in that the method for processing the data of the list comprises the following steps:

when a list creation synchronous task request sent by a terminal is received, a target function set is obtained according to the list creation synchronous task request, codes of the target function set are spliced, and a list synchronous task is obtained, wherein the target function set comprises a source table value function, a duplicate removal function, a verification function and a target table processing function;

when a request for executing the list synchronization task sent by the terminal is received, the list data to be processed is obtained from an initial list data table through the source table value-taking function;

carrying out duplicate removal processing on the to-be-processed list data through the duplicate removal function, and synchronizing the duplicate removed list data into a first target list data table to obtain first list data;

performing blacklist verification on the first list data through the verification function to obtain second list data, wherein the second list data comprises target numbers of filtered blacklists;

and synchronizing the second list data to a second target list data table through the target list processing function, and carrying out voice call on the target number of the filtered blacklist.

2. The method for processing the list data according to claim 1, wherein when receiving a request for creating the list synchronization task sent by a terminal, the method obtains a target function set according to the request for creating the list synchronization task, and performs code splicing on the target function set to obtain the list synchronization task, wherein the target function set includes a source table value function, a deduplication function, a check function, and a target table processing function, and includes:

3. The method for processing the list data according to claim 1, wherein the obtaining the list data to be processed from an initial list data table through the source table value-taking function when receiving the request for executing the list synchronization task sent by the terminal comprises:

when a request for executing the list synchronization task sent by the terminal is received, generating a first query statement for a first preset list type and an initial list data table according to a structured query language grammar rule;

and setting the first query statement as a parameter value corresponding to the source table value-taking function, and executing the source table value-taking function based on the parameter value to obtain the to-be-processed list data, wherein the to-be-processed list data comprises a to-be-screened number.

4. The method for processing the list data according to claim 3, wherein the performing the deduplication processing on the list data to be processed by the deduplication function and synchronizing the deduplicated list data into the first target list data table to obtain the first list data includes:

extracting the numbers to be screened from the list data to be processed;

judging whether the number to be screened exists in a first target name list data table or not through the duplication removal function;

if the difference duration is greater than or equal to a preset duration, keeping the list data corresponding to the number to be screened;

and packaging the removed list data according to a preset data format, and updating the packaged list data into the first target list data table to obtain first list data.

5. The method for processing the list data according to claim 1, wherein the performing the blacklist check on the first list data through the check function to obtain second list data, the second list data including a target number of a filtered blacklist, includes:

reading a number to be verified from the first list data, and judging whether the number to be verified exists in a preset blacklist or not through the verification function;

if the number to be verified does not exist in a preset blacklist, reserving the list data corresponding to the number to be verified;

if the number to be verified exists in a preset blacklist, deleting the list data corresponding to the number to be verified;

6. The method for processing the list data according to any one of claims 1 to 5, wherein when receiving a request for creating the list synchronization task sent by a terminal, the method for processing the list data further includes, before obtaining a target function set according to the request for creating the list synchronization task and performing code splicing on the target function set to obtain the list synchronization task, the target function set including a source table value-taking function, a deduplication function, a check function, and a target table processing function:

circularly extracting a plurality of preset codes from a plurality of developed source codes according to a preset service type, and preprocessing the plurality of preset codes, wherein the plurality of preset codes are used for indicating code blocks for realizing a plurality of synchronous data processing according to the preset service type;

carrying out syntax analysis on the preprocessed plurality of preset codes to obtain a plurality of abstract syntax trees;

extracting any two abstract syntax trees from the plurality of abstract syntax trees, and calculating cosine distances based on vocabularies in the any two abstract syntax trees to obtain similar preset codes;

performing the same category identification on the similar preset codes based on a preset rule to obtain a plurality of category identification codes;

screening and de-duplicating the identified preset codes according to the plurality of category identification codes to obtain code blocks to be encapsulated corresponding to each category;

and performing function encapsulation on the code block to be encapsulated corresponding to each category according to the preset service type, and adding corresponding preset parameters to the encapsulated functions to obtain a target function set.

7. The method of processing roster data according to any of claims 1-5, characterized in that after the synchronizing of the second roster data into a second target roster data table by the target table processing function and the voice call to the target number of the filtered black list, the method further comprises:

when a list pushing request sent by the terminal is received, setting a second query statement based on a data list to be pushed and a second preset list type;

executing the second query statement through the source table value-taking function to obtain list data to be pushed;

and acquiring identification information of a target terminal through the target table processing function, and pushing the list data to be pushed to the target terminal according to the identification information.

8. A list data processing apparatus, characterized in that the list data processing apparatus comprises:

the system comprises a creating module, a list synchronizing task processing module and a list synchronizing task processing module, wherein the creating module is used for acquiring a target function set according to a list synchronizing task creating request sent by a terminal and performing code splicing on the target function set to obtain a list synchronizing task, and the target function set comprises a source list value-taking function, a duplicate-removing function, a verification function and a target list processing function;

the acquisition module is used for acquiring the to-be-processed list data from an initial list data table through the source table value-taking function when receiving the list synchronization task execution request sent by the terminal;

the duplication removing module is used for carrying out duplication removing processing on the to-be-processed list data through the duplication removing function and synchronizing the duplicated list data into a first target list data table to obtain first list data;

the verification module is used for performing blacklist verification on the first list data through the verification function to obtain second list data, and the second list data comprises target numbers of filtered blacklists;

and the calling module is used for synchronizing the second list data into a second target list data table through the target list processing function and carrying out voice calling on the target number of the filtered blacklist.

9. A roster data processing apparatus, characterized in that the roster data processing apparatus comprises: a memory having instructions stored therein and at least one processor, the memory and the at least one processor interconnected by a line;

the at least one processor invokes the instructions in the memory to cause the roster data processing device to perform the roster data processing method of any of claims 1-7.

10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the method for processing roster data according to any one of claims 1-7.