CN113268231A - Data acquisition and comparison method - Google Patents

Abstract

The invention relates to the field of computer software and data processing, and particularly provides a data acquisition and comparison method, which comprises the following steps: s1, analyzing and collecting requirements; s2, compiling a configuration file; s3, fine-tuning a configuration file and a Python script; s4, running comparison logic; and S5, displaying the comparison result in foreground. Compared with the prior art, the invention can improve the reusability of the codes in the development process of the communication equipment data acquisition comparison business codes and reduce the repeated work in the development process.

Description

Data acquisition and comparison method

Technical Field

The invention relates to the field of computer software and data processing, and particularly provides a data acquisition and comparison method.

Background

The acquisition comparison takes the acquisition comparison service requirements as a starting point, and each sql script needs to be modified and then the shell script needs to be modified when the new acquisition comparison related service requirements are processed before. The files need to be changed every time of acquiring and comparing services, so that the workload for processing new acquiring and comparing services every time is high, and the problems of low code reusability, low efficiency and the like are caused.

Disclosure of Invention

Aiming at the defects of the prior art, the invention provides a data acquisition and comparison method with strong practicability.

The technical scheme adopted by the invention for solving the technical problems is as follows:

a data acquisition and comparison method comprises the following steps:

s1, analyzing and collecting requirements;

s2, compiling a configuration file;

s3, fine-tuning a configuration file and a Python script;

s4, running comparison logic;

and S5, displaying the comparison result in foreground.

Further, in step S1, different acquisition tasks correspond to different acquisition tables and resource tables, each acquisition requires analyzing the table and field mapping relationship corresponding to the current acquisition package, the dependency relationship between resource tables and acquisition tables, and the relationship between the client requirement and the communication resource, and then corresponds to the data table one by one.

Furthermore, under the same service acquisition scene, the service public logic is extracted and divided into two parts, namely a configurable attribute and the public service logic, which are respectively designed into a configuration file and a Python script.

Preferably, under different acquisition service scenarios, part of logic needs to be finely tuned for configuration files and Python scripts.

Further, in step S2, according to the acquisition requirement, configuring the database connection information, the acquisition package, the resource table and the acquisition table mapping and the in-table field mapping information in the configuration file, and the Python script automatically runs the comparison logic according to the configuration file information and records the result into the database.

Further, in step S3, the Python script is the core of the comparison logic, and only some special fields need to be modified or added to adapt to the current scene acquisition requirement.

Further, in step S4, first, the configuration information of each part of the configuration file is analyzed, then, the reusable part of the collection and comparison service logic code is obtained, the loop body includes the analysis of the table and field mapping of the configuration file, the comparison logic and comparison result of the fields in the table, and the sql statement is stored in the database logic.

Further, in step S5, when displaying in foreground, the sql is first parsed into a real comparison result, and then displayed on the user operation interface;

and the user checks the comparison result and then the background takes out the sql statement corresponding to the result item to execute, thereby completing the acquisition and comparison business.

Compared with the prior art, the data acquisition and comparison method has the following outstanding beneficial effects:

the data acquisition and comparison method can improve the reusability of codes in the development process of the data acquisition and comparison business codes of the communication equipment and reduce the repeated work in the development process.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.

FIG. 1 is a schematic flow chart of a data acquisition and comparison method;

FIG. 2 is a schematic flow chart of compiling configuration files in a data acquisition comparison method;

FIG. 3 is a schematic flow chart of comparison logic in a data acquisition and comparison method.

Detailed Description

The present invention will be described in further detail with reference to specific embodiments in order to better understand the technical solutions of the present invention. It is to be understood that the described embodiments are merely exemplary of the invention, and not restrictive of the full scope of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

A preferred embodiment is given below:

as shown in fig. 1, in the same acquisition service scenario, the data acquisition and comparison method in this embodiment is embodied in that a service common logic is extracted and divided into two parts, namely, a configurable attribute and a common service logic, which are respectively designed as a configuration file and a Python script.

Under different acquisition service scenes, only a part of logic needs to be finely adjusted for the configuration file and the Python script.

And splicing the comparison result into sql and inputting the sql into a database, displaying the comparison result on a foreground after the foreground preliminarily analyzes the sql, checking the data to be input by a user, namely checking the sql to be executed, and further finishing acquisition and comparison.

The configuration file structure can be adjusted according to actual requirements so as to adapt to the requirements of acquiring and comparing services in different scenes.

The method specifically comprises the following steps:

s1, analyzing and collecting requirements:

different acquisition tasks correspond to different acquisition tables and resource tables, and the table and field mapping relation corresponding to the current acquisition packet and the dependency relation between the resource tables and the acquisition tables need to be analyzed during each acquisition. The relationship between the customer requirements and the communication resources needs to be analyzed and then mapped to the database tables one-to-one.

S2, writing a configuration file:

as shown in fig. 2, the core of the method is a configuration file, and the Python script automatically runs a comparison logic according to the configuration file information and records the result into a database. In this step, according to the acquisition requirement of the first step of analysis, information such as database connection information, acquisition packages, resource table and acquisition table mapping, and in-table field mapping, etc. is configured in the configuration file. Therefore, repeated code development can be reduced, and reusability of logic codes is improved.

S3, fine-tuning the configuration file and Python script:

the Python script is the core of the comparison logic of the method. Configuration information for a general configuration file may not be enough to satisfy acquisition comparison services in other scenarios, so that the configuration file and a corresponding comparison script analysis part code may need to be flexibly modified for a current scenario. Only some special fields and the like need to be modified or added to adapt to the current scene acquisition requirement, and the configuration and the overall comparison logic do not need to be modified.

S4, running comparison logic:

as shown in fig. 3, the configuration information of each part of the configuration file is firstly analyzed, then the reusable part of the collection and comparison service logic code is used, and the loop body comprises the table mapping of the configuration file, the field comparison logic in the table and the comparison result-sql statement stored in the database logic. The partial code is a core service code of the scheme, is a basic logic for acquiring services, can be used in each acquisition scene, and does not need to be modified during each development.

Inf.ini: a configuration file.

CollectTables: and the acquisition table is used for storing the data acquired by the communication equipment.

Resource tables: and the resource table stores the equipment information recorded in the current database.

TempMapperRecordList: and temporarily storing the results of the previous cycle, and only judging whether the current comparison resources are recorded or not for the following cycle so as to judge whether to execute insert or update format splicing.

Commit: and (4) submitting database transactions, wherein the database submission is placed at the end of the whole comparison logic in order to prevent errors in the comparison process.

And S5, displaying the comparison result by the foreground, checking by the user, and executing sql. The last step is just to save the spliced sql statement. When foreground display is carried out, the sql is firstly analyzed into a real comparison result, and then the real comparison result is displayed on a user operation interface. And the user checks the comparison result and then the background takes out the sql statement corresponding to the result item to execute, thereby completing the acquisition and comparison business.

The above embodiments are only specific cases of the present invention, and the scope of the present invention includes but is not limited to the above embodiments, and any suitable changes or substitutions that are consistent with the claims of a data acquisition and comparison method of the present invention and are made by those skilled in the art should fall within the scope of the present invention.

Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.

Claims (8)

1. A data acquisition and comparison method is characterized by comprising the following steps:

s1, analyzing and collecting requirements;

s2, compiling a configuration file;

s3, fine-tuning a configuration file and a Python script;

s4, running comparison logic;

and S5, displaying the comparison result in foreground.

2. A data collection comparison method according to claim 1, wherein in step S1, different collection tasks correspond to different collection tables and resource tables, and each collection requires analysis of the tables and field mapping relationships corresponding to the current collection package, the dependencies between the resource tables and the collection tables, analysis of the relationships between the customer requirements and the communication resources, and then one-to-one correspondence with the data tables.

3. The data acquisition and comparison method according to claim 2, wherein in the same acquisition service scenario, the service common logic is extracted and divided into two parts, namely a configurable attribute and a common service logic, which are respectively designed into a configuration file and a Python script.

4. The data collection comparison method of claim 3, wherein part of the logic needs to be trimmed for configuration files and Python scripts under different collection service scenarios.

5. The method according to claim 1, wherein in step S2, according to the collection requirement, the database connection information, the collection package, the resource table and the collection table mapping and the in-table field mapping information are configured in the configuration file, and the Python script automatically runs the comparison logic according to the configuration file information and records the result into the database.

6. The method according to claim 1, wherein in step S3, the Python script is the core of the comparison logic, and only some special fields need to be modified or added to adapt to the current scene acquisition requirement.

7. The method according to claim 1, wherein in step S4, the configuration information of each part of the configuration file is parsed, then the reusable part of the collection and comparison service logic code is parsed, the loop body includes parsing the table and field mapping of the configuration file, the comparison logic and comparison result of the fields in the table, and the sql statement is saved in the database logic.

8. The data acquisition and comparison method according to claim 1, wherein in step S5, in foreground display, sql is parsed into real comparison results and then displayed on the user interface;

and the user checks the comparison result and then the background takes out the sql statement corresponding to the result item to execute, thereby completing the acquisition and comparison business.

Images