WO2018059056A1

WO2018059056A1 - Service system data processing method and device

Info

Publication number: WO2018059056A1
Application number: PCT/CN2017/091916
Authority: WO
Inventors: 廉宜果; 郝建明; 马平清; 王巍; 韩智东; 李昂
Original assignee: 中国银联股份有限公司
Priority date: 2016-09-29
Filing date: 2017-07-06
Publication date: 2018-04-05
Also published as: CN106446184A; CN106446184B

Abstract

A service system data processing method and device. The method comprises: extracting data to be processed from a service system and adding same to a data table template to form a basic data table; establishing a label item table associated with the basic data table, wherein the label item table comprises label items set according to the data to be processed; setting optional values of the label items; according to a pre-set pushing rule, pushing the label items to a data filling person for value assignment, wherein the data filling person selects corresponding values from the set optional values of the label items to assign values to the label items; and pushing the label items assigned with values to an auditor for auditing according to the pre-set pushing rule, when the auditing is passed, the label items assigned with values coming into force, and when the auditing is not passed, pushing the label items that do not pass the auditing to the data filling person for value re-assignment. By means of the solution, data is extracted from a service system, and classification information about the data is perfected without developing and changing the service system, so that the operation of the service system is not affected.

Description

Business system data processing method and device

The present application claims the priority of the Chinese Patent Application Serial No. No. No. No. No. No. No. No. No. No. No. No. No. No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No

Technical field

The present invention relates to the field of data processing technologies, and in particular, to a service system data processing method and apparatus.

Background technique

Business systems (such as personnel systems, financial systems, material systems, etc.) generate a large amount of business data and process data during the operation, usually in the business system for statistical and analysis of these data, will be encountered in statistics and analysis The problem of insufficient information due to the classification of data. In order to improve the data and meet the needs of statistics and analysis, people usually transform the business system and develop and change it. This has the following problems: 1) As the business develops and the market changes, the needs of statistics and analysis also change. The development method cannot meet the statistical and analytical requirements of data, and it cannot adapt flexibly. Changes in demand for statistics and analysis; 2) Changes to the business system are risky, and if the change fails, it will fall back, which will affect the operation of the entire business system and cause immeasurable losses; 3) continuous development means continuous The human, material and financial resources have increased operating costs for the company.

Summary of the invention

The embodiment of the invention provides a data processing method for a business system, which satisfies the requirements for statistics and analysis of data, and does not need to develop and change a business system, thereby saving development costs. The method includes:

Extracting the to-be-processed data from the service system, and adding the to-be-processed data to the data table template to form a basic data table;

Establishing a label item table associated with the basic data table, the label item table including label items set according to data to be processed;

Setting an optional value of the label item in the label item table;

And the label item is pushed to the data filling person to perform label item assignment according to a preset pushing rule, wherein the data filling person selects a corresponding value from the set value of the set label item to assign a value to the label item;

The assigned label item is pushed to the auditor for review according to the preset push rule. When the audit is passed, the assigned label item takes effect, and when the audit fails, the approved label item that has not passed the audit is pushed. Re-assign the data entry personnel.

The embodiment of the invention further provides a service system data processing device, which satisfies the statistical and analysis requirements of the data, and does not need to develop and change the business system, thereby saving development cost. The device includes:

a data extraction module, configured to extract data to be processed from a service system, and add the to-be-processed data to a data table template to form a basic data table;

a tag item table establishing module, configured to establish a tag item table associated with the basic data table, where the tag item table includes a tag item set according to data to be processed;

a tag item value setting module, configured to set an optional value of the tag item in the tag item table;

a pushing module, configured to: push the label item to a data filling person to perform label item assignment according to a preset pushing rule, wherein the data filling person selects a corresponding value from the set value of the set label item to assign a value to the label item. ;

The push module is further configured to: push the assigned label item to the auditor for review according to a preset push rule, and when the audit is passed, the assigned label item takes effect, and when the audit fails, the license item fails. The audited assigned tag item is pushed to the data entry person for reassignment.

In the embodiment of the present invention, the to-be-processed data may be extracted from the service system, and then the to-be-processed data is added to the data table template to form a basic data table, and a label item table associated with the basic data table is established, wherein The tag item table includes a tag item set according to the data to be processed; setting an optional value of the tag item in the tag item table; and pushing the tag item to the data filling person according to a preset pushing rule to perform tag item assignment, The data filling personnel selects the corresponding value from the set value of the set label item to assign a value to the label item; according to the preset pushing rule, the assigned label item is pushed to the auditing personnel for review, and when the audit is passed, the The tagged tag item takes effect, and when the audit fails, the unassigned tagged item that has not passed the audit is pushed to the data filling person for reassignment. Compared with the prior art, the present invention extracts data from the business system, and then completes the classification information of the data, thereby satisfying the statistical and analysis requirements of the data, and without developing and changing the business system. It will not affect the operation of the entire business system, nor will it cause immeasurable losses and save development costs.

DRAWINGS

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly described below. It is obvious that the drawings in the following description are only some embodiments of the present invention. Correct Those skilled in the art can also obtain other drawings based on these drawings without paying any creative work. In the drawing:

1 is a flowchart of a data processing method for a service system according to an embodiment of the present invention;

2 is a schematic diagram of a basic data table and a label item table according to an embodiment of the present invention;

3 is a schematic diagram 1 of setting a label item table according to an embodiment of the present invention;

4 is a schematic diagram 2 of setting a label item table according to an embodiment of the present invention;

FIG. 5 is a schematic diagram 3 of setting a label item table according to an embodiment of the present invention; FIG.

6 is a schematic diagram of a push label item table according to an embodiment of the present invention;

FIG. 7 is a schematic diagram 1 of assigning a label item table according to an embodiment of the present invention; FIG.

FIG. 8 is a second schematic diagram of assigning a label item table according to an embodiment of the present invention; FIG.

FIG. 9 is a schematic diagram of reviewing a label item table according to an embodiment of the present invention; FIG.

FIG. 10 is a schematic structural diagram of a device for processing a data system of a service system according to an embodiment of the present invention.

detailed description

The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention. It is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.

The existing data is statistically analyzed and analyzed within the business system. When the data does not meet the statistical and analytical requirements, the business system needs to be developed and changed. This will bring about the following problems: 1) With the development of the business and the market The needs of change, statistics and analysis are also constantly changing. The development method cannot meet the statistical and analytical requirements of data, and it cannot flexibly adapt to changes in demand for statistics and analysis. 2) The change to the business system is If there is risk, it will fall back after the failure of the change, which will affect the operation of the entire business system and cause immeasurable losses. Based on this, the present invention provides a service system data processing method and apparatus.

FIG. 1 is a flowchart of a data processing method of a service system according to an embodiment of the present invention. As shown in FIG. 1 , the method includes:

Step 101: Extract data to be processed from the service system, and add the to-be-processed data to the data table template to form a basic data table.

Step 102: Establish a label item table associated with the basic data table, where the label item table includes a label item set according to data to be processed;

Step 103: Set an optional value of the label item in the label item table.

Step 104: Push the label item to the data filling person to perform label item assignment according to a preset pushing rule, wherein the data filling person selects a corresponding value from the set value of the set label item to assign a value to the label item;

Step 105: Push the assigned label item to the auditor for review according to the preset push rule. When the audit is passed, the assigned label item takes effect, and when the audit fails, the approved value that has not passed the audit is assigned. The tag item is pushed to the data entry person for reassignment.

In the specific implementation, the network connection with the target service system needs to be established first, and then the appropriate ETL tool is selected according to the actual situation to extract the data to be processed from the target service system (that is, the data to be added with the data label), which is to be processed. After the data is cleaned and converted, it is loaded into the target library table (data table template) to form the basic data table, as shown in Figure 2. When the data to be processed does not meet the requirements of the statistical analysis service, a label item table associated with the basic data table is created, as shown in FIG. 2, wherein the label item table includes a label set according to the data to be processed. Item (that is, data that satisfies the business needs of statistical analysis), the name of the tag item table may be the name of the underlying data table + check.

The ETL (Extract-Transform-Load) tool is used to describe the process of extracting, cleaning, transforming, and loading data from the source to the destination. The ETL tool can include DataStage and Kettle. , Informatica, and more.

The label items included in the label item table are defined by the user according to actual needs, and the user can add or delete the label items in the label item table as needed. After the user defines the label item, save it to the created label item table, and define the optional value of the label item and the configuration query condition. The value of the label item may include "yes" and "no". Figure 3-5 shows a schematic diagram of setting up a label item.

In the specific implementation, after the basic data table and the label item table are established, the value of the label item in the label item table is empty, and the label item table needs to be assigned. The invention will push the corresponding label item in the label item table to the corresponding data filling person according to the preset pushing rule (can be a person or a group of people), and the data filling person needs to be selected in advance. ) View the data content in the underlying data table item by item, and then select the value of the tag item from the optional values of the set tag item. The data filling personnel can be personnel of different departments within the business system, or people outside the business system. Figure 7-8 shows the assignment of the label item table.

Specifically, you can use the rule configuration page to customize the push rules in the foreground. The principle is to use the foreground page to write sql scripts. The preset push rule may include selecting a data filling person, selecting a tag item table name to be pushed, and the like, and then saving the push rule, as shown in FIG. 6.

In the specific implementation, after assigning the label items in the label item table, it is also necessary to review the assigned label items. The invention pushes the assigned label items to the auditing personnel according to the preset pushing rules, and the auditing personnel review the assigned label items one by one. At this time, the push rule may further include selecting an auditor, etc., as shown in FIG. 6.

Specifically, there are two cases in the audit: when the audit is passed, it indicates that the value of the assigned label item is valid; when the audit fails, the value of the assigned label item is not valid, and the assigned value that has not passed the audit is required. The tag item is pushed to the data entry person for reassignment. The failure to pass the audit can include two situations: one is that the data entry personnel do not assign values to all the tag items to be assigned; the other is that the tag items are assigned errors. A review opinion can also be attached when reviewing the assigned label item table, as shown in Figure 9.

In a specific implementation, the method may further include: outputting the label item table and the basic data table passed by the audit; or combining the basic data table and the label item table passed through the audit to obtain a service data table, and outputting the service data table. .

In specific implementation, the method can be applied to data analysis scenarios and data mining scenarios. When applied to a data analysis scenario, the method further includes: performing statistical analysis on the basic data table and the label item table passed through the audit; or performing statistical analysis on the service data table.

Based on the same inventive concept, an embodiment of the present invention further provides a service system data processing apparatus, as described in the following embodiments. Since the principle of solving the problem in the data processing device of the service system is similar to the data processing method of the service system, the implementation of the data processing device of the service system can be referred to the implementation of the data processing method of the service system, and the repeated description is not repeated. The term "unit" or "module" as used hereinafter may be a combination of software and/or hardware that implements a predetermined function. Although the apparatus described in the above embodiments is preferably implemented in software, hardware, or a combination of software and hardware, is also possible and conceivable.

FIG. 10 is a schematic structural diagram of a device for processing a data processing method according to an embodiment of the present invention. As shown in FIG. 10, the device includes:

The data extraction module 1001 is configured to extract data to be processed from the service system, and add the to-be-processed data to the data table template to form a basic data table.

a label item table creating module 1002, configured to establish a label item table associated with the basic data table, where the label item table includes a label item set according to data to be processed;

a tag item value setting module 1003, configured to set an optional value of the tag item in the tag item table;

The pushing module 1004 is configured to push the label item to the data filling person to perform label item assignment according to a preset pushing rule, wherein the data filling person selects a corresponding value from the set value of the set label item as the label item. Assignment

The pushing module 1004 is further configured to: push the assigned label item to the auditing personnel for review according to a preset pushing rule, and when the audit is passed, the assigned label item takes effect, and when the audit fails, the The data entry person is re-assigned by the approved assigned tag item.

In a specific implementation, the data extraction module 1001 is specifically configured to:

The data to be processed is cleaned and converted, and the data after the cleaning and conversion processing is loaded into the data table template to form a basic data table.

In a specific implementation, the apparatus further includes: an output module, configured to output a label item table and a basic data table that are approved for review.

In a specific implementation, the output module is further configured to: merge the basic data table and the label item table passed by the audit, obtain a service data table, and output the service data table.

The embodiment of the present invention further provides a computer device, including a memory, a processor, and a computer program stored on the memory and operable on the processor, where the processor implements the foregoing service when executing the computer program System data processing method.

Also provided in the embodiment of the present invention is a computer readable storage medium storing a computer program for executing the above-described business system data processing method.

In summary, the device of the present invention can be separated from the service system as a separate system; the device of the present invention extracts the data to be processed from the service system, and adds or deletes the data to the device of the present invention. In progress, it does not have any impact on the business system, effectively shielding the risks that may exist when the business system changes; using the device of the present invention can label the data of any business system, which is universal; the device of the present invention provides A series of modules, such as data extraction, label item generation, push rule configuration, label generation, label review, etc., fully meet the data labeling business requirements; using the device of the invention, no need to carry out business system development, and saving development costs.

Those skilled in the art will appreciate that embodiments of the present invention can be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or a combination of software and hardware. Moreover, the invention can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) including computer usable program code.

The present invention has been described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (system), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or FIG. These computers are available Program instructions to a processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing device to produce a machine such that instructions executed by a processor of a computer or other programmable data processing device are generated for implementation in a process A device or a plurality of processes and/or block diagrams of a device in a block or a plurality of blocks.

The computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture comprising the instruction device. The apparatus implements the functions specified in one or more blocks of a flow or a flow and/or block diagram of the flowchart.

These computer program instructions can also be loaded onto a computer or other programmable data processing device such that a series of operational steps are performed on a computer or other programmable device to produce computer-implemented processing for execution on a computer or other programmable device. The instructions provide steps for implementing the functions specified in one or more of the flow or in a block or blocks of a flow diagram.

The above description is only the preferred embodiment of the present invention, and is not intended to limit the present invention, and various changes and modifications may be made to the embodiments of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and scope of the present invention are intended to be included within the scope of the present invention.

Claims

A business system data processing method, comprising:

Extracting the to-be-processed data from the service system, and adding the to-be-processed data to the data table template to form a basic data table;

Establishing a label item table associated with the basic data table, the label item table including label items set according to data to be processed;

Setting an optional value of the label item in the label item table;

And the label item is pushed to the data filling person to perform label item assignment according to a preset pushing rule, wherein the data filling person selects a corresponding value from the set value of the set label item to assign a value to the label item;

The assigned label item is pushed to the auditor for review according to the preset push rule. When the audit is passed, the assigned label item takes effect, and when the audit fails, the approved label item that has not passed the audit is pushed. Re-assign the data entry personnel.
The service system data processing method according to claim 1, wherein the data to be processed is added to the data table template to form a basic data table, including:

The data to be processed is cleaned and converted, and the data after the cleaning and conversion processing is loaded into the data table template to form a basic data table.
The service system data processing method according to claim 1, further comprising:

The label item table and the underlying data table passed by the audit are output.
The service system data processing method according to claim 3, further comprising:

The basic data table and the label item table passed through the audit are merged to obtain a business data table, and the business data table is output.
A service system data processing device, comprising:

a data extraction module, configured to extract data to be processed from a service system, and add the to-be-processed data to a data table template to form a basic data table;

a tag item table establishing module, configured to establish a tag item table associated with the basic data table, where the tag item table includes a tag item set according to data to be processed;

a tag item value setting module, configured to set an optional value of the tag item in the tag item table;

a pushing module, configured to: push the label item to a data filling person to perform label item assignment according to a preset pushing rule, wherein the data filling person selects a corresponding value from the set value of the set label item to assign a value to the label item. ;

The push module is further configured to: push the assigned label item to the auditor for review according to a preset push rule, and when the audit is passed, the assigned label item takes effect, and when the audit fails, the license item fails. The audited assigned tag item is pushed to the data entry person for reassignment.
The service system data processing apparatus according to claim 5, wherein the data extraction module is specifically configured to:

The data to be processed is cleaned and converted, and the data after the cleaning and conversion processing is loaded into the data table template to form a basic data table.
The service system data processing apparatus of claim 5, further comprising:

An output module for outputting the label item table and the basic data table passed by the audit.
The service system data processing apparatus according to claim 7, wherein the output module is further configured to: merge the basic data table and the label item list passed through the audit, obtain a service data table, and output the service data. table.