WO2018223678A1

WO2018223678A1 - Data processing method and processing device

Info

Publication number: WO2018223678A1
Application number: PCT/CN2017/120155
Authority: WO
Inventors: 张磊; 陈少杰; 张文明
Original assignee: 武汉斗鱼网络科技有限公司
Priority date: 2017-06-07
Filing date: 2017-12-29
Publication date: 2018-12-13
Also published as: CN107368523A; CN107368523B

Abstract

Disclosed are a data processing method and system, the method comprising: storing data content to a bean file and transferring the bean file to a data warehouse controller, wherein there is a strong coupling relationship between the bean file and the data warehouse controller (S1); the data warehouse controller selecting, according to an externally introduced service logic parameter instruction, a corresponding data processing module to process the data content in the bean file, wherein the data warehouse controller is coupled to each data processing module via a unified protocol (S2). The data warehouse controller is coupled to each data processing module via a unified protocol, ensuring that all of the data is processed according to a uniform principle; abstract unification of data processing logic facilitates use of a flow logic related to follow-up maintenance data, and increases the maintainability and expandability of procedures during a development process.

Description

Data processing method and processing device

cross reference

The present application is hereby incorporated by reference in its entirety in its entirety in its entirety in the the the the the the the the the the the the

Technical field

The present invention relates to the field of data control technologies, and in particular, to a data processing method and system.

Background technique

In the development process, the data processing is often encountered, and the different processing of the data corresponds to different data processing logics. For example, some need to store the data in a disk file, and some store the data in a database. Processing logic.

In the traditional scheme, the data of these data is decentralized, and there is no unified management. One type of data processing needs to design a corresponding set of data processing logic, so the data processing logic of the whole code is very confusing.

Summary of the invention

The present invention provides a data processing method and system that overcomes the above problems or at least partially solves the above problems, and overcomes the problem that the data processing logic is confusing due to the lack of unified management of data.

According to a first aspect of the present invention, a data processing method is provided, comprising:

S1, storing the data content in the bean file, and passing the bean file to the data warehouse controller;

S2. The data warehouse controller selects a data processing module corresponding to the business logic parameter instruction from the plurality of data processing modules, and performs corresponding business logic processing on the data content in the bean file by using the selected data processing module;

The bean file and the data warehouse controller are strongly coupled, and the data warehouse controller and each data processing module are coupled by a unified protocol.

The invention has the beneficial effects that a perfect data control design scheme is proposed, and the data warehouse controller is coupled with each data processing module through a unified protocol, which can ensure that all data can be processed according to a unified rule, and the data is processed. The logic is abstracted and unified, which facilitates the maintenance of data-related flow logic in the later stage, and improves the maintainability of the program and the scalability of the program during the development process.

Based on the above technical solutions, the present invention can also be improved as follows.

Further, the data warehouse controller and each data processing module are coupled by a unified protocol, including:

An interface object for data processing logic is defined on the data warehouse controller, and an interface function is defined in the interface object, and specific data processing logic is implemented through the interface function, wherein the data processing logic has a correspondence with the data processing module relationship.

Further, the step S2 includes:

S21. The data warehouse controller switches from the coupled plurality of data processing modules to the data processing module corresponding to the service logic parameter instruction according to the external incoming business logic parameter instruction, where each data processing module corresponds to Data processing logic class;

S22. The interface function in the interface object is duplicated in the data processing logic class to implement data logical processing on the data content, wherein the data processing logic class inherits the interface object in the data warehouse controller.

Further, the step S21 includes:

Design a switch class in the data warehouse controller and write a switch function in the switch class;

The interface object is input to the switching function, and an interface function in the interface object is called in the switching function to implement switching of the data processing logic, thereby implementing switching of the data processing module.

Further, the data content in the bean file has multiple types, and the parameter of the interface function in the interface object is an object object, and multiple types of data content in the bean file are transmitted to the data processing module through the bean object.

Further, each of the data processing modules has a one-to-one correspondence with the processed data type, and the step S22 further includes:

The data processing module judges the data type of all the data contents in the bean file, and performs corresponding business logic processing on the data of the data type corresponding to the data processing module.

According to a second aspect of the present invention, there is also provided a data processing system comprising:

a storage module for storing data content in a bean file and passing the bean file to the data warehouse controller;

a data warehouse controller for selecting, by the data warehouse controller, a data processing module corresponding to the business logic parameter instruction from the plurality of data processing modules;

a data processing module, configured to perform corresponding business logic processing on the data content in the bean file;

Further, the data warehouse controller is further configured to:

An interface object for data processing logic is defined, and an interface function is defined in the interface object, and data processing logic is implemented by the interface function, wherein the data processing logic has a corresponding relationship with the data processing module.

Further, the data warehouse controller is specifically configured to:

Switching from the coupled plurality of data processing modules to the data processing module corresponding to the service logic parameter instruction according to the external incoming business logic parameter instruction, wherein each data processing module corresponds to a data processing logic class;

Each of the data processing modules is specifically configured to:

Defining a data processing logic class and rewriting an interface function in the interface in the data processing logic class to implement data logical processing on the data content, wherein the data processing logic class inherits an interface object in the data warehouse controller.

According to a third aspect, an electronic device is provided, comprising:

At least one processor, at least one memory, and a bus; wherein

The processor and the memory complete communication with each other through the bus;

The memory stores program instructions executable by the processor, the processor invoking the program instructions to perform the following methods:

Store the data content in a bean file and pass the bean file to the data warehouse controller;

The data warehouse controller selects a data processing module corresponding to the business logic parameter instruction from the plurality of data processing modules, and performs corresponding business logic processing on the data content in the bean file through the selected data processing module;

DRAWINGS

1 is a flow chart of a data processing method according to an embodiment of the present invention;

2 is a block diagram showing the connection of a data processing system according to another embodiment of the present invention;

FIG. 3 is a block diagram of an electronic device connection according to an embodiment of the present invention.

detailed description

The specific embodiments of the present invention are further described in detail below with reference to the drawings and embodiments. The following examples are intended to illustrate the invention but are not intended to limit the scope of the invention.

Referring to FIG. 1, FIG. 1 provides a data processing method according to an embodiment of the present invention, comprising: S1, storing data content in a bean file, and passing the bean file to a data warehouse controller; S2, the data warehouse controller is from more than The data processing module corresponding to the business logic parameter instruction is selected in the data processing module, and the corresponding data processing is performed on the data content in the bean file by the selected data processing module. The bean file and the data warehouse controller are strongly coupled, and the data warehouse controller and each data processing module are coupled by a unified protocol.

The entire process of data control and processing is to first store all the data content that needs to be processed into a bean file, where the bean file is an object that stores data in memory, and then the bean file is passed to the data warehouse controller. The data warehouse controller is responsible for selecting the corresponding data processing module to perform corresponding business logic processing on the data content in the bean file. The data warehouse controller is the core of the entire data processing flow process, and the real data processing process is performed on the data in the bean file by selecting the corresponding data processing module. In this scheme, there is a direct coupling relationship between the bean file and the data warehouse controller, which is the traditional strong coupling relationship. The reason why this level is designed as a strong coupling relationship is because only the simple data transfer does not involve Any data processing logic is simply designed as a strong coupling relationship to facilitate data transfer. The data warehouse controller and each data processing module are coupled by a unified protocol, and the data warehouse controller selects the corresponding data processing module to perform the data content in the bean file according to the external incoming business logic parameter instruction. Processing, each data processing module has a different set of business logic processing, and can implement various data processing logics through a unified protocol.

This embodiment proposes a complete design scheme of data control. The data warehouse controller is coupled with each data processing module through a unified protocol, which can ensure that all data can be processed according to the unified rules, and the data processing logic is abstracted. The uniformity facilitates the maintenance of data-related flow logic in the later stage, improving the maintainability of the program and the scalability of the program during the development process.

In an embodiment of the present invention, the data warehouse controller and each data processing module are coupled by a unified protocol, including: defining an interface object for data processing logic on the data warehouse controller, and The interface object defines an interface function, and the data processing logic is implemented by the interface function, wherein the data processing logic has a corresponding relationship with the data processing module.

In order to enable unified processing logic between the data warehouse controller and each data processing module, and in order to reduce the coupling relationship between the data warehouse controller and each data processing module, the embodiment implements data through interface design. The coupling between the warehouse controller and each data processing module.

Specifically, an interface object for data processing logic is designed on the data warehouse controller. In this embodiment, the name of the designed interface object is IDataHander, and an interface function is defined in the interface object, and the interface function is The definition is as follows:

Pubic viod handleData(Object data);

The name of the interface function is handleData, and the parameter of the interface function is an object object. In java, object is a basic data type, that is, the parent class of all data types, which can be used to carry any type of data object, so that all types The data can be passed to the corresponding data processing module through the object object.

The specific data processing logic can be realized by the interface function in the defined interface object, wherein the data processing logic has a corresponding relationship with the data processing module, and a specific data processing logic corresponds to a specific data processing module for processing.

In this embodiment, the data warehouse controller and each data processing module adopt an interface coupling manner to pave the way for the subsequent data processing logic switching, and can switch different data processing modules through a unified switching manner, thereby achieving data unification. The purpose of management.

In another embodiment of the present invention, the step S2 includes: S21, according to an external incoming service logic parameter instruction, the data warehouse controller switches from the coupled plurality of data processing modules to the service logic parameter instruction. Corresponding data processing module, wherein each data processing module corresponds to a data processing logic class; S22, in the data processing logic class, an interface function in the interface is overwritten to implement data logical processing on the data content, wherein The data processing logic class inherits interface objects in the data warehouse controller.

In the specific business processing logic, the data warehouse controller receives the business logic parameter instruction introduced from the outside, wherein the business logic parameter instruction refers to what kind of business processing needs to be performed on the data content in the bean file. After receiving the business logic parameter instruction, the data warehouse controller switches from all the coupled data processing modules to the data processing module corresponding to the business logic parameter instruction, and the data processing module performs specific data processing.

Wherein, each data processing module involves a corresponding data processing logic class, these data processing logic classes inherit the interface object in the data warehouse controller, and then overwrite the interface function in the interface object in the data processing logic class, To achieve specific logical processing of the data content of the bean file.

Each data processing logic is a separate business function module. In order to facilitate the data warehouse controller to quickly switch different processing logics, in this embodiment, all data processing logics are designed into functional modules having the same structure, which is convenient. The data logic is switched later.

For example, externally, a request is passed to the data warehouse controller. To store the data in the bean file into the database, a data processing logic class A can be designed, corresponding to the data processing module A, and the data processing logic class A inherits the IDataHander. The interface object, then overwrites the interface function handledata in the IDataHander interface object in the data processing logic class A, and finally completes the real database storage logic in the interface function handledata.

For another example, to store the data of the bean file into the disk file, a data processing logic class B can be designed, corresponding to the data processing module B, the data processing logic class B inherits the IDataHander interface object, and then in the data processing logic class B. The interface function handledata in the IDataHander interface object is rewritten, and the real disk file storage logic is finally completed in the interface function handledata.

Through the description of the above two examples, it can be found that all the data processing modules have the same structure, and the corresponding data processing logic functions are completed strictly according to the designed interface object and interface function, and the data is unified. The purpose of management.

In an embodiment of the present invention, the step S21 includes: designing a switching class in the data warehouse controller, and writing a switching function in the switching class; transmitting the interface object to the switching function, and switching The interface function in the interface object is called to realize the switching of the data processing logic, thereby implementing the switching of the data processing module.

In the process of the data warehouse controller switching to the data processing module, the switching of the data processing module is designed in an abstract manner. In this embodiment, the switching class is designed in the data warehouse controller, and the name of the switching class is SwitchUtils. And write a switch function switchDataControl(IDataHander idataHandler) in the switch class SwitchUtils. Wherein, the parameter in the switching function is an interface object, and the interface object of the foregoing design is used to implement data processing logic. Therefore, the interface object is passed to the switching function, that is, the data processing logic parameter is passed, according to the incoming Data processing logic parameters, the data warehouse controller can know which data processing logic class to switch to, that is, which data processing module to switch to. Then, in the switch function switchDataControl, the interface function handledata in the IDataHander of the interface object is called, thereby implementing the call of the data processing logic.

In this embodiment, when the data processing module is switched, the corresponding data processing logic parameter needs to be input into the switching function, for example, the data processing logic class name is input to the switching function, and the data processing logic class name and The data processing module has a corresponding relationship, so the data warehouse controller can know which data processing module should be switched to.

In another embodiment of the present invention, each of the data processing modules has a one-to-one correspondence with the processed data type, and the step S22 further includes: the data processing module performs the data type of all the data contents in the bean file. Judging and performing business logic processing on data of a data type corresponding to the data processing module.

It should be noted that the data content in the bean file usually has multiple types. Since a data processing module can only process one type of data, the data warehouse controller passes all data types in the bean file to the data processing module. After the data, the data processing module judges the data type of all the data in the bean file, and performs specific business logic processing only on the data of the type that can be processed by itself.

Referring to FIG. 2, a data processing system according to another embodiment of the present invention includes a storage module 21, a data warehouse controller 22, and a data processing module 23.

The storage module 21 is configured to store the data content in the bean file and pass the bean file to the data warehouse controller.

The data warehouse controller 22 is configured to select a data processing mode corresponding to the business logic parameter instruction from the plurality of data processing modules.

The data processing module 23 is configured to perform corresponding business logic processing on the data content in the bean file; wherein the bean file and the data warehouse controller are strongly coupled, and the data warehouse controller and each data processing module Coupling through a unified protocol.

The data warehouse controller 22 is further configured to:

The data warehouse controller 22 is specifically configured to:

Each of the data processing modules 23 is specifically configured to:

Defining a data processing logic class and rewriting an interface function in the interface in the data processing logic class to implement data logical processing on the data content, wherein the data processing logic class inherits the interface object in the data warehouse controller.

Referring to FIG. 3, an electronic device according to the present invention includes: a processor 301, a memory 302, and a bus 303. The processor 301 and the memory 302 complete communication with each other through the bus 303. .

The processor 301 is configured to invoke program instructions in the memory 302 to perform the methods provided by the foregoing method embodiments, for example, including: storing data content in a bean file, and passing the bean file to a data warehouse control The data warehouse controller selects a data processing module corresponding to the business logic parameter instruction from the plurality of data processing modules, and performs corresponding business logic processing on the data content in the bean file through the selected data processing module; The bean file and the data warehouse controller are strongly coupled, and the data warehouse controller and each data processing module are coupled by a unified protocol.

The invention discloses a computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions, the computer being capable of executing the corresponding implementation when the program instructions are executed by the computer The intelligent filtering control layout method provided by the example includes, for example, storing the data content in the bean file and passing the bean file to the data warehouse controller; the data warehouse controller is selected from the plurality of data processing modules and the business logic a data processing module corresponding to the parameter instruction, and performing corresponding business logic processing on the data content in the bean file by the selected data processing module; wherein the bean file and the data warehouse controller are strongly coupled, the data warehouse The controller is coupled to each data processing module by a uniform protocol.

The present invention also provides a non-transitory computer readable storage medium storing computer instructions for causing a computer to execute the smart layout control method provided by the corresponding embodiment, for example, including : storing the data content in the bean file, and passing the bean file to the data warehouse controller; the data warehouse controller selects a data processing module corresponding to the business logic parameter instruction from the plurality of data processing modules, and passes the selected The data processing module performs corresponding business logic processing on the data content in the bean file; wherein the bean file and the data warehouse controller are strongly coupled, and the data warehouse controller and each data processing module are unified The protocol is coupled.

A person skilled in the art can understand that all or part of the steps of implementing the above method embodiments may be completed by using hardware related to the program instructions. The foregoing program may be stored in a computer readable storage medium, and the program is executed when executed. The foregoing steps include the steps of the foregoing method embodiments; and the foregoing storage medium includes: a medium that can store program codes, such as a ROM, a RAM, a magnetic disk, or an optical disk.

The embodiments described above are merely illustrative, wherein the units illustrated as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, ie may be located in one place. Or it can be distributed to multiple network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the embodiment. Those of ordinary skill in the art can understand and implement without deliberate labor.

Through the description of the above embodiments, those skilled in the art can clearly understand that the various embodiments can be implemented by means of software plus a necessary general hardware platform, and of course, by hardware. Based on such understanding, the above-described technical solutions may be embodied in the form of software products in essence or in the form of software products, which may be stored in a computer readable storage medium such as ROM/RAM, magnetic Discs, optical discs, etc., include instructions for causing a computer device (which may be a personal computer, server, or network device, etc.) to perform various embodiments of the various embodiments or embodiments.

The invention provides a data processing method and system. By designing a data warehouse controller, the original distributed data processing is unified and merged and managed. The scheme firstly gives the overall scheme design by introducing the coupling relationship between the levels. The framework then introduces the specific implementation methods of data processing logic and data processing logic switching, and finally achieves the purpose of design, improving the maintainability and scalability of the program.

Finally, the method of the present application is only a preferred embodiment and is not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and scope of the present invention are intended to be included within the scope of the present invention.

Claims

A data processing method, comprising:

S1, storing the data content in the bean file, and passing the bean file to the data warehouse controller;

S2. The data warehouse controller selects a data processing module corresponding to the business logic parameter instruction from the plurality of data processing modules, and performs corresponding business logic processing on the data content in the bean file by using the selected data processing module;

The bean file and the data warehouse controller are strongly coupled, and the data warehouse controller and each data processing module are coupled by a unified protocol.
The data processing method according to claim 1, wherein the coupling between the data warehouse controller and each of the data processing modules by a unified protocol comprises:

An interface object for data processing logic is defined on the data warehouse controller, and an interface function is defined in the interface object, and the data processing logic is implemented by the interface function, wherein the data processing logic has a corresponding relationship with the data processing module.
The data processing method according to claim 2, wherein the step S2 comprises:

S21. The data warehouse controller switches from the coupled plurality of data processing modules to the data processing module corresponding to the service logic parameter instruction according to the external incoming business logic parameter instruction, where each data processing module corresponds to Data processing logic class;

S22. The interface function in the interface object is duplicated in the data processing logic class to implement data logical processing on the data content, wherein the data processing logic class inherits the interface object in the data warehouse controller.
The data processing method according to claim 3, wherein said step S21 comprises:

Design a switch class in the data warehouse controller and write a switch function in the switch class;

The interface object is input to the switching function, and an interface function in the interface object is called in the switching function to implement switching of the data processing logic, thereby implementing switching of the data processing module.
The data processing method according to claim 4, wherein the data content in the bean file has multiple types, and the parameter of the interface function in the interface object is an object object, and the bean file is used in the bean file. Types of data content are passed to the data processing module.
The data processing method according to claim 5, wherein each of the data processing modules has a one-to-one correspondence with the processed data type, and the step S22 further includes:

The data processing module judges the data type of all the data contents in the bean file, and performs corresponding business logic processing on the data of the data type corresponding to the data processing module.
A data processing system, comprising:

a storage module for storing data content in a bean file and passing the bean file to the data warehouse controller;

a data warehouse controller, configured to select, from a plurality of data processing modules, a data processing module corresponding to the business logic parameter instruction;

a data processing module, configured to perform corresponding business logic processing on the data content in the bean file;

The bean file and the data warehouse controller are strongly coupled, and the data warehouse controller and each data processing module are coupled by a unified protocol.
A data processing system according to claim 7, wherein said data warehouse controller is further configured to:

An interface object for data processing logic is defined, and an interface function is defined in the interface object, and data processing logic is implemented by the interface function, wherein the data processing logic has a corresponding relationship with the data processing module.
A data processing system according to claim 8, wherein said data warehouse controller is specifically configured to:

Switching from the coupled plurality of data processing modules to the data processing module corresponding to the service logic parameter instruction according to the external incoming business logic parameter instruction, wherein each data processing module corresponds to a data processing logic class;

Each of the data processing modules is specifically configured to:

A data processing logic class is defined, and an interface function in the interface is overwritten in the data processing logic class to implement data logical processing on the data content, wherein the data processing logic class inherits the interface object in the data warehouse controller.
A device for a toy control system, comprising:

At least one processor, at least one memory, a communication interface, and a bus;

Wherein the processor, the memory, and the communication interface complete communication with each other through the bus;

The communication interface is used for information transmission between the device and a communication device of the toy control system;

The memory stores program instructions executable by the processor, the processor invoking the program instructions to perform the method of claims 1-6.
A computer program product, comprising: a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions, when the program instructions are executed by a computer, The computer instructions cause the computer to perform the method of claims 1-6.
A non-transitory computer readable storage medium, characterized in that the non-transitory computer readable storage medium stores computer instructions that cause the computer to perform the method of claims 1-6.