CN100587693C - Method and system for obtaining data from multiple data sources - Google Patents

Abstract

The invention discloses a method and system for obtaining data from multiple data sources, which solves the problem of poor flexibility in the current method of obtaining data by using fixed codes. The method includes: obtaining an entity file; obtaining all attributes of the entity from the entity file, wherein the entity attribute is called a data source; parsing the entity attribute, constructing an entity structure tree, and displaying it; querying the entity structure tree, and selecting The data source to get entity property values. Through the present invention, users can flexibly define the corresponding relationship between entity attributes and business parameters according to requirements, and extract corresponding attribute values from entities in an active way of "pull" according to the defined corresponding relationship when running the business parameters. Composition of business parameters, so as to realize the variability of business control without fixed coding.

Description

Method and system for obtaining data from multiple data sources

technical field

The invention relates to computer data processing technology, in particular to a method and system for acquiring data from multiple data sources.

Background technique

In the data processing process of the business application system, it is often necessary to obtain data from multiple data sources for processing. For example, in financial applications, many business documents require budget control, and the budget control system realizes the budget control function by processing the budget data of documents. In the budget control system, budget control parameters required for budget control are defined according to business conditions, which correspond to some attribute information of business documents. When performing budget processing, attribute information corresponding to budget control parameters needs to be obtained from business documents to form budget control Parameters for budget control processing.

For example, the budget control parameters include budget organization, budget item, budget element, budget period, and currency, which respectively correspond to the organization, business item, budget type, business occurrence date, and currency information of the document. When performing budget control on the document, the budget control system searches for the corresponding budget data in the document according to the mapping relationship, and then controls whether the specific operation of the document (such as payment) meets the occurrence conditions according to certain rules, and then makes a decision. Corresponding decision-making, such as deducting the budget if the budget is sufficient, the document enters the next business processing link, and returning to manual decision-making if the budget is insufficient.

In the above example of budget control, the business document is used as a data source to provide budget data to the budget control system, wherein various attribute information of the document can be used as a data source of budget control parameters. If the document needs to participate in budget control, the existing implementation method is implemented by code, by adding the function of finding budget data in the business system where the document is located, that is, each business document development team completes the whole process of business document search budget . This method of obtaining attributes on documents through fixed codes to form budget control parameters in the program is a passive "push" method, which has the following disadvantages:

First, budgetary control has poor variability and cannot adapt to changing requirements because the control logic is fixed in code. For example, in a voucher document, the source of the budget item may have many different forms: account, account + material, account + organization, etc. It will be very difficult to complete these combinations through code, because the code cannot exhaustively enumerate one by one. The combination is relatively fixed. Once the combination that the user needs is not in the combination written by the code, the code has to be re-modified, and the maintenance cost is very high;

Second, the development workload is heavy, that is, all modules that enable budget control must write codes related to budget control;

Third, the budget business knowledge requirements of developers of various business systems are very high. They must clearly understand the details of many budget modules, otherwise the development cannot be carried out;

Fourth, the codes of various business systems and budget control systems are highly coupled, making it very difficult to reconstruct the budget system.

Contents of the invention

The technical problem to be solved by the present invention is to provide a method and a system for obtaining data from multiple data sources, so as to solve the problems existing in the current method of obtaining data by using fixed codes.

In order to solve the above technical problems, according to the specific embodiments provided by the invention, the invention discloses the following technical solutions:

Methods for obtaining data from multiple data sources, including:

Get the entity file;

Get all the properties of the entity from the entity file, where said entity properties are called data sources;

Analyzing entity attributes, building an entity structure tree, and displaying it; specifically including: predefining rules for accessing entities; analyzing entity attribute properties according to the rules; constructing entity structure trees according to the attribute properties; and saving the parsed file according to the rules entity information;

The entity structure tree is queried to obtain entity attribute values from the selected data source.

Wherein, the entity attribute value is obtained from the selected data source according to the following steps: select the entity attribute to establish a mapping relationship with the business parameter; when executing the business parameter, obtain the corresponding entity attribute value according to the mapping relationship.

A system that acquires data from multiple sources, including:

The general interface is used to predefine the rules for accessing entities; the entity parsing unit is used to obtain entity files; obtain all attributes of entities from entity files, wherein the entity attributes are called data sources; parse entity attributes to build entity structure trees , specifically includes: analyzing entity attribute properties according to the rules defined by the general interface, constructing an entity structure tree according to the attribute properties, and saving the parsed entity information according to the rules;

An interactive unit, configured to display the entity structure tree and provide a function of querying the entity structure tree;

The data acquisition unit is used to acquire entity attribute values from the selected data source.

Wherein, the general interface can be extended.

Wherein, the interaction unit establishes the mapping relationship between the selected entity attribute and the business parameter when providing the query function; when executing the business parameter, the data acquisition unit acquires the entity attribute value from the corresponding data source according to the mapping relationship.

A system for obtaining data from multiple data sources, including: a client and a server, the client is used to initiate a data acquisition request to the server, and the server is used to obtain data from multiple data sources and return; wherein the server It includes an entity resolution engine, which is used to obtain entity files and obtain all attributes of entities therefrom, resolve entity attributes, and build entity structure trees, wherein the entity attributes are called data sources; the client uses the user interface to convert the entity structure Tree display; also includes a parser, which is used to call the entity resolution engine on the server side to resolve entities according to the client request, and obtain the corresponding attribute value from the server side according to the entity attribute selected by the user in the entity structure tree;

Wherein, the entity parsing engine parses entity attributes and constructs an entity structure tree, which specifically includes: analyzing entity attribute properties according to predefined rules for accessing entities, constructing an entity structure tree according to attribute properties, and constructing an entity structure tree according to the rules. Entity information is saved.

A system for acquiring data from multiple data sources, including: a client and a server, the client is used to initiate a data acquisition request to the server, and the server acquires data from multiple data sources through components and returns them to the client;

Wherein, the component includes an entity parsing engine, which is used to obtain entity files and obtain all attributes of entities therefrom, parse entity attributes, and build entity structure trees, wherein the entity attributes are called data sources; client interface is responsible for communicating with users interact, and display the entity structure tree to the user; the resolver is used to call the entity resolution engine to resolve the entity according to the client request, and obtain the corresponding attribute from the server according to the entity attribute selected by the user in the entity structure tree value;

Wherein, the entity parsing engine parses entity attributes and constructs an entity structure tree, which specifically includes: analyzing entity attribute properties according to predefined rules for accessing entities, constructing an entity structure tree according to attribute properties, and constructing an entity structure tree according to the rules. Entity information is saved.

According to the specific embodiments provided by the invention, the invention discloses the following technical effects:

First, obtain the entity file and obtain all the attributes of the entity, wherein the entity attribute is called a data source, and then parse the entity attribute to construct an entity structure tree and display it to the user. The user can flexibly define the entity attribute according to the requirement (such as the document attribute ) and business parameters (such as budget control parameters); according to the defined correspondence when running the business parameters, the corresponding attribute values are extracted from entities (such as documents) through a "pull" active way to form Business parameters (budget control parameters), thereby realizing the variability of budget control without fixed coding.

Secondly, the amount of code is greatly reduced, and the code maintenance is small.

Thirdly, since the user is provided with an interface for flexibly setting document attributes and budget control parameters, the function of obtaining data from documents in the budget control system is separated from the original business systems, providing a unified platform, and developers do not need to understand related budgetary business knowledge, thus reducing the requirements for developers.

Finally, since a unified data acquisition platform is provided, the code coupling between each business system and the budget control system is greatly reduced, and the reconfiguration of the budget control system is provided.

Description of drawings

FIG. 1 is a flowchart of a method for acquiring data from multiple data sources according to an embodiment of the present invention;

Fig. 2 is a schematic diagram of a configuration interface of the budget control system in an embodiment of the present invention;

Fig. 3 is a schematic diagram of the entity structure tree of the voucher document in the embodiment of the present invention;

Fig. 4 is a structural block diagram of a system for acquiring data from multiple data sources according to an embodiment of the present invention.

Detailed ways

In order to make the above objects, features and advantages of the present invention more comprehensible, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

The embodiment of the present invention provides a method for obtaining data from multiple data sources. Driven by the MDA model, the corresponding relationship between entity attributes (that is, data sources) and business parameters is defined, and the business parameters are dynamically executed according to the defined The corresponding relationship actively extracts the corresponding attribute values from the entity through a "pull" method, composes business parameters and performs subsequent processing, thereby realizing the variability of budget control without fixed coding, and fundamentally changing budget control Variability, greatly reducing the amount of code.

Among them, the above-mentioned MDA (Model Driven Architecture) is a model-driven architecture, which is a software development framework defined by OMG (Object Management Group, Object Management Organization), which separates the specification of system operation from the details of how the system utilizes the capabilities of the underlying platform Out of the idea, MDA provides a way (through related tools) to standardize a platform-independent system, standardize the platform, select a specific implementation platform for the system, and convert the system specification to the specific implementation platform. The three main goals of MDA are: portability, interoperability, and reusability through architectural separation. The method described in the embodiment of the present invention is based on the idea of model-driven, and the following will still use budget control as an example for illustration.

In order to facilitate the understanding of the technical solution of the present invention, the processing method of budget control is firstly introduced. When performing budget control, if the document is to participate in the budget, it is driven by the MDA model. The user needs to select in the definition interface provided by the system which system and which document is subject to budget control, and then select the budget control parameter -- budget Which fields of the document correspond to organization, budget item, currency, time, business application value, etc. The budget control system will extract information from corresponding documents according to the user's configuration, and assemble them into budget control parameters for budget control. In this process, the fundamental difference between the present invention and the prior art lies in the way of setting the corresponding relationship between budget control parameters and document attributes. The prior art implements this through fixed coding, but the present invention implements this configuration function on the client The foreground is implemented, and users can flexibly configure it according to their needs.

The following will describe in detail how to implement the configuration function for users. Referring to FIG. 1 , it is a flow chart of the method for acquiring data from multiple data sources. In the embodiment of the present invention, the application environment of the budget control system includes a client and a server. The client provides a configuration interface for budget control parameters. When a user needs to select a document to participate in budget control, the user enters the configuration interface and initiates a transaction to the server through the client. request, the server will perform the following steps:

Step 101, obtain the entity file according to the request.

In the system driven by the MDA model, the document is the entity data, and the entity data is stored in the form of a file, generally with ".entity" as the extension name, corresponding to a business document entity. The user will first set which document of which system needs to be used for budget control on the configuration interface, and the server side will obtain the corresponding entity file from the corresponding business system according to the user's settings.

The configuration interface can be shown in Figure 2. The business system selected by the user in the interface is "management accounting", and the business document is "travel expense reimbursement form". The corresponding control parameters of these two options in the budget control system are budget organization.

Step 102, obtain all attributes of the entity from the entity file. For example, the document entity contains multiple attribute information, such as account, material, etc., and these attribute information constitute the data source of budget control parameters. According to the needs of budget control, the budget control system needs to extract data from multiple data sources, that is, multiple entity attributes to form control parameters.

Step 103, parse the obtained entity attributes, construct an entity structure tree, and display it on the client.

The entity structure tree is displayed to the user in a visual window according to the structure of the entity in the business system. Referring to Figure 3, it is a schematic diagram of the entity structure tree of a voucher document. The voucher document includes sub-attributes such as "posting", "cashier", and "entry", and each sub-attribute contains several attributes. way to display.

This embodiment provides a method for constructing an entity structure tree according to entity attributes, and predefines general rules for accessing entities. The rules define how to obtain field names and field types in entity attributes, determine whether they have field aliases, and Node and other methods, and then analyze the entity attributes according to the rules, so as to construct the entity structure tree. The specific steps include: first build the root node, then analyze all the attributes obtained by the entity file according to step 102, and analyze the attribute properties. If the current attribute has no children, then add it to the entity object; if the current attribute has children, it means that the child is this An associated attribute of an attribute (the "entry" attribute in Figure 3 is an associated attribute of the "voucher" entity), first add a virtual node in the entity object, and then obtain the child entity according to the association relationship to continue the analysis, so recursively Go down and build an entity structure tree. Wherein, the entity object is used to store parsed entity information (such as field name and other information) according to the above rules. Of course, other methods can also be used to parse entities and build entity structure trees.

Step 104, query the entity structure tree, and obtain entity attribute values from the selected data source.

Referring to the entity structure tree shown in Figure 3, the user can select each attribute of the voucher document in the pop-up "source document attribute selection" window to correspond to the control parameters such as budget item, currency, and time of the budget control system. For example, select the "Entry" attribute to correspond to the budget item, or select the "Cashier + Posting" attribute to correspond to the budget item, so that the user's settings are very flexible. After the user configures the corresponding relationship between document attributes and budget control parameters, the server will extract real data from the entity object according to the corresponding relationship.

Refer to Figure 3 to illustrate the whole process. In the control strategy, select "responsibility center source", the system will pop up the attribute source window, select the document attribute, and the system will pop up the parsed entity structure tree, which is the real structure of the credential entity. You can select the "company" attribute on the credential entity, the key of this attribute in the credential entity is company.id, and this key is stored in the database. When the voucher is approved, the budget control system will go to the voucher to extract the real value of company.id, then use the id to find the budget organization, and then search the budget data in the organization according to certain rules (which budget will be found depends on the control information origin of).

Through the above method, the user can flexibly configure the budget control, and other business system personnel do not need to write codes related to the budget control system, which greatly reduces the workload.

The embodiment of the present invention also provides a system for acquiring data from multiple data sources. As a part of the budget control system, budget data can be extracted from the business system where the data source is located for budget control. Referring to Fig. 4, it is a structural block diagram of the system. The system mainly includes an entity resolution unit 401 , an interaction unit 402 and a data acquisition unit 403 .

The entity parsing unit 401 is used to obtain the entity file, obtain all attributes of the entity from the entity file, and analyze the entity attributes to build an entity structure tree; the interaction unit 402 is responsible for interacting with the user, and is used to provide a configuration interface and display the entity structure tree. The entity structure tree is described above, and the function of querying the entity structure tree is provided, and the user can set the corresponding relationship between entity attributes and budget control parameters; the data acquisition unit 403 is used to acquire entity attribute values from the selected data source. Wherein, when resolving entities, the entity parsing unit 401 analyzes entity attribute properties according to entity access rules defined by the general interface 404, constructs an entity structure tree according to attribute properties, and saves the resolved entity information into entity objects according to the rules. The general interface 404 is used to predefine rules for accessing entities, and provides extended functions, and users can further expand according to the general access rules defined by the interface.

According to the actual application, the present invention also provides another system embodiment. The system includes a client and a server. The client is used to initiate a data acquisition request to the server. Obtain data and return; wherein the server end includes an entity resolution engine for obtaining entity files and obtaining all attributes of the entity therefrom, resolving entity attributes, and building an entity structure tree; the client through the client UI (User Interface, The user interface) displays the entity structure tree; it also includes a parser, which is used to call the entity resolution engine on the server side to resolve entities according to the client request, and obtain corresponding attributes from the server side according to the entity attributes selected by the user in the entity structure tree value.

Preferably, in the system environment of the application component, it can be realized by creating a component, and the server obtains data from multiple data sources through the component and returns it to the client. This component includes an entity resolution engine, which is used to obtain entity files and obtain all attributes of the entity, parse entity attributes, and construct an entity structure tree; the client UI is responsible for interacting with users, and calls the server-side function to convert the entity structure tree Displayed to the user; the resolver is used to call the entity resolution engine to resolve the entity according to the client request, and obtain the corresponding attribute value from the server according to the entity attribute selected by the user in the entity structure tree; the entity object is used to store the resolution Good entity information, such as field names and other information.

It should be noted that, in the present invention, the concept of "component" refers to a related entity applied to a computer, such as hardware, a combination of hardware and software, software or software in execution, and the like. In detail, for example, a component may be, but is not limited to being, a process running on a processor, a processor, an object, an executable component, a thread of execution, a program and/or a computer. Also, the application program or script program running on the server, and the server can both be components. One or more components can be in a process and/or thread of execution, and a component can be localized on one computer and/or distributed between two or more computers and run from various computer-readable media . Components can also be based on a signal having one or more data packets, for example, from a signal that interacts with another component in a local system, a distributed system, and/or interacts with other systems over a network of the Internet via a signal. local and/or remote procedures to communicate.

For the parts not described in detail in the system shown in FIG. 4 , reference may be made to the relevant parts of the method shown in FIG. 1 , which are not described in detail here for the sake of space.

The method and system for obtaining data from multiple data sources provided by the present invention have been introduced in detail above. In this paper, specific examples have been used to illustrate the principle and implementation of the present invention. The descriptions of the above embodiments are only used to help Understand the method of the present invention and its core idea; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation and application scope. In summary, the contents of this specification should not be construed as limiting the present invention.

Claims (7)

1. A method for obtaining data from multiple data sources, comprising: Get the entity file; Get all the properties of the entity from the entity file, where said entity properties are called data sources; Analyzing entity attributes, building an entity structure tree, and displaying it; specifically including: predefining rules for accessing entities; analyzing entity attribute properties according to the rules; constructing entity structure trees according to the attribute properties; and saving the parsed file according to the rules entity information; The entity structure tree is queried to obtain entity attribute values from the selected data source. 2. The method according to claim 1, wherein the entity attribute value is obtained from the selected data source according to the following steps: Select entity attributes to establish a mapping relationship with business parameters; When executing the business parameter, the corresponding entity attribute value is obtained according to the mapping relationship. 3. A system for obtaining data from multiple data sources, characterized in that it includes: Common interface for pre-defining rules for accessing entities; The entity parsing unit is used to obtain the entity file; obtain all the attributes of the entity from the entity file, wherein the entity attribute is called a data source; parse the entity attribute and construct an entity structure tree, specifically including: analyzing the entity according to the rules defined by the general interface Attribute properties, constructing an entity structure tree according to the attribute properties, and saving the parsed entity information according to the rules; An interactive unit, configured to display the entity structure tree and provide a function of querying the entity structure tree; The data acquisition unit is used to acquire entity attribute values from the selected data source. 4. The system according to claim 3, characterized in that the general interface is expandable. 5. The system according to claim 3, characterized in that: when providing the query function, the interaction unit establishes the mapping relationship between the selected entity attribute and the business parameter; when executing the business parameter, the data acquisition unit according to the mapping Relationships get entity property values from the corresponding data sources. 6. A system for obtaining data from multiple data sources, characterized in that it includes: a client and a server, the client is used to initiate a data acquisition request to the server, and the server is used to obtain data from multiple data sources and return ; Wherein the server end includes an entity resolution engine, which is used to obtain entity files and obtain all attributes of entities therefrom, resolve entity attributes, and build entity structure trees, wherein the entity attributes are called data sources; The interface displays the entity structure tree; it also includes a resolver for invoking the entity resolution engine on the server side to resolve the entity according to the client request, and obtains the corresponding attribute value from the server side according to the entity attribute selected by the user in the entity structure tree; Wherein, the entity parsing engine parses entity attributes and constructs an entity structure tree, which specifically includes: analyzing entity attribute properties according to predefined rules for accessing entities, constructing an entity structure tree according to attribute properties, and constructing an entity structure tree according to the rules. Entity information is saved. 7. A system for acquiring data from multiple data sources, characterized in that it includes: a client and a server, the client is used to initiate a data acquisition request to the server, and the server acquires data from multiple data sources through components and returns to the client; Wherein, the component includes an entity parsing engine, which is used to obtain entity files and obtain all attributes of entities therefrom, parse entity attributes, and build entity structure trees, wherein the entity attributes are called data sources; client interface is responsible for communicating with users interact, and display the entity structure tree to the user; the resolver is used to call the entity resolution engine to resolve the entity according to the client request, and obtain the corresponding attribute from the server according to the entity attribute selected by the user in the entity structure tree value; Wherein, the entity parsing engine parses entity attributes and constructs an entity structure tree, which specifically includes: analyzing entity attribute properties according to predefined rules for accessing entities, constructing an entity structure tree according to attribute properties, and constructing an entity structure tree according to the rules. Entity information is saved.

Images