CN112541031A - Unified data service method and system based on automatic data assembly - Google Patents

Unified data service method and system based on automatic data assembly Download PDF

Info

Publication number
CN112541031A
CN112541031A CN202011517534.3A CN202011517534A CN112541031A CN 112541031 A CN112541031 A CN 112541031A CN 202011517534 A CN202011517534 A CN 202011517534A CN 112541031 A CN112541031 A CN 112541031A
Authority
CN
China
Prior art keywords
data
interface
data service
unified
management module
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202011517534.3A
Other languages
Chinese (zh)
Inventor
曾远征
张健
方月
徐志华
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sichuan XW Bank Co Ltd
Original Assignee
Sichuan XW Bank Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sichuan XW Bank Co Ltd filed Critical Sichuan XW Bank Co Ltd
Priority to CN202011517534.3A priority Critical patent/CN112541031A/en
Publication of CN112541031A publication Critical patent/CN112541031A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/25Integrating or interfacing systems involving database management systems
    • G06F16/258Data format conversion from or to a database

Landscapes

  • Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Stored Programmes (AREA)

Abstract

The invention relates to the field of data service, and discloses a unified data service method and a system based on automatic data assembly, which comprises the following steps that S101, a bottom layer data system enters a data standardization module of a unified data service system through an interface and transmits the data standardization module to a data source management module; step S102, a system administrator empowers system application users according to a data security principle; step S103, the system application user sets attributes in the data service interface management module according to the application requirements of the system application user, and the data service interface management module realizes the automatic assembly of the required data; and step S104, completing the assembly of the basic data and transmitting the basic data to a specified unified data service interface after the steps. According to the invention, by building the system, the underlying basic data of an enterprise can be uniformly collected and integrated, the development cost of the data service background is reduced through automatic data assembly, and the data standard and the application standard can be landed when the data service background is assembled.

Description

Unified data service method and system based on automatic data assembly
Technical Field
The invention relates to the field of data service, in particular to a unified data service method and a system based on automatic data assembly.
Background
The traditional data calling mode is that each service system is independently linked with a bottom data system, the bottom data system carries out data processing for customizing the service system, and after the data processing is finished, the service system realizes the data transmission by calling the interface of each data system and puts the data into production.
However, the data call mode has the following three problems:
1. the whole process has low efficiency: each service system needs to be in contact with a bottom data system when developing a project, but partial data requirements of different service systems and different projects can be communicated and integrated, however, due to the relative independence of the systems, sharing of data resources cannot be realized, so that the whole data calling process has long time from development to online and low efficiency;
2. background development costs for data services are high: aiming at each data requirement, a bottom layer data system needs to be developed in a customized manner, so that a large burden and a large pressure are brought to the bottom layer data system, and the development cost is high;
3. data call interface management is difficult; the bottom layer data system provides data support for each business system through the autonomous open interface, the situations of multi-open, false open and the like may exist, and abnormal information generated when the business department calls the interface is difficult to be uniformly summarized to an upstream system, so that the problems such as data safety are caused.
Therefore, a unified data service method and a unified data service system based on automatic data assembly, which can uniformly collect and integrate underlying basic data of an enterprise, reduce background development cost of data services, provide standardized services for business systems of the enterprise, and promote the data services, are urgently needed.
Disclosure of Invention
Based on the above problems, the present invention provides a unified data service method and system based on automated data assembly. According to the invention, by building the system, the underlying basic data of an enterprise can be uniformly collected and integrated, the background development cost of data service is reduced through automatic data assembly, the data standard and the application standard are grounded while the data standard and the application standard are assembled, the standardized service is provided for the business system of the enterprise, and the efficiency, the accuracy and the normalization of the data service are improved.
In order to solve the technical problems, the technical scheme adopted by the invention is as follows:
a unified data service system based on automatic data assembly comprises a bottom layer data system, a data processing system and a data processing system, wherein the bottom layer data system is used for providing bottom layer basic data; the unified data service system comprises a data source management module, a data standardization module and a data service interface management module, wherein the data source management module is used for interacting with a bottom data system, the data standardization module is used for realizing a data standard and an application standard, the data service interface management module is used for realizing the automatic assembly of data required by a system application user, and the data service interface management module is used for finishing the assembly of basic data and transmitting the basic data to a specified unified data service interface; and the downstream business system calls the corresponding unified data service interface to realize the calling of the required data.
Preferably, the data source management module includes a data source list management function, a data source interface adding function, a data source interface debugging function, and a data source interface testing function.
Preferably, the data normalization module performs normalization processing on data transmitted by the underlying data system.
Preferably, the unified data service system further comprises a system management module, wherein the system management module comprises a user management function, a role management function and a permission management function and is responsible for user, role and permission management.
A unified data service method based on automatic data assembly comprises the following steps: step S101, a bottom layer data system enters a data standardization module of a unified data service system through an interface, the data standardization module carries out standardization processing on the input data and transmits the data to a data source management module, and the interface is displayed to be enabled; step S102, a system administrator empowers system application users according to a data security principle, and confirms the use permission of the system application users to data in a data source management module; step S103, the system application user sets the attribute in the data service interface management module according to the application requirement of the system application user, the data service interface management module realizes the automatic assembly of the required data, and displays the 'enabled' of the data service interface; and step S104, after the steps are carried out, the basic data are assembled and transmitted to a specified unified data service interface, and a downstream business system can call the related interface to realize the calling of the required data.
Preferably, in step S101, the underlying data system enters the data standardization module of the unified data service system through an API interface or an interface calling mode preset by a data engineer.
Preferably, in step S101, if the data source transmitted by the underlying data system is a newly added data source or the input attribute of the data source is adjusted, the data source testing function is invoked after the data source is transmitted to the data source management module, and the interface state is changed to the enabled state only if the feedback data and the data dictionary have consistency.
Preferably, in step S103, the attributes set in the data service interface management module by the system application user include one or more of creator, creation group, creation time, last update time, update personnel, dependent interface, required field, and assembly rule.
Preferably, in step S103, if the data service interface is a newly added interface or the assembling attribute of the interface is adjusted, after the new or adjustment is completed, the module automatically invokes the online testing function of the assembled service interface to verify whether the interface is usable, and only if the testing result meets the requirement, the interface state is changed to the enabled state.
The invention has the beneficial effects that: according to the invention, by building the system, the bottom-layer basic data of an enterprise can be uniformly collected and integrated, the bottom-layer basic data can be one or more of a data warehouse, a data mart or an external data source, the background development cost of the data service can be effectively reduced through automatic data assembly in the uniform data service system, the uniform data service system realizes the landing of a data standard and an application standard during assembly, and can provide standardized service for a downstream business system of the enterprise, the downstream business system can be one or more of a credit maintenance system, a financial system and a personnel system according to the self condition of the enterprise, and the efficiency, the accuracy and the standardization of the data service can be improved.
Drawings
A better understanding of the features and advantages of the present invention will be obtained by reference to the following detailed description that sets forth illustrative embodiments, in which the principles of the invention are utilized, and the accompanying drawings of which:
FIG. 1 is a service flow diagram shown in accordance with some embodiments of the present description.
FIG. 2 is a system framework diagram shown in accordance with some embodiments of the present description.
Detailed Description
In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings used in the description of the embodiments will be briefly described below. It is obvious that the drawings in the following description are only examples or embodiments of the present description, and that for a person skilled in the art, the present description can also be applied to other similar scenarios on the basis of these drawings without inventive effort.
A unified data service system based on automatic data assembly comprises a bottom layer data system, a data processing system and a data processing system, wherein the bottom layer data system is used for providing bottom layer basic data; the unified data service system comprises a data source management module, a data standardization module and a data service interface management module, wherein the data source management module is used for interacting with a bottom data system, the data standardization module is used for realizing a data standard and an application standard, the data service interface management module is used for realizing the automatic assembly of data required by a system application user, and the data service interface management module is used for finishing the assembly of basic data and transmitting the basic data to a specified unified data service interface; and the downstream business system calls the corresponding unified data service interface to realize the calling of the required data.
It should be noted that, in some embodiments, the data source management module includes a data source list management function, a data source interface adding function, a data source interface debugging function, and a data source interface testing function. The module is mainly responsible for the interaction between the underlying data system and the system. The interaction between the data source and the system is realized by setting the interface name, the interface state, the interface data source, the interface IP address and the interface calling mode in the module. However, it should be noted that when a data source is newly added, information such as a data source interface name, a status, a source, and a person needs to be transmitted to the module. When configuring information of each field of the data source interface, uploading of a digital dictionary in a standard format can be supported, and field attributes can also be added and configured through a single field, wherein the field attributes comprise field names, field lengths, field types and field code values. After the interface input parameters are input for the first time, the data source testing function is called up, and the interface state is changed into the starting state only if the feedback data and the data dictionary have consistency.
It should also be noted that, in some embodiments, the data service interface management module is mainly responsible for the interaction between the downstream business system and the present system. And the application system user sets a dependent interface, a required field and an assembly rule from the authorized data source interface list according to the requirements of the self service, so as to realize data conversion and automatic assembly of the bottom data. After data assembly parameters are input for the first time, the module can automatically call the online test function of the assembled service interface so as to verify whether the interface is available. The specific functions of the data service interface management module comprise a data service interface calling authority display function, a data service interface list management function, a data service interface newly-added function, a data service interface debugging function and a data service interface testing function.
It is noted that, in some embodiments, the data normalization module normalizes data incoming from the underlying data system. The data standardization module is mainly responsible for realizing data standards and application standards. After the enterprise sets the external data standard and the service interface standard specification, the module carries out standardization processing on the exported data so as to facilitate the use of a subsequent system. The standardization process mainly comprises the scenes of mapping of data value ranges, field naming and the like.
It should be noted that, in some embodiments, the unified data service system further includes a system management module, where the system management module includes a user management function, a role management function, and a permission management function, and is responsible for user, role, and permission management. The module mainly relates to the contents of users, roles, authority management and the like. The system mainly comprises three types of users: system administrators, data engineers, and application system users. The system administrator is mainly responsible for daily management functions such as user creation, user authority configuration and the like; the data engineer is mainly responsible for the functions of configuration of a data source interface, assembly of an external service interface, interface calling relation and calling threshold of an application system, configuration of an interface abnormity alarm threshold and the like; and the application system user performs the restricted calling according to the configured authority.
It should be further noted that the unified data service system may further set one or more of a log management module, a system monitoring module, and a system data storage query module as needed.
And the log management module comprises a system operation log part and a manual operation log part. The system operation log part mainly records the operation and maintenance conditions of the system; the manual operation log records the operation log of the system user and can be inquired according to the management authority of the user.
And the system monitoring module is mainly responsible for monitoring the operation of the relevant modules of the system interface, including the called condition, the searching rate, the success rate, the change condition of the calling amount of each interface and the like of each data source interface and the service interface.
The system data storage query module is mainly used for storing the calling relation of data and the data value after data standard, asynchronously and structurally storing the index database data and asynchronously and structurally storing the calling log.
A unified data service method based on automatic data assembly comprises the following steps: step S101, a bottom layer data system enters a data standardization module of a unified data service system through an interface, the data standardization module carries out standardization processing on the input data and transmits the data to a data source management module, and the interface is displayed to be enabled; step S102, a system administrator empowers system application users according to a data security principle, and confirms the use permission of the system application users to data in a data source management module; step S103, the system application user sets the attribute in the data service interface management module according to the application requirement of the system application user, the data service interface management module realizes the automatic assembly of the required data, and displays the 'enabled' of the data service interface; and step S104, after the steps are carried out, the basic data are assembled and transmitted to a specified unified data service interface, and a downstream business system can call the related interface to realize the calling of the required data.
It should be noted that, in some embodiments, in step S101, the underlying data system enters the data standardization module of the unified data service system through an API interface or an interface calling mode preset by a data engineer. The module carries out standardization processing such as value domain mapping, field naming and the like on the incoming data, and transmits the data to the data source management module to display that the interface is enabled. If the data source is a newly added data source or the input attribute of the data source is adjusted, the data source testing function is called after the data source is transmitted to the data source management module, and the interface state is changed into the starting state only if the feedback data is consistent with the data dictionary.
It should be noted that, in some embodiments, in step S101, if the data source transmitted by the underlying data system is a new data source or the input attribute of the data source is adjusted, the data source testing function is invoked after the data source is transmitted to the data source management module, and the interface state is changed to the enabled state only if the feedback data and the data dictionary have consistency.
It should be noted that, in some embodiments, in step S103, the attributes set in the data service interface management module by the system application user include one or more of creator, creation group, creation time, last update time, update person, dependent interface, required field, and assembly rule.
It should be noted that, in some embodiments, in step S103, if the data service interface is a new interface or the interface assembly attribute is adjusted, after the new or adjustment is completed, the module automatically invokes the assembled service interface online test function to verify whether the interface is available, and only after the test result meets the requirement, the interface state is changed to the enabled state.
Having thus described the basic concept, it will be apparent to those skilled in the art that the foregoing detailed disclosure is to be regarded as illustrative only and not as limiting the present specification. Various modifications, improvements and adaptations to the present description may occur to those skilled in the art, although not explicitly described herein. Such modifications, improvements and adaptations are proposed in the present specification and thus fall within the spirit and scope of the exemplary embodiments of the present specification.
Also, the description uses specific words to describe embodiments of the description. Reference throughout this specification to "one embodiment," "an embodiment," and/or "some embodiments" means that a particular feature, structure, or characteristic described in connection with at least one embodiment of the specification is included. Therefore, it is emphasized and should be appreciated that two or more references to "an embodiment" or "one embodiment" or "an alternative embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, some features, structures, or characteristics of one or more embodiments of the specification may be combined as appropriate.
Additionally, the order in which the elements and sequences of the process are recited in the specification, the use of alphanumeric characters, or other designations, is not intended to limit the order in which the processes and methods of the specification occur, unless otherwise specified in the claims. While various presently contemplated embodiments of the invention have been discussed in the foregoing disclosure by way of example, it is to be understood that such detail is solely for that purpose and that the appended claims are not limited to the disclosed embodiments, but, on the contrary, are intended to cover all modifications and equivalent arrangements that are within the spirit and scope of the embodiments herein. For example, although the system components described above may be implemented by hardware devices, they may also be implemented by software-only solutions, such as installing the described system on an existing server or mobile device.
Similarly, it should be noted that in the preceding description of embodiments of the present specification, various features are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure aiding in the understanding of one or more of the embodiments. This method of disclosure, however, is not intended to imply that more features than are expressly recited in a claim. Indeed, the embodiments may be characterized as having less than all of the features of a single embodiment disclosed above.
Finally, it should be understood that the embodiments described herein are merely illustrative of the principles of the embodiments of the present disclosure. Other variations are also possible within the scope of the present description. Thus, by way of example, and not limitation, alternative configurations of the embodiments of the specification can be considered consistent with the teachings of the specification. Accordingly, the embodiments of the present description are not limited to only those embodiments explicitly described and depicted herein.

Claims (9)

1. A unified data service system based on automated data assembly is characterized by comprising,
the bottom data system is used for providing bottom basic data;
the unified data service system comprises a data source management module, a data standardization module and a data service interface management module, wherein the data source management module is used for interacting with a bottom data system, the data standardization module is used for realizing a data standard and an application standard, the data service interface management module is used for realizing the automatic assembly of data required by a system application user, and the data service interface management module is used for finishing the assembly of basic data and transmitting the basic data to a specified unified data service interface;
and the downstream business system calls the corresponding unified data service interface to realize the calling of the required data.
2. The unified data service system based on automated data assembly according to claim 1, wherein the data source management module comprises a data source list management function, a data source interface adding function, a data source interface debugging function, and a data source interface testing function.
3. A unified data service system based on automated data assembly according to claim 1, characterized by: the data standardization module is used for standardizing data transmitted by the bottom layer data system.
4. A unified data service system based on automated data assembly according to claim 1, characterized by: the unified data service system also comprises a system management module, wherein the system management module comprises a user management function, a role management function and an authority management function and is responsible for user, role and authority management.
5. A unified data service method based on automatic data assembly is characterized by comprising the following steps:
step S101, a bottom layer data system enters a data standardization module of a unified data service system through an interface, the data standardization module carries out standardization processing on the input data and transmits the data to a data source management module, and the interface is displayed to be enabled;
step S102, a system administrator empowers system application users according to a data security principle, and confirms the use permission of the system application users to data in a data source management module;
step S103, the system application user sets the attribute in the data service interface management module according to the application requirement of the system application user, the data service interface management module realizes the automatic assembly of the required data, and displays the 'enabled' of the data service interface;
and step S104, after the steps are carried out, the basic data are assembled and transmitted to a specified unified data service interface, and a downstream business system can call the related interface to realize the calling of the required data.
6. A unified data service method based on automated data assembly according to claim 5, characterized in that: in step S101, the underlying data system enters a data standardization module of the unified data service system through an API interface or an interface calling mode preset by a data engineer.
7. A unified data service method based on automated data assembly according to claim 5, characterized in that: in step S101, if the data source transmitted by the underlying data system is a newly added data source or the input attribute of the data source is adjusted, the data source testing function is invoked after the data source is transmitted to the data source management module, and the interface state is changed to the enabled state only if the feedback data and the data dictionary have consistency.
8. A unified data service method based on automated data assembly according to claim 5, characterized in that: in step S103, the attributes set in the data service interface management module by the system application user include one or more of creator, creation group, creation time, last update time, update person, dependent interface, required field, and assembly rule.
9. A unified data service method based on automated data assembly according to claim 5, characterized in that: in step S103, if the data service interface is a new interface or the assembling attribute of the interface is adjusted, after the new or adjustment is completed, the module will automatically invoke the online testing function of the assembled service interface to verify whether the interface is usable, and only if the testing result meets the requirement, the interface status will be changed to the enabled status.
CN202011517534.3A 2020-12-21 2020-12-21 Unified data service method and system based on automatic data assembly Pending CN112541031A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011517534.3A CN112541031A (en) 2020-12-21 2020-12-21 Unified data service method and system based on automatic data assembly

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011517534.3A CN112541031A (en) 2020-12-21 2020-12-21 Unified data service method and system based on automatic data assembly

Publications (1)

Publication Number Publication Date
CN112541031A true CN112541031A (en) 2021-03-23

Family

ID=75019255

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011517534.3A Pending CN112541031A (en) 2020-12-21 2020-12-21 Unified data service method and system based on automatic data assembly

Country Status (1)

Country Link
CN (1) CN112541031A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113220343A (en) * 2021-04-14 2021-08-06 深圳新贝奥数字科技有限公司 Enterprise cross-system data integration processing system

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110765337A (en) * 2019-11-15 2020-02-07 中科院计算技术研究所大数据研究院 Service providing method based on internet big data
CN111027284A (en) * 2019-12-05 2020-04-17 集奥聚合(北京)人工智能科技有限公司 Standardized output method based on flexible data access
CN111143444A (en) * 2018-11-06 2020-05-12 上海顶澄科技有限公司 API-based data exchange platform
CN111880950A (en) * 2020-07-30 2020-11-03 浪潮云信息技术股份公司 Cloud service API management method and system based on government affair cloud platform

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111143444A (en) * 2018-11-06 2020-05-12 上海顶澄科技有限公司 API-based data exchange platform
CN110765337A (en) * 2019-11-15 2020-02-07 中科院计算技术研究所大数据研究院 Service providing method based on internet big data
CN111027284A (en) * 2019-12-05 2020-04-17 集奥聚合(北京)人工智能科技有限公司 Standardized output method based on flexible data access
CN111880950A (en) * 2020-07-30 2020-11-03 浪潮云信息技术股份公司 Cloud service API management method and system based on government affair cloud platform

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113220343A (en) * 2021-04-14 2021-08-06 深圳新贝奥数字科技有限公司 Enterprise cross-system data integration processing system

Similar Documents

Publication Publication Date Title
US10754932B2 (en) Centralized consent management
EP3617884B1 (en) Adapter extension for inbound messages from robotic automation platforms to unified automation platform
US20190251759A1 (en) Vehicle data aggregation and analysis platform providing dealership service provider dashboard
JP5503875B2 (en) Process control system and process control method
US9767441B2 (en) System for monitoring elevators and maintaining elevators
US20040128583A1 (en) Method and system for monitoring, diagnosing, and correcting system problems
US10007491B2 (en) Systems and/or methods for cloud-based event-driven integration
US20030212654A1 (en) Data integration system and method for presenting 360° customer views
US20210133622A1 (en) Ml-based event handling
US20100332509A1 (en) Systems and methods for integrated management of governmental projects
US20070208574A1 (en) System and method for managing master data information in an enterprise system
US9805083B1 (en) Method and computer program product for allowing a software application to interact with a product
CN102946415A (en) Implementation method and device for local application of mobile terminal
CN112541031A (en) Unified data service method and system based on automatic data assembly
CN111045652A (en) Power distribution network development and service system
US20180121441A1 (en) Accessing application services from forms
CN111338717B (en) Interface calling method, application program upgrading method, server and client
CN106656592A (en) Service management method and device based on role configuration
US20070271229A1 (en) System and method for data searching among multiple enterprise applications
CN111045928A (en) Interface data testing method, device, terminal and storage medium
CN112988705B (en) Data middlebox construction method for enterprise-level production
CN105335145A (en) Operation result processing method, device and system
CN110569026A (en) Service-oriented system architecture method based on J2EE framework
CN110825354A (en) Intellectual property agency service management system based on SOA system
US20190392361A1 (en) Method and device for generating data services from heterogeneous systems

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20210323

RJ01 Rejection of invention patent application after publication