CN106126351B - A kind of data resource sharing management system based on message queue - Google Patents
A kind of data resource sharing management system based on message queue Download PDFInfo
- Publication number
- CN106126351B CN106126351B CN201610431536.8A CN201610431536A CN106126351B CN 106126351 B CN106126351 B CN 106126351B CN 201610431536 A CN201610431536 A CN 201610431536A CN 106126351 B CN106126351 B CN 106126351B
- Authority
- CN
- China
- Prior art keywords
- data
- platform
- resource
- user
- downloading
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000005540 biological transmission Effects 0.000 claims abstract description 14
- 238000012544 monitoring process Methods 0.000 claims abstract description 14
- 238000003032 molecular docking Methods 0.000 claims abstract description 8
- 238000000605 extraction Methods 0.000 claims abstract description 6
- 238000010606 normalization Methods 0.000 claims description 6
- 238000007726 management method Methods 0.000 abstract description 53
- 238000005516 engineering process Methods 0.000 abstract description 4
- 238000000034 method Methods 0.000 abstract description 4
- 230000008569 process Effects 0.000 abstract description 3
- 238000013523 data management Methods 0.000 abstract description 2
- 238000013500 data storage Methods 0.000 abstract 1
- 230000006870 function Effects 0.000 description 10
- 230000008859 change Effects 0.000 description 3
- 238000010276 construction Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000012800 visualization Methods 0.000 description 3
- 230000005856 abnormality Effects 0.000 description 1
- 238000013475 authorization Methods 0.000 description 1
- 230000004888 barrier function Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 239000006185 dispersion Substances 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/546—Message passing systems or structures, e.g. queues
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The present invention discloses a kind of data resource sharing management system based on message queue, it is related to computer data management technology, utilize active MQ and FTP technology, it is proposed that a data sharing manages platform, user passes through the data source and data storage position that the platform configuration to be downloaded well, and data can be realized and be automatically downloaded to designated position, while platform carries out complete monitoring to this process, by exception handling, guarantee system operates normally with security and stability.The Data Resources Management System provides the push download function to contents such as file, pictures, there is very big promotion in safe and stable, efficiency；The docking for supporting common IDI extraction tool supports the data of multiple data sources to transmit, and realizes that the transmission of different data resource is shared, user is facilitated to improve the utilization rate of data information to the use of information resources.
Description
Technical field
The present invention relates to computer data management technology, specifically a kind of data resource sharing based on message queue
Management system.
Background technique
With the development of national information engineering construction, government and each functional department all establish respective internal applications
And information resources, and these application systems be by different manufacturers on different platforms, developed using different language
, it is difficult to realize information sharing each other, results in and forms a large amount of information island in e-government development.As it can be seen that
Due to the limitation of initial stage system Construction between business enterprice sector, realize data it is shared be it is relatively difficult, therefore, realize different
The demand of data sharing between application system is very urgent.How this barrier is got through, by several application systems of dispersion construction
It is integrated, the data transmission downloading between realization department just becomes important.
Summary of the invention
The present invention is directed to the demand and shortcoming of current technology development, provides a kind of data resource based on message queue
Sharing Management system.
A kind of data resource sharing management system based on message queue of the present invention solves above-mentioned technical problem and uses
Technical solution it is as follows: a kind of data resource sharing management system based on message queue, propose a data sharing management
Platform, system architecture mainly include data normalization layer, network layer, MQ engine, monitoring center, interface layer and application layer；
Wherein, the data normalization layer is the data supporting layer of data sharing management platform, the branch including data sharing management platform
Support library and shared data resource；Reliable data transmission is realized by the network layer；The MQ engine is data sharing pipe
The core of platform mainly includes MQ message queue, the reliable transmission and automation of data is realized by it；In the monitoring
The heart is mainly responsible for the transmission of monitoring data and the operating condition of data sharing management platform, can grasp system in real time by it
Operating condition；Interface layer docking third party's tool, application or service, provide the channel of data downloading client；It is described to answer
Visualization interface is provided a user with layer, user is facilitated to customize or use data resource；
Preferably, the interface layer docking IDI extraction tool and information resources service platform.
Preferably, the data sharing management platform mainly includes FMQ management module, data resource management module, user
Authority management module, operation monitoring module and system management module；Wherein, the FMQ management module is responsible for MQ configuration, FTP
Configuration, data downloading and incremental data downloading；The resource management module is mainly responsible for configuration data source information and data
Resource information；The user authority management module is mainly responsible for the control that data permission access is realized to the user for logging in platform；
The operation monitoring module is responsible for the case where using data to user in system, platform operating condition, system exception carry out in real time
Monitoring；The system management module, which is mainly responsible for, accesses menu and function progress permission control to platform user.
Preferably, the MQ configuration mainly configures the address of MQ；FTP is configured to complete the address FTP and user configuration；
The full dose downloading of data resource is realized in data downloading, and the full dose that the period by formulating resource and downloading carries out data is downloaded；
Incremental data downloading mainly passes through the increment downloading of the realization data of configuring timing tasks timing.
What a kind of data resource sharing management system based on message queue of the present invention had compared with prior art
Beneficial effect is: the Data Resources Management System is provided to the push download functions of the contents such as file, picture, it is safe and stable,
There is very big promotion in efficiency；The docking for supporting common IDI extraction tool supports the data of multiple data sources to transmit, to entirety
Message and resource supplying process provide monitoring, realize the asynchronous of data resource, distributed treatment and active push function；User
Only need to manage the path of the customized resource to be downloaded of platform and downloading storage by data sharing, system automatically passes through data
Full dose or the mode of increment are pushed to user, realize that the transmission of different data resource is shared, facilitate user to information resources
It uses；Different from traditional message push, this platform supports the resource transmission of the accessory class such as text, picture, so that user obtains
The mode and content of data are more abundant；
The present invention is data safety efficiently using service platform is provided, so that data between enterprise or government's various pieces
Information sharing reduces information redundancy；Unified inter-sectional data consistency；By monitoring in real time and counting, provided for decision-making level
Intuitive information is shown, and then provides foundation for upper layer work arrangements and strategy.
Figure of description
Attached drawing 1 is the system architecture schematic diagram of the data resource sharing management system；
Attached drawing 2 is the operation schematic diagram of the data resource sharing management system.
Specific embodiment
To make the objectives, technical solutions, and advantages of the present invention clearer, below in conjunction with specific embodiment, to this hair
A kind of bright data resource sharing management system further description based on message queue.
For the relatively difficult problem of data sharing between business enterprice sector, the invention discloses a kind of based on message queue
Data resource sharing management system improves traditional MQ message push service, relies on the skills such as big data, FTP, MQ
Art proposes that a data sharing manages platform, and by platform configuration management function, user can easily customize the number of oneself needs
According to resource；System provides the push download function to contents such as file, pictures, and data are passed through full dose or the side of increment automatically
Formula is pushed to user；Also, the docking for supporting common IDI extraction tool, supports the data of multiple data sources to transmit, to entirety
Message and resource supplying process provide monitoring, realize the asynchronous of data resource, distributed treatment and active push function, finally
Realize that transmitting data resources are shared.
Embodiment:
A kind of data resource sharing management system based on message queue described in the present embodiment, user are total by logon data
Enjoy the path of the management platform resource to be downloaded of setting and downloading storage, can pending file active push, and with message
Form notifies user；Realize the shared of different data sources；As shown in Fig. 1, system architecture mainly includes data standard
Change layer, network layer, MQ engine, monitoring center, interface layer and the several aspects of application layer；Wherein, the data normalization layer is
Data sharing manages the data supporting layer of platform, support library and shared data resource including data sharing management platform,
Unified data resource is provided for different user；Reliable data transmission is realized by the network layer；The MQ engine is data
The core of Sharing Management platform mainly includes MQ message queue, the activeMQ open source frame for the apache that MQ message queue uses
Frame realizes message push function, and the reliable transmission and automation of data are realized by it；The monitoring center is mainly responsible for prison
The transmission of data and the operating condition of data sharing management platform are controlled, running situation can be grasped by it in real time；Institute
Interface layer docking third party's tool, application or service are stated, the channel of data downloading client is provided；The application layer is mentioned to user
For visualization interface, user is facilitated to customize or use data resource.
In data resource sharing management system described in the present embodiment, the interface layer docking IDI extraction tool and information
Resource service platform.
In data resource sharing management system described in the present embodiment, the data normalization layer is data sharing management platform
Data supporting layer, unified data resource can be provided by this layer for each department of enterprise, guarantee the consistent of data
Property, the redundancy of data is reduced, the quality of data is improved；Reliable data transmission is ensured by network layer；The MQ engine is that user is total
The tie of data is enjoyed, mainly includes MQ message queue, the active MQ Open Framework for the apache that MQ message queue uses is real
Existing message push function；Third party's tool, application or service are docked by interface layer, while interface layer is also data downloading client
The channel at end；And running situation is grasped by monitoring center in real time, it is handled to note abnormalities in time；Pass through application
The visualization interface of layer facilitates user to customize and use data resource.
Attached drawing 2 is the operation schematic diagram of the data resource sharing management system, and as shown in Fig. 2, user passes through described
Data sharing manages the customized required resource of platform, and after system starts downloading task, data sharing management platform passes through data standard
Change layer and search qualified data resource, and data resource arrange being packaged and caches in a manner of file；Then,
The ID of data resource will be stored on file push to ftp server, in message queue by MQ engine；Ftp server is main
It is the terminal of data downloading.Client is recipient's (machine for the reception file that user specifies) of data resource, works as message
When data resource needed for having user in queue, client then passes through the ID of data resource in message queue from ftp server
Downloading data resource realizes the automatic downloading of data to locally.
The data sharing that the data resource sharing management system proposes manages platform, as user's subscription data resource,
Administrator is that user distributes permission, and the portal of configuration data resource, structure mainly includes FMQ management module, data resource pipe
Manage module, user authority management module, operation monitoring module and system management module；
Wherein, the FMQ management module is responsible for MQ configuration, FTP configuration, data downloading and incremental data downloading；MQ matches
Set the address for mainly configuring MQ；FTP is configured to complete the address FTP and user configuration；Data resource is realized in data downloading
The full dose downloading of data can be realized by formulating the period of resource and downloading in full dose downloading；Incremental data downloading is main logical
Cross the increment downloading of the realization data of configuring timing tasks timing；
The resource management module is mainly responsible for configuration data source information and data asset information, and data source information is main
The contents such as address, type, version including database, data asset information are mainly to create to provide the data resource letter of downloading
Breath；The user authority management module is mainly responsible for the control that data permission access is realized to the user for logging in platform, guarantees number
According to safe handling, family rights management include user management, access authorization for resource management；The operation monitoring module is responsible for in system
The case where user is using data, platform operating condition, system exception monitor in real time, and data use can be provided for user and is become
The content of policy decision such as gesture, and can find the problem of system is run in time to handle in time；The system management module is mainly responsible for
Menu and function are accessed to platform user and carry out permission control, and realizes the management function in terms of other platforms.
It is flat to need to build data sharing management in the specific implementation for data resource sharing management system described in the present embodiment
The parts such as platform, MQ engine, FIP server, client, support library.Resource-sharing manages the separation of platform front and back, front and back resource
On Sharing Management Platform deployment to different server, it is ok using windows, linux operating system；Used in background server
Machine performance is determined with specific reference to user's usage amount, generally uses the server of high requirement.Supporting library is data sharing management
The basis of platform operation is deployed in the general server of performance using MySql database, and ensures data sharing pipe
Platform has the access authority to the server.MQ engine and FIP can dispose cluster, MQ server requirement high concurrent and
High-throughput, ftp server need biggish hard drive space on this basis.The file of client primary recipient downloading, service
Device has certain hard drive space to can satisfy storage file.
Above-mentioned specific embodiment is only specific case of the invention, and scope of patent protection of the invention includes but is not limited to
Above-mentioned specific embodiment, any person of an ordinary skill in the technical field that meet claims of the present invention and any
The appropriate change or replacement done to it, all shall fall within the protection scope of the present invention.
Claims (2)
1. a kind of data resource sharing management system based on message queue, which is characterized in that propose that a data sharing management is flat
Platform, system architecture include data normalization layer, network layer, MQ engine, monitoring center, interface layer and application layer；Wherein, institute
State data normalization layer be data sharing management platform data supporting layer, including data sharing management platform support library and
Shared data resource；Reliable data transmission is realized by the network layer；The MQ engine is data sharing management platform
Core, including MQ message queue realize the reliable transmission and automation of data by it；The monitoring center is responsible for monitoring number
According to transmission and data sharing management platform operating condition, running situation can be grasped by it in real time；It is described to connect
Mouth layer docking third party's tool, application or service, provide the channel of data downloading client；The application layer provides a user can
Depending on changing interface, user is facilitated to customize or use data resource；
The data sharing management platform includes FMQ management module, data resource management module, user authority management module, fortune
Row monitoring module and system management module；Wherein, the FMQ management module be responsible for MQ configuration, FTP configuration, data downloading with
And incremental data downloading；The resource management module is mainly responsible for configuration data source information and data asset information；The use
Family authority management module is mainly responsible for the control that data permission access is realized to the user for logging in platform；The operation monitoring module
The case where being responsible for user in system using data, platform operating condition, system exception monitor in real time；The system administration
Module, which is mainly responsible for, accesses menu and function progress permission control to platform user；
The address of configuration MQ is responsible in the MQ configuration；FTP is configured to complete the address FTP and user configuration；Data downloading is realized
The full dose of data resource is downloaded, and the full dose that the period by formulating resource and downloading carries out data is downloaded；Incremental data downloading
The increment downloading of the main realization data by configuring timing tasks timing.
2. a kind of data resource sharing management system based on message queue according to claim 1, which is characterized in that described
Interface layer docks IDI extraction tool and information resources service platform.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610431536.8A CN106126351B (en) | 2016-06-17 | 2016-06-17 | A kind of data resource sharing management system based on message queue |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610431536.8A CN106126351B (en) | 2016-06-17 | 2016-06-17 | A kind of data resource sharing management system based on message queue |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106126351A CN106126351A (en) | 2016-11-16 |
| CN106126351B true CN106126351B (en) | 2019-06-04 |
Family
ID=57469951
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610431536.8A Active CN106126351B (en) | 2016-06-17 | 2016-06-17 | A kind of data resource sharing management system based on message queue |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106126351B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108984725A (en) * | 2018-07-11 | 2018-12-11 | 浪潮软件股份有限公司 | Cross-gatekeeper data synchronization method |
| CN109981354A (en) * | 2019-03-07 | 2019-07-05 | 浪潮软件集团有限公司 | A kind of intelligence based on business model is deployed to ensure effective monitoring and control of illegal activities method for early warning |
| CN111416763A (en) * | 2020-02-21 | 2020-07-14 | 北京值得买科技股份有限公司 | Message pushing method |
| CN111556019B (en) * | 2020-03-27 | 2022-06-14 | 天津市普迅电力信息技术有限公司 | Vehicle-mounted machine data encryption transmission and processing method under distributed environment |
| CN114710311B (en) * | 2022-02-11 | 2024-01-02 | 浙江高信技术股份有限公司 | Multi-project message management method and system |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103024014A (en) * | 2012-12-03 | 2013-04-03 | 北京京东世纪贸易有限公司 | Method and system for distributing and processing mass data passing through message queues |
| CN103136249A (en) * | 2011-11-30 | 2013-06-05 | 北京航天长峰科技工业集团有限公司 | System and method of multiplex mode isomerous data integration |
| CN103139251A (en) * | 2011-11-29 | 2013-06-05 | 神州数码信息系统有限公司 | Method of city-level data sharing exchange platform technology |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7613830B2 (en) * | 2004-12-10 | 2009-11-03 | Microsoft Corporation | Reliably transferring queued application messages |
-
2016
- 2016-06-17 CN CN201610431536.8A patent/CN106126351B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103139251A (en) * | 2011-11-29 | 2013-06-05 | 神州数码信息系统有限公司 | Method of city-level data sharing exchange platform technology |
| CN103136249A (en) * | 2011-11-30 | 2013-06-05 | 北京航天长峰科技工业集团有限公司 | System and method of multiplex mode isomerous data integration |
| CN103024014A (en) * | 2012-12-03 | 2013-04-03 | 北京京东世纪贸易有限公司 | Method and system for distributing and processing mass data passing through message queues |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106126351A (en) | 2016-11-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106126351B (en) | A kind of data resource sharing management system based on message queue | |
| US11501057B2 (en) | Enabling file attachments in calendar events | |
| CN105531694B (en) | Show that multiple authors' is on the scene in electrical form | |
| US8606897B2 (en) | Systems and methods for exporting usage history data as input to a management platform of a target cloud-based network | |
| US8769127B2 (en) | Cross-domain solution (CDS) collaborate-access-browse (CAB) and assured file transfer (AFT) | |
| CN109413043B (en) | Method and device for realizing dynamic configuration of database, electronic equipment and storage medium | |
| US10263929B2 (en) | Cloud based chat governance system based on behavioral patterns and situational-awareness | |
| WO2016048414A1 (en) | Managing change events for devices in an enterprise system | |
| US20100313249A1 (en) | Methods and apparatus for distributing, storing, and replaying directives within a network | |
| CA3120996C (en) | Synchronization of data between local and remote computing environment buffers | |
| US10135763B2 (en) | System and method for secure and efficient communication within an organization | |
| US11599353B2 (en) | Hosting event-based applications | |
| US20100311393A1 (en) | Methods and apparatus for distributing, storing, and replaying directives within a network | |
| US20100312813A1 (en) | Methods and apparatus for distributing, storing, and replaying directives within a network | |
| WO2022133811A1 (en) | Protecting sensitive data using conversational history | |
| US10715494B2 (en) | Orchestrating work across domains and work streams | |
| US11768692B2 (en) | Systems and methods for automated application launching | |
| US10127293B2 (en) | Collaborative data intelligence between data warehouse models and big data stores | |
| US20220417200A1 (en) | Contextual discovery and design of application workflow | |
| EP2785019A1 (en) | Managing mobile telecommunication devices with a general purpose messaging transport protocol in digital cellular telecommunication networks | |
| CN107077490B (en) | Data query job submission management | |
| US20170093782A1 (en) | Electronic-messaging system interceptor forwarding client notifications | |
| Chang et al. | The design of an XMPP-based service integration scheme | |
| US11659021B2 (en) | Asynchronous distributed modular function calling | |
| US20240143352A1 (en) | Unified Management Interface |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |