CN106850788B - Integrated framework and integrated approach towards multi-source heterogeneous geographic information resources - Google Patents
Integrated framework and integrated approach towards multi-source heterogeneous geographic information resources Download PDFInfo
- Publication number
- CN106850788B CN106850788B CN201710045358.XA CN201710045358A CN106850788B CN 106850788 B CN106850788 B CN 106850788B CN 201710045358 A CN201710045358 A CN 201710045358A CN 106850788 B CN106850788 B CN 106850788B
- Authority
- CN
- China
- Prior art keywords
- data
- service
- module
- message
- geographic information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/29—Geographical information databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/445—Program loading or initiating
- G06F9/44505—Configuring for program initiating, e.g. using registry, configuration files
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/2866—Architectures; Arrangements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/2866—Architectures; Arrangements
- H04L67/30—Profiles
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/52—Network services specially adapted for the location of the user terminal
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/562—Brokering proxy services
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/60—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Software Systems (AREA)
- Remote Sensing (AREA)
- Information Transfer Between Computers (AREA)
Abstract
Integrated framework and integrated approach towards multi-source heterogeneous geographic information resources, including Services Integration module, message managing module, structural data synchronization module, unstructured data transmission module and the Lightweight J 2 EE Framework for integrating four modules.The present invention is mainly directed towards the multi-source heterogeneous geographical spatial data and geographic information services of magnanimity, provides a kind of modular Distributed Information Resource technology of sharing solution.The factors such as obtaining means, distributing position, architectural characteristic based on resource in existing area of geographic information, corresponding scheme module is designed and Implemented from cross-domain data entity transmission, distributed geographic information synchronization, remote service calling, multidimensional platform message management etc., and then realize the geography information heterogeneous resource integrated framework of complete set, solve calling between different platform of service, message, structural data and unstructured data, transmission with it is synchronous the problems such as, networked resource utilization efficiency is improved, resource-sharing cost is reduced.
Description
Technical field
The present invention relates to technical field of geographic information, especially a kind of integrated frame towards multi-source heterogeneous geographic information resources
Frame and integrated approach.
Background technique
In recent years, with production and living advanced IT application, geographic information services are in public transport, social platform, shopping joy
The fields such as happy and smart city, government decision are increasingly becoming the basic module that all kinds of services are relied on.Every profession and trade, department's accumulation
A large amount of geography information related resource and be served by, but due to lacking effective resource integrated method, result in for
Number numerous " resource isolation ", results in waste of resources and unnecessary repeated construction.
To improve geographic information resources utilization efficiency, production management cost is reduced, is mentioned by resource consolidation scheme for user
Become new trend for a kind of shared service mode towards multidimensional data, the resource sharing platform for having high integration ability is answered
It transports and gives birth to.However due to the continuous expansion of geographical spatial data scale, the increasingly sophisticated while enterprise of geographic information services framework
Because the difference of business demand cause platform language version it is different situations such as, brought to the process of resource-sharing between different platform all
More problems.
The resource sharing service of building interconnection needs integrated remote method call, data transmission, data synchronization, message
A variety of key technologies such as distribution, establish complete framework technology system.Previous framework mostly uses greatly service-oriented
Architectural framework (service oriented architecture, SOA) thought, specific implementation have Web Service,
The modes such as CORBA, DCOM and RMI.But with continuous expansion, the continuous promotion of performance indicator of service scale, above-mentioned realization side
Formula more or less be there are problems that, be made a concrete analysis of as follows:
The performance of 1.Web Service, the defects of having a single function, gradually appear.Either opened in network bandwidth or CPU
Pin aspect all can cause serious performance issue to application, so that the technology can not be suitable for many to high-performance and mutual
Operation requires extremely frequent system.
2.CORBA and DCOM because its numerous and jumbled technical standard and platform compatibility aspect the shortcomings that due to be difficult to push away extensively
Extensively.
3.RMI is the far call of Java language, and the program language at both ends must be that Java is realized, can not support different languages
The intercommunication of speech.
In terms of data integration, the mode (XML) that previous integrated framework attempts to convert data resource to standard is carried out
The problem of accessing, but being still faced with inefficiency.
In data synchronization aspects, the data synchronization solutions of mainstream have database manufacturer provide data synchronization technology and
Data synchronization technology based on capture change technique customized exploitation.However, there is also following problems:
1. the synchronization that the simultaneous techniques that manufacturer provides often does not support heterogeneous database, it is difficult to meet the requirements.
2. many scholars are proposed the database synchronization technology based on capture technique, as Shen Min utilizes trigger and storage
Process combines and obtains delta data from source database subsystem, Liu Yongyi propose an improved main text updating method and when
Between stab the strategy combined.But the ununified technical standard of these methods, there are various restrictive conditions, are not suitable for large-scale system
System.
In addition, previous framework focuses mainly on the integrated of Service Source and database resource, for message
Resource and being transmitted without for unstructured data resource provide rigorous, available implementation.
In conclusion service call, message communicating, data synchronization and data transmission between isomerous environment, at present
Geographic information resources shared platform there is no a set of perfect solutions and frame system.
Summary of the invention
In view of the above-mentioned deficiencies of the prior art, the technical problem to be solved by the present invention is to provide one kind towards multi-source heterogeneous
The integrated framework of geographic information resources should be directed to current information towards the integrated framework of multi-source heterogeneous geographic information resources and integrate skill
Requirement of the deficiency and resource consolidation existing for art to exploitation complexity and performance etc., provides a kind of modular distribution
Information resources share technical solution, it is same in service call, message communicating, information for solving all kinds of geographic information resources
The problem during step, data transmission etc., improves networked resource utilization efficiency, reduces resource-sharing cost.
In order to solve the above technical problems, the technical solution adopted by the present invention is that:
Integrated framework towards multi-source heterogeneous geographic information resources is integrated including four modules and by four modules
Lightweight J 2 EE Framework.
Four modules are respectively Services Integration module, message managing module, structural data synchronization module and unstructured
Data transmission module.
Services Integration module carries out the exploitation and deployment of distributed application program using ICE middleware Technology；Services Integration
Module includes the interface UI, registration center and service deployment tool；Wherein, registration center is by being deployed in the IceNode of server-side
Obtain host and information on services；The interface UI can start service for showing information acquired in registration center
Or it stops operation；The new IceNode node serve of service deployment tool energy rapid deployment, and to original IceNode node serve
Carry out upgrading or rolling back action.
Message managing module is a series of message queue cluster being made of message queues, and GIS-Geographic Information System passes through message
Queue cluster carries out message communicating, so that the message-driven between each business module provides infrastructural support；In message queue
Message content derive from each business module, each business module sends a message to the dedicated tunnel of message managing module
On, call corresponding message middleware to complete message communicating.
Structural data synchronization module includes log collection analytical tool CANAL and several Node nodes；Log collection
Analytical tool CANAL is deployed in the synchronous originating end of data；Each Node node includes data access, data are extracted, data turn
It changes and is loaded into four-stage with data；Structural data synchronization module obtains the data of types of databases by the way of Incremental Log
Variation realizes that the data in Various types of data source are synchronous by simulation SETL data warehousing process；Meanwhile file is as in database
The attachment of certain data record, synchronizes when tables of data changes with data.
Unstructured data transmission module includes being deployed in the CTRANS transmission service and third party's memory module of server-side；
High efficiency of transmission of more category image resources between different platform, batch are realized based on udp protocol using CTRANS transmission service
Transmission and breakpoint transmission service.
Services Integration module can also provide starting server, agency's distribution, asynchronous event distribution, application configuration on demand and answer
With the service of patch upgrading.
Business module includes order status change, job state change, plan is assigned or abnormal alarm.
Various types of data source involved in structural data synchronization module include mysql, oracle, media and
Hbase。
A kind of integrated approach towards multi-source heterogeneous geographic information resources, GIS-Geographic Information System are real by above-mentioned integrated framework
Now all kinds of geographic information resources are integrated；Client can call all kinds of resources after integrating by the corresponding interface；Geography letter
The integrated approach for ceasing resource be that Services Integration module is integrated, message managing module is integrated, structural data synchronization module is integrated and
One or more combination during unstructured data transmission module is integrated.
The integrated method of geographic information resources is carried out using Services Integration module, is included the following steps.
Step F1: server-side defines Slice interface description file and compiles the interface document for generating corresponding language, server-side
Respective service realizes function after birdsmouth.
Step F2: client use Slice interface description file identical with server-side in step F1 simultaneously compiles generation pair
The interface document of language is answered, client inherits calling interface and realizes that service object calls.
Step F3: service definition ICE target adapter ties up service object and target adapter that step F2 is called
It is fixed, complete service publication.
Step F4: the step F3 service issued and IceNode are deployed in each server using service deployment tool.
Step F5: configuring and starts registration center.
Step F6: carrying out service registration, configures application.xml according to service implementation and is loaded into registration
The heart.
Step F7: host and information on services are checked by the interface UI, and service is started, is stopped operation.
Step F8: client configures registration center address.
Step F9: when client call service, request query service terminal point information is sent to registration center.
Step F10: client establishes connection using IceNode where terminal point information and server-side.
Service invocation request is sent to respective service by step F11:IceNode.
Step F12: service is completed to return result to client by IceNode after responding.
In the step F6, service implementation is using JBOSS as service container.
The integrated method of geographic information resources is carried out using message managing module, is included the following steps.
Step X1: various types of messages queue is configured in server-side.
Step X2: business module creation link factory.
Step X3: business module is using point-to-point or publish/subscribe mode creation connection and session.
Step X4: creation message sends a message to the destination of message server.
Step X5: message server forwards the message to message recipient or message subscribing person.
Step X6: connection is closed.
The integrated method of geographic information resources is carried out using structural data synchronization module, is included the following steps.
Step T1: deployment log collection analytical tool CANAL and Node node.
Step T2: disposition data source, the data source for configuring completion may act as initiating synchronous source, can also be used as
Synchronous target database.
Step T3: configuration data table fills in and needs synchronous data table information.
Step T4: configuration log collects analytical tool CANAL, according to service function, from the data source that step T2 is configured
Selection needs to obtain the data source of log, then disposition data source account, password and character set, initializes log acquisition position
It sets.
Step T5: channel is configured, establishes a channel between every two data source, every channel can wrap
Pipeline containing two different directions, each pipeline are single direction channel.
Step T6: configuration pipeline selects Node node and log collection analytical tool CANAL.
Step T7: configuration relation mapping table selects source data table and step T3 in the data source configured in step T2 to match
The target database table set.
Step T8: starting channel, it is synchronous to start data, and when source data table data variation can be synchronous according to mapping ruler
To target matrix.
The integrated method of geographic information resources is carried out using unstructured data transmission module, is included the following steps.
Step C1: " CTRANS_IP " is configured in server-side configuration file and is attached with third party's memory module, is configured
More category data download address " DOWNLOAD_PATH ".
Step C2: more category data are issued in service, are called the more category data entities of CTRANS transmission service acquisition and are stored in
Third party's memory module.
Step C3:CTRANS transmission service makes seed according to the storage condition of more category data entities, includes in seed
The relevant information for obtaining more category data entities is stored from third party.
Step C4: service selects corresponding message queue to complete the publication of seed, transmission by message managing module.
Step C5: recipient obtains the seed message of service publication by message queue and seed information is transferred to reception
CTRANS where person transmits service.
Step C6: recipient calls CTRANS transmission service to obtain more category data entities from third party's memory module, more
Category data entity is stored in the specified path " DOWNLOAD_PATH ", and recipient can call more category data entities to carry out
Other operation flows.
The present invention is mainly directed towards the multi-source heterogeneous geographical spatial data and geographic information services of magnanimity, provides a kind of modularization
Distributed Information Resource technology of sharing solution.Based on the obtaining means of resource, distribution position in existing area of geographic information
It sets, the factors such as architectural characteristic, from cross-domain data entity transmission, distributed geographic information is synchronous, remote service is called, multidimensional platform
Message management etc. designs and Implements corresponding scheme module, is then integrated by Lightweight J 2 EE Framework to each module,
And then it achieves that:
1) the geography information heterogeneous resource integrated framework for realizing complete set, solves service, message, structural data
With calling of the unstructured data between different platform, transmission with it is synchronous the problems such as.
2) the multiple terminals service call modes such as mobile phone, plate, computer are supported.
3) towards the different platforms framework such as C/S, B/S, compared with existing solution, the present invention has apparent performance excellent
Gesture.
4) compared with prior art, the present invention is better than similar technique scheme in terms of scalability and exploitation simplification.
Detailed description of the invention
Fig. 1 shows a kind of structural schematic diagram of the integrated framework towards multi-source heterogeneous geographic information resources of the present invention.
Fig. 2 shows that geographic information resources integrated framework implements architecture diagram.
Fig. 3 shows the test knot of geographic information resources integrated framework service response time Yu online user's quantitative relation
Fruit.
Fig. 4 shows the test result of geographic information resources integrated framework file synchronization time and quantity of documents relationship.
Fig. 5 shows the test result of geographic information resources integrated framework file synchronization time and file size relationship.
Specific embodiment
Xia Mianjiehefutuhejuti compare Jia Shishifangshiduibenfamingzuojinyibuxiangxishuoming.
As depicted in figs. 1 and 2, a kind of integrated framework towards multi-source heterogeneous geographic information resources, including four modules and
The Lightweight J 2 EE Framework that four modules are integrated.Lightweight J 2 EE Framework is a more mature prior art, this
In will not be described in further detail.
Four modules are respectively Services Integration module, message managing module, structural data synchronization module and unstructured
Data transmission module.
Services Integration module carries out the exploitation and deployment of distributed application program using ICE middleware Technology, realizes service
Far call.As shown in Figure 1, Services Integration module has multinomial service, such as service 1, service 2, service 3 and service 4, lead to
The Route Selection and load balancing between above-mentioned multinomial service are realized in the service dispatch center crossed in ICENODE, thus needed for calling
Service Source.
As shown in Fig. 2, Services Integration module includes the interface UI, registration center and service deployment tool.Wherein, registration center
IceNode by being deployed in server-side obtains host and information on services；The interface UI is for showing acquired in registration center
Information, and service can be carried out to start or stop operation；The new IceNode node clothes of service deployment tool energy rapid deployment
Business, and upgrading or rolling back action are carried out to original IceNode node serve.
Services Integration module is supported to call the Service Source for operating in different platform, multilingual exploitation, realizes and service money
Source integration and service interoperation.Services Integration module can also provide on-demand starting server, agency distributes, asynchronous event is distributed,
The service of application configuration and application patch upgrading.
Message managing module can manage the message generated in Services Integration module integrating process concentratedly, realize clothes
The function when operations such as business communication, asynchronous process, log collection, permission control is supported.
Message managing module is a series of message queue cluster being made of message queues, and GIS-Geographic Information System passes through message
Queue cluster carries out message communicating, so that the message-driven between each business module provides infrastructural support, and solves and is
The problems such as between system using coupling, asynchronous message and flow peak clipping.
Message content in message queue derives from each business module, and each business module sends a message to message pipe
In the dedicated tunnel for managing module, corresponding message middleware is called to complete message communicating.
Above-mentioned business module includes order status change, job state change, plan is assigned or abnormal alarm etc..
Structural data synchronization module is to pass through table level number towards the structural data in sql, part Nosql database
According to virtualization integration, the functions such as log parsing, recording synchronism are realized.
Structural data synchronization module includes log collection analytical tool CANAL and several Node nodes.
Log collection analytical tool CANAL is deployed in the synchronous originating end of data；Each Node node include data access,
Data are extracted, data conversion and data are loaded into four-stage.Structural data synchronization module is obtained by the way of Incremental Log
The data variation of types of databases realizes that the data in Various types of data source are synchronous by simulation SETL data warehousing process.Meanwhile
The attachment that file (such as picture, binary stream) is recorded as certain data in database, when tables of data changes with number
According to synchronizing together.Data resources synchronization module supports that real-time data synchronization, multipoint data insertion, attachment is synchronous, remote room is same
A variety of data resources synchronization demands such as step, data mapping, filtering.
Various types of data source involved in structural data synchronization module includes mysql, oracle, media and Hbase
Deng.
Unstructured data transmission module includes being deployed in the CTRANS transmission service and third party's memory module of server-side.
Unstructured data transmission module transmits service by CTRANS and realizes document entity towards large-data documents such as remote sensing images
Breakpoint transmission, the functions such as bulk transfer are supported in transmission, mainly solve in geographic information services a large amount of image solid datas not
With the transmission between platform, these image data formats are varied, and size is differed by tens KB to tens GB.It is passed using CTRANS
Defeated service realizes that high efficiency of transmission, bulk transfer and breakpoint of more category image resources between different platform are continuous based on udp protocol
Pass service.
A kind of integrated approach towards multi-source heterogeneous geographic information resources, GIS-Geographic Information System are real by above-mentioned integrated framework
Now all kinds of geographic information resources are integrated；Client can call all kinds of resources after integrating by the corresponding interface；Geography letter
The integrated approach for ceasing resource be that Services Integration module is integrated, message managing module is integrated, structural data synchronization module is integrated and
One or more combination during unstructured data transmission module is integrated.
The integrated method of geographic information resources is carried out using Services Integration module, is included the following steps.
Step F1: server-side defines Slice interface description file and compiles the interface document for generating corresponding language, server-side
Respective service realizes function after birdsmouth.
Step F2: client use Slice interface description file identical with server-side in step F1 simultaneously compiles generation pair
The interface document of language is answered, client inherits calling interface and realizes that service object calls.
Step F3: service definition ICE target adapter ties up service object and target adapter that step F2 is called
It is fixed, complete service publication.
Step F4: the step F3 service issued and IceNode are deployed in each server using service deployment tool.
Step F5: configuring and starts registration center.
Step F6: carrying out service registration, configures application.xml according to service implementation and is loaded into registration
The heart.Service implementation preferably uses JBOSS as service container.
Step F7: host and information on services are checked by the interface UI, and service is started, is stopped operation.
Step F8: client configures registration center address.
Step F9: when client call service, request query service terminal point information is sent to registration center.
Step F10: client establishes connection using IceNode where terminal point information and server-side.
Service invocation request is sent to respective service by step F11:IceNode.
Step F12: service is completed to return result to client by IceNode after responding.
The integrated method of geographic information resources is carried out using message managing module, is included the following steps.
Step X1: various types of messages queue is configured in server-side.
Step X2: business module is using prior art creation link factory.
Step X3: business module is using point-to-point or publish/subscribe mode creation connection and session.
Step X4: creation message namely refers to the information transmitted between service at message here；Then, message is sent
To the destination of message server.
Step X5: message server forwards the message to message recipient or message subscribing person.
Step X6: connection is closed.
The integrated method of geographic information resources is carried out using structural data synchronization module, is included the following steps.
Step T1: deployment log collection analytical tool CANAL and Node node.
Step T2: disposition data source, the data source for configuring completion may act as initiating synchronous source, can also be used as
Synchronous target database.
Step T3: configuration data table fills in and needs synchronous data table information.
Step T4: configuration log collects analytical tool CANAL, according to service function, from the data source that step T2 is configured
Selection needs to obtain the data source of log, then disposition data source account, password and character set, initializes log acquisition position
It sets.
Step T5: channel is configured, establishes a channel between every two data source, every channel can wrap
Pipeline containing two different directions, each pipeline are single direction channel.
Step T6: configuration pipeline selects Node node and log collection analytical tool CANAL.
Step T7: configuration relation mapping table selects source data table and step T3 in the data source configured in step T2 to match
The target database table set.
Step T8: starting channel, it is synchronous to start data, and when source data table data variation can be synchronous according to mapping ruler
To target matrix.
The integrated method of geographic information resources is carried out using unstructured data transmission module, is included the following steps.
Step C1: " CTRANS_IP " is configured in server-side configuration file and is attached with third party's memory module, is configured
More category data download address " DOWNLOAD_PATH ".
Step C2: more category data are issued in service, are called the more category data entities of CTRANS transmission service acquisition and are stored in
Third party's memory module.
Step C3:CTRANS transmission service makes seed according to the storage condition of more category data entities, includes in seed
The relevant information for obtaining more category data entities is stored from third party.
Step C4: service selects corresponding message queue to complete the publication of seed, transmission by message managing module.
Step C5: recipient obtains the seed message of service publication by message queue and seed information is transferred to reception
CTRANS where person transmits service.
Step C6: recipient calls CTRANS transmission service to obtain more category data entities from third party's memory module, more
Category data entity is stored in the specified path " DOWNLOAD_PATH ", and recipient can call more category data entities to carry out
Other operation flows.
As illustrated in fig. 2, it is assumed that there are two back end A and B, operation service 1 and service 2, data section in back end A
Operation service 3 and service 4 in point B, operation support container is all Jboss middleware.Service 1 or service 2 in back end A is logical
It crosses for the resource that the integrated framework of the invention towards multi-source heterogeneous geographic information resources calls in back end B, to this hair
Bright integrated approach is described as follows.
The integrated approach of geographic information resources is preferably integrated including Services Integration module simultaneously, message managing module is integrated,
Structural data synchronization module is integrated and unstructured data transmission module is integrated.
1. service 1-4 needs are registered in the registration center in Services Integration module first.
2. then, service call: IceNode 1 is responsible for call request being sent to service 1, and operation support container is
Jboss service 1；IceNode 2 is responsible for for call request being sent to service 2, and operation support container is Jboss service 2；
IceNode 3 is responsible for for call request being sent to service 3, and operation support container is Jboss service 3；IceNode 4 is responsible for adjust
It is sent to service 4 with request, operation support container is Jboss service 4.
When the Service Source in service call back end B, integrated framework passes through the service dispatch center in ICENODE
Required Service Source in returned data node B.
3. message management: the message communicating between back end A and B, which is then recorded in message queue cluster, to be managed collectively.
4. data are synchronous: if access resource passes through structural data synchronization module, by two there are in database table
The data that the processes such as data access, data extraction, data conversion, the data loading at end realize source to destination are synchronous.
5. data are transmitted: and for the equal unstructured datas of some image resources, then pass through CTRANS service realization source
Hold the equity transmission of destination.
Fig. 3 is mainly by taking place name service and login service in geographic information resources shared service as an example, in same hardware
Under environment by with RMI frame compare, integrated framework medium-long range service calling module resources occupation rate, in terms of
It will be better than technologies such as RMI；Database synchronization performance indicator depends on target database performance, size of data, network environment etc.
Multiple factors, it is 100tps that single machine room, which synchronizes preliminary performance, postpones 100ms, 5000tps, postpones 1s, data 1kb size, the limit
Tps is up to 10,000 or more.The attachment that file is recorded as certain data in database, when tables of data changes with data one
It rises and synchronizes.The attachment of a variety of data formats can be supported synchronous, picture (gif, jpg, png, bmp), compressed file (xls, doc,
Exe) and file is supported.Dependence test result such as Fig. 4, Fig. 5.
The preferred embodiment of the present invention has been described above in detail, still, during present invention is not limited to the embodiments described above
Detail a variety of equivalents can be carried out to technical solution of the present invention within the scope of the technical concept of the present invention, this
A little equivalents all belong to the scope of protection of the present invention.
Claims (10)
1. a kind of integrated system towards multi-source heterogeneous geographic information resources, it is characterised in that: including four modules and by four
The Lightweight J 2 EE Framework that module is integrated；
Four modules are respectively Services Integration module, message managing module, structural data synchronization module and unstructured data
Transmission module；
Services Integration module carries out the exploitation and deployment of distributed application program using ICE middleware Technology；Services Integration module
Including the interface UI, registration center and service deployment tool；Wherein, registration center is obtained by being deployed in the IceNode of server-side
Host and information on services；The interface UI can be started or be stopped to service for showing information acquired in registration center
Only operate；The new IceNode node serve of service deployment tool energy rapid deployment, and original IceNode node serve is carried out
Upgrading or rolling back action；
Message managing module is a series of message queue cluster being made of message queues, and GIS-Geographic Information System passes through message queue
Cluster carries out message communicating, so that the message-driven between each business module provides infrastructural support；Disappearing in message queue
Content sources are ceased in each business module, and each business module is sent a message in the dedicated tunnel of message managing module, is adjusted
Message communicating is completed with corresponding message middleware；
Structural data synchronization module includes log collection analytical tool CANAL and several Node nodes；Log collection parsing
Tool CANAL is deployed in the synchronous originating end of data；Each Node node include data access, data extract, data conversion and
Data are loaded into four-stage；The data that structural data synchronization module obtains types of databases by the way of Incremental Log become
Change, realizes that the data in Various types of data source are synchronous by simulation SETL data warehousing process；Meanwhile file is as certain in database
The attachment of data record, synchronizes when tables of data changes with data；
Unstructured data transmission module includes being deployed in the CTRANS transmission service and third party's memory module of server-side；It utilizes
CTRANS transmission service based on udp protocol realize high efficiency of transmission of more category image resources between different platform, bulk transfer and
Breakpoint transmission service.
2. the integrated system according to claim 1 towards multi-source heterogeneous geographic information resources, it is characterised in that: services set
At module can also provide on demand starting server, agency distribution, asynchronous event distribution, application configuration and application patch upgrading clothes
Business.
3. the integrated system according to claim 1 towards multi-source heterogeneous geographic information resources, it is characterised in that: business mould
Block includes order status change, job state change, plan is assigned or abnormal alarm.
4. the integrated system according to claim 1 towards multi-source heterogeneous geographic information resources, it is characterised in that: structuring
Various types of data source involved in data simultaneous module includes mysql, oracle, media and Hbase.
5. a kind of integrated approach towards multi-source heterogeneous geographic information resources, it is characterised in that: GIS-Geographic Information System passes through right
It is required that the 1 integrated system realization integrates all kinds of geographic information resources；Client can be called integrated by the corresponding interface
All kinds of resources afterwards；The integrated approach of geographic information resources is that Services Integration module is integrated, message managing module is integrated, structuring
Data simultaneous module is integrated and unstructured data transmission module it is integrated in one or more combination.
6. the integrated approach according to claim 5 towards multi-source heterogeneous geographic information resources, it is characterised in that: using clothes
Business integration module carries out the integrated method of geographic information resources, includes the following steps:
Step F1: server-side defines Slice interface description file and compiles the interface document for generating corresponding language, and server-side is corresponding
Service realizes function after birdsmouth；
Step F2: client use Slice interface description file identical with server-side in step F1 simultaneously compiles the corresponding language of generation
The interface document of speech, client inherit calling interface and realize that service object calls；
Step F3: service definition ICE target adapter binds service object and target adapter that step F2 is called,
Complete service publication；
Step F4: the step F3 service issued and IceNode are deployed in each server using service deployment tool；
Step F5: configuring and starts registration center；
Step F6: carrying out service registration, configures application.xml according to service implementation and is loaded into registration center；
Step F7: host and information on services are checked by the interface UI, and service is started, is stopped operation；
Step F8: client configures registration center address；
Step F9: when client call service, request query service terminal point information is sent to registration center；
Step F10: client establishes connection using IceNode where terminal point information and server-side；
Service invocation request is sent to respective service by step F11:IceNode；
Step F12: service is completed to return result to client by IceNode after responding.
7. the integrated approach according to claim 6 towards multi-source heterogeneous geographic information resources, it is characterised in that: the step
In rapid F6, service implementation is using JBOSS as service container.
8. the integrated approach according to claim 5 towards multi-source heterogeneous geographic information resources, it is characterised in that: use and disappear
It ceases management module and carries out the integrated method of geographic information resources, include the following steps:
Step X1: various types of messages queue is configured in server-side；
Step X2: business module creation link factory；
Step X3: business module is using point-to-point or publish/subscribe mode creation connection and session；
Step X4: creation message sends the message of creation to the destination of message server；
Step X5: message server forwards the message to message recipient or message subscribing person；
Step X6: connection is closed.
9. the integrated approach according to claim 5 towards multi-source heterogeneous geographic information resources, it is characterised in that: using knot
Structure data simultaneous module carries out the integrated method of geographic information resources, includes the following steps:
Step T1: deployment log collection analytical tool CANAL and Node node；
Step T2: disposition data source, the data source for configuring completion may act as initiating synchronous source, can also be used as synchronization
Target database；
Step T3: configuration target database table is filled in and needs synchronous data table information；
Step T4: configuration log collects analytical tool CANAL, according to service function, selects from the data source that step T2 is configured
It needs to obtain the data source of log, then disposition data source account, password and character set, initializes log acquisition position；
Step T5: channel is configured, establishes a channel between every two data source, every channel may include two
The pipeline of a different directions, each pipeline are single direction channel；
Step T6: configuration pipeline selects Node node and log collection analytical tool CANAL；
Step T7: configuration relation mapping table selects source data table and step T3 in the data source configured in step T2 to configure
Target database table；
Step T8: starting channel, it is synchronous to start data, and when source data table data variation can be synchronized to mesh according to mapping ruler
Mark tables of data.
10. the integrated approach according to claim 5 towards multi-source heterogeneous geographic information resources, it is characterised in that: use
Unstructured data transmission module carries out the integrated method of geographic information resources, includes the following steps:
Step C1: " CTRANS_IP " is configured in server-side configuration file and is attached with third party's memory module, more product are configured
Class data download address " DOWNLOAD_PATH "；
Step C2: more category data are issued in service, are called the more category data entities of CTRANS transmission service acquisition and are stored in third
Square memory module；
Step C3:CTRANS transmission service makes seed according to the storage condition of more category data entities, comprising from the in seed
Tripartite stores the relevant information for obtaining more category data entities；
Step C4: service selects corresponding message queue to complete the publication of seed, transmission by message managing module；
Step C5: recipient obtains the seed message of service publication by message queue and seed information is transferred to recipient institute
CTRANS transmit service；
Step C6: recipient calls CTRANS transmission service to obtain more category data entities, more categories from third party's memory module
Data entity is stored in the specified path " DOWNLOAD_PATH ", and recipient can call more category data entities to carry out other
Operation flow.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710045358.XA CN106850788B (en) | 2017-01-22 | 2017-01-22 | Integrated framework and integrated approach towards multi-source heterogeneous geographic information resources |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710045358.XA CN106850788B (en) | 2017-01-22 | 2017-01-22 | Integrated framework and integrated approach towards multi-source heterogeneous geographic information resources |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106850788A CN106850788A (en) | 2017-06-13 |
| CN106850788B true CN106850788B (en) | 2019-08-20 |
Family
ID=59119539
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710045358.XA Active CN106850788B (en) | 2017-01-22 | 2017-01-22 | Integrated framework and integrated approach towards multi-source heterogeneous geographic information resources |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106850788B (en) |
Families Citing this family (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107609834A (en) * | 2017-07-21 | 2018-01-19 | 东莞新奥燃气有限公司 | project management system and project management method |
| CN107526786A (en) * | 2017-08-01 | 2017-12-29 | 江苏速度信息科技股份有限公司 | The method and system that place name address date based on multi-source data is integrated |
| CN107832903A (en) * | 2017-08-28 | 2018-03-23 | 中国石油化工股份有限公司 | The integrated method of application system modularization |
| CN108052399A (en) * | 2017-12-14 | 2018-05-18 | 郑州云海信息技术有限公司 | Method for message transmission and system between a kind of program module |
| CN107948309A (en) * | 2017-12-15 | 2018-04-20 | 神思电子技术股份有限公司 | A kind of integrated management method and system of the server resource based on Restful API |
| CN109962957B (en) * | 2017-12-26 | 2022-04-05 | 珠海全志科技股份有限公司 | Data synchronization method and device for multi-region multi-active service |
| CN108170846B (en) * | 2018-01-17 | 2018-12-11 | 清华大学 | Extensive multi-source heterogeneous data persistence method based on elastic table model |
| CN113407633A (en) * | 2018-09-13 | 2021-09-17 | 华东交通大学 | Distributed data source heterogeneous synchronization method |
| CN109522386B (en) * | 2018-12-25 | 2020-10-20 | 武汉众智鸿图科技有限公司 | Method and system for generating spatial information service across GIS platform |
| CN109684435B (en) * | 2019-01-11 | 2019-09-10 | 长沙市规划勘测设计研究院 | Specialized multi-source heterogeneous space-time geodata is gone to integrate system and method |
| CN111435354A (en) * | 2019-01-14 | 2020-07-21 | 北京京东尚科信息技术有限公司 | Data export method and device, storage medium and electronic equipment |
| CN110297871A (en) * | 2019-06-20 | 2019-10-01 | 常州冰鉴信息科技有限公司 | A kind of method that isomeric data acquires in real time |
| CN111988315A (en) * | 2020-08-19 | 2020-11-24 | 青岛易来智能科技股份有限公司 | Synchronous calling method, device, electronic equipment, system and storage medium |
| CN112487070A (en) * | 2020-10-30 | 2021-03-12 | 中科星图股份有限公司 | Access integration device and method for multi-source heterogeneous data |
| CN112506630A (en) * | 2020-12-01 | 2021-03-16 | 国科晋云技术有限公司 | Multi-domain service platform architecture |
| CN113032598B (en) * | 2021-04-12 | 2023-07-14 | 郑州航空工业管理学院 | Image design visual conveying system based on big data |
| CN114138258B (en) * | 2021-10-13 | 2024-07-23 | 同济大学 | Complex software coupling modeling method |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101976199A (en) * | 2010-10-15 | 2011-02-16 | 德讯科技股份有限公司 | Object-oriented middleware distributed system development platform |
| CN102684930A (en) * | 2012-06-08 | 2012-09-19 | 山东科汇电力自动化有限公司 | Distributed application high-reliability realization mechanism and operation method based on in-circuit emulator (ICE) |
| CN102750147A (en) * | 2012-06-08 | 2012-10-24 | 山东科汇电力自动化有限公司 | Internet communications engine (ICE) middleware based distributed application management framework and operation method |
-
2017
- 2017-01-22 CN CN201710045358.XA patent/CN106850788B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101976199A (en) * | 2010-10-15 | 2011-02-16 | 德讯科技股份有限公司 | Object-oriented middleware distributed system development platform |
| CN102684930A (en) * | 2012-06-08 | 2012-09-19 | 山东科汇电力自动化有限公司 | Distributed application high-reliability realization mechanism and operation method based on in-circuit emulator (ICE) |
| CN102750147A (en) * | 2012-06-08 | 2012-10-24 | 山东科汇电力自动化有限公司 | Internet communications engine (ICE) middleware based distributed application management framework and operation method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106850788A (en) | 2017-06-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106850788B (en) | Integrated framework and integrated approach towards multi-source heterogeneous geographic information resources | |
| CN107301048B (en) | Internal control management system of application response type shared application architecture | |
| CN107172187B (en) | A kind of SiteServer LBS and method | |
| CN105809356A (en) | Information system resource management method based on application integrated cloud platform | |
| CN109542865A (en) | Distributed cluster system configuration file synchronous method, device, system and medium | |
| CN103617255B (en) | A kind of business datum for power information system exchanges and synchronizes system and method | |
| CN107169069B (en) | Distributed hierarchical extraction multi-application method and data extraction applicator | |
| CN110262902A (en) | Information processing method and system, medium and calculating equipment | |
| CN110213156A (en) | A kind of span centre heart group's instant communicating method and system | |
| CN113794652B (en) | Data processing method, device, electronic equipment and storage medium | |
| CN109246220A (en) | Message pushing system and method | |
| US8924530B2 (en) | Multi-phase monitoring of hybrid system landscapes | |
| CN101834765B (en) | System and method for remote file transfer | |
| JP2009153126A (en) | System, apparatus and method for synchronizing presence information with schedule information | |
| CN110413677A (en) | A kind of Distributed Data Synchronization method and system for supporting concurrently to apply | |
| CN112711625B (en) | Bidirectional self-adaptive multi-source heterogeneous big data dynamic processing method | |
| CN103685485A (en) | Mobile terminal VoIP (voice over internet protocol) communication audio management system | |
| CN112351106A (en) | Service grid platform containing event grid and communication method thereof | |
| CN103152428A (en) | Method for carrying out service communication among nodes on cloud platform | |
| CN107632991A (en) | A kind of public material database method for building up and system | |
| CN103713903A (en) | Hard disk dynamic loading method and device | |
| CN107786661B (en) | Information synchronization method | |
| CN102938723A (en) | Microblog aggregation method and system based on group number | |
| CN110555201B (en) | Knowledge document generation method, device, electronic equipment and storage medium | |
| CN113765958B (en) | Job task processing method and job client |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |