CN106850788B - Integrated framework and integrated approach towards multi-source heterogeneous geographic information resources - Google Patents

Integrated framework and integrated approach towards multi-source heterogeneous geographic information resources Download PDF

Info

Publication number
CN106850788B
CN106850788B CN201710045358.XA CN201710045358A CN106850788B CN 106850788 B CN106850788 B CN 106850788B CN 201710045358 A CN201710045358 A CN 201710045358A CN 106850788 B CN106850788 B CN 106850788B
Authority
CN
China
Prior art keywords
data
service
module
message
geographic information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201710045358.XA
Other languages
Chinese (zh)
Other versions
CN106850788A (en
Inventor
丁赤飚
付琨
胡岩峰
朱利鲁
丁士伟
张伟
刘振
吴亚军
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Suzhou Research Institute Institute Of Electronics Chinese Academy Of Sciences
Original Assignee
Suzhou Research Institute Institute Of Electronics Chinese Academy Of Sciences
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Suzhou Research Institute Institute Of Electronics Chinese Academy Of Sciences filed Critical Suzhou Research Institute Institute Of Electronics Chinese Academy Of Sciences
Priority to CN201710045358.XA priority Critical patent/CN106850788B/en
Publication of CN106850788A publication Critical patent/CN106850788A/en
Application granted granted Critical
Publication of CN106850788B publication Critical patent/CN106850788B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • H04L67/1095Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/29Geographical information databases
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/445Program loading or initiating
    • G06F9/44505Configuring for program initiating, e.g. using registry, configuration files
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • H04L67/1097Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/2866Architectures; Arrangements
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/2866Architectures; Arrangements
    • H04L67/30Profiles
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/50Network services
    • H04L67/52Network services specially adapted for the location of the user terminal
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/50Network services
    • H04L67/56Provisioning of proxy services
    • H04L67/562Brokering proxy services
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/50Network services
    • H04L67/60Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Software Systems (AREA)
  • Remote Sensing (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

Integrated framework and integrated approach towards multi-source heterogeneous geographic information resources, including Services Integration module, message managing module, structural data synchronization module, unstructured data transmission module and the Lightweight J 2 EE Framework for integrating four modules.The present invention is mainly directed towards the multi-source heterogeneous geographical spatial data and geographic information services of magnanimity, provides a kind of modular Distributed Information Resource technology of sharing solution.The factors such as obtaining means, distributing position, architectural characteristic based on resource in existing area of geographic information, corresponding scheme module is designed and Implemented from cross-domain data entity transmission, distributed geographic information synchronization, remote service calling, multidimensional platform message management etc., and then realize the geography information heterogeneous resource integrated framework of complete set, solve calling between different platform of service, message, structural data and unstructured data, transmission with it is synchronous the problems such as, networked resource utilization efficiency is improved, resource-sharing cost is reduced.

Description

Integrated framework and integrated approach towards multi-source heterogeneous geographic information resources
Technical field
The present invention relates to technical field of geographic information, especially a kind of integrated frame towards multi-source heterogeneous geographic information resources Frame and integrated approach.
Background technique
In recent years, with production and living advanced IT application, geographic information services are in public transport, social platform, shopping joy The fields such as happy and smart city, government decision are increasingly becoming the basic module that all kinds of services are relied on.Every profession and trade, department's accumulation A large amount of geography information related resource and be served by, but due to lacking effective resource integrated method, result in for Number numerous " resource isolation ", results in waste of resources and unnecessary repeated construction.
To improve geographic information resources utilization efficiency, production management cost is reduced, is mentioned by resource consolidation scheme for user Become new trend for a kind of shared service mode towards multidimensional data, the resource sharing platform for having high integration ability is answered It transports and gives birth to.However due to the continuous expansion of geographical spatial data scale, the increasingly sophisticated while enterprise of geographic information services framework Because the difference of business demand cause platform language version it is different situations such as, brought to the process of resource-sharing between different platform all More problems.
The resource sharing service of building interconnection needs integrated remote method call, data transmission, data synchronization, message A variety of key technologies such as distribution, establish complete framework technology system.Previous framework mostly uses greatly service-oriented Architectural framework (service oriented architecture, SOA) thought, specific implementation have Web Service, The modes such as CORBA, DCOM and RMI.But with continuous expansion, the continuous promotion of performance indicator of service scale, above-mentioned realization side Formula more or less be there are problems that, be made a concrete analysis of as follows:
The performance of 1.Web Service, the defects of having a single function, gradually appear.Either opened in network bandwidth or CPU Pin aspect all can cause serious performance issue to application, so that the technology can not be suitable for many to high-performance and mutual Operation requires extremely frequent system.
2.CORBA and DCOM because its numerous and jumbled technical standard and platform compatibility aspect the shortcomings that due to be difficult to push away extensively Extensively.
3.RMI is the far call of Java language, and the program language at both ends must be that Java is realized, can not support different languages The intercommunication of speech.
In terms of data integration, the mode (XML) that previous integrated framework attempts to convert data resource to standard is carried out The problem of accessing, but being still faced with inefficiency.
In data synchronization aspects, the data synchronization solutions of mainstream have database manufacturer provide data synchronization technology and Data synchronization technology based on capture change technique customized exploitation.However, there is also following problems:
1. the synchronization that the simultaneous techniques that manufacturer provides often does not support heterogeneous database, it is difficult to meet the requirements.
2. many scholars are proposed the database synchronization technology based on capture technique, as Shen Min utilizes trigger and storage Process combines and obtains delta data from source database subsystem, Liu Yongyi propose an improved main text updating method and when Between stab the strategy combined.But the ununified technical standard of these methods, there are various restrictive conditions, are not suitable for large-scale system System.
In addition, previous framework focuses mainly on the integrated of Service Source and database resource, for message Resource and being transmitted without for unstructured data resource provide rigorous, available implementation.
In conclusion service call, message communicating, data synchronization and data transmission between isomerous environment, at present Geographic information resources shared platform there is no a set of perfect solutions and frame system.
Summary of the invention
In view of the above-mentioned deficiencies of the prior art, the technical problem to be solved by the present invention is to provide one kind towards multi-source heterogeneous The integrated framework of geographic information resources should be directed to current information towards the integrated framework of multi-source heterogeneous geographic information resources and integrate skill Requirement of the deficiency and resource consolidation existing for art to exploitation complexity and performance etc., provides a kind of modular distribution Information resources share technical solution, it is same in service call, message communicating, information for solving all kinds of geographic information resources The problem during step, data transmission etc., improves networked resource utilization efficiency, reduces resource-sharing cost.
In order to solve the above technical problems, the technical solution adopted by the present invention is that:
Integrated framework towards multi-source heterogeneous geographic information resources is integrated including four modules and by four modules Lightweight J 2 EE Framework.
Four modules are respectively Services Integration module, message managing module, structural data synchronization module and unstructured Data transmission module.
Services Integration module carries out the exploitation and deployment of distributed application program using ICE middleware Technology;Services Integration Module includes the interface UI, registration center and service deployment tool;Wherein, registration center is by being deployed in the IceNode of server-side Obtain host and information on services;The interface UI can start service for showing information acquired in registration center Or it stops operation;The new IceNode node serve of service deployment tool energy rapid deployment, and to original IceNode node serve Carry out upgrading or rolling back action.
Message managing module is a series of message queue cluster being made of message queues, and GIS-Geographic Information System passes through message Queue cluster carries out message communicating, so that the message-driven between each business module provides infrastructural support;In message queue Message content derive from each business module, each business module sends a message to the dedicated tunnel of message managing module On, call corresponding message middleware to complete message communicating.
Structural data synchronization module includes log collection analytical tool CANAL and several Node nodes;Log collection Analytical tool CANAL is deployed in the synchronous originating end of data;Each Node node includes data access, data are extracted, data turn It changes and is loaded into four-stage with data;Structural data synchronization module obtains the data of types of databases by the way of Incremental Log Variation realizes that the data in Various types of data source are synchronous by simulation SETL data warehousing process;Meanwhile file is as in database The attachment of certain data record, synchronizes when tables of data changes with data.
Unstructured data transmission module includes being deployed in the CTRANS transmission service and third party's memory module of server-side; High efficiency of transmission of more category image resources between different platform, batch are realized based on udp protocol using CTRANS transmission service Transmission and breakpoint transmission service.
Services Integration module can also provide starting server, agency's distribution, asynchronous event distribution, application configuration on demand and answer With the service of patch upgrading.
Business module includes order status change, job state change, plan is assigned or abnormal alarm.
Various types of data source involved in structural data synchronization module include mysql, oracle, media and Hbase。
A kind of integrated approach towards multi-source heterogeneous geographic information resources, GIS-Geographic Information System are real by above-mentioned integrated framework Now all kinds of geographic information resources are integrated;Client can call all kinds of resources after integrating by the corresponding interface;Geography letter The integrated approach for ceasing resource be that Services Integration module is integrated, message managing module is integrated, structural data synchronization module is integrated and One or more combination during unstructured data transmission module is integrated.
The integrated method of geographic information resources is carried out using Services Integration module, is included the following steps.
Step F1: server-side defines Slice interface description file and compiles the interface document for generating corresponding language, server-side Respective service realizes function after birdsmouth.
Step F2: client use Slice interface description file identical with server-side in step F1 simultaneously compiles generation pair The interface document of language is answered, client inherits calling interface and realizes that service object calls.
Step F3: service definition ICE target adapter ties up service object and target adapter that step F2 is called It is fixed, complete service publication.
Step F4: the step F3 service issued and IceNode are deployed in each server using service deployment tool.
Step F5: configuring and starts registration center.
Step F6: carrying out service registration, configures application.xml according to service implementation and is loaded into registration The heart.
Step F7: host and information on services are checked by the interface UI, and service is started, is stopped operation.
Step F8: client configures registration center address.
Step F9: when client call service, request query service terminal point information is sent to registration center.
Step F10: client establishes connection using IceNode where terminal point information and server-side.
Service invocation request is sent to respective service by step F11:IceNode.
Step F12: service is completed to return result to client by IceNode after responding.
In the step F6, service implementation is using JBOSS as service container.
The integrated method of geographic information resources is carried out using message managing module, is included the following steps.
Step X1: various types of messages queue is configured in server-side.
Step X2: business module creation link factory.
Step X3: business module is using point-to-point or publish/subscribe mode creation connection and session.
Step X4: creation message sends a message to the destination of message server.
Step X5: message server forwards the message to message recipient or message subscribing person.
Step X6: connection is closed.
The integrated method of geographic information resources is carried out using structural data synchronization module, is included the following steps.
Step T1: deployment log collection analytical tool CANAL and Node node.
Step T2: disposition data source, the data source for configuring completion may act as initiating synchronous source, can also be used as Synchronous target database.
Step T3: configuration data table fills in and needs synchronous data table information.
Step T4: configuration log collects analytical tool CANAL, according to service function, from the data source that step T2 is configured Selection needs to obtain the data source of log, then disposition data source account, password and character set, initializes log acquisition position It sets.
Step T5: channel is configured, establishes a channel between every two data source, every channel can wrap Pipeline containing two different directions, each pipeline are single direction channel.
Step T6: configuration pipeline selects Node node and log collection analytical tool CANAL.
Step T7: configuration relation mapping table selects source data table and step T3 in the data source configured in step T2 to match The target database table set.
Step T8: starting channel, it is synchronous to start data, and when source data table data variation can be synchronous according to mapping ruler To target matrix.
The integrated method of geographic information resources is carried out using unstructured data transmission module, is included the following steps.
Step C1: " CTRANS_IP " is configured in server-side configuration file and is attached with third party's memory module, is configured More category data download address " DOWNLOAD_PATH ".
Step C2: more category data are issued in service, are called the more category data entities of CTRANS transmission service acquisition and are stored in Third party's memory module.
Step C3:CTRANS transmission service makes seed according to the storage condition of more category data entities, includes in seed The relevant information for obtaining more category data entities is stored from third party.
Step C4: service selects corresponding message queue to complete the publication of seed, transmission by message managing module.
Step C5: recipient obtains the seed message of service publication by message queue and seed information is transferred to reception CTRANS where person transmits service.
Step C6: recipient calls CTRANS transmission service to obtain more category data entities from third party's memory module, more Category data entity is stored in the specified path " DOWNLOAD_PATH ", and recipient can call more category data entities to carry out Other operation flows.
The present invention is mainly directed towards the multi-source heterogeneous geographical spatial data and geographic information services of magnanimity, provides a kind of modularization Distributed Information Resource technology of sharing solution.Based on the obtaining means of resource, distribution position in existing area of geographic information It sets, the factors such as architectural characteristic, from cross-domain data entity transmission, distributed geographic information is synchronous, remote service is called, multidimensional platform Message management etc. designs and Implements corresponding scheme module, is then integrated by Lightweight J 2 EE Framework to each module, And then it achieves that:
1) the geography information heterogeneous resource integrated framework for realizing complete set, solves service, message, structural data With calling of the unstructured data between different platform, transmission with it is synchronous the problems such as.
2) the multiple terminals service call modes such as mobile phone, plate, computer are supported.
3) towards the different platforms framework such as C/S, B/S, compared with existing solution, the present invention has apparent performance excellent Gesture.
4) compared with prior art, the present invention is better than similar technique scheme in terms of scalability and exploitation simplification.
Detailed description of the invention
Fig. 1 shows a kind of structural schematic diagram of the integrated framework towards multi-source heterogeneous geographic information resources of the present invention.
Fig. 2 shows that geographic information resources integrated framework implements architecture diagram.
Fig. 3 shows the test knot of geographic information resources integrated framework service response time Yu online user's quantitative relation Fruit.
Fig. 4 shows the test result of geographic information resources integrated framework file synchronization time and quantity of documents relationship.
Fig. 5 shows the test result of geographic information resources integrated framework file synchronization time and file size relationship.
Specific embodiment
Xia Mianjiehefutuhejuti compare Jia Shishifangshiduibenfamingzuojinyibuxiangxishuoming.
As depicted in figs. 1 and 2, a kind of integrated framework towards multi-source heterogeneous geographic information resources, including four modules and The Lightweight J 2 EE Framework that four modules are integrated.Lightweight J 2 EE Framework is a more mature prior art, this In will not be described in further detail.
Four modules are respectively Services Integration module, message managing module, structural data synchronization module and unstructured Data transmission module.
Services Integration module carries out the exploitation and deployment of distributed application program using ICE middleware Technology, realizes service Far call.As shown in Figure 1, Services Integration module has multinomial service, such as service 1, service 2, service 3 and service 4, lead to The Route Selection and load balancing between above-mentioned multinomial service are realized in the service dispatch center crossed in ICENODE, thus needed for calling Service Source.
As shown in Fig. 2, Services Integration module includes the interface UI, registration center and service deployment tool.Wherein, registration center IceNode by being deployed in server-side obtains host and information on services;The interface UI is for showing acquired in registration center Information, and service can be carried out to start or stop operation;The new IceNode node clothes of service deployment tool energy rapid deployment Business, and upgrading or rolling back action are carried out to original IceNode node serve.
Services Integration module is supported to call the Service Source for operating in different platform, multilingual exploitation, realizes and service money Source integration and service interoperation.Services Integration module can also provide on-demand starting server, agency distributes, asynchronous event is distributed, The service of application configuration and application patch upgrading.
Message managing module can manage the message generated in Services Integration module integrating process concentratedly, realize clothes The function when operations such as business communication, asynchronous process, log collection, permission control is supported.
Message managing module is a series of message queue cluster being made of message queues, and GIS-Geographic Information System passes through message Queue cluster carries out message communicating, so that the message-driven between each business module provides infrastructural support, and solves and is The problems such as between system using coupling, asynchronous message and flow peak clipping.
Message content in message queue derives from each business module, and each business module sends a message to message pipe In the dedicated tunnel for managing module, corresponding message middleware is called to complete message communicating.
Above-mentioned business module includes order status change, job state change, plan is assigned or abnormal alarm etc..
Structural data synchronization module is to pass through table level number towards the structural data in sql, part Nosql database According to virtualization integration, the functions such as log parsing, recording synchronism are realized.
Structural data synchronization module includes log collection analytical tool CANAL and several Node nodes.
Log collection analytical tool CANAL is deployed in the synchronous originating end of data;Each Node node include data access, Data are extracted, data conversion and data are loaded into four-stage.Structural data synchronization module is obtained by the way of Incremental Log The data variation of types of databases realizes that the data in Various types of data source are synchronous by simulation SETL data warehousing process.Meanwhile The attachment that file (such as picture, binary stream) is recorded as certain data in database, when tables of data changes with number According to synchronizing together.Data resources synchronization module supports that real-time data synchronization, multipoint data insertion, attachment is synchronous, remote room is same A variety of data resources synchronization demands such as step, data mapping, filtering.
Various types of data source involved in structural data synchronization module includes mysql, oracle, media and Hbase Deng.
Unstructured data transmission module includes being deployed in the CTRANS transmission service and third party's memory module of server-side. Unstructured data transmission module transmits service by CTRANS and realizes document entity towards large-data documents such as remote sensing images Breakpoint transmission, the functions such as bulk transfer are supported in transmission, mainly solve in geographic information services a large amount of image solid datas not With the transmission between platform, these image data formats are varied, and size is differed by tens KB to tens GB.It is passed using CTRANS Defeated service realizes that high efficiency of transmission, bulk transfer and breakpoint of more category image resources between different platform are continuous based on udp protocol Pass service.
A kind of integrated approach towards multi-source heterogeneous geographic information resources, GIS-Geographic Information System are real by above-mentioned integrated framework Now all kinds of geographic information resources are integrated;Client can call all kinds of resources after integrating by the corresponding interface;Geography letter The integrated approach for ceasing resource be that Services Integration module is integrated, message managing module is integrated, structural data synchronization module is integrated and One or more combination during unstructured data transmission module is integrated.
The integrated method of geographic information resources is carried out using Services Integration module, is included the following steps.
Step F1: server-side defines Slice interface description file and compiles the interface document for generating corresponding language, server-side Respective service realizes function after birdsmouth.
Step F2: client use Slice interface description file identical with server-side in step F1 simultaneously compiles generation pair The interface document of language is answered, client inherits calling interface and realizes that service object calls.
Step F3: service definition ICE target adapter ties up service object and target adapter that step F2 is called It is fixed, complete service publication.
Step F4: the step F3 service issued and IceNode are deployed in each server using service deployment tool.
Step F5: configuring and starts registration center.
Step F6: carrying out service registration, configures application.xml according to service implementation and is loaded into registration The heart.Service implementation preferably uses JBOSS as service container.
Step F7: host and information on services are checked by the interface UI, and service is started, is stopped operation.
Step F8: client configures registration center address.
Step F9: when client call service, request query service terminal point information is sent to registration center.
Step F10: client establishes connection using IceNode where terminal point information and server-side.
Service invocation request is sent to respective service by step F11:IceNode.
Step F12: service is completed to return result to client by IceNode after responding.
The integrated method of geographic information resources is carried out using message managing module, is included the following steps.
Step X1: various types of messages queue is configured in server-side.
Step X2: business module is using prior art creation link factory.
Step X3: business module is using point-to-point or publish/subscribe mode creation connection and session.
Step X4: creation message namely refers to the information transmitted between service at message here;Then, message is sent To the destination of message server.
Step X5: message server forwards the message to message recipient or message subscribing person.
Step X6: connection is closed.
The integrated method of geographic information resources is carried out using structural data synchronization module, is included the following steps.
Step T1: deployment log collection analytical tool CANAL and Node node.
Step T2: disposition data source, the data source for configuring completion may act as initiating synchronous source, can also be used as Synchronous target database.
Step T3: configuration data table fills in and needs synchronous data table information.
Step T4: configuration log collects analytical tool CANAL, according to service function, from the data source that step T2 is configured Selection needs to obtain the data source of log, then disposition data source account, password and character set, initializes log acquisition position It sets.
Step T5: channel is configured, establishes a channel between every two data source, every channel can wrap Pipeline containing two different directions, each pipeline are single direction channel.
Step T6: configuration pipeline selects Node node and log collection analytical tool CANAL.
Step T7: configuration relation mapping table selects source data table and step T3 in the data source configured in step T2 to match The target database table set.
Step T8: starting channel, it is synchronous to start data, and when source data table data variation can be synchronous according to mapping ruler To target matrix.
The integrated method of geographic information resources is carried out using unstructured data transmission module, is included the following steps.
Step C1: " CTRANS_IP " is configured in server-side configuration file and is attached with third party's memory module, is configured More category data download address " DOWNLOAD_PATH ".
Step C2: more category data are issued in service, are called the more category data entities of CTRANS transmission service acquisition and are stored in Third party's memory module.
Step C3:CTRANS transmission service makes seed according to the storage condition of more category data entities, includes in seed The relevant information for obtaining more category data entities is stored from third party.
Step C4: service selects corresponding message queue to complete the publication of seed, transmission by message managing module.
Step C5: recipient obtains the seed message of service publication by message queue and seed information is transferred to reception CTRANS where person transmits service.
Step C6: recipient calls CTRANS transmission service to obtain more category data entities from third party's memory module, more Category data entity is stored in the specified path " DOWNLOAD_PATH ", and recipient can call more category data entities to carry out Other operation flows.
As illustrated in fig. 2, it is assumed that there are two back end A and B, operation service 1 and service 2, data section in back end A Operation service 3 and service 4 in point B, operation support container is all Jboss middleware.Service 1 or service 2 in back end A is logical It crosses for the resource that the integrated framework of the invention towards multi-source heterogeneous geographic information resources calls in back end B, to this hair Bright integrated approach is described as follows.
The integrated approach of geographic information resources is preferably integrated including Services Integration module simultaneously, message managing module is integrated, Structural data synchronization module is integrated and unstructured data transmission module is integrated.
1. service 1-4 needs are registered in the registration center in Services Integration module first.
2. then, service call: IceNode 1 is responsible for call request being sent to service 1, and operation support container is Jboss service 1;IceNode 2 is responsible for for call request being sent to service 2, and operation support container is Jboss service 2; IceNode 3 is responsible for for call request being sent to service 3, and operation support container is Jboss service 3;IceNode 4 is responsible for adjust It is sent to service 4 with request, operation support container is Jboss service 4.
When the Service Source in service call back end B, integrated framework passes through the service dispatch center in ICENODE Required Service Source in returned data node B.
3. message management: the message communicating between back end A and B, which is then recorded in message queue cluster, to be managed collectively.
4. data are synchronous: if access resource passes through structural data synchronization module, by two there are in database table The data that the processes such as data access, data extraction, data conversion, the data loading at end realize source to destination are synchronous.
5. data are transmitted: and for the equal unstructured datas of some image resources, then pass through CTRANS service realization source Hold the equity transmission of destination.
Fig. 3 is mainly by taking place name service and login service in geographic information resources shared service as an example, in same hardware Under environment by with RMI frame compare, integrated framework medium-long range service calling module resources occupation rate, in terms of It will be better than technologies such as RMI;Database synchronization performance indicator depends on target database performance, size of data, network environment etc. Multiple factors, it is 100tps that single machine room, which synchronizes preliminary performance, postpones 100ms, 5000tps, postpones 1s, data 1kb size, the limit Tps is up to 10,000 or more.The attachment that file is recorded as certain data in database, when tables of data changes with data one It rises and synchronizes.The attachment of a variety of data formats can be supported synchronous, picture (gif, jpg, png, bmp), compressed file (xls, doc, Exe) and file is supported.Dependence test result such as Fig. 4, Fig. 5.
The preferred embodiment of the present invention has been described above in detail, still, during present invention is not limited to the embodiments described above Detail a variety of equivalents can be carried out to technical solution of the present invention within the scope of the technical concept of the present invention, this A little equivalents all belong to the scope of protection of the present invention.

Claims (10)

1. a kind of integrated system towards multi-source heterogeneous geographic information resources, it is characterised in that: including four modules and by four The Lightweight J 2 EE Framework that module is integrated;
Four modules are respectively Services Integration module, message managing module, structural data synchronization module and unstructured data Transmission module;
Services Integration module carries out the exploitation and deployment of distributed application program using ICE middleware Technology;Services Integration module Including the interface UI, registration center and service deployment tool;Wherein, registration center is obtained by being deployed in the IceNode of server-side Host and information on services;The interface UI can be started or be stopped to service for showing information acquired in registration center Only operate;The new IceNode node serve of service deployment tool energy rapid deployment, and original IceNode node serve is carried out Upgrading or rolling back action;
Message managing module is a series of message queue cluster being made of message queues, and GIS-Geographic Information System passes through message queue Cluster carries out message communicating, so that the message-driven between each business module provides infrastructural support;Disappearing in message queue Content sources are ceased in each business module, and each business module is sent a message in the dedicated tunnel of message managing module, is adjusted Message communicating is completed with corresponding message middleware;
Structural data synchronization module includes log collection analytical tool CANAL and several Node nodes;Log collection parsing Tool CANAL is deployed in the synchronous originating end of data;Each Node node include data access, data extract, data conversion and Data are loaded into four-stage;The data that structural data synchronization module obtains types of databases by the way of Incremental Log become Change, realizes that the data in Various types of data source are synchronous by simulation SETL data warehousing process;Meanwhile file is as certain in database The attachment of data record, synchronizes when tables of data changes with data;
Unstructured data transmission module includes being deployed in the CTRANS transmission service and third party's memory module of server-side;It utilizes CTRANS transmission service based on udp protocol realize high efficiency of transmission of more category image resources between different platform, bulk transfer and Breakpoint transmission service.
2. the integrated system according to claim 1 towards multi-source heterogeneous geographic information resources, it is characterised in that: services set At module can also provide on demand starting server, agency distribution, asynchronous event distribution, application configuration and application patch upgrading clothes Business.
3. the integrated system according to claim 1 towards multi-source heterogeneous geographic information resources, it is characterised in that: business mould Block includes order status change, job state change, plan is assigned or abnormal alarm.
4. the integrated system according to claim 1 towards multi-source heterogeneous geographic information resources, it is characterised in that: structuring Various types of data source involved in data simultaneous module includes mysql, oracle, media and Hbase.
5. a kind of integrated approach towards multi-source heterogeneous geographic information resources, it is characterised in that: GIS-Geographic Information System passes through right It is required that the 1 integrated system realization integrates all kinds of geographic information resources;Client can be called integrated by the corresponding interface All kinds of resources afterwards;The integrated approach of geographic information resources is that Services Integration module is integrated, message managing module is integrated, structuring Data simultaneous module is integrated and unstructured data transmission module it is integrated in one or more combination.
6. the integrated approach according to claim 5 towards multi-source heterogeneous geographic information resources, it is characterised in that: using clothes Business integration module carries out the integrated method of geographic information resources, includes the following steps:
Step F1: server-side defines Slice interface description file and compiles the interface document for generating corresponding language, and server-side is corresponding Service realizes function after birdsmouth;
Step F2: client use Slice interface description file identical with server-side in step F1 simultaneously compiles the corresponding language of generation The interface document of speech, client inherit calling interface and realize that service object calls;
Step F3: service definition ICE target adapter binds service object and target adapter that step F2 is called, Complete service publication;
Step F4: the step F3 service issued and IceNode are deployed in each server using service deployment tool;
Step F5: configuring and starts registration center;
Step F6: carrying out service registration, configures application.xml according to service implementation and is loaded into registration center;
Step F7: host and information on services are checked by the interface UI, and service is started, is stopped operation;
Step F8: client configures registration center address;
Step F9: when client call service, request query service terminal point information is sent to registration center;
Step F10: client establishes connection using IceNode where terminal point information and server-side;
Service invocation request is sent to respective service by step F11:IceNode;
Step F12: service is completed to return result to client by IceNode after responding.
7. the integrated approach according to claim 6 towards multi-source heterogeneous geographic information resources, it is characterised in that: the step In rapid F6, service implementation is using JBOSS as service container.
8. the integrated approach according to claim 5 towards multi-source heterogeneous geographic information resources, it is characterised in that: use and disappear It ceases management module and carries out the integrated method of geographic information resources, include the following steps:
Step X1: various types of messages queue is configured in server-side;
Step X2: business module creation link factory;
Step X3: business module is using point-to-point or publish/subscribe mode creation connection and session;
Step X4: creation message sends the message of creation to the destination of message server;
Step X5: message server forwards the message to message recipient or message subscribing person;
Step X6: connection is closed.
9. the integrated approach according to claim 5 towards multi-source heterogeneous geographic information resources, it is characterised in that: using knot Structure data simultaneous module carries out the integrated method of geographic information resources, includes the following steps:
Step T1: deployment log collection analytical tool CANAL and Node node;
Step T2: disposition data source, the data source for configuring completion may act as initiating synchronous source, can also be used as synchronization Target database;
Step T3: configuration target database table is filled in and needs synchronous data table information;
Step T4: configuration log collects analytical tool CANAL, according to service function, selects from the data source that step T2 is configured It needs to obtain the data source of log, then disposition data source account, password and character set, initializes log acquisition position;
Step T5: channel is configured, establishes a channel between every two data source, every channel may include two The pipeline of a different directions, each pipeline are single direction channel;
Step T6: configuration pipeline selects Node node and log collection analytical tool CANAL;
Step T7: configuration relation mapping table selects source data table and step T3 in the data source configured in step T2 to configure Target database table;
Step T8: starting channel, it is synchronous to start data, and when source data table data variation can be synchronized to mesh according to mapping ruler Mark tables of data.
10. the integrated approach according to claim 5 towards multi-source heterogeneous geographic information resources, it is characterised in that: use Unstructured data transmission module carries out the integrated method of geographic information resources, includes the following steps:
Step C1: " CTRANS_IP " is configured in server-side configuration file and is attached with third party's memory module, more product are configured Class data download address " DOWNLOAD_PATH ";
Step C2: more category data are issued in service, are called the more category data entities of CTRANS transmission service acquisition and are stored in third Square memory module;
Step C3:CTRANS transmission service makes seed according to the storage condition of more category data entities, comprising from the in seed Tripartite stores the relevant information for obtaining more category data entities;
Step C4: service selects corresponding message queue to complete the publication of seed, transmission by message managing module;
Step C5: recipient obtains the seed message of service publication by message queue and seed information is transferred to recipient institute CTRANS transmit service;
Step C6: recipient calls CTRANS transmission service to obtain more category data entities, more categories from third party's memory module Data entity is stored in the specified path " DOWNLOAD_PATH ", and recipient can call more category data entities to carry out other Operation flow.
CN201710045358.XA 2017-01-22 2017-01-22 Integrated framework and integrated approach towards multi-source heterogeneous geographic information resources Active CN106850788B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710045358.XA CN106850788B (en) 2017-01-22 2017-01-22 Integrated framework and integrated approach towards multi-source heterogeneous geographic information resources

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710045358.XA CN106850788B (en) 2017-01-22 2017-01-22 Integrated framework and integrated approach towards multi-source heterogeneous geographic information resources

Publications (2)

Publication Number Publication Date
CN106850788A CN106850788A (en) 2017-06-13
CN106850788B true CN106850788B (en) 2019-08-20

Family

ID=59119539

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710045358.XA Active CN106850788B (en) 2017-01-22 2017-01-22 Integrated framework and integrated approach towards multi-source heterogeneous geographic information resources

Country Status (1)

Country Link
CN (1) CN106850788B (en)

Families Citing this family (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107609834A (en) * 2017-07-21 2018-01-19 东莞新奥燃气有限公司 project management system and project management method
CN107526786A (en) * 2017-08-01 2017-12-29 江苏速度信息科技股份有限公司 The method and system that place name address date based on multi-source data is integrated
CN107832903A (en) * 2017-08-28 2018-03-23 中国石油化工股份有限公司 The integrated method of application system modularization
CN108052399A (en) * 2017-12-14 2018-05-18 郑州云海信息技术有限公司 Method for message transmission and system between a kind of program module
CN107948309A (en) * 2017-12-15 2018-04-20 神思电子技术股份有限公司 A kind of integrated management method and system of the server resource based on Restful API
CN109962957B (en) * 2017-12-26 2022-04-05 珠海全志科技股份有限公司 Data synchronization method and device for multi-region multi-active service
CN108170846B (en) * 2018-01-17 2018-12-11 清华大学 Extensive multi-source heterogeneous data persistence method based on elastic table model
CN113407633A (en) * 2018-09-13 2021-09-17 华东交通大学 Distributed data source heterogeneous synchronization method
CN109522386B (en) * 2018-12-25 2020-10-20 武汉众智鸿图科技有限公司 Method and system for generating spatial information service across GIS platform
CN109684435B (en) * 2019-01-11 2019-09-10 长沙市规划勘测设计研究院 Specialized multi-source heterogeneous space-time geodata is gone to integrate system and method
CN111435354A (en) * 2019-01-14 2020-07-21 北京京东尚科信息技术有限公司 Data export method and device, storage medium and electronic equipment
CN110297871A (en) * 2019-06-20 2019-10-01 常州冰鉴信息科技有限公司 A kind of method that isomeric data acquires in real time
CN111988315A (en) * 2020-08-19 2020-11-24 青岛易来智能科技股份有限公司 Synchronous calling method, device, electronic equipment, system and storage medium
CN112487070A (en) * 2020-10-30 2021-03-12 中科星图股份有限公司 Access integration device and method for multi-source heterogeneous data
CN112506630A (en) * 2020-12-01 2021-03-16 国科晋云技术有限公司 Multi-domain service platform architecture
CN113032598B (en) * 2021-04-12 2023-07-14 郑州航空工业管理学院 Image design visual conveying system based on big data
CN114138258B (en) * 2021-10-13 2024-07-23 同济大学 Complex software coupling modeling method

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101976199A (en) * 2010-10-15 2011-02-16 德讯科技股份有限公司 Object-oriented middleware distributed system development platform
CN102684930A (en) * 2012-06-08 2012-09-19 山东科汇电力自动化有限公司 Distributed application high-reliability realization mechanism and operation method based on in-circuit emulator (ICE)
CN102750147A (en) * 2012-06-08 2012-10-24 山东科汇电力自动化有限公司 Internet communications engine (ICE) middleware based distributed application management framework and operation method

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101976199A (en) * 2010-10-15 2011-02-16 德讯科技股份有限公司 Object-oriented middleware distributed system development platform
CN102684930A (en) * 2012-06-08 2012-09-19 山东科汇电力自动化有限公司 Distributed application high-reliability realization mechanism and operation method based on in-circuit emulator (ICE)
CN102750147A (en) * 2012-06-08 2012-10-24 山东科汇电力自动化有限公司 Internet communications engine (ICE) middleware based distributed application management framework and operation method

Also Published As

Publication number Publication date
CN106850788A (en) 2017-06-13

Similar Documents

Publication Publication Date Title
CN106850788B (en) Integrated framework and integrated approach towards multi-source heterogeneous geographic information resources
CN107301048B (en) Internal control management system of application response type shared application architecture
CN107172187B (en) A kind of SiteServer LBS and method
CN105809356A (en) Information system resource management method based on application integrated cloud platform
CN109542865A (en) Distributed cluster system configuration file synchronous method, device, system and medium
CN103617255B (en) A kind of business datum for power information system exchanges and synchronizes system and method
CN107169069B (en) Distributed hierarchical extraction multi-application method and data extraction applicator
CN110262902A (en) Information processing method and system, medium and calculating equipment
CN110213156A (en) A kind of span centre heart group's instant communicating method and system
CN113794652B (en) Data processing method, device, electronic equipment and storage medium
CN109246220A (en) Message pushing system and method
US8924530B2 (en) Multi-phase monitoring of hybrid system landscapes
CN101834765B (en) System and method for remote file transfer
JP2009153126A (en) System, apparatus and method for synchronizing presence information with schedule information
CN110413677A (en) A kind of Distributed Data Synchronization method and system for supporting concurrently to apply
CN112711625B (en) Bidirectional self-adaptive multi-source heterogeneous big data dynamic processing method
CN103685485A (en) Mobile terminal VoIP (voice over internet protocol) communication audio management system
CN112351106A (en) Service grid platform containing event grid and communication method thereof
CN103152428A (en) Method for carrying out service communication among nodes on cloud platform
CN107632991A (en) A kind of public material database method for building up and system
CN103713903A (en) Hard disk dynamic loading method and device
CN107786661B (en) Information synchronization method
CN102938723A (en) Microblog aggregation method and system based on group number
CN110555201B (en) Knowledge document generation method, device, electronic equipment and storage medium
CN113765958B (en) Job task processing method and job client

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant