CN107967166A - Remote sensing image processing service flow implementation method under a kind of cloud environment - Google Patents
Remote sensing image processing service flow implementation method under a kind of cloud environment Download PDFInfo
- Publication number
- CN107967166A CN107967166A CN201710971582.1A CN201710971582A CN107967166A CN 107967166 A CN107967166 A CN 107967166A CN 201710971582 A CN201710971582 A CN 201710971582A CN 107967166 A CN107967166 A CN 107967166A
- Authority
- CN
- China
- Prior art keywords
- remote sensing
- image processing
- sensing image
- virtual machine
- task
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/45562—Creating, deleting, cloning virtual machine instances
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/4557—Distribution of virtual machine instances; Migration and load balancing
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Facsimiles In General (AREA)
Abstract
The present invention relates to field of remote sensing image processing, more particularly, to remote sensing image processing service flow implementation method under a kind of cloud environment, includes following steps：Remote sensing images image file is uploaded in Openstack, creates the cluster virtual machine example of different remote sensing images collaboration processing under cloud environment in Openstack with this；The close transmission for logging in, realizing remote sensing images intermediate data between different virtual machine is exempted from by SSH between different virtual machine；Remote sensing image processing service flow dispatches system after the remote sensing image processing service flow job that Web client receives remote sensing image processing user structure, and task is distributed on different virtual machines and carries out dispatch deal by Telemetry Service BPMN flows resolver, Jenkins REST API and openstack rest API.The present invention is isolated by virtualizing, and is realized that multiple remote sensing image processing algorithms are run in same physical machine, is improved the resource utilization of physical machine multinuclear；By building cluster virtual machine on more physical machine servers, carry out realizing that the remote sensing task cooperation on Virtual Cluster is handled by REST API modes, so as to form the cooperative scheduling service flow under a cloud environment.
Description
Technical field
The present invention relates to field of remote sensing image processing, more particularly, to remote sensing image processing service under a kind of cloud environment
Flow implementation method.
Background technology
In order to realize that the efficient of remote sensing resources utilizes, first have in the case of limited resources so that remote sensing image processing
Calculate handling capacity and reach maximum.In the prior art, since the different disposal step of remote sensing image processing algorithm flow and processing are appointed
Business is under many circumstances using serial processing mode, therefore remote sensing image processing engineer does not have when designing remote sensing algorithm
Have and remove design remote sensing image processing algorithm using the mode of multinuclear.This design method is necessarily caused in operation remote sensing image processing
During algorithm, treatment facility only has a monokaryon being handled, and reduces hardware resource utilization, is unfavorable for the height of remote sensing resources
Effect utilizes.
The content of the invention
The present invention is at least one defect overcome described in the above-mentioned prior art, there is provided under a kind of cloud environment at remote sensing images
Service flow implementation method is managed, the resource utilization of physical machine multinuclear is provided by virtualizing means, while is isolated by virtualizing,
Realize that multiple remote sensing image processing algorithms are run in same physical machine, by carrying out task scheduling point to distributed virtual machine
Hair, each task run is on different virtual machine, and then between whole cluster virtual machine, at the collaboration that carries out remote sensing task
Reason, forms the cooperative scheduling service flow under a cloud environment.
In order to solve the above technical problems, the technical solution adopted by the present invention is：Remote sensing image processing takes under a kind of cloud environment
Business stream implementation method, includes following steps：
S1：Remote sensing image processing software and remote sensing algorithm are configured on a virtual machine, by the remote sensing images mirror image text of virtual machine processing
Part is uploaded in Openstack mirror images and managed；Build http and download network address, and remotely-sensed data is classified and is preserved；
S2：Different remote sensing images collaboration processing are respectively created under cloud environment according to remote sensing images image file in Openstack
Cluster virtual machine example；Realize that the virtual machine of remote sensing image processing exempts from cryptographic acess in Openstack, realize different virtual
The mutual transmission of remote sensing images intermediate data after being handled between machine；
S3：Build and realize Jenkins system of the dynamic expansion from node, the virtual machine under Openstack is passed through into node label
It is marked；To establishment task and the triggering mode of each task is defined in Jenkins, appointed task operation node, and structure
It build on present node and uses the good configuration information of node environment variable-definition, and writes out present node operation and distribute holding for task
Travel far and wide this.
S4：Virtual machine manager is built, the remote sensing image processing node in Jenkins is carried out by virtual machine manager
Management；The BPMN flow resolvers of BPMN XML files can be parsed by creating at the same time, be parsed by BPMN flows resolver different
Remote sensing processing service flow between front and rear continuous dependence execution relation perform relation with parallel；Create remote sensing image processing service flow
Scheduling system, carries out remote sensing image processing service flow by the way of multi-process the processing of task, is parsed according to BPMN flows
Device parses remote sensing image processing running environment parameter information, is sent to remote sensing image processing virtual machine manager and meets running environment
Node query requests；
S5：After remote sensing image processing service flow scheduling system receives virtual machine manager query result, new process is initiated, new
In process dispatch deal is carried out by calling Jenkins rest api interfaces that task is distributed on different virtual machines；
S6：After virtual machine receives an assignment, if task proceeds by processing from initial data, classify from the remotely-sensed data of establishment and protect
Deposit terminal and obtain data；If task proceeds by processing from pilot process, intermediate processing data is obtained from precursor dependence.
In one embodiment, in step sl, when being configured to virtual machine, virtual machine configuration SSH is serviced, is led to
Cross the mutual access that virtual machine is realized in SSH services；Ubuntu image files are selected to create from Openstack Dashboard
Original processing data are carried out classification preservation by http download servers.
By creating http download servers, remotely-sensed data is classified according to particular requirement.Installation and configuration in system
Good remote sensing image processing software and remote sensing algorithm carry out adaptability configuration to realize to plan machine, will pass through remote sensing made of virtual machine
Image specialty image file is uploaded in Openstack.
In one embodiment, in step s 2, by associating Floating IP address for different cluster virtual machine examples, realize
SSH between virtual machine is accessed, and forms a remote sensing images collaboration processing cluster virtual machine.
In Openstack, required mirror image text is selected in the remote sensing image processing specialty image file that user uploads
Part, is respectively created the cluster virtual machine example of different remote sensing images collaboration processing under cloud environment, different instances association Floating IP address, structure
Into a remote sensing images collaboration processing cluster virtual machine.
Preferably, in step s 2, the mutual access between different virtual machine is realized by SSH；Using SCP telecopies
Order coordinates with SSH, realizes the transmission remote sensing image processing intermediate data between different virtual machine, realizes that remote sensing image processing takes
The Intranet flowing of data is handled between business stream；
The SSH public keys obtained according to virtual machine in Openstack by Metadata Service, provide and access needed for the virtual machine
Private key, obtains mutually accessing required private key between different virtual machine.Using the order of SCP telecopies plus private key as parameter
Mode, remote sensing image processing intermediate data is transmitted between different virtual machine, realizes and locates between remote sensing image processing service flow
Manage the Intranet flowing of data.
In one embodiment, in step s3, Ubuntu operating system conducts are selected from Openstack
The master nodes of Jenkins, are all added remote sensing image processing virtual machine by the slave node addition manner in node administration
Unified management is got up in Jenkins systems；It will be marked from the remote sensing image processing virtual machine in node by label；
Preferably, in step s3, created in Jenkins according to the flow of different remote sensing image processings at user's remote sensing images
The different task of service flow is managed, the task triggering mode is defined and determines that present node runs the perform script of the task.
The different task of user's remote sensing image processing service flow is created in Jenkins, defines the triggering side of each task
Formula, according to the difference of remote sensing image processing software and environment needed for current task, selects it to run node, and build and working as prosthomere
The good configuration information of node environment variable-definition is used on point, and writes out the perform script that present node operation distributes task.
In one embodiment, in step s 4, the virtual machine management system of a remote sensing image processing is built, it is described
Management system realizes that the remote sensing image processing node in Openstack such as is inquired about, added, deleted, configured at the management.
Preferably, in step s 4, remote sensing image processing service flow scheduling system constituting method is as follows：
S41：Scheduling system reads nodal information and the dependency information that Telemetry Service BPMN flow resolvers parse, structure
Go out a business processing flow and perform tree；
S42：The nodal information parsed according to resolver builds task system, gives each node to build a process, should be into
Cheng Faqi remote sensing image processing virtual machines performing tasks；
S43：The remote sensing image processing for the present node that scheduling system call is parsed according to remote sensing image processing resolution system
Running environment parameter information, the node query requests for meeting running environment are sent to remote sensing image processing virtual machine management system；
S44：After scheduling system call receives the nodal information of virtual machine management system return, the IP address of nodal information is believed
The parameter that the information such as breath, account, password and port are performed as remote sensing image processing task scheduling；
S45：Scheduling system performs the frontier juncture system in tree between leaf node according to flow, between structure task parallel with it is serial
Execution order.
In one embodiment, in step s 5, finger daemon is built, passes through the finger daemon monitoring and dispatching system
The task status of system task pool, and task is distributed on different virtual machines by the finger daemon and carries out dispatch deal.
Pass through the shapes such as the preparation of the task in finger daemon monitoring and dispatching system, hang-up, solution extension, ready, execution, stopping
State.When finger daemon performs task, task is distributed on different virtual machines and carries out dispatch deal.
In one embodiment, in step s 6, virtual machine performs distant according to the perform script configured in Openstack
Feel image processing operations, if task obtains intermediate processing data from precursor dependence, will be passed to according to the machine of transferring in task
Predecessor node IP address information from predecessor node by intermediate result data copy come to be performed.
Beneficial effects of the present invention are：
1st, the function that the present invention is provided based on Openstack, has carried out some configurations towards remote sensing image processing professional domain,
Enable the scheduling of remote sensing image processing service flow that openstack supported under cloud environment, reduce remote sensing personnel to using
Cloud environment builds the learning cost of remote sensing image processing service flow.
2nd, this dispatching method by developing the virtual machine manager of remote sensing image processing, Telemetry Service BPMN flows parse
Device, remote sensing image processing service flow scheduling system module, realize the collaboration of the remote sensing image processing service flow under cluster virtual machine
Scheduling.Since the collocation method of the remote sensing algorithm in different platform and remote sensing processing method have been integrated into remote sensing by designer respectively
Image processing services stream dispatches system and Jenkins integrated systems, thus uses the user of system caused by patent of the present invention
It need not again go to understand the configuration of each remote sensing image processing software runtime environment, the component relied on and operation bag, reduce remote sensing
The learning cost of user of service.
3rd, by the way of the present invention is using openstack operation virtual machines, realize on physical server memory and hard disk it
Between occupation mode virtualization isolation, improve the handling capacity of single server, it is less to examine when solving due to programmers design
Consider the isolation of multiprogramming run-time memory, can cause, when running multiple remote sensing image processing algorithms in a physical machine, to deposit
The problem of conflicting, causing operation failure.
When the 4th, going scheduling remote sensing image processing using server using the present invention, since openstack bottoms have invoked void
Planization technology, thus more nuclear superiority of server can be made full use of to go to carry out remote sensing image processing service flow parallel computation,
Improve the utilization rate of resource.
Brief description of the drawings
Fig. 1 is present invention method flow schematic diagram in one embodiment.
Embodiment
Attached drawing is only for illustration, it is impossible to is interpreted as limitation of the present invention；It is attached in order to more preferably illustrate the present embodiment
Scheme some components to have omission, zoom in or out, do not represent the size of actual product；To those skilled in the art,
Some known features and its explanation may be omitted and will be understood by attached drawing.Being given for example only property of position relationship described in attached drawing
Explanation, it is impossible to be interpreted as limitation of the present invention.
Embodiment 1：
The present invention provides remote sensing image processing service flow implementation method under a kind of cloud environment, wherein, include following steps：
S1：Remote sensing image processing software and remote sensing algorithm are configured on a virtual machine, by the remote sensing images mirror image text of virtual machine processing
Part is uploaded in Openstack mirror images and managed；Build http and download network address, and remotely-sensed data is classified and is preserved；
S2：Different remote sensing images collaboration processing are respectively created under cloud environment according to remote sensing images image file in Openstack
Cluster virtual machine example；Realize that the virtual machine of remote sensing image processing exempts from cryptographic acess in Openstack, realize different virtual
The mutual transmission of remote sensing images intermediate data after being handled between machine；
S3：Build and realize Jenkins system of the dynamic expansion from node, the virtual machine under Openstack is passed through into node label
It is marked；To establishment task and the triggering mode of each task is defined in Jenkins, appointed task operation node, and structure
It build on present node and uses the good configuration information of node environment variable-definition, and writes out present node operation and distribute holding for task
Travel far and wide this.
S4：Virtual machine manager is built, the remote sensing image processing node in Jenkins is carried out by virtual machine manager
Management；The BPMN flow resolvers of BPMN XML files can be parsed by creating at the same time, be parsed by BPMN flows resolver different
Remote sensing processing service flow between front and rear continuous dependence execution relation perform relation with parallel；Create remote sensing image processing service flow
Scheduling system, carries out remote sensing image processing service flow by the way of multi-process the processing of task, is parsed according to BPMN flows
Device parses remote sensing image processing running environment parameter information, is sent to remote sensing image processing virtual machine manager and meets running environment
Node query requests；
S5：After remote sensing image processing service flow scheduling system receives virtual machine manager query result, new process is initiated, new
In process dispatch deal is carried out by calling Jenkins rest api interfaces that task is distributed on different virtual machines；
S6：After virtual machine receives an assignment, if task proceeds by processing from initial data, classify from the remotely-sensed data of establishment and protect
Deposit terminal and obtain data；If task proceeds by processing from pilot process, intermediate processing data is obtained from precursor dependence.
In step sl, following steps are included：
Step S11：User installs and has configured remote sensing image processing software and remote sensing algorithm on VME operating system, and will
Native virtual machine is fabricated to the professional image file of remote sensing image processing, and uploads in Openstack mirror images and manage；
Step S12：One http of structure downloads network address, and remotely-sensed data is classified according to space and temporal information,
Processing data needed for being provided to remote sensing image processing algorithm.
In the present embodiment, in step sl, user installs and has configured remote sensing image processing on VME operating system
Software and remote sensing algorithm, and native virtual machine is fabricated to the professional image file of remote sensing image processing, and upload to
It is managed in Openstack.
In step s 2, following steps are included：
Step S21：According to the remote sensing image processing specialty image file of making in Openstack, it is respectively created under cloud environment
The cluster virtual machine example of different remote sensing images collaboration processing；
Step S22：In the same users of Openstack carry out remote sensing image processing virtual machine between, realize virtual machine it
Between exempt from password and mutually access, easy to transmit processed remote sensing images intermediate data between different virtual machine.
In the present embodiment, in the step s 21, include following steps：
S211：Openstack Dashboard are logged in, are selected in the remote sensing image processing specialty image file uploaded according to user
Required image file, creates the different running examples needed for remote sensing image processing service flow；
S212：Associate Floating IP address to different example, different remote sensing image processing virtual machines by this IP, realize its virtual machine it
Between SSH access, remote sensing images collaboration processing cluster virtual machine is formed with this.
In the present embodiment, in step S22：
S221：The SSH public keys obtained according to virtual machine from Openstack Metadata Services, provide and access needed for the virtual machine
Private key, required private key is mutually accessed between realizing different virtual machine with this.
S222：By the way of the order of SCP telecopies adds private key as parameter, transmitted between different virtual machine distant
Feel image procossing intermediate data, realize the Intranet flowing of processing data between remote sensing image processing service flow.
In step s3, following steps are included：
S31：Jenkins system that can be by dynamic expansion from node is built, belonging under the same users of Openstack
The virtual machine of Linux and Windows environment is according to the difference of its remote sensing image processing professional software installed, using different
Node label is marked；
S32：To the different task for creating user's remote sensing image processing service flow in Jenkins, the triggering of each task is defined
Mode, according to the difference of remote sensing image processing software and environment needed for current task, specifies it to run node, and build current
The good configuration information of node environment variable-definition is used on node, and writes out the perform script that present node operation distributes task.
In the present embodiment, node dynamic expansion system elects Jenkins systems as.
In the present embodiment, in step S31：
S311：The Ubuntu operating systems without dress remote sensing image processing special-purpose software are selected from Openstack mirror images,
As the master nodes of Jenkins, set in its global configuration between main and subordinate node by way of account/password into
Row SSH is accessed, and remote sensing image processing virtual machine is all added Jenkins systems by the slave node addition manner in node administration
Unified management is got up in system；
S312：It will be classified from the remote sensing image processing virtual machine in node according to its purposes, stamp different labels, the mark
Node purposes ID number when label are used to distribute Job, the ID number be remote sensing image processing service flow dispatch system assignments task to
Keyword with node.
In the present embodiment, in step s 32：
S321：According to the flow of remote sensing image processing, one each remote sensing image processing step is abstracted as in Jenkins
Task, defines the mode of task triggering；
S322：The difference of the remote sensing image processing special-purpose software of required by task in remote sensing image processing flow and want
The difference for the running environment asked, the label for meeting condition is selected from tally set, is somebody's turn to do so that system enforcement engine randomly chooses out
Idle node operation current task under label；
S323：Select script to perform configuration interface in task structure configuration, fill in current task and perform required script letter
Breath and configuration information, write out the script dispatched and remote sensing professional treatment software processing remotely-sensed data is performed in the node.
In step s 4, following steps are included：
S41：Build a remote sensing image processing virtual machine manager, the manager can be to the remote sensing images in Jenkins at
Reason node, which is inquired about, added, deleted, configure etc., to be managed.
S42：Design a business processing flow BPMN XML that can parse relation between remote sensing image processing service flow
The Telemetry Service BPMN flow resolvers of file, the resolver parse the front and rear continuous dependence between different remote sensing processing service flows
Execution relation performs relation with parallel.
S43：Remote sensing image processing service flow scheduling system is designed, to remote sensing image processing service by the way of multi-process
The processing of carry out task is flowed, remote sensing image processing service flow scheduling system parses distant according to remote sensing image processing resolver
Feel image procossing running environment parameter information, sent to remote sensing image processing virtual machine manager and meet that the node of running environment is looked into
Ask request.
In step S41：
S411：The virtual machine manager of remote sensing image processing is write using python, the manager is by calling openstack
Rest api interfaces, inquire about the relevant configuration information of virtual machine instance；
S412：The image file according to used in calling openstack rest API to inquire current virtual machine realization, judges
Go out the remote sensing professional image software that current virtual machine is run；
S413：The virtual machine manager of remote sensing image processing calls openstack rest API, externally provides distant under cloud environment
Feel inquiry, addition, deletion, the configuration operation needed for image processing services stream.
In step S42：
S421：The remote sensing image processing operation flow BPMN XML files of input are carried out according to BPMN specifications using Java language
Parsing, parses nodal information and procedure information；
S422：According to the nodal information and flow Dependency Specification parsed, the task scheduling of structure remote sensing image processing service flow
Data structure and tasks carrying stream parallel with serial scheduling information.
In step S43：
Remote sensing image processing scheduling system constituting method is as follows：
S431：Scheduling system reads nodal information and the dependency information that Telemetry Service BPMN flow resolvers parse, structure
Build out a business processing flow and perform tree；
S432：The nodal information parsed according to resolver builds task system, gives each node to build a process, should be into
Cheng Faqi remote sensing image processing virtual machines performing tasks；
S433：The remote sensing image processing fortune for the present node that scheduling system call is parsed according to remote sensing image processing resolver
Row ambient parameter information, the node query requests for meeting running environment are sent to remote sensing image processing virtual machine manager；
S434：After scheduling system call receives the nodal information of virtual machine manager return, the IP address of nodal information is believed
The parameter that the information such as breath, account, password and port are performed as remote sensing image processing task scheduling；
S435：Scheduling system performs the frontier juncture system in tree between leaf node according to flow, between structure task parallel with serially
Execution order.
In step s 5, following steps are included：
S51：Finger daemon is built, the task of the task pool in the finger daemon monitoring and dispatching system prepares, hangs up, solution is hung, just
Thread, execution, halted state；
S52：, will be by calling Jenkins rest api interfaces to be distributed to task when finger daemon performs task in task pool
Dispatch deal is carried out on different virtual machines.
In step s 6, following steps are included：
S61：Remote sensing image processing virtual machine belongs to the slave node of Jenkins, when Jenkins host nodes are dispatched to node work
When, it performs remote sensing image processing according to the perform script configured in Jenkins tasks and operates；
S62：When the remote sensing image processing job in Jenkins is handled from initial data, then from the remotely-sensed data built
Http downloads network address and obtains data；
S63：When remote sensing image processing step obtains intermediate processing data from its precursor dependence, it will be according to transferring machine transmission
Intermediate result data copy is come to perform from predecessor node to the predecessor node IP address information in task.
Obviously, the above embodiment of the present invention is only intended to clearly illustrate example of the present invention, and is not pair
The restriction of embodiments of the present invention.For those of ordinary skill in the field, may be used also on the basis of the above description
To make other variations or changes in different ways.There is no necessity and possibility to exhaust all the enbodiments.It is all this
All any modification, equivalent and improvement made within the spirit and principle of invention etc., should be included in the claims in the present invention
Protection domain within.
Claims (10)
1. remote sensing image processing service flow implementation method under a kind of cloud environment, it is characterised in that include following steps：
S1：Remote sensing image processing software and remote sensing algorithm are configured on a virtual machine, by the remote sensing images mirror image text of virtual machine processing
Part is uploaded in Openstack mirror images and managed；Build http and download network address, and remotely-sensed data is classified and is preserved；
S2：Different remote sensing images collaboration processing are respectively created under cloud environment according to remote sensing images image file in Openstack
Cluster virtual machine example；Realize that the virtual machine of remote sensing image processing exempts from cryptographic acess in Openstack, realize different virtual
The mutual transmission of remote sensing images intermediate data after being handled between machine；
S3：Build and realize Jenkins system of the dynamic expansion from node, the virtual machine under Openstack is passed through into node label
It is marked；To establishment task and the triggering mode of each task is defined in Jenkins, appointed task operation node, and structure
It build on present node and uses the good configuration information of node environment variable-definition, and writes out present node operation and distribute holding for task
Travel far and wide this；
S4：Virtual machine manager is built, pipe is carried out to the remote sensing image processing node in Jenkins by virtual machine manager
Reason；The BPMN flow resolvers of BPMN XML files can be parsed by creating at the same time, be parsed by BPMN flows resolver different
Front and rear continuous dependence execution relation between remote sensing processing service flow performs relation with parallel；Create remote sensing image processing service flow tune
Degree system, carries out remote sensing image processing service flow by the way of multi-process the processing of task, according to BPMN flow resolvers
Remote sensing image processing running environment parameter information is parsed, is sent to remote sensing image processing virtual machine manager and meets running environment
Node query requests；
S5：After remote sensing image processing service flow scheduling system receives virtual machine manager query result, new process is initiated, new
In process dispatch deal is carried out by calling Jenkins rest api interfaces that task is distributed on different virtual machines；
S6：After virtual machine receives an assignment, if task proceeds by processing from initial data, classify from the remotely-sensed data of establishment and protect
Deposit terminal and obtain data；If task proceeds by processing from pilot process, intermediate processing data is obtained from precursor dependence.
2. remote sensing image processing service flow implementation method according to claim 1, it is characterised in that in step sl, right
When virtual machine is configured, virtual machine configuration SSH is serviced, realizes that the mutual of virtual machine accesses by SSH services；From
Select ubuntu image files to create http download servers in Openstack Dashboard, original processing data are carried out
Classification preserves.
3. remote sensing image processing service flow implementation method according to claim 1, it is characterised in that in step s 2, lead to
Cross and associate Floating IP address for different cluster virtual machine examples, realize that the SSH between virtual machine is accessed, form a remote sensing images association
With processing cluster virtual machine.
4. remote sensing image processing service flow implementation method according to claim 3, it is characterised in that in step s 2, lead to
Cross SSH and realize mutual access between different virtual machine；Coordinated using the order of SCP telecopies and SSH, realized different virtual
Remote sensing image processing intermediate data is transmitted between machine, realizes the Intranet flowing of processing data between remote sensing image processing service flow.
5. remote sensing image processing service flow implementation method according to claim 1, it is characterised in that in step s3, from
Master node of the Ubuntu operating systems as Jenkins is selected in Openstack, passes through the slave node in node administration
Remote sensing image processing virtual machine is all added in Jenkins systems and is managed collectively by addition manner；By from the remote sensing in node
Image procossing virtual machine is marked by label.
6. remote sensing image processing service flow implementation method according to claim 5, it is characterised in that in step s3,
The different task of user's remote sensing image processing service flow, definition are created in Jenkins according to the flow of different remote sensing image processings
The task triggering mode simultaneously determines that present node runs the perform script of the task.
7. remote sensing image processing service flow implementation method according to claim 1, it is characterised in that in step s 4, structure
The virtual machine management system of a remote sensing image processing is built, the management system realizes the remote sensing image processing in Openstack
Node such as is inquired about, is added, being deleted, being configured at the management.
8. remote sensing image processing service flow implementation method according to claim 7, it is characterised in that in step s 4, distant
It is as follows to feel image processing services stream scheduling system constituting method：
S41：A multithread pool is built first, this thread pool can be inquired about from remote sensing image processing virtual machine manager works as
Preceding virtual machine operation conditions, can also inquire about from Telemetry Service BPMN flow resolvers the nodal information parsed and be closed with relying on
It is information；
S42：The nodal information parsed according to resolver builds task system, gives each node to build a process, should be into
Cheng Faqi remote sensing image processing virtual machines performing tasks；
S43：The remote sensing image processing for the present node that scheduling system call is parsed according to remote sensing image processing resolution system
Running environment parameter information, the node query requests for meeting running environment are sent to remote sensing image processing virtual machine management system；
S44：After scheduling system call receives the nodal information of virtual machine management system return, the IP address of nodal information is believed
The parameter that the information such as breath, account, password and port are performed as remote sensing image processing task scheduling；
S45：Scheduling system performs the frontier juncture system in tree between leaf node according to flow, between structure task parallel with it is serial
Execution order.
9. remote sensing image processing service flow implementation method according to claim 1, it is characterised in that in step s 5, structure
Finger daemon is built, by the task status in the finger daemon monitoring and dispatching system task pond, and will by the finger daemon
Task, which is distributed on different virtual machines, carries out dispatch deal.
10. remote sensing image processing service flow implementation method according to claim 1, it is characterised in that in step s 6, empty
Plan machine performs remote sensing image processing operation according to the perform script configured in Openstack, if task is obtained from precursor dependence
During intermediate processing data, by the predecessor node IP address information passed to according to machine of transferring in task by centre from predecessor node
Result data copy comes to be performed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710971582.1A CN107967166A (en) | 2017-10-18 | 2017-10-18 | Remote sensing image processing service flow implementation method under a kind of cloud environment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710971582.1A CN107967166A (en) | 2017-10-18 | 2017-10-18 | Remote sensing image processing service flow implementation method under a kind of cloud environment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107967166A true CN107967166A (en) | 2018-04-27 |
Family
ID=61996759
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710971582.1A Pending CN107967166A (en) | 2017-10-18 | 2017-10-18 | Remote sensing image processing service flow implementation method under a kind of cloud environment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107967166A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108491259A (en) * | 2018-03-30 | 2018-09-04 | 北京航天宏图信息技术股份有限公司 | Remote sensing algorithm flow Method of Scheduling Parallel and device |
| CN111385598A (en) * | 2018-12-29 | 2020-07-07 | 富泰华工业（深圳）有限公司 | Cloud device, terminal device and image classification method |
| CN112597162A (en) * | 2020-12-25 | 2021-04-02 | 平安银行股份有限公司 | Data set acquisition method, system, device and storage medium |
| CN113887396A (en) * | 2021-09-29 | 2022-01-04 | 上海商汤智能科技有限公司 | Image processing method and device, computer equipment and storage medium |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103051714A (en) * | 2012-12-24 | 2013-04-17 | 河海大学 | Implementation method of water conservation cloud platform |
-
2017
- 2017-10-18 CN CN201710971582.1A patent/CN107967166A/en active Pending
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103051714A (en) * | 2012-12-24 | 2013-04-17 | 河海大学 | Implementation method of water conservation cloud platform |
Non-Patent Citations (1)
| Title |
|---|
| 阎继宁: "多数据中心架构下遥感云数据管理及产品生产关键技术研究", 《中国博士学位论文全文数据库信息科技辑》 * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108491259A (en) * | 2018-03-30 | 2018-09-04 | 北京航天宏图信息技术股份有限公司 | Remote sensing algorithm flow Method of Scheduling Parallel and device |
| CN111385598A (en) * | 2018-12-29 | 2020-07-07 | 富泰华工业（深圳）有限公司 | Cloud device, terminal device and image classification method |
| CN112597162A (en) * | 2020-12-25 | 2021-04-02 | 平安银行股份有限公司 | Data set acquisition method, system, device and storage medium |
| CN112597162B (en) * | 2020-12-25 | 2023-08-08 | 平安银行股份有限公司 | Data set acquisition method, system, equipment and storage medium |
| CN113887396A (en) * | 2021-09-29 | 2022-01-04 | 上海商汤智能科技有限公司 | Image processing method and device, computer equipment and storage medium |
| WO2023050745A1 (en) * | 2021-09-29 | 2023-04-06 | 上海商汤智能科技有限公司 | Image processing method and apparatus, device, medium, and program |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11182152B2 (en) | Methods and systems that share resources among multiple, interdependent release pipelines | |
| Sharma et al. | A complete survey on software architectural styles and patterns | |
| JP5277251B2 (en) | Model-based composite application platform | |
| CA3000422C (en) | Workflow service using state transfer | |
| CN107967166A (en) | Remote sensing image processing service flow implementation method under a kind of cloud environment | |
| CN112668386A (en) | Long running workflows for document processing using robotic process automation | |
| CN105765578B (en) | Parallel access of data in a distributed file system | |
| JP2005504455A (en) | Adaptive multi-protocol communication system | |
| WO2013063098A2 (en) | Federated, policy-driven service meshes for distributed software systems | |
| CA2802361A1 (en) | Method and system for workload distributing and processing across a network of replicated virtual machines | |
| JP2018133084A (en) | System for optimizing distribution of processing of automated process | |
| US11456914B2 (en) | Implementing affinity and anti-affinity with KUBERNETES | |
| US11595299B2 (en) | System and method of suppressing inbound payload to an integration flow of an orchestration based application integration | |
| CN109302321A (en) | Server, business demand processing system, method and monitoring system | |
| US11695840B2 (en) | Dynamically routing code for executing | |
| US11271895B1 (en) | Implementing advanced networking capabilities using helm charts | |
| US20070089107A1 (en) | Database communication method | |
| CN112256414A (en) | Method and system for connecting multiple computing storage engines | |
| CN113672240A (en) | Container-based multi-machine-room batch automatic deployment application method and system | |
| Wang et al. | Transformer: a new paradigm for building data-parallel programming models | |
| CN114510321A (en) | Resource scheduling method, related device and medium | |
| Bezirgiannis et al. | ABS: A high-level modeling language for cloud-aware programming | |
| WO2018188607A1 (en) | Stream processing method and device | |
| CN105577807A (en) | Cloud computing data resource scheduling WEB management platform | |
| US10417051B2 (en) | Synchronizing shared resources in an order processing environment using a synchronization component |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180427 |
|
| RJ01 | Rejection of invention patent application after publication |