CN108234607B - File distribution method and system based on micro-service architecture mode - Google Patents
File distribution method and system based on micro-service architecture mode Download PDFInfo
- Publication number
- CN108234607B CN108234607B CN201711367068.3A CN201711367068A CN108234607B CN 108234607 B CN108234607 B CN 108234607B CN 201711367068 A CN201711367068 A CN 201711367068A CN 108234607 B CN108234607 B CN 108234607B
- Authority
- CN
- China
- Prior art keywords
- file
- rule
- server
- module
- client
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/60—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources
Abstract
The invention discloses a file distribution method and a file distribution system based on a micro-service architecture mode, which are designed and developed by combining open source system software saltstack on the basis of a micro-service distributed architecture. The invention aims at a scene, the file copying is carried out between different network mapping disks (including a physical disk) of a single file server, the file servers exist in a cluster mode, no central server exists, all the servers are not related and not influenced mutually, and the scene cannot be well met by looking up the existing scheme. The scheme is used as a scene supplement for the current existing scheme, not only enriches the existing scheme, but also carries out optimization innovation on the architecture, adopts the current popular micro-service architecture, fully decouples the services, greatly improves the stability and reliability of the system, and enriches the existing file distribution system and method.
Description
Technical Field
The invention relates to a file distribution method and a file distribution system, in particular to a file distribution method and a file distribution system based on a micro-service architecture mode, and belongs to the technical field of automatic operation and maintenance application.
Background
The existing file distribution system and method basically adopt a star topology structure with a file server as a center, or perform file distribution and pushing from a centralized file server to each child node by means of a CDN node, focus is on selection of a distribution path, and have a great difference in structure from the file distribution method related to the system. The existing method lacks consideration of the situation of a network mapping disk file server, the architecture situation is ubiquitous in the field of security dealer, a high-efficiency solution is lacked at present, and higher labor time cost is consumed.
Moreover, the existing file servers are dispersed, under the condition that the file distribution rule is complex, the servers and the tasks are difficult to manage in a unified manner, and the distribution tasks are difficult to operate at high concurrency, high efficiency and stability, so that a file distribution method and a file distribution system based on a micro-service architecture mode are provided for solving the problems.
Disclosure of Invention
The invention aims to solve the problems and provide a file distribution method and a file distribution system based on a micro-service architecture mode, which are designed and developed by combining open source system software saltstack on the basis of a micro-service distributed architecture and have the characteristics of high efficiency, stability, convenience and easiness in use.
The invention realizes the aim through the following technical scheme, and a file distribution method based on a micro-service architecture mode comprises the following steps:
step A, establishing a rule, judging whether the rule check is legal or not, if so, performing an online rule, and if not, continuously establishing the rule;
b, judging the online rule in the step A, judging whether the rule can be online, if so, continuing to scan the file, and if not, returning to the online rule;
step C, judging whether the scanning file in the step B is found, if so, copying is executed, and if not, ending;
and D, reporting the state of the file copied in the step C, checking the result, and ending.
A file distribution system based on a micro-service architecture mode comprises a set server, a Client and a Portal interface, wherein the set server is connected with the Portal interface, and the set server is connected with a Client module; the server side comprises a RuleManager module, an API module and a master module, and the Api module is connected with the master module; the Client end comprises a Quartz module, a File-parameter module and an Executer module inside, and is connected with a RuleManager module, an API module and a master module inside the server end.
Preferably, the Server is a central control scheduling center of the system, is composed of a rule management module, a salt-api module and a saltmaster module, and is mainly responsible for scheduling and issuing rules;
the method has the following basic functions:
and (3) rule management: the rule management module is mainly responsible for creating, deleting, modifying, inquiring, online and offline rules, firstly, the legality of the rules is verified, and the rules can be issued to the client through online operation after being successfully verified;
and (3) authority management: the system controls the authority according to the principle of dividing the authority into domains, different users belong to different user groups, different machines have different authorities, and centralized management and attribute group division are carried out through an authority module;
parameter configuration: the parameter configuration module is mainly used for specific business services, users in different business scenes can perform custom configuration of rule parameters according to different business scenes, and the rule module generates corresponding rules according to the parameter configuration and sends the corresponding rules to the client.
Preferably, the Client terminal mainly comprises a scheduler, a scanner and an actuator, the scheduler starts a scheduling task according to the configuration of a rule after receiving the rule issued by the server terminal and transmits the rule to the file scanner, the file scanner returns a result to the server terminal after finding a corresponding file or folder, the server terminal issues the task to the actuator for execution according to the returned result, and after the execution of the actuator is completed, the execution result is reported to the server terminal, and the task is completed; the method comprises the following main functions:
1) scanning a file: searching and matching of files are supported according to file types and file name keywords, and deep-level traversal matching is supported for folders;
2) task scheduling: the single client supports multi-task concurrent execution, tasks are not affected mutually, and task scheduling supports concurrent execution of tasks of more than ten thousand orders of magnitude at most;
3) and (3) task execution: the task execution module is realized by means of saltstack mining, can support the copying of various files, and is efficient, stable and reliable;
a. a user clicks an online rule of an online button through an interface, and a server issues the rule to a corresponding client;
b. the client task scheduling module creates a corresponding scheduling task according to the rule;
c. scheduling a task to start a file scanner and scanning files according to a configuration rule;
d. the file scanner reports the searched file to the server through the corresponding service of the client, then the server issues the corresponding task to the salt-minus through the salt-master, and the minus returns the corresponding result to the server after the execution is completed.
A method and a system for distributing files based on a micro-service architecture mode are provided, a client establishes a copy state of a file under a corresponding scheduling task polling inquiry corresponding rule, the file copy state is refreshed according to a result returned by a server, the service adopts a mode of combining BS and CS architectures, the client is deployed on file servers, each file server is deployed with a client program, the server is provided with a background server and a salt-master server, and the client and the server transmit messages through a system bus.
The invention has the beneficial effects that: the invention aims at a scene, the file copying is carried out between different network mapping disks (including a physical disk) of a single file server, the file servers exist in a cluster mode, no central server exists, all the servers are not related and not influenced mutually, and the scene cannot be well met by looking up the existing scheme. The scheme is used as a scene supplement for the current existing scheme, not only enriches the existing scheme, but also carries out optimization innovation on the architecture, adopts the current popular micro-service architecture, fully decouples the services, greatly improves the stability and reliability of the system, and enriches the existing file distribution system and method.
Drawings
FIG. 1 is a flow chart of the present invention;
FIG. 2 is a system frame connection diagram of the present invention;
FIG. 3 is a data flow diagram of the present invention;
FIG. 4 is a schematic diagram of a Client terminal according to the present invention;
FIG. 5 is a schematic diagram of a Server terminal according to the present invention;
FIG. 6 is a schematic diagram of a deployment architecture of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1-6, a file distribution method based on micro service architecture mode includes the following steps:
step A, establishing a rule, judging whether the rule check is legal or not, if so, performing an online rule, and if not, continuously establishing the rule;
b, judging the online rule in the step A, judging whether the rule can be online, if so, continuing to scan the file, and if not, returning to the online rule;
step C, judging whether the scanning file in the step B is found, if so, copying is executed, and if not, ending;
and D, reporting the state of the file copied in the step C, checking the result, and ending.
A file distribution system based on a micro-service architecture mode comprises a set server, a Client and a Portal interface, wherein the set server is connected with the Portal interface, and the set server is connected with a Client module; the server side comprises a RuleManager module, an API module and a master module, and the Api module is connected with the master module; the Client end comprises a Quartz module, a File-parameter module and an Executer module inside, and is connected with a RuleManager module, an API module and a master module inside the server end.
Preferably, the Server is a central control scheduling center of the system, is composed of a rule management module, a salt-api module and a saltmaster module, and is mainly responsible for scheduling and issuing rules;
the method has the following basic functions:
and (3) rule management: the rule management module is mainly responsible for creating, deleting, modifying, inquiring, online and offline rules, firstly, the legality of the rules is verified, and the rules can be issued to the client through online operation after being successfully verified;
and (3) authority management: the system controls the authority according to the principle of dividing the authority into domains, different users belong to different user groups, different machines have different authorities, and centralized management and attribute group division are carried out through an authority module;
parameter configuration: the parameter configuration module is mainly used for specific business services, users in different business scenes can perform custom configuration of rule parameters according to different business scenes, and the rule module generates corresponding rules according to the parameter configuration and sends the corresponding rules to the client.
Preferably, the Client terminal mainly comprises a scheduler, a scanner and an actuator, the scheduler starts a scheduling task according to the configuration of a rule after receiving the rule issued by the server terminal and transmits the rule to the file scanner, the file scanner returns a result to the server terminal after finding a corresponding file or folder, the server terminal issues the task to the actuator for execution according to the returned result, and after the execution of the actuator is completed, the execution result is reported to the server terminal, and the task is completed; the method comprises the following main functions:
1) scanning a file: searching and matching of files are supported according to file types and file name keywords, and deep-level traversal matching is supported for folders;
2) task scheduling: the single client supports multi-task concurrent execution, tasks are not affected mutually, and task scheduling supports concurrent execution of tasks of more than ten thousand orders of magnitude at most;
3) and (3) task execution: the task execution module is realized by means of saltstack mining, can support the copying of various files, and is efficient, stable and reliable;
a. a user clicks an online rule of an online button through an interface, and a server issues the rule to a corresponding client;
b. the client task scheduling module creates a corresponding scheduling task according to the rule;
c. scheduling a task to start a file scanner and scanning files according to a configuration rule;
d. the file scanner reports the searched file to the server through the corresponding service of the client, then the server issues the corresponding task to the salt-minus through the salt-master, and the minus returns the corresponding result to the server after the execution is completed.
A method and a system for distributing files based on a micro-service architecture mode are provided, a client establishes a copy state of a file under a corresponding scheduling task polling inquiry corresponding rule, the file copy state is refreshed according to a result returned by a server, the service adopts a mode of combining BS and CS architectures, the client is deployed on file servers, each file server is deployed with a client program, the server is provided with a background server and a salt-master server, and the client and the server transmit messages through a system bus.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim concerned.
Furthermore, it should be understood that although the present description refers to embodiments, not every embodiment may contain only a single embodiment, and such description is for clarity only, and those skilled in the art should integrate the description, and the embodiments may be combined as appropriate to form other embodiments understood by those skilled in the art.
Claims (5)
1. A file distribution method based on micro-service architecture mode is characterized in that: the file distribution system is realized based on a micro-service architecture mode; the file distribution system based on the micro-service architecture mode comprises a service server, a Client and a Portal interface; the Client end comprises a scheduler, a scanner and an actuator, the scheduler starts a scheduling task according to the configuration of a rule after receiving the rule issued by the service end and transmits the rule to the scanner, the scanner returns a result to the service end after finding a corresponding file or folder, the service end issues the task to the actuator for execution according to the returned result, the actuator reports the execution result to the service end after the execution is completed, and the task is completed; the file distribution method based on the micro-service architecture mode specifically comprises the following steps:
step A, establishing a rule, judging whether the rule check is legal or not, if so, performing an online rule, and if not, continuously establishing the rule;
step B, judging the online rule in the step A, judging whether the rule can be online or not, if so, issuing the rule to the Client side by the server side, starting a scheduling task according to the configuration of the rule after the scheduler of the Client side receives the rule issued by the server side, transmitting the rule to the scanner, continuously scanning the file by the scanner, and returning to the online rule if the rule can not be online;
step C, judging whether the scanning file in the step B is searched, if so, after the scanner of the Client end searches the corresponding file or folder, returning the result to the service end, sending the task to the actuator to be executed by the service end according to the returned result, and executing file copying by the actuator; after the execution of the executor is finished, reporting the execution result to the server, finishing the task, and if the file is not searched, finishing the process;
and D, reporting the state of the file copied in the step C, checking the result, and ending.
2. A file distribution system based on micro-service architecture mode is characterized in that: the system comprises a Server server, a Client terminal and a Portal interface, wherein the Server server is connected with the Portal interface and is connected with a Client module; the Server server comprises a Rule Manager module, an API module and a master module, wherein the API module is connected with the master module; the Client end comprises a Quartz module, a File-parameter module and an Executer module inside, and is connected with a Rule Manager module, an API module and a master module inside the service server; the Client side comprises a scheduler, a scanner and an actuator, the scheduler starts a scheduling task according to the configuration of the rule after receiving the rule issued by the server side and transmits the rule to the file scanner, the file scanner returns the result to the server side after finding the corresponding file or folder, the server side issues the task to the actuator to execute according to the returned result, and after the execution of the actuator is completed, the execution result is reported to the server side and the task is completed.
3. The file distribution system based on the microservice architecture mode of claim 2, wherein: the Server server is a central control dispatching center of the system, consists of a rule management module, a salt-api module and a saltmaster module, and is responsible for dispatching and issuing rules:
the method has the following basic functions:
1) and (3) rule management: the rule management module is responsible for creating, deleting, modifying, inquiring, online and offline rules, firstly, the legality of the rules is verified, and the rules are issued to the client through online operation after being successfully verified;
2) and (3) authority management: the system controls the authority according to the principle of dividing the authority into domains, different users belong to different user groups, different machines have different authorities, and centralized management and attribute group division are carried out through an authority module;
3) parameter configuration: the parameter configuration module is used for specific business services, users in different business scenes carry out custom configuration of rule parameters according to different business scenes, and the rule module generates corresponding rules according to the parameter configuration and sends the corresponding rules to the client.
4. The file distribution system based on the microservice architecture mode of claim 2, wherein: the Client terminal comprises the following functions:
1) scanning a file: searching and matching of files are supported according to file types and file name keywords, and deep-level traversal matching is supported for folders;
2) task scheduling: the single client supports multi-task concurrent execution, tasks are not affected mutually, and task scheduling supports concurrent execution of tasks of more than ten thousand orders of magnitude at most;
3) and (3) task execution: the task execution module is realized by means of saltstack mining, supports the copying of various files, and is efficient, stable and reliable;
a. a user clicks an online rule of an online button through an interface, and a server issues the rule to a corresponding client;
b. the client task scheduling module creates a corresponding scheduling task according to the rule;
c. scheduling a task to start a file scanner and scanning files according to a configuration rule;
d. the file scanner reports the searched file to the server through the corresponding service of the client, then the server issues the corresponding task to the salt-minus through the salt-master, and the minus returns the corresponding result to the server after the execution is completed.
5. The file distribution system in the microservice architecture mode of claim 2, wherein: the client establishes a new copy state of the file under the corresponding scheduling task polling inquiry corresponding rule, the file copy state is refreshed according to the result returned by the server, the service adopts a mode of combining BS and CS frameworks, the client is deployed on the file servers, each file server is deployed with a client program, the server is provided with a background server and a salt-master server, and the client and the server transmit messages through a system bus.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711367068.3A CN108234607B (en) | 2017-12-18 | 2017-12-18 | File distribution method and system based on micro-service architecture mode |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711367068.3A CN108234607B (en) | 2017-12-18 | 2017-12-18 | File distribution method and system based on micro-service architecture mode |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108234607A CN108234607A (en) | 2018-06-29 |
| CN108234607B true CN108234607B (en) | 2020-06-05 |
Family
ID=62649774
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711367068.3A Active CN108234607B (en) | 2017-12-18 | 2017-12-18 | File distribution method and system based on micro-service architecture mode |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108234607B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110287144B (en) * | 2019-06-06 | 2022-12-09 | 深圳证券通信有限公司 | Distribution method of data summary file |
| CN111338656B (en) * | 2020-02-25 | 2023-12-15 | 平安科技（深圳）有限公司 | Method and device for installing software package to target host and computer equipment |
| CN112448961A (en) * | 2020-12-18 | 2021-03-05 | 上海中畅数据技术有限公司 | Micro-service authentication method and system based on authority management |
| CN116185668B (en) * | 2023-04-26 | 2023-06-30 | 上海帆声图像科技有限公司 | Efficient multi-model matching deployment method based on grpc |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2013159703A1 (en) * | 2012-04-24 | 2013-10-31 | Tencent Technology (Shenzhen) Company Limited | Offline download method, multimedia file download method and system thereof |
| CN104283975A (en) * | 2014-11-06 | 2015-01-14 | 福建合诚信息科技有限公司 | File distribution method and device |
| CN106331184A (en) * | 2016-12-01 | 2017-01-11 | 网宿科技股份有限公司 | Big data distribution method and distribution platform based on internet |
| CN106372778A (en) * | 2016-08-23 | 2017-02-01 | 江苏亿科达科技发展有限公司 | Vehicle scheduling system and method based on dynamic rule engine |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101216838B (en) * | 2008-01-15 | 2011-08-10 | 中兴通讯股份有限公司 | XML interface configuration file distribute system and method based on WEB marker |
| JP5132375B2 (en) * | 2008-03-18 | 2013-01-30 | キヤノン株式会社 | Document management system and method enabling document operation using shortcut template |
| CN101452487B (en) * | 2008-12-31 | 2011-03-16 | 中国建设银行股份有限公司 | Data loading method and system, and data loading unit |
| CN101777142A (en) * | 2009-09-01 | 2010-07-14 | 厦门东南融通系统工程有限公司 | Method for converting rules and regulations of enterprises and public institutions into computer-recognizable programs |
-
2017
- 2017-12-18 CN CN201711367068.3A patent/CN108234607B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2013159703A1 (en) * | 2012-04-24 | 2013-10-31 | Tencent Technology (Shenzhen) Company Limited | Offline download method, multimedia file download method and system thereof |
| CN104283975A (en) * | 2014-11-06 | 2015-01-14 | 福建合诚信息科技有限公司 | File distribution method and device |
| CN106372778A (en) * | 2016-08-23 | 2017-02-01 | 江苏亿科达科技发展有限公司 | Vehicle scheduling system and method based on dynamic rule engine |
| CN106331184A (en) * | 2016-12-01 | 2017-01-11 | 网宿科技股份有限公司 | Big data distribution method and distribution platform based on internet |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108234607A (en) | 2018-06-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108234607B (en) | File distribution method and system based on micro-service architecture mode | |
| CN107370786B (en) | General information management system based on micro-service architecture | |
| US9389903B2 (en) | Method, system and apparatus for creating virtual machine | |
| AU2015419073B2 (en) | Life cycle management method and device for network service | |
| US8725798B2 (en) | Provisioning high performance computing clusters | |
| US10303450B2 (en) | Systems and methods for a policy-driven orchestration of deployment of distributed applications | |
| WO2017167100A1 (en) | Data migration method and device | |
| KR102000990B1 (en) | Micro grid energy management system using dds middleware | |
| CN105827428A (en) | Management method and device of asset information | |
| CN113515350A (en) | Hybrid cloud computing management platform | |
| CN110716983B (en) | Information processing method and block chain network service platform | |
| CN105208090A (en) | Zookeeper-based Leader selection method | |
| CN102546839B (en) | Efficient and reliable software distribution method for large scale network | |
| CN104468838A (en) | P2P network and Web service selection method in publish-subscribe pattern | |
| US20130227568A1 (en) | Systems and methods involving virtual machine host isolation over a network | |
| CN103685535A (en) | Interface design of cloud database system for large-scale distributed heterogeneous data treatment | |
| CN105787632A (en) | Method and system for managing conference room | |
| WO2021135279A1 (en) | Asymmetric configuration management method, apparatus and device, and readable storage medium | |
| CN107025134B (en) | Database service system and method compatible with multiple databases | |
| CN105159974A (en) | Method for automatically generating cross-data-source web service | |
| Ådahl | Shared resource for collaborative editing over a wireless network | |
| WO2014019545A1 (en) | Content distribution system and content distribution method therefor | |
| Tang et al. | Application centric lifecycle framework in cloud | |
| US20140351792A1 (en) | Version Construction System and Method | |
| CN112882765A (en) | Digital twin model scheduling method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |