CN111416842B - Automatic resource cluster distribution and hot update system and method - Google Patents
Automatic resource cluster distribution and hot update system and method Download PDFInfo
- Publication number
- CN111416842B CN111416842B CN202010153327.8A CN202010153327A CN111416842B CN 111416842 B CN111416842 B CN 111416842B CN 202010153327 A CN202010153327 A CN 202010153327A CN 111416842 B CN111416842 B CN 111416842B
- Authority
- CN
- China
- Prior art keywords
- resource
- uploading
- updated
- module
- redis
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2365—Ensuring data consistency and integrity
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/12—Applying verification of the received information
- H04L63/123—Applying verification of the received information received data contents, e.g. message integrity
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
- H04L9/32—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials
- H04L9/3236—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials using cryptographic hash functions
- H04L9/3239—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials using cryptographic hash functions involving non-keyed hash functions, e.g. modification detection codes [MDCs], MD5, SHA or RIPEMD
Landscapes
- Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Computer Hardware Design (AREA)
- Computing Systems (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The invention discloses a method and a system for automatically distributing and thermally updating a resource cluster, wherein the method comprises the following steps: s1, acquiring the compiled resource to be updated and the compiled Redis state from the resource pool; s2, judging whether the uploading condition is met; the uploading condition comprises the following steps: uploading service is online, a target node exists, and the Redis state meets Redis detection during uploading; if so, uploading the resource to be updated to an ftp server; if not, no uploading operation is carried out; s3, judging whether the uploading is successful; if so, updating the Redis state; if not, uploading again until uploading is successful or the set uploading times are reached; s4, resource distribution and hot update are performed. The invention can reduce the risk of system stability and improve the reliability of data.
Description
Technical Field
The invention relates to the technical field of automatic distribution of resource clusters, in particular to an automatic distribution and hot update system and method of a resource cluster.
Background
In recent years, with the rapid development of mobile internet technology, the degree of informatization is increasing day by day, and the artificial intelligence industry is facing another wave of development climax. Among them, natural language understanding technology, which is one of the most important directions in the field of artificial intelligence, has been a focus of research by researchers in the related fields. The machine can understand natural language, the goals of reducing labor input, improving industrial intelligence and the like are achieved, and the method is always a hotspot for research of researchers in related fields. In natural language processing application, data is important food for driving an artificial intelligence technology, and how to ensure real-time and efficient transmission, loading and rollback of the data is significant. For example, in a telephone navigation project, which is one of natural language technology application projects, a client often requires isolation of a semantic optimization test environment from an online environment of telephone navigation, so that a workflow of making and optimizing resources in one environment to a desired effect and then releasing the resources online is realized. To meet the huge traffic access demand, the online navigation environment often deploys multiple engines on multiple servers to form a cluster system. The traditional method for copying resources manually has the defects of complex flow, long period and inconvenient operation, and the existing scheme based on directory sharing has certain defects of stability and reliability. In addition, the update or rollback of the resource needs to be manually executed, the engine system needs to be restarted every time, and the time consumption and the perception are poor. In view of the above scenario, how to quickly and automatically distribute data resources to a cluster system and implement hot update or hot rollback of resources is a key problem that needs to be solved urgently.
Network File System (NFS) shared data is a very common way of data transfer in applications. The NFS accesses the remote client over the network in a similar manner as accessing the local file system. It allows sharing of a common file system between multiple users and provides advantages in data aggregation. Specifically, NFS configuration is performed first, a remote system directory of a resource to be updated is shared to a local compiling directory, then a knowledge base is compiled, and a resource on which a semantic understanding engine depends is shared to a remote environment, so that resource release is realized.
The resource distribution technology based on the NFS file system has the advantages of simple operation, easy operation, low maintenance cost and the like, but has many defects. First, if the NFS server is down, all clients cannot access the shared directory, so there is a risk of system stability. Secondly, the NFS does not check the data itself, there is a file exception, and if the resource synchronized to the client side is damaged, a major production accident may be caused, so there is a risk of data reliability. In addition, the above scheme only realizes the data transmission function, and cannot solve the problem of hot update of resources, and the rollback of the resources is realized by copying the backed-up resources to the corresponding directory for transmission, which lacks automation and intellectualization.
Disclosure of Invention
An object of the present invention is to provide a method and a system for automatically distributing and hot-updating a resource cluster, which can reduce the risk of system stability and improve the reliability of data.
The invention provides a resource cluster automatic distribution and hot update method, which comprises the following steps:
s1, acquiring the compiled resource to be updated and the compiled Redis state from the resource pool;
s2, judging whether the uploading condition is met;
the uploading condition comprises the following steps: uploading service is online, a target node exists, and the Redis state meets Redis detection during uploading; if so, uploading the resource to be updated to an ftp server;
if not, no uploading operation is carried out;
s3, judging whether the uploading is successful; if so, updating the Redis state; if not, uploading again until uploading is successful or the set uploading times are reached;
s4, resource distribution and hot update are performed.
The method for automatically distributing and hot-updating the resource cluster as described above, wherein optionally, the step S4 includes the following specific steps:
s41, acquiring a Redis state;
s42, judging whether the Redis state accords with Redis detection during downloading, if so, downloading the resource to be updated to a receiving end;
s43, judging whether the downloaded resource to be updated is a new resource, if so, sequentially performing resource check, resource decompression and integrity check on the resource to be updated, and executing the step S44 after the check is successful;
and S44, performing hot updating.
The method for automatically distributing and hot-updating the resource cluster as described above, wherein optionally, step S44 is:
and performing resource head verification and information abstract algorithm verification on the resource to be updated, and sequentially performing new resource loading, old resource deletion and resource head recording after successful verification.
The method for automatically distributing and hot-updating the resource cluster as described above, wherein optionally, step S41 includes the following specific steps:
s411, judging whether the downloading service is on line, if so, executing step S412; if not, the download service is pulled up through the pull-up module;
s412, obtaining the Redis state.
The method for automatically distributing and hot-updating the resource cluster as described above, optionally, further includes the following steps:
s5, after receiving the rollback instruction, searching resources;
s6, judging whether the target resource exists, if yes, executing step S7; if not, feeding back rollback failure information;
and S7, sequentially modifying the resource header, distributing the resource and updating the resource, and feeding back rollback success information after the resource is updated successfully.
The method for automatically distributing and hot-updating the resource cluster as described above, wherein optionally, step S2 includes the following specific steps:
s21, judging whether the uploading service is on line, if yes, executing step S23; if not, go to step S22;
s22, restarting the uploading service through the re-module so as to enable the uploading service to be online;
s23, judging whether the target node exists or not; if yes, go to step S24; if not, not uploading;
s24, judging whether the Redis state meets Redis detection during uploading; if so, uploading the resource to be updated to the ftp server, and executing S3; if not, not uploading.
The invention also provides a system for automatically distributing and thermally updating the resource cluster, which comprises the following steps:
the knowledge base management module is used for copying the resources to be updated which reach the standard through testing from the local user;
the resource compiling module is connected with the knowledge base management module and used for compiling the resource to be updated and setting a corresponding Redis state after successful compiling and storage;
the resource storage module is connected with the resource compiling module and is used for temporarily storing the compiled resources to be updated;
the resource distribution module is connected with the resource compiling module and used for acquiring the compiled resources to be updated from the resource storage module, judging whether the uploading condition is met, if so, uploading the resources to be updated, and if not, not uploading the resources to be updated; wherein, the uploading condition comprises: uploading service is online, a target node exists, and the Redis state meets Redis detection during uploading;
and the resource receiving and hot updating module is connected with the resource distribution module and is used for acquiring the resource to be updated, verifying the resource to be updated and carrying out hot updating.
The system for automatically distributing and thermally updating the resource cluster as described above, wherein optionally, an ftp server is further included;
the resource distribution module, the resource receiving and hot updating module are connected with the ftp server, and the resource distribution module is used for uploading the resource to be updated to the ftp server;
the resource receiving and hot updating module is further configured to download the resource to be updated from the ftp server.
The system for automatically distributing resource clusters and thermally updating resource clusters as described above, wherein optionally, the resource compiling module is configured to add a resource source when compiling the resource to be updated, and the resource source is configured to perform verification in the processes of resource distribution and thermal updating;
the resource source head comprises: data type, parity bits, and compile time.
The system for automatically distributing and thermally updating resource clusters as described above, optionally further includes:
the user corpus collection module is used for automatically collecting user data;
the knowledge mining module is connected with the user corpus collecting module; the knowledge mining module is used for automatically expanding the structured knowledge according to the collected user data;
the recommendation module is connected with the knowledge mining module and used for recommending the expanded structured knowledge to the user and acquiring feedback information of whether the recommendation is accepted or not;
the effect detection module is connected with the recommendation module and used for comparing and evaluating the current resource effect; and outputting the resource which suggests the user to hot update the current version or fall back to the historical version to the user according to the evaluation result.
Compared with the prior art, the method and the system can realize automatic distribution of local resources to the remote server, ensure quick data transmission and improve the stability of system operation and the reliability of data transmission. By setting the Redis state, the state of data transmission can be fed back to a user in real time, and the method has the functions of error monitoring, repeated requests and the like, and ensures the safety and reliability of transmission.
When the resource to be uploaded is compiled, the resource header is added for subsequent verification, so that the completeness and consistency of data transmission and the legality of the data can be ensured.
And after the data transmission is completed, updating the resources to be updated to each engine of the system cluster. Because the data updating is completed in a hot updating mode, a system in use cannot be interrupted due to data updating, and therefore non-sensing operation can be achieved, working efficiency can be improved, and user experience can be improved.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description, serve to explain the principles of the invention.
FIG. 1 is a flowchart illustrating steps of a method for automatically distributing and hot-updating a resource cluster according to the present invention;
FIG. 2 is a flowchart of the steps of resource distribution in the present invention;
FIG. 3 is a flowchart illustrating the steps of resource reception according to the present invention;
FIG. 4 is a flowchart illustrating the steps of resource heat update according to the present invention;
FIG. 5 is a flowchart of the steps of the resource rollback of the present invention;
FIG. 6 is a schematic structural diagram of an automatic resource cluster distribution and hot-update system according to the present invention;
FIG. 7 is a flowchart of the steps of the intelligent recommendation proposed by the present invention;
Detailed Description
Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that: the relative arrangement of the components and steps, the numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present invention unless specifically stated otherwise.
The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses.
Techniques, methods, and apparatus known to those of ordinary skill in the relevant art may not be discussed in detail but are intended to be part of the specification where appropriate.
In all examples shown and discussed herein, any particular value should be construed as merely illustrative, and not limiting. Thus, other examples of the exemplary embodiments may have different values.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, further discussion thereof is not required in subsequent figures. The connection referred to in this embodiment is not limited to wired or wireless, and means that both having a connection relationship can communicate before.
Referring to fig. 1 to fig. 5, a method for automatically distributing and hot-updating a resource cluster according to an embodiment of the present invention includes the following steps:
and S1, acquiring the compiled resource to be updated and the compiled Redis state from the resource pool. Specifically, the resource to be updated is temporarily stored in the resource pool, specifically, the resource to be updated is still stored in the resource pool for a period of time after the update is completed, and during this period, there may be a higher version of the resource to be updated, so that, at the same time point, multiple versions of the resource are stored in the resource pool, and these multiple versions of the resource are used for the resource rollback. When the method is implemented, a local user firstly tests the resources to ensure that the resources reach the standard after detection, then compiles the resources to be updated, and temporarily stores the compiled resources to be updated into a resource pool. Meanwhile, after the compilation is successful, the Redis state is set. Specifically, the Redis state can be set in the Redis server, and the Redis server is used as a third-party server and is connected with the resource distribution module in a communication mode, so that information can be stored and acquired more simply and rapidly. It should be noted that Redis a key-value storage system.
S2, judging whether the uploading condition is met; specifically, the upload conditions include: uploading service is online, a target node exists, and the Redis state meets Redis detection during uploading; if so, uploading the resource to be updated to an ftp server; and if not, not carrying out uploading operation. Specifically, step S2 includes the following specific steps:
s21, judging whether the uploading service is on line, if yes, executing step S23; if not, go to step S22; s22, restarting the uploading service through the re-module so as to enable the uploading service to be online; namely, whether the uploading service is in a state or not is monitored in real time during implementation, and if the uploading service is disconnected, the restarting module is automatically called to enable the uploading service to be on line, so that a stable working state can be guaranteed.
S23, judging whether the target node exists or not; if yes, go to step S24; if not, not uploading; specifically, the target node does not exist or the configuration is illegal, and the target node does not exist. And when the target node does not exist, uploading is not carried out, and meanwhile, the user is reminded that the target node does not exist or the configuration is illegal.
S24, judging whether the Redis state meets Redis detection during uploading; if so, uploading the resource to be updated to the ftp server, and executing S3; if not, not uploading. After the compiling is successful, the set Redis state is actually a successfully compiled and stored mark information, and when the mark information is detected, the data transmission can be carried out.
Further, in the process of uploading the resources, any abnormal state of data transmission can be captured, if the data transmission fails, the abnormal state is fed back to the knowledge base management module to inform a user or maintenance personnel, and if the data transmission succeeds, the Redis state can be updated to the Redis server to inform a receiving end of preparing to download the resources. Specifically, in the uploading process, the method comprises the following steps: s3, judging whether the uploading is successful; if so, updating the Redis state; if not, the uploading is carried out again until the uploading is successful or the set uploading times are reached. When the method is implemented specifically, the maximum uploading times are preset, and if the uploading times reach the preset maximum uploading times and cannot be successfully uploaded, the uploading is stopped, and uploading failure is fed back to the knowledge base management module so as to inform a user or maintenance personnel of the uploading failure.
S4, resource distribution and hot update are performed. The method comprises the following specific steps:
s41, acquiring a Redis state; in specific implementation, the resource receiving tool may acquire the Redis state from the Redis server, or the Redis server may send the Reids state to the resource receiving tool after updating the Redis state. More specifically, S411, determining whether the download service is online, if yes, executing step S412; if not, the downloading service is pulled up through the pulling-up module, so that smooth downloading and subsequent updating of the data can be ensured; s412, obtaining the Redis state.
And S42, judging whether the Redis state accords with Redis detection during downloading, and if so, downloading the resource to be updated to the receiving end. That is, the Redis state of the resource to be updated of each version has at least two different states, one is a state after compiling and storing are successful; one is the status after successful upload. The Redis state is in accordance with Redis detection during downloading, namely the Redis state is the state after uploading is successful. The detection of the Redis state is carried out in real time, and when the Redis state is the state after the uploading is successful, the downloading is carried out immediately.
And S43, judging whether the downloaded resource to be updated is a new resource, if so, sequentially performing resource check, resource decompression and integrity check on the resource to be updated, and executing the step S44 after the check is successful. After the data downloading is finished, the MD5 value is checked first to prevent the data transmission packet loss, and if the data is abnormal, the resources are not updated. And decompressing the downloaded data and carrying out data integrity check to prevent abnormal data decompression if the data is normal.
And S44, performing hot updating. Specifically, resource header verification and information summary algorithm (MD5) verification are performed on the resources to be updated, and new resource loading, old resource deletion and resource source recording are sequentially performed after verification is successful. Specifically, the successful verification here refers to successful verification through the resource header verification and the information digest algorithm. In specific implementation, the resource manager is called to complete the check and loading of the resource for calling. And (5) utilizing the resource manager to check the repeatability and the validity of the resource. The resource manager records the time information of the resources used by the system, compares the time header information of the new and old resources, and can load the new resources if the time of the new resources is newer, or cannot update the new resources. In addition, the resource manager can identify the resource type and version information supported by the system, and only the resource which passes the verification of the resource manager can be updated and used.
Specifically, the present embodiment further includes the following steps:
and S5, after receiving the rollback instruction, searching resources. In particular, the rollback instruction may be user initiated through the knowledge base management module. When the user wishes to roll back to a historical resource, the resource corresponding to the point in time may be selected via the interface. And then, notifying the resource pool manager to search the resource of the corresponding version from the resource pool according to the time point, wherein the resource pool is used for temporarily storing the resource to be updated, but does not represent the resource to be updated of only one version in the resource pool, and specifically, a plurality of resources to be updated released in the latest period of time, such as resources to be updated in the last two years, may exist in the resource pool at the same time. S6, judging whether the target resource exists, if yes, executing step S7; if not, the specified version of the resource does not exist, and at the moment, rollback failure information is fed back.
And S7, sequentially modifying the resource header, distributing the resource and updating the resource, and feeding back rollback success information after the resource is updated successfully. Specifically, the time of the resource head is modified to the latest time, and the resource is updated by using the updated Redis state notification. The resource distribution tool distributes the data to the cluster server, and after the receiving end receives the resources, the time head is found to be the newest, namely the time head is updated to the engine system.
Referring to fig. 6, an embodiment of the present invention further provides an automatic resource cluster distribution and hot update system, including:
the knowledge base management module is used for copying the resources to be updated which reach the standard after the test from the local user; specifically, the function of the knowledge base management module is to provide a set of visual data management platform so as to provide more convenient automatic and intelligent services for users, and the knowledge base management module is connected with local users during specific implementation.
And the resource compiling module is connected with the knowledge base management module and used for compiling the resource to be updated and setting a corresponding Redis state after successful compiling and storage. Specifically, in the compiling process, resource source checking information, such as data type, check bits, compiling time, and the like, is added to the resource to be updated. The method is used for performing resource verification, validity verification and the like in the uploading and updating processes.
The resource storage module is connected with the resource compiling module and is used for temporarily storing the compiled resources to be updated; when the method is specifically implemented, the method comprises a resource pool and a resource pool manager, wherein the resource pool stores all resources compiled within a period of time, and the resource pool manager maintains time node information of each resource to be updated, provides information input and update operations, and provides maintenance management for searching, importing and exporting the resources. When a user wants to rollback a resource, the resource storage module searches for the resource to be rolled back, and then the resource distribution module distributes the resource. For the resource rollback, refer to step 5 in the foregoing method for automatically distributing and hot-updating resource clusters, which is not described herein again.
The resource distribution module is connected with the resource compiling module and used for acquiring the compiled resources to be updated from the resource storage module, judging whether the uploading condition is met, uploading the resources to be updated if the uploading condition is met, and not uploading the resources to be updated if the uploading condition is not met; wherein, the uploading condition comprises: the uploading service is online, a target node exists, and the Redis state meets Redis detection during uploading. Namely, the resource distribution module is mainly used for verifying and uploading data to the ftp server. The system is provided with functions of resource verification, node identification and transmission, Redis detection and the like, and provides reliable transmission capability for data. The resource distribution module, the resource receiving and hot updating module are connected with the ftp server, and the resource distribution module is used for uploading the resource to be updated to the ftp server; the resource receiving and hot updating module is further configured to download the resource to be updated from the ftp server. By taking the ftp server as a transfer station of the data, automatic uploading and receiving of the data based on the sftp protocol are realized. And the data transmission is ensured to be safe and reliable.
And the resource receiving and hot updating module is connected with the resource distribution module and is used for acquiring the resource to be updated, verifying the resource to be updated and carrying out hot updating. The resource receiving is responsible for downloading and verifying the data, and the hot updating loads the resources into the cluster system in a hot mode, so that the condition limitation of resource updating is removed, and the data starting efficiency is improved.
As a better implementation manner, the resource compiling module is configured to add a resource source head when compiling a resource to be updated, where the resource source head is configured to perform verification in the resource distribution and hot update processes;
the resource source head comprises: data type, parity bits, and compile time.
Referring to fig. 7, the system for automatically distributing and hot-updating resource clusters further includes:
the user corpus collection module is used for automatically collecting user data; the knowledge mining module is connected with the user corpus collecting module; the knowledge mining module is used for automatically expanding the structured knowledge according to the collected user data; the recommendation module is connected with the knowledge mining module and used for recommending the expanded structured knowledge to the user and acquiring feedback information of whether the recommendation is accepted or not; the effect detection module is connected with the recommendation module and used for comparing and evaluating the current resource effect; and outputting the resource which suggests the user to hot update the current version or fall back to the historical version to the user according to the evaluation result. Referring to fig. 7, the system provided by the present invention also has an intelligent recommendation function, specifically, the system automatically collects user data, automatically expands structured knowledge through mining technology, recommends to the user, and enters the warehouse after the administrator confirms. Then, the system automatically calls an effect detection module to compare and evaluate the resource effect of the current system, and finally, the user is recommended to update the semantic resource of the current version or return to a certain historical version.
As apparent from the foregoing description, the present invention has the following effects:
1. the automatic distribution of local resources to the remote server is realized, and the stability of system operation and the reliability of data transmission are improved while the rapid data transmission is ensured. And based on the sftp protocol, automatic uploading and receiving of data are realized. And information header check codes are added during resource compiling, and the integrity and the legality of data transmission are ensured by checking the information summary algorithm (MD5) and checking the resource headers. In the system, the PMI automatic pull-up module is utilized to monitor the resource distribution service in real time, so that the stability of data transmission work is ensured. The data uploading and receiving end has the function of a Redis state real-time monitoring module, feeds the state of data transmission back to a user in real time, has the functions of error monitoring, repeated requests and the like, and ensures safe and reliable transmission.
2. Resource heat updating improves user experience perception. And after the transmission of the data is completed, utilizing the resource management module to thermally update the data at the receiving end of the ftp tool to each engine of the system cluster. Firstly, the legality and the effectiveness of data are verified by detecting time information and other verification information of a resource head. And if the requirements are met, loading the resources into the system, otherwise, not loading. The advantages of hot update are mainly that the system in use is not interrupted by data update, no-sense operation is realized, and work efficiency is improved.
3. The resource rollback function can be realized. And the resource of the historical time node is selected to roll back. Every time the resource is on-line, the resource compiling module stores the resource data into the resource pool, and the resource pool management module is used for recording, searching and modifying the resource time. When the latest online resource is not in accordance with the expectation, selecting historical resources to roll back on the user interface, searching the resource at the corresponding time by calling the resource pool manager, taking out the resource from the resource pool, and modifying the head check time information of the resource into the current calling time. And then calling a resource uploading and receiving tool to carry out resource inspection and transmission. And finally, detecting that the resource is updated by the hot updating module, and hot updating the resource to a local or remote cluster system.
4. An update or rollback function is automatically recommended. The system automatically constructs structured knowledge through a knowledge digging technology according to corpora accumulated by user interaction track information, directionally pushes the structured knowledge to users, and stores the structured knowledge in a warehouse for updating after the users confirm the structured knowledge. In addition, the system automatically compares the semantic effect of the new resource and the old resource, and recommends whether the user is updated hot or the resources are returned.
Although some specific embodiments of the present invention have been described in detail by way of examples, it should be understood by those skilled in the art that the above examples are for illustrative purposes only and are not intended to limit the scope of the present invention. It will be appreciated by those skilled in the art that modifications may be made to the above embodiments without departing from the scope and spirit of the invention. The scope of the invention is defined by the appended claims.
Claims (7)
1. A resource cluster automatic distribution and hot update method is characterized by comprising the following steps:
s1, acquiring the compiled resource to be updated and the compiled Redis state from the resource pool;
s2, judging whether the uploading condition is met;
the uploading condition comprises the following steps: uploading service online, a target node exists, and Redis state meets Redis detection during uploading; if so, uploading the resource to be updated to an ftp server;
if not, no uploading operation is carried out;
s3, judging whether the uploading is successful; if so, updating the Redis state; if not, uploading again until uploading is successful or the set uploading times are reached;
s4, performing resource distribution and hot updating, including downloading the resource to be updated to a receiving end when the acquired Redis state is judged to be in accordance with the Redis detection during downloading; if the downloaded resource to be updated is a new resource, sequentially performing resource verification, resource decompression and integrity verification on the resource to be updated, and performing the following hot updating operations after the verification is successful: and performing resource head verification and information abstract algorithm verification on the resource to be updated, and performing new resource loading, old resource deletion and resource head recording after the verification is passed.
2. The method for automatically distributing and hot-updating resource clusters according to claim 1, wherein in the hot-update stage, acquiring the Redis state comprises the following specific steps:
s411, judging whether the downloading service is on line, if so, executing step S412; if not, the download service is pulled up through the pull-up module;
s412, obtaining the Redis state.
3. The method for automatic distribution and hot-update of resource clusters according to claim 1, further comprising the steps of:
s5, after receiving the rollback instruction, searching resources;
s6, judging whether the target resource exists, if yes, executing step S7; if not, feeding back rollback failure information;
and S7, sequentially modifying the resource header, distributing the resource and updating the resource, and feeding back rollback success information after the resource is updated successfully.
4. The method for automatically distributing and hot-updating resource clusters according to any one of claims 1 to 3, wherein the step S2 comprises the following specific steps:
s21, judging whether the uploading service is on line, if yes, executing step S23; if not, go to step S22;
s22, restarting the uploading service through the re-module so as to enable the uploading service to be online;
s23, judging whether the target node exists or not; if yes, go to step S24; if not, not uploading;
s24, judging whether the Redis state meets Redis detection during uploading; if so, uploading the resource to be updated to the ftp server, and executing S3; if not, not uploading.
5. A system for automatically distributing and thermally updating resource clusters, comprising:
the knowledge base management module is used for copying the resources to be updated which reach the standard after the test from the local user;
the resource compiling module is connected with the knowledge base management module and used for compiling the resource to be updated and setting a corresponding Redis state after successful compiling and storage;
the resource storage module is connected with the resource compiling module and is used for temporarily storing the compiled resources to be updated;
the resource distribution module is connected with the resource compiling module and used for acquiring the compiled resources to be updated from the resource storage module, judging whether the uploading condition is met, uploading the resources to be updated if the uploading condition is met, and not uploading the resources to be updated if the uploading condition is not met; wherein, the uploading condition comprises: uploading service is online, a target node exists, and the Redis state meets Redis detection during uploading;
the resource receiving and hot updating module is connected with the resource distribution module and used for acquiring the resource to be updated, verifying the resource to be updated and carrying out hot updating, wherein the hot updating comprises the step of downloading the resource to be updated to a receiving end when the acquired Redis state is judged to be in accordance with Redis detection during downloading; if the downloaded resource to be updated is a new resource, sequentially performing resource verification, resource decompression and integrity verification on the resource to be updated, and performing the following hot updating operations after the verification is successful: performing resource head verification and information abstract algorithm verification on the resource to be updated, and performing new resource loading, old resource deletion and resource head recording after the verification is passed;
the resource distribution module, the resource receiving module and the hot updating module are connected with the ftp server, and the resource distribution module is used for uploading the resource to be updated to the ftp server; the resource receiving and hot updating module is further configured to download the resource to be updated from the ftp server.
6. The system according to claim 5, wherein the resource compiling module is configured to add a resource source header when compiling the resource to be updated, and the resource header is configured to perform verification during the resource distributing and hot updating process;
the resource source head comprises: data type, parity bits, and compile time.
7. The system for automatic distribution and hot-update of resource clusters according to any of claims 5-6, further comprising:
the user corpus collection module is used for automatically collecting user data;
the knowledge mining module is connected with the user corpus collecting module; the knowledge mining module is used for automatically expanding the structured knowledge according to the collected user data;
the recommendation module is connected with the knowledge mining module and used for recommending the expanded structured knowledge to the user and acquiring feedback information of whether the recommendation is accepted or not;
the effect detection module is connected with the recommendation module and used for comparing and evaluating the current resource effect; and outputting the resource which suggests the user to hot update the current version or fall back to the historical version to the user according to the evaluation result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010153327.8A CN111416842B (en) | 2020-03-06 | 2020-03-06 | Automatic resource cluster distribution and hot update system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010153327.8A CN111416842B (en) | 2020-03-06 | 2020-03-06 | Automatic resource cluster distribution and hot update system and method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111416842A CN111416842A (en) | 2020-07-14 |
| CN111416842B true CN111416842B (en) | 2022-08-05 |
Family
ID=71492836
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010153327.8A Active CN111416842B (en) | 2020-03-06 | 2020-03-06 | Automatic resource cluster distribution and hot update system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111416842B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114070843A (en) * | 2021-10-29 | 2022-02-18 | 盐城金堤科技有限公司 | Data updating method, client, server and FTP server |
Family Cites Families (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CA2867585A1 (en) * | 2013-10-15 | 2015-04-15 | Coho Data Inc. | Methods, devices and systems for coordinating network-based communication in distributed server systems with sdn switching |
| CN104657170A (en) * | 2013-11-25 | 2015-05-27 | 腾讯科技（上海）有限公司 | Data updating method, device and system |
| CN104077131B (en) * | 2014-06-13 | 2017-08-25 | 科大讯飞股份有限公司 | A kind of deployment of components system and method |
| CN105871966A (en) * | 2015-09-22 | 2016-08-17 | 乐视网信息技术（北京）股份有限公司 | Message distribution method, device and system |
| CN105162878B (en) * | 2015-09-24 | 2018-08-31 | 网宿科技股份有限公司 | Document distribution system based on distributed storage and method |
| CN105472412B (en) * | 2015-11-20 | 2018-05-08 | 四川长虹电器股份有限公司 | A kind of big data processing method for distinguishing smart television state |
| CN105677415B (en) * | 2016-01-06 | 2020-07-17 | 网易（杭州）网络有限公司 | Hot update method and device |
| CN106131612B (en) * | 2016-06-30 | 2019-03-05 | 武汉斗鱼网络科技有限公司 | The method and system of Android app dynamically load resource function module |
| CN106506191A (en) * | 2016-09-23 | 2017-03-15 | 郑州云海信息技术有限公司 | The automatically dispose serviced based on virtual platform Redis and maintaining method |
| CN107315668B (en) * | 2017-06-26 | 2021-06-29 | 郑州云海信息技术有限公司 | Automatic rapid detection method and device for data consistency of distributed storage system |
| CN108881477B (en) * | 2018-07-16 | 2020-09-29 | 上海新炬网络技术有限公司 | Distributed file acquisition monitoring method |
| CN109032638A (en) * | 2018-07-18 | 2018-12-18 | 广州宸瑞软件科技有限公司 | Hot update method, server and the mobile terminal of mobile application software |
-
2020
- 2020-03-06 CN CN202010153327.8A patent/CN111416842B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN111416842A (en) | 2020-07-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111399873B (en) | Model updating method and device | |
| US8180762B2 (en) | Database tuning methods | |
| CN108881477B (en) | Distributed file acquisition monitoring method | |
| CN106462575A (en) | Design and implementation of clustered in-memory database | |
| CN109144785B (en) | Method and apparatus for backing up data | |
| CN105763369A (en) | Terminal equipment remote software version distribution method and system | |
| CN111416821A (en) | Internet of things equipment information acquisition method, system and device | |
| CN110162334B (en) | Code management method, device and storage medium | |
| CN104423960A (en) | Continuous project integration method and continuous project integration system | |
| CN103457979B (en) | Distribution type file storage system and method | |
| CN110545207B (en) | Synchronous automatic intelligent DNS system and configuration method | |
| CN112506870B (en) | Data warehouse increment updating method and device and computer equipment | |
| CN111737227B (en) | Data modification method and system | |
| CN106339273A (en) | Application program restoration method, terminal and server | |
| CN110659202A (en) | Client automatic testing method and device | |
| CN115712623B (en) | Batch data fault-tolerant collection method based on captured metadata change | |
| CN111416842B (en) | Automatic resource cluster distribution and hot update system and method | |
| CN106126419A (en) | The adjustment method of a kind of application program and device | |
| US20060123387A1 (en) | Apparatus and method for producing application software for streaming service and system and method for providing software streaming service with network fault tolerance | |
| CN112579289B (en) | Distributed analysis engine method and device capable of being intelligently scheduled | |
| CN112202909B (en) | Online upgrading method and system for computer storage system | |
| CN112148745B (en) | Multi-HBase cluster access method, device and storage medium | |
| CN115129574A (en) | Code testing method and device | |
| CN109145054A (en) | A kind of method of managing customer end data | |
| CN110795332A (en) | Automatic testing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |