CN116610652A - Information migration method, information generation method, information migration program, and information migration program apparatus, device and readable medium - Google Patents

Information migration method, information generation method, information migration program, and information migration program apparatus, device and readable medium Download PDF

Info

Publication number
CN116610652A
CN116610652A CN202210120958.9A CN202210120958A CN116610652A CN 116610652 A CN116610652 A CN 116610652A CN 202210120958 A CN202210120958 A CN 202210120958A CN 116610652 A CN116610652 A CN 116610652A
Authority
CN
China
Prior art keywords
target
information
processing system
identification information
message processing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210120958.9A
Other languages
Chinese (zh)
Inventor
褚继辉
闫莉刚
马晓敏
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Jingdong Century Trading Co Ltd
Beijing Wodong Tianjun Information Technology Co Ltd
Original Assignee
Beijing Jingdong Century Trading Co Ltd
Beijing Wodong Tianjun Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Jingdong Century Trading Co Ltd, Beijing Wodong Tianjun Information Technology Co Ltd filed Critical Beijing Jingdong Century Trading Co Ltd
Priority to CN202210120958.9A priority Critical patent/CN116610652A/en
Publication of CN116610652A publication Critical patent/CN116610652A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/21—Design, administration or maintenance of databases
    • G06F16/214—Database migration support
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22—Indexing; Data structures therefor; Storage structures
    • G06F16/2282—Tablespace storage structures; Management thereof
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

The embodiment of the disclosure discloses an information migration method, an information generation device, an information migration device, an information generation device and a readable medium. One embodiment of the information migration method comprises the following steps: determining whether a target topic exists in the target message processing system in response to receiving the migration information; creating a target topic in the target message processing system in response to determining that the target topic does not exist; determining whether information of a new processor exists in the target message processing system, wherein the new processor is used for processing messages belonging to a target theme; creating information of the new processor in the target message processing system in response to determining that the information of the new processor does not exist; and adjusting the related information in the target message processing system according to the configuration information of the original processor of the target theme. The implementation mode can realize the automatic migration of the theme and the processor among the message processing systems, and reduce the manual operation. Therefore, not only can the information migration efficiency be improved, but also the occurrence of human errors can be avoided or reduced.

Description

Information migration method, information generation method, information migration program, and information migration program apparatus, device and readable medium
Technical Field
Embodiments of the present disclosure relate to the field of computer technology, and in particular, to an information migration method, an information generation method, an apparatus, a device, and a readable medium.
Background
With the continuous development of internet technology, the trend of internet big data is increasingly remarkable. Timeliness, accuracy, stability of data processing are becoming increasingly important. With this trend, big data real-time computing platforms have evolved. To meet the requirement of services on data stability, a platform is generally required to be able to switch topics (topic) between different message processing systems (such as kafka, distributed publish-subscribe messaging systems) so as to achieve the purpose of topic upgrade or disaster recovery.
Currently, for version upgrades, manual operations are often relied upon. Not only is the step complicated, but also the error is easy to occur, and the overall stability is influenced. In addition, the prior art generally adopts a double-flow mode to realize a disaster recovery mechanism. I.e. apply for a set of topic, publishers and subscribers in advance in other systems. In the event of a system failure, the user is also required to manually modify the configuration in the publish and subscribe tasks. This mechanism has a problem of wasting resources and still has a risk of failure.
Disclosure of Invention
The disclosure is in part intended to introduce concepts in a simplified form that are further described below in the detailed description. The disclosure is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. Some embodiments of the present disclosure propose information migration methods, information generation methods, apparatuses, electronic devices, and computer-readable media to solve one or more of the technical problems mentioned in the background section above.
In a first aspect, some embodiments of the present disclosure provide an information migration method, including: determining whether a target theme exists in the target message processing system or not in response to receiving migration information, wherein the migration information comprises the target theme, an identification information set of an original processor of the target theme and identification information of the target message processing system; creating a target topic in the target message processing system in response to determining that the target topic does not exist; determining whether information of a new processor exists in the target message processing system, wherein the new processor is used for processing messages belonging to a target theme; creating information of the new processor in the target message processing system in response to determining that the information of the new processor does not exist; and adjusting the related information in the target message processing system according to the configuration information of the original processor of the target theme.
In some embodiments, adjusting relevant information in the target message processing system based on the configuration information of the original processor of the target subject includes: determining whether an original processor of the target theme has a target data synchronization task, wherein the target data synchronization task is a data synchronization task related to the target theme; in response to determining that there is a target data synchronization task, a new data synchronization task is constructed in accordance with the target data synchronization task, wherein the new data synchronization task is used to indicate a target topic in the target message processing system.
In some embodiments, adjusting relevant information in the target message processing system based on the configuration information of the original processor of the target subject includes: determining whether an original processor of the target theme has a data interaction task; and in response to determining that the data interaction task exists, modifying a message processing system corresponding to the original processor in the data interaction task into a target message processing system.
In some embodiments, adjusting relevant information in the target message processing system based on the configuration information of the original processor of the target subject includes: and modifying the monitoring parameters in the target message processing system according to the monitoring configuration information of the original processor of the target theme.
In some embodiments, creating information of a new processor in a target message processing system includes: in the target message processing system, creating information of a new subscriber of the target topic, and resetting the location of the new subscriber to a preset position.
In some embodiments, the method further comprises: and generating migration records and storing the migration records in a database.
In some embodiments, the method further comprises: determining whether a target record is contained in the database, wherein the target record is a record characterizing a target topic created in the target message processing system; generating and storing the target record in response to determining that the target record is not included; establishing an association relation between the new processor and the target theme; and generating and storing a modification record according to the association relation.
In a second aspect, some embodiments of the present disclosure provide an information migration apparatus, including: a topic determination unit configured to determine whether a target topic exists in the target message processing system in response to receiving migration information, wherein the migration information includes the target topic, an identification information set of an original processor of the target topic, and identification information of the target message processing system; a topic creation unit configured to create a target topic in the target message processing system in response to determining that the target topic does not exist; a processor determining unit configured to determine whether information of a new processor exists in the target message processing system, wherein the new processor is used for processing a message belonging to the target subject; a processor creation unit configured to create information of a new processor in the target message processing system in response to determining that the information of the new processor does not exist; and the adjusting unit is configured to adjust the related information in the target message processing system according to the configuration information of the original processor of the target theme.
In a third aspect, some embodiments of the present disclosure provide an information generating method, including: receiving a migration request, wherein the migration request comprises an identification information set of a processor to be migrated and identification information of a target message processing system; determining a target theme according to the identification information in the identification information set to obtain a target theme set; selecting an identification information group from the identification information set according to the target topics in the target topic set to obtain an identification information group set; and generating migration information according to the identification information groups in the identification information group set to obtain a migration information set, wherein the migration information comprises the target theme, the identification information set of the processor of the target theme and the identification information of the target message processing system.
In some embodiments, determining the target topic according to the identification information in the identification information set to obtain the target topic set includes: inquiring a theme corresponding to the publisher indicated by the identification information in an original message processing system according to the identification information in the identification information set of the publisher to be migrated, and taking the theme as a candidate theme to obtain a candidate theme set, wherein the original message processing system is a message processing system with information to be migrated; for each candidate topic in the set of candidate topics, the following query steps are performed: querying a data interaction task related to the candidate theme in an original message processing system; inquiring topics related to a data interaction task to obtain a topic set; determining whether the topic set contains other topics, wherein the other topics are topics which correspond to the to-be-migrated publisher and are outside the candidate topic set; in response to determining not to include, a candidate topic set is determined as a target topic set.
In some embodiments, selecting an identification information set from the identification information set according to the target topics in the target topic set to obtain an identification information set, including: in an original message processing system, inquiring identification information of a subscriber of a target topic in a target topic set to obtain an identification information set of the subscriber; grouping target topics in the target topic set according to the data interaction task to obtain a plurality of target topic subsets; and selecting an identification information group from the identification information set of the publisher and the identification information set of the subscriber according to the target topic in each target topic subset in the target topic subsets to obtain an identification information group set.
In some embodiments, the method further comprises: and carrying out migration processing on the migration information in the migration information set by adopting the information migration method described in any implementation manner of the first aspect.
In a fourth aspect, some embodiments of the present disclosure provide an information generating apparatus including: a receiving unit configured to receive a migration request, wherein the migration request includes an identification information set of a processor to be migrated and identification information of a target message processing system; the determining unit is configured to determine a target theme according to the identification information in the identification information set to obtain a target theme set; the selecting unit is configured to select an identification information group from the identification information set according to the target subjects in the target subject set to obtain an identification information group set; the generation unit is configured to generate migration information according to the identification information groups in the identification information group set to obtain a migration information set, wherein the migration information comprises a target theme, the identification information set of a processor of the target theme and the identification information of the target message processing system.
In a fifth aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device having one or more programs stored thereon, which when executed by one or more processors causes the one or more processors to implement the method described in any of the above implementations of the first or third aspects.
In a sixth aspect, some embodiments of the present disclosure provide a computer readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method described in any of the above-described implementations of the first or third aspects.
The above embodiments of the present disclosure have the following advantageous effects: the information migration method of some embodiments of the present disclosure provides an implementation manner in which a topic and a processor (i.e., a publisher and/or subscriber) of the topic automatically switch between systems, so that migration efficiency can be improved, and stability of data can be ensured. Specifically, the reasons for the inefficiency of information migration and data instability among the message processing systems are as follows: mainly relies on manual operation. Based on this, the information migration method of some embodiments of the present disclosure may automatically create information of topics and new processors in the target (i.e., to-be-migrated) message processing system. And the related information in the target message processing system can be automatically adjusted according to the configuration information of the original processor. Thus, manual operation can be greatly reduced, and further, the situation of human error is reduced. Migration efficiency can be improved, and stability of data can be guaranteed.
Drawings
The above and other features, advantages, and aspects of embodiments of the present disclosure will become more apparent by reference to the following detailed description when taken in conjunction with the accompanying drawings. The same or similar reference numbers will be used throughout the drawings to refer to the same or like elements. It should be understood that the figures are schematic and that elements and components are not necessarily drawn to scale.
FIG. 1 is an architecture diagram of an exemplary system in which some embodiments of the present disclosure may be applied;
FIG. 2 is a flow chart of some embodiments of an information migration method according to the present disclosure;
FIG. 3 is a schematic structural view of some embodiments of an information migration apparatus according to the present disclosure;
FIG. 4 is a flow chart of some embodiments of an information generation method according to the present disclosure;
FIG. 5 is a schematic diagram of the structure of some embodiments of an information generating apparatus according to the present disclosure;
fig. 6A and 6B are schematic diagrams of application scenarios of an information migration method and an information generation method according to some embodiments of the present disclosure;
fig. 7 is a schematic structural diagram of an electronic device suitable for use in implementing some embodiments of the present disclosure.
Detailed Description
Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete. It should be understood that the drawings and embodiments of the present disclosure are for illustration purposes only and are not intended to limit the scope of the present disclosure.
It should be noted that, for convenience of description, only the portions related to the present invention are shown in the drawings. Embodiments of the present disclosure and features of embodiments may be combined with each other without conflict.
It should be noted that the terms "first," "second," and the like in this disclosure are merely used to distinguish between different devices, modules, or units and are not used to define an order or interdependence of functions performed by the devices, modules, or units.
It should be noted that references to "one", "a plurality" and "a plurality" in this disclosure are intended to be illustrative rather than limiting, and those of ordinary skill in the art will appreciate that "one or more" is intended to be understood as "one or more" unless the context clearly indicates otherwise.
The names of messages or information interacted between the various devices in the embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.
The present disclosure will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.
Fig. 1 illustrates an exemplary system architecture 100 to which the information migration method or information migration apparatus of some embodiments of the present disclosure may be applied.
As shown in fig. 1, the system architecture 100 may include a terminal device 101, networks 102, 104, a server 103, and a server cluster 105 (servers 1051, 1052). Network 102 may be the medium used to provide communication links between terminal device 101 and server 103. Network 104 may be a medium used to provide communication links between servers 103 and server cluster 105. The networks 102, 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.
A user may interact with the server 103 via the network 102 using the terminal device 101 to receive or send messages or the like. Various client applications such as a big data real-time computing class application, a message processing system management class application, a web browser, a shopping class application, an instant messaging tool, and the like may be installed on the terminal device 101.
The terminal device 101 may be hardware or software. When the terminal device 101 is hardware, it may be various electronic devices with a display screen including, but not limited to, a smart phone, a tablet computer, an electronic book reader, a laptop portable computer, a desktop computer, and the like. When the terminal apparatus 101 is software, it can be installed in the above-listed electronic apparatus. It may be implemented as a plurality of software or software modules, for example, for providing distributed services, or as a single software or software module. The present invention is not particularly limited herein.
Servers 1051, 1052 in server cluster 105 may be servers that provide various services, such as servers that provide support for message processing systems, such as a distributed publish-subscribe messaging system (kafka).
The server 103 may be a server providing various services, and may be a background server providing support for applications installed on the terminal device 101, for example. The background server, upon receiving the migration information, may determine whether the target topic and the information of the new processor of the target topic are present in the target message processing system (e.g., server 1051). And further, according to the determination result, migration processing of corresponding information can be performed. And may transmit the processing result (e.g., the adjustment result or the information of the adjusted target message processing system) to the terminal device 101 used by the user.
The servers 1051 and 1052 in the server 103 and the server cluster 105 may be hardware or software. When the server 103 and the servers 1051 and 1052 are hardware, the server may be implemented as a distributed server cluster including a plurality of servers, or as a single server. When the server 103 and the servers 1051 and 1052 are software, they may be implemented as a plurality of software or software modules for providing distributed services, for example, or as a single software or software module. The present invention is not particularly limited herein.
It should be noted that, the method provided by the embodiment of the present disclosure may be performed by the server 103 or may be performed by the terminal device 101. Accordingly, the apparatus may be provided in the server 103 or in the terminal device 101. The present invention is not particularly limited herein.
Note that, in the case where the servers in the server cluster 105 have the functions of the server 103, the server 103 may not be provided in the system architecture 100.
It should be understood that the terminal devices, networks, servers, server clusters, and the number of servers in a server cluster in fig. 1 are merely illustrative. There may be any number of terminal devices, networks, servers, server clusters, and servers in a server cluster, as practical.
With continued reference to fig. 2, a flow 200 of some embodiments of information migration methods according to the present disclosure is shown. The method comprises the following steps:
in response to receiving the migration information, a determination is made as to whether a target topic exists in the target message processing system, step 201.
In some embodiments, the execution body of the information migration method (e.g., server 103 shown in fig. 1) may determine whether a target topic exists in the target message processing system upon receiving the migration information. Wherein the target theme is generally the theme to be migrated. The target message handling system is typically the message handling system into which the target topic is to be migrated.
Here, the message processing system may be a distributed publish-subscribe message system (e.g., kafka). It is typically characterized by high throughput, low latency, scalability, message persistence, high concurrency, etc. One or more servers may typically be included in a message processing system. Such servers are commonly referred to as middlemen (brooker). It can receive messages from publishers, typically set offsets for the messages, and submit the messages to disk storage. Typically in a message processing system there will be one broker that acts as a broker cluster controller at the same time. And each message posted to the message processing system has a topic (also called a category).
In some embodiments, the executing body may migrate the information by way of a wired connection or a wireless connection. The migration information may include the target topic, an identification information set of an original processor of the target topic, and identification information of the target message processing system. The identification information herein may generally be used to uniquely indicate what it represents. The identification information may be at least one of a number, letter, word or symbol. In addition, the original processor is mainly distinguished from the subsequent new processor. The original processor is typically used to process messages belonging to a target topic in a message processing system from which the information is to be migrated (i.e., the original message processing system).
Herein, a processor may generally include a publisher and/or a subscriber. The publisher (i.e., producer) typically publishes a message to a message processing system. Before the message is published, the subject to which the message belongs is determined. And subscribers (i.e., consumers) will typically read messages of the subscribed subject matter from the message processing system. Typically, a publisher corresponds to a topic. While a topic may correspond to multiple publishers and multiple subscribers.
It should be noted that, when the execution body receives the migration information, the execution body may also lock the authentication operation of the related processor, so as to avoid data loss in the subsequent migration processing process. The locking mode is not limited herein, such as tagging the identification information of the relevant processor, or disconnecting the association of the processor with the target subject in the message processing system to be migrated, etc.
In some embodiments, if the execution body has data of the target message processing system stored locally, such as if the execution body is located on the same server as the target message processing system, it may be determined whether there is a topic in the target message processing system that matches (e.g., is the same as) the target topic. Conversely, the executing entity may obtain the subject data thereon from the target message processing system (e.g., server 1051 shown in fig. 1) by way of a wired connection or a wireless connection. And further determines whether a target topic exists in the topic data. Or the executing entity may send a query request containing the target topic to the target message processing system. Thus, according to the returned query result, whether the target theme exists in the query result can be determined.
In some embodiments, if it is determined that the target topic is not present in the target message processing system, step 202 may be performed. If it is determined that the target topic exists, step 203 may be performed.
Step 202, creating a target topic in a target message processing system.
In some embodiments, if it is determined that the target topic does not exist, the execution body may create the target topic in the target message processing system. For example, the execution body may send creation instructions of the target topic to the target message processing system. As an example, the target theme may be created by invoking a corresponding interface (e.g., JSF, java Server Faces, a standard framework for building applications). Or the executing entity may send a message to the target message handling system, the subject to which the message pertains being the target subject. Thus enabling the presence of a target topic in the target message processing system. The relevant configuration in the creation process can be according to the configuration information in the original message processing system.
Step 203 determines if there is information for a new processor in the target message processing system.
In some embodiments, the executing entity may further determine if there is information for a new processor in the target message processing system. Wherein the new processor is used to process messages belonging to the target topic in the target message processing system.
In some embodiments, the execution body may determine whether there is information of a processor corresponding to the target topic in the target message processing system. The handler here corresponds to the new handler required. I.e. if the new processor is a subscriber, the processor here is also a subscriber. If so, it can be stated that there is information of the new processor. Otherwise, the description is not given.
Further, in order to improve the accuracy of the determination result, if it is determined that the information of the processor corresponding to the target topic is included, the execution subject may further determine whether the identification information of the processor matches with the identification information of the original processor of the target topic, that is, whether the identification information of the processor corresponding to the target topic (corresponding to the processor to be migrated) in the original message processing system. If there is a match (e.g., the same), then it may be indicated that there is new processor information in the target message processing system. If there is no match (e.g., different), then no can be indicated.
Alternatively, the execution body may determine whether other topics are present in the target message processing system. Other topics herein may be topics having the same processor as the target topic. For example, topics subscribed to by the same subscriber. If there are other topics as described above, it may be stated that there is new processor information in the target message processing system. Because the new processor's information has already been created at the time other topics are created in the target message processing system. There is no need to repeatedly create this. This helps to improve the migration efficiency of information.
Here, if it is determined that there is no information of a new processor, step 204 may be performed. If it is determined that there is information of a new processor, step 205 may be performed.
Step 204, creating information of the new processor in the target message processing system.
In some embodiments, if it is determined that no new processor information exists, the execution body may create the new processor information in the target message processing system. And may perform a series of configuration operations on the new processor. Such as authorization of the new processor, i.e. authentication. Such as limiting the speed of the new processor, i.e., limiting its processing speed or occupied memory resources, etc. Therefore, the balanced distribution of resources can be realized, and the stability of the target message processing system can be improved. And can establish an association between the new processor and the target subject. In this way, the new processor may publish the message of the target topic to the target message processing system, or the new processor may read the message of the target topic from the target message processing system.
In some embodiments, where the identification information set of migration information includes identification information of a subscriber of the target topic, information specifying a new subscriber of the target topic may be included in the information of a new processor that needs to be created in the target message processing system. And in the case of the completion of creation, the execution body may also reset the location of the new subscriber to a preset location. Here, a site is typically a metadata, which is an increasing integer value, used to record the location of a subscriber when a rebalancing occurs, for use in recovering data. That is, the new subscriber may read the message in the target message processing system from a preset location. The preset position may be any position, for example, may be a starting position of message reading or a position set by a user.
Alternatively, the executing entity may also determine whether there is information about an idle or useless processor in the target message processing system. If so, the execution body may take the processor as a new processor and may perform configuration operations for some columns. Or the processor which corresponds to the target theme and does not match the identification information can be used as a new processor. Therefore, the creation efficiency can be improved, and the information migration efficiency can be further improved.
It will be appreciated that if a topic has previously been migrated from message processing system a to message processing system B. Later, the topic is migrated back from message processing system B to message processing system A. Then there will typically be information about the topic and the processor of the topic in message processing system a.
Step 205, according to the configuration information of the original processor of the target theme, the relevant information in the target message processing system is adjusted.
In some embodiments, the execution subject may obtain or find configuration information of the original processor according to the identification information of the original processor of the target subject. And further, according to the configuration information, the relevant information in the target message processing system can be adjusted.
As an example, the execution body may modify the monitoring parameters of the target message processing system (or new processor) based on the monitoring configuration information of the original processor. For example, the execution body may store the monitoring program in the target message processing system. Or the monitoring may be performed by a separate monitoring node (software program or hardware) in general. In this case, the executing body may invoke the interface provided by the monitoring node, thereby enabling monitoring of the target message processing system. This can improve the modification efficiency.
In some embodiments, the execution body may also determine whether the original processor of the target subject matter has the target data synchronization task. Wherein the target data synchronization task is typically a data synchronization task related to the target topic. A data synchronization task generally refers to synchronizing message data in one message processing system (e.g., a brooker cluster) in real-time to another message processing system (e.g., a brooker cluster). For example, the message published by the publisher is stored in system a. And subscribers can read these messages from system B. Such data synchronization tasks are often performed in the background and are not readily noticeable to the user. If so, the execution subject can construct a new data synchronization task according to the target data synchronization task. Wherein the new data synchronization task may be used to indicate a target topic in the target message processing system. For example, the execution body may replicate the target data synchronization task and modify the target topic therein to the target topic in the target message processing system, such that a new data synchronization task may be obtained and initiated. I.e., modifying the data source or target source to a target topic in the target message processing system.
In some application scenarios, the execution body may also determine whether the original processor has a data interaction task. Here, a data interaction task is generally a type of task that exchanges data between multiple data sources (e.g., fregata access/distribution task). Data interaction among various data sources such as a relational database management system (MySQL), a non-relational database (NoSQL), a theme (Topic) and the like is usually supported, and data integration/synchronization can be integrated. Such data interaction tasks are often user-specified. For example, the message is stored in system a by default and the user may indicate storage in system B. If yes, the execution body can modify the message processing system corresponding to the original processor in the data interaction task into a target message processing system. To ensure the validity of the modified data interaction task, the execution subject may restart the data interaction task.
It can be appreciated that, in order to further improve the migration efficiency, if the migration information includes a plurality of target topics, the data synchronization task and the data interaction task may be performed after a plurality of or all of the target topics are migrated to the target message processing system. So that the repetition of the process can be reduced. In addition, in the case where the information creation of the new processor is completed or the entire migration is completed, the execution subject may also generate a hint information. And may send the hint information to the terminal device of the corresponding processor to restart the message processing (publish and/or subscribe) task. The presentation mode of the prompt information can include, but is not limited to, at least one of pictures, words, audio or video. And the sending mode may include, but is not limited to, mail, short message, telephone, etc. In this case, the user may select whether to restart immediately according to the actual situation. For example, the user may choose to restart after reading the data in the original message processing system to read the data in the target message processing system. If the user can accept a certain amount of data loss, an immediate restart may be selected.
From the above description, it can be appreciated that the information migration method provided by some embodiments of the present disclosure may implement automatic migration of a subject and a processor between message processing systems. A set of topics and processors are not required to be applied in advance in other message processing systems; nor does it require manual manipulation. Therefore, the utilization rate of resources and the information migration efficiency can be improved, and human errors can be avoided or reduced, so that the stability of data can be improved.
In some embodiments, the execution body may also generate the migration record according to the migration process described above. And may store the migration record in a database. This may facilitate the user's retrospective query. Further, the execution body may also send the migration processing procedure to the terminal device for presentation to the user. Therefore, the user can know the migration progress, and the user can conveniently find problems or places needing improvement in time.
As an example, before performing a creation operation of the information of the new processor in the target message processing system, the execution body may determine whether the target record is contained in the database. Wherein the target record may be a record characterizing a target topic created in the target message processing system. If it is determined that the target record is not included, the execution subject may generate the target record and store it in a database. For another example, the execution body may also generate the modification record and store according to an association relationship between the new processor and the target subject. I.e. the ID (Identity document, identity) of the target subject in the new handler record in the database is modified.
It is understood that the database may be any database (e.g., hbase database). The manner in which the migration records (or logs) are maintained may include, but is not limited to, a log system (e.g., log), a topic, or a distributed full text search engine (e.g., elastosearch)
With further reference to fig. 3, as an implementation of the method shown in fig. 2 described above, the present disclosure provides some embodiments of an information migration apparatus, which apparatus embodiments correspond to those method embodiments shown in fig. 2, and the apparatus is particularly applicable to various electronic devices.
As shown in fig. 3, the information migration apparatus 300 of some embodiments may include: a topic determination unit 301 configured to determine whether a target topic exists in the target message processing system in response to receiving migration information, wherein the migration information includes the target topic, an identification information set of an original processor of the target topic, and identification information of the target message processing system; a topic creation unit 302 configured to create a target topic in the target message processing system in response to determining that the target topic does not exist; a processor determining unit 303 configured to determine whether or not there is information of a new processor in the target message processing system, wherein the new processor is used for processing a message belonging to the target subject; a processor creation unit 304 configured to create information of a new processor in the target message processing system in response to determining that the information of the new processor does not exist; an adjusting unit 305 is configured to adjust relevant information in the target message processing system according to the configuration information of the original processor of the target subject.
In some embodiments, the adjustment unit 305 may be further configured to determine whether the original processor of the target subject has a target data synchronization task, wherein the target data synchronization task is a data synchronization task related to the target subject; in response to determining that there is a target data synchronization task, a new data synchronization task is constructed in accordance with the target data synchronization task, wherein the new data synchronization task is used to indicate a target topic in the target message processing system.
In some embodiments, the adjustment unit 305 may be further configured to determine whether the original processor of the target subject matter has a data interaction task; and in response to determining that the data interaction task exists, modifying a message processing system corresponding to the original processor in the data interaction task into a target message processing system.
In some embodiments, the adjustment unit 305 may be further configured to modify the monitoring parameters in the target message processing system according to the monitoring configuration information of the original processor of the target subject matter.
In some embodiments, the processor creation unit 304 may be configured to create information of a new subscriber of the target topic in the target message processing system and reset the location of the new subscriber to a preset location.
In some embodiments, the apparatus 300 may further comprise a storage unit (not shown in fig. 3) configured to generate and store the migration record to the database.
In some embodiments, the storage unit may be further configured to determine whether a target record is contained in the database, wherein the target record is a record characterizing creation of a target topic in the target message processing system; generating and storing the target record in response to determining that the target record is not included; establishing an association relation between the new processor and the target theme; and generating and storing a modification record according to the association relation.
It will be appreciated that the elements described in the apparatus 300 correspond to the various steps in the method described with reference to fig. 2. Thus, the operations, features and resulting benefits described above with respect to the method are equally applicable to the apparatus 300 and the units contained therein, and are not described in detail herein.
With continued reference to fig. 4, a flow 400 of some embodiments of information generation methods according to the present disclosure is shown. The method may further comprise the steps of:
step 401, a migration request is received.
In some embodiments, the execution body of the information generating method may receive the migration request through a wired connection manner or a wireless connection manner. Wherein the migration request may include an identification information set of processors to be migrated and identification information of the target message processing system. The processors herein may include publishers and/or subscribers. The target message processing system is typically the message processing system to be migrated.
It is to be understood that the execution body of the present embodiment may or may not be the same execution body as the execution body in the embodiment of fig. 2 (e.g., the server 103 shown in fig. 1). The migration request may be sent by the user, or may be sent by other electronic devices detecting a trigger signal. For example, when the message processing system a (e.g., the server 1051 shown in fig. 1) detects that a certain operation index reaches a preset value, the migration request may be sent to the execution body, so that information therein may be migrated. The operation index here is generally related to the operation condition of the system, and when the index reaches a preset value, it indicates that the system may malfunction or the operation processing efficiency may be significantly reduced. In this case, the target message processing system may be preset by the user, or may be selected according to the current operation conditions of other systems.
Step 402, determining a target theme according to the identification information in the identification information set, and obtaining a target theme set.
In some embodiments, the execution body may determine the target topic based on the identification information in the migration request, such that a target topic set may be obtained. The target topic here is generally the topic to be migrated.
As an example, if the migration request includes an identification information set of the subscriber to be migrated, that is, information migration of the subscriber, the execution body may query the original message processing system for a topic related to the identification information according to each or more identification information in the identification information set. I.e. the topic of the message to be read by the subscriber indicated by the identification information. Thus, the queried theme can be used as the target theme to obtain the target theme set. The original message handling system here is typically the message handling system from which the information is to be migrated.
For another example, if the migration request includes an identification information set of the publisher to be migrated, that is, information migration of the publisher, the execution subject may query, in the original message processing system, a topic corresponding to the publisher indicated by the identification information according to the identification information in the identification information set. I.e. the subject of the message issued by the issuer indicated by the identification information. Thus, the queried theme can be used as a candidate theme to obtain a candidate theme set.
Further, for each candidate topic in the set of candidate topics, the following query steps may be performed: querying a data interaction task related to the candidate topic in an original message processing system; inquiring topics related to a data interaction task to obtain a topic set; determining whether the set of topics contains other topics; in response to determining not to include, a candidate topic set is determined as a target topic set. Wherein the other topics are typically topics outside the candidate topic set corresponding to the publisher to be migrated. The publisher to be migrated here is the publisher indicated by the migration request. Here, if the topic set includes other topics, the executing body may use the other topics as candidate topics, and execute the query step again.
Step 403, selecting the identification information group from the identification information set according to the target subject in the target subject set, and obtaining the identification information group set.
In some embodiments, the executing entity may select the set of identification information groups from the set of identification information based on one or more target topics in the set of target topics, resulting in a set of sets of identification information groups.
As an example, if the migration request includes an identification information set of the subscriber to be migrated, the execution body may select an identification information set from the identification information set according to each target topic in the target topic set. I.e. selecting the identification information of all subscribers to the target topic. For example, if subscriber a and subscriber B are both consumers of topic T1, then subscriber a and subscriber B may be grouped together. That is, subscribers in the same group of identification information all subscribe to the same target topic.
Optionally, if the migration request includes the identification information set of the publisher to be migrated, the execution body may first query, in the original message processing system, the identification information of the subscriber of the target topic in the target topic set, to obtain the identification information set of the subscriber. Then, according to the data interaction task, the target topics in the target topic set can be grouped to obtain a plurality of target topic subsets. And then, selecting an identification information group from the identification information set of the publisher and the identification information set of the subscriber according to the target topic in each target topic subset in the target topic subsets to obtain an identification information group set. For example, topics T1 and T2 are involved in data interaction task 1. At this time, the topics T1 and T2 may be divided into a group (target topic subset). Thus, publishers and subscribers of the topics T1 and T2 can be divided into a group to obtain the identification information group. I.e. the identification information of the processors involved is grouped in a task + topic way.
It can be appreciated that, for information migration of publishers, the information generating method in this embodiment may link publishing and subscribing. That is, in the case of receiving the identification information of the publisher to be migrated, the identification information of the related subscriber (i.e., the subscriber to be migrated) may be automatically determined, thereby generating migration information. In this way, one-touch migration switching of topics, publishers and subscribers can be achieved. Therefore, the manual operation can be further reduced, and the migration efficiency is further improved. And can avoid that subscribers cannot consume data after the upstream publisher is switched.
And step 404, generating migration information according to the identification information groups in the identification information group set to obtain a migration information set.
In some embodiments, based on the set of identification information groups obtained in step 403, the executing body may generate migration information, and thus a set of information to be migrated, from each of the identification information groups therein. The migration information may include, among other things, the target topic, a set of identifying information for a processor of the target topic (i.e., a set of identifying information), and identifying information for the target message processing system. It should be noted that the target theme in the migration information is the target theme when the identification information group is selected.
As an example, if both the subscriber a and the subscriber B are consumers of the topic T1, the identification information group including the subscriber a and the subscriber B may be selected according to the topic T1. The migration information thus generated may include the topic T1, identification information of the subscriber a, identification information of the subscriber B, and identification information of the target message processing system.
For another example, an identification information group including publisher a, publisher B, and subscriber C is selected from a subset of topics including topics T1 and T2. The migration information thus generated may include topics T1 and T2, identification information of publisher A, identification information of publisher B, identification information of subscriber C, and identification information of the target message processing system
In some application scenarios, the execution body may further perform migration processing on each migration information in the migration information set by using the information migration method described in any implementation manner in the embodiment of fig. 2. It is understood that, to enhance the migration processing efficiency, the execution bodies may perform processing by using multithreading, respectively. For example, a thread may perform the operations of the information migration method described above on a migration message. Alternatively, the execution body may send each migration information in the migration information set to each server (such as servers 1051, 1052 shown in fig. 1) in the server cluster, respectively. Each server can execute the information migration method, so that information migration processing is carried out on the received migration information. That is, by grouping, one migration request can be split, thereby obtaining a plurality of migration information. Further, a plurality of threads of a plurality of machines can process a plurality of pieces of migration information, respectively. Thereby, the optimization processing of information migration can be realized.
As can be seen from the above description, the information generating method provided by some embodiments of the present disclosure may implement automatic acquisition and generation of migration information. This can reduce man-made operations. Not only can the information generation efficiency be improved, but also the integrity and the comprehensiveness of the information can be ensured. Therefore, the influence on the use of the message processing system after the migration caused by incomplete migration of the subsequent information due to omission can be avoided. In addition, by grouping and splitting the information related to the migration request, the subsequent batch processing of the migration information can be facilitated. Thereby contributing to an improvement in processing efficiency.
With further reference to fig. 5, as an implementation of the method shown in fig. 4 described above, the present disclosure provides some embodiments of information generating apparatus, which correspond to those method embodiments shown in fig. 4, and which are particularly applicable in various electronic devices.
As shown in fig. 5, the information generating apparatus 500 of some embodiments may include: a receiving unit 501 configured to receive a migration request, wherein the migration request includes an identification information set of a processor to be migrated and identification information of a target message processing system; a determining unit 502 configured to determine a target topic according to the identification information in the identification information set, to obtain a target topic set; a selecting unit 503 configured to select an identification information group from the identification information set according to the target subjects in the target subject set, to obtain an identification information group set; a generating unit 504, configured to generate migration information according to the identification information groups in the identification information group set, so as to obtain a migration information set, where the migration information includes the target topic, the identification information set of the processor of the target topic, and the identification information of the target message processing system.
In some embodiments, the determining unit 502 may be further configured to query, according to the identification information in the identification information set of the publisher to be migrated, a topic corresponding to the publisher indicated by the identification information in the original message processing system, as a candidate topic, to obtain a candidate topic set, where the original message processing system is the message processing system to which the information is to be migrated; for each candidate topic in the set of candidate topics, the following query steps are performed: querying a data interaction task related to the candidate theme in an original message processing system; inquiring topics related to a data interaction task to obtain a topic set; determining whether the topic set contains other topics, wherein the other topics are topics which correspond to the to-be-migrated publisher and are outside the candidate topic set; in response to determining not to include, a candidate topic set is determined as a target topic set.
In some embodiments, the selecting unit 503 may be further configured to query, in the original message processing system, identification information of a subscriber of the target topic in the target topic set, to obtain an identification information set of the subscriber; grouping target topics in the target topic set according to the data interaction task to obtain a plurality of target topic subsets; and selecting an identification information group from the identification information set of the publisher and the identification information set of the subscriber according to the target topic in each target topic subset in the target topic subsets to obtain an identification information group set.
In some embodiments, the apparatus 500 may further include a processing unit (not shown in fig. 5) configured to perform migration processing on the migration information in the migration information set using the information migration method described in any implementation of the embodiment of fig. 2.
It will be appreciated that the elements described in the apparatus 500 correspond to the various steps in the method described with reference to fig. 4. Thus, the operations, features and resulting benefits described above with respect to the method are equally applicable to the apparatus 500 and the units contained therein, and are not described in detail herein.
Referring to fig. 6A and 6B, schematic diagrams of an application scenario of the information migration method and the information generation method of the present disclosure are shown, respectively.
In this application scenario, as shown in fig. 6A, a user may manage a page at a producer, select an identification information set of a publisher to be migrated in the message processing system a, and select a message processing system B to be migrated (hereinafter referred to as system B). The host server 601 may invoke a task interface (e.g., fregata interface) to query related topics and subscribers to obtain a target topic set and an identification information set of subscribers to be migrated. And the identification information of the publishers and subscribers can be grouped according to the task and theme mode, so that migration information is generated, and a migration information set is obtained. The main server 601 may send migration information to the sub servers 602, 603, respectively.
As can be seen in fig. 6A, the sub-server 602, upon receiving the migration information, may first lock the authentication of the involved publishers and subscribers. Next, it may be determined whether the target topic is present in system B. If not, a target topic is created in system B. If so, it may be determined whether a target record (i.e., a record that creates a target topic in system B) exists in database 604. If not, a target record may be generated and inserted into database 604. Thereafter, creation, authorization, speed limiting operations, etc. of the publisher information may be performed in the system B, and a corresponding modification record may be generated. Then, creation, authorization, speed limiting operation, etc. of subscriber information can be performed in the system B, and a corresponding modification record is generated. And may reset the subscriber's location to the starting location. Notifying the subscriber that the user has restarted the program. And modifying the monitoring information configured by the publishers and subscribers to the system B. Thereafter, it can be determined whether the original publisher has a data sync task. If so, a data synchronization task can be newly established. And adds the relevant records in database 604. If not, it can be further determined whether the original publisher has a data interaction task. If so, the data interaction task may be modified. If not, the migration is ended.
In addition, as shown in fig. 6B, the user may select, on the producer management page, an identification information set of subscribers to be migrated in the message processing system a, and select a message processing system B to be migrated (hereinafter referred to as system B), and a time to reset a site in the system B. The main server 601 may query related topics to obtain a target set of topics. And the identification information of the subscribers can be grouped according to the topic mode, so that migration information is generated, and a migration information set is obtained. The main server 601 may send migration information to the sub servers 602, 603, respectively.
As can be seen in fig. 6B, the sub-server 602, upon receiving the migration information, may first lock the authentication of the subscriber involved. Next, it may be determined whether the target topic is present in system B. If not, a target topic is created in system B. If so, it may be determined whether a target record (i.e., a record that creates a target topic in system B) exists in database 604. If not, a target record may be generated and inserted into database 604. Thereafter, creation, authorization, rate limiting operations, etc. of subscriber information may be performed in system B, and a corresponding modification record may be generated. And may reset the subscriber's location to the time set by the user. And modifying the monitoring information of the subscriber configuration to the system B. Thereafter, it may be determined whether the original subscriber has a data synchronization task. If so, a data synchronization task can be newly established. And adds the relevant records in database 604. If not, it can be further determined whether the original subscriber has the data interaction task. If so, the data interaction task may be modified. If not, the migration is ended.
It should be noted that the migration process may be modularized, for example, in fig. 6A and 6B, a process flow included in a dashed box may be completed by a node. The node may be a software program or may be hardware. Thus, the coupling between the processing flows can be reduced, and the flexibility of the expansion of the processing flows is improved. Therefore, the quick access of the new module can be realized, and the difficulty level of the method expansion is reduced.
Referring now to fig. 7, a schematic diagram of an electronic device (e.g., server 103 of fig. 1) 700 suitable for use in implementing some embodiments of the present disclosure is shown. The electronic device shown in fig. 7 is only one example and should not impose any limitations on the functionality and scope of use of embodiments of the present disclosure.
As shown in fig. 7, the electronic device 700 may include a processing means (e.g., a central processor, a graphics processor, etc.) 701, which may perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 702 or a program loaded from a storage means 708 into a Random Access Memory (RAM) 703. In the RAM703, various programs and data required for the operation of the electronic device 700 are also stored. The processing device 701, the ROM 702, and the RAM703 are connected to each other through a bus 704. An input/output (I/O) interface 705 is also connected to bus 704.
In general, the following devices may be connected to the I/O interface 705: input devices 706 including, for example, a touch screen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, and the like; an output device 707 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage 708 including, for example, magnetic tape, hard disk, etc.; and a communication device 709. The communication means 709 may allow the electronic device 700 to communicate wirelessly or by wire with other devices to exchange data. While fig. 7 shows an electronic device 700 having various means, it is to be understood that not all of the illustrated means are required to be implemented or provided. More or fewer devices may be implemented or provided instead. Each block shown in fig. 7 may represent one device or a plurality of devices as needed.
In particular, according to some embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, some embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flow chart. In such embodiments, the computer program may be downloaded and installed from a network via communications device 709, or from storage 708, or from ROM 702. The above-described functions defined in the methods of some embodiments of the present disclosure are performed when the computer program is executed by the processing means 701.
It should be noted that, the computer readable medium described in some embodiments of the present disclosure may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In some embodiments of the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In some embodiments of the present disclosure, however, the computer-readable signal medium may comprise a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, fiber optic cables, RF (radio frequency), and the like, or any suitable combination of the foregoing.
In some implementations, the clients, servers may communicate using any currently known or future developed network protocol, such as HTTP (HyperText Transfer Protocol ), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the internet (e.g., the internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed networks.
The computer readable medium may be contained in the electronic device; or may exist alone without being incorporated into the electronic device. The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: determining whether a target theme exists in the target message processing system or not in response to receiving migration information, wherein the migration information comprises the target theme, an identification information set of an original processor of the target theme and identification information of the target message processing system; creating a target topic in the target message processing system in response to determining that the target topic does not exist; determining whether information of a new processor exists in the target message processing system, wherein the new processor is used for processing messages belonging to a target theme; creating information of the new processor in the target message processing system in response to determining that the information of the new processor does not exist; and adjusting the related information in the target message processing system according to the configuration information of the original processor of the target theme. Or receiving a migration request, wherein the migration request comprises an identification information set of a processor to be migrated and identification information of a target message processing system; determining a target theme according to the identification information in the identification information set to obtain a target theme set; selecting an identification information group from the identification information set according to the target topics in the target topic set to obtain an identification information group set; and generating migration information according to the identification information groups in the identification information group set to obtain a migration information set, wherein the migration information comprises the target theme, the identification information set of the processor of the target theme and the identification information of the target message processing system.
Furthermore, computer program code for carrying out operations of some embodiments of the present disclosure may be written in one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in some embodiments of the present disclosure may be implemented by means of software, or may be implemented by means of hardware. The described units may also be provided in a processor, for example, described as: a processor includes a theme determining unit, a theme creating unit, a processor determining unit, a processor creating unit, and an adjusting unit. Or a receiving unit, a determining unit, a selecting unit and a generating unit. Wherein the names of these units do not constitute a limitation of the unit itself in some cases, for example, the topic determination unit may also be described as "unit determining whether a target topic is present in the target message processing system".
The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: a Field Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), an Application Specific Standard Product (ASSP), a system on a chip (SOC), a Complex Programmable Logic Device (CPLD), and the like.
The foregoing description is only of the preferred embodiments of the present disclosure and description of the principles of the technology being employed. It will be appreciated by those skilled in the art that the scope of the invention in the embodiments of the present disclosure is not limited to the specific combination of the above technical features, but encompasses other technical features formed by any combination of the above technical features or their equivalents without departing from the spirit of the invention. Such as the above-described features, are mutually substituted with (but not limited to) the features having similar functions disclosed in the embodiments of the present disclosure.

Claims (15)

1. An information migration method, comprising:
determining whether a target topic exists in a target message processing system or not in response to receiving migration information, wherein the migration information comprises the target topic, an identification information set of an original processor of the target topic and identification information of the target message processing system;
creating the target topic in the target message processing system in response to determining that the target topic is not present;
determining whether information of a new processor exists in the target message processing system, wherein the new processor is used for processing messages belonging to the target theme;
creating information of the new processor in the target message processing system in response to determining that the information of the new processor does not exist;
and adjusting the related information in the target message processing system according to the configuration information of the original processor of the target theme.
2. The information migration method of claim 1, wherein the adjusting the related information in the target message processing system according to the configuration information of the original processor of the target subject includes:
determining whether an original processor of the target theme has a target data synchronization task, wherein the target data synchronization task is a data synchronization task related to the target theme;
In response to determining that the target data synchronization task exists, a new data synchronization task is constructed according to the target data synchronization task, wherein the new data synchronization task is used for indicating the target theme in the target message processing system.
3. The information migration method of claim 1, wherein the adjusting the related information in the target message processing system according to the configuration information of the original processor of the target subject includes:
determining whether an original processor of the target theme has a data interaction task or not;
and in response to determining that the data interaction task exists, modifying a message processing system corresponding to the original processor in the data interaction task into the target message processing system.
4. The information migration method of claim 1, wherein the adjusting the related information in the target message processing system according to the configuration information of the original processor of the target subject includes:
and modifying the monitoring parameters in the target message processing system according to the monitoring configuration information of the original processor of the target theme.
5. The information migration method of claim 1, wherein the creating the information of the new processor in the target message processing system comprises:
In the target message processing system, creating information of a new subscriber of the target topic, and resetting the location of the new subscriber to a preset position.
6. The information migration method of one of claims 1 to 5, wherein the method further comprises:
and generating migration records and storing the migration records in a database.
7. The information migration method of claim 6, wherein the method further comprises:
determining whether a target record is contained in the database, wherein the target record is a record characterizing creation of the target topic in the target message processing system;
generating and storing the target record in response to determining not to include the target record; and
establishing an association relationship between the new processor and the target subject;
and generating and storing a modification record according to the association relation.
8. An information migration apparatus comprising:
a topic determination unit configured to determine whether a target topic exists in a target message processing system in response to receiving migration information, wherein the migration information includes the target topic, an identification information set of an original processor of the target topic, and identification information of the target message processing system;
A topic creation unit configured to create the target topic in the target message processing system in response to determining that the target topic is not present;
a processor determining unit configured to determine whether information of a new processor exists in the target message processing system, wherein the new processor is used for processing a message belonging to the target subject;
a processor creation unit configured to create information of the new processor in the target message processing system in response to determining that the information of the new processor does not exist;
and the adjusting unit is configured to adjust the related information in the target message processing system according to the configuration information of the original processor of the target theme.
9. An information generation method, comprising:
receiving a migration request, wherein the migration request comprises an identification information set of a processor to be migrated and identification information of a target message processing system;
determining a target theme according to the identification information in the identification information set to obtain a target theme set;
selecting an identification information group from the identification information set according to the target topics in the target topic set to obtain an identification information group set;
And generating migration information according to the identification information groups in the identification information group set to obtain a migration information set, wherein the migration information comprises a target theme, the identification information set of a processor of the target theme and the identification information of the target message processing system.
10. The information generating method according to claim 9, wherein the determining a target topic according to the identification information in the identification information set, to obtain a target topic set, includes:
inquiring a theme corresponding to the publisher indicated by the identification information in an original message processing system according to the identification information in the identification information set of the publisher to be migrated, and taking the theme as a candidate theme to obtain a candidate theme set, wherein the original message processing system is a message processing system with information to be migrated;
for each candidate topic in the set of candidate topics, performing the following query steps:
querying the original message processing system for data interaction tasks related to the candidate topics;
inquiring topics related to the data interaction task to obtain a topic set;
determining whether the topic set contains other topics, wherein the other topics are topics which correspond to the publisher to be migrated and are outside the candidate topic set;
In response to determining not to include, the candidate topic set is determined to be a target topic set.
11. The information generating method according to claim 10, wherein the selecting, according to the target topics in the target topic set, an identification information group from the identification information set to obtain an identification information group set includes:
querying identification information of a subscriber of a target topic in the target topic set in the original message processing system to obtain an identification information set of the subscriber;
grouping the target topics in the target topic set according to the data interaction task to obtain a plurality of target topic subsets;
and selecting an identification information group from the identification information set of the publisher and the identification information set of the subscriber according to the target topic in each target topic subset in the target topic subsets to obtain an identification information group set.
12. The information generating method according to one of claims 9 to 11, wherein the method further comprises:
the information migration method according to any one of claims 1 to 7, wherein migration information in the migration information set is subjected to migration processing.
13. An information generating apparatus comprising:
A receiving unit configured to receive a migration request, wherein the migration request includes an identification information set of a processor to be migrated and identification information of a target message processing system;
the determining unit is configured to determine a target theme according to the identification information in the identification information set to obtain a target theme set;
a selecting unit configured to select an identification information group from the identification information set according to the target topics in the target topic set, to obtain an identification information group set;
the generation unit is configured to generate migration information according to the identification information groups in the identification information group set to obtain a migration information set, wherein the migration information comprises a target theme, the identification information set of a processor of the target theme and the identification information of the target message processing system.
14. An electronic device, comprising:
one or more processors;
a storage device having one or more programs stored thereon,
when executed by the one or more processors, cause the one or more processors to implement the method of any of claims 1-7 or 9-12.
15. A computer readable medium having stored thereon a computer program, wherein the program when executed by a processor implements the method of any of claims 1-7 or 9-12.
CN202210120958.9A 2022-02-09 2022-02-09 Information migration method, information generation method, information migration program, and information migration program apparatus, device and readable medium Pending CN116610652A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210120958.9A CN116610652A (en) 2022-02-09 2022-02-09 Information migration method, information generation method, information migration program, and information migration program apparatus, device and readable medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210120958.9A CN116610652A (en) 2022-02-09 2022-02-09 Information migration method, information generation method, information migration program, and information migration program apparatus, device and readable medium

Publications (1)

Publication Number Publication Date
CN116610652A true CN116610652A (en) 2023-08-18

Family

ID=87675203

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210120958.9A Pending CN116610652A (en) 2022-02-09 2022-02-09 Information migration method, information generation method, information migration program, and information migration program apparatus, device and readable medium

Country Status (1)

Country Link
CN (1) CN116610652A (en)

Similar Documents

Publication Publication Date Title
CN110609872B (en) Method and apparatus for synchronizing node data
CN111143382B (en) Data processing method, system and computer readable storage medium
US20240012641A1 (en) Model construction method and apparatus, and medium and electronic device
CN110909521B (en) Online document information synchronous processing method and device and electronic equipment
CN111679990B (en) Test data generation method and device, readable medium and electronic equipment
CN111309747A (en) Data synchronization method, system and device
US20240045879A1 (en) Presentation method, apparatus and electronic device
CN111338834B (en) Data storage method and device
CN113190517B (en) Data integration method and device, electronic equipment and computer readable medium
CN111752939A (en) Data processing method, device, computer system and medium for multiple systems
CN112148744A (en) Page display method and device, electronic equipment and computer readable medium
CN110855562A (en) Internet of things data distribution management method and system, storage medium and management terminal
US11277300B2 (en) Method and apparatus for outputting information
CN116610652A (en) Information migration method, information generation method, information migration program, and information migration program apparatus, device and readable medium
CN112241332B (en) Interface compensation method and device
CN110324425B (en) Hybrid cloud transaction route processing method and device
CN115017149A (en) Data processing method and device, electronic equipment and storage medium
CN112817874A (en) User interface test method, device, equipment and medium
CN115079873A (en) Information display method and device, electronic equipment and storage medium
CN113064987A (en) Data processing method, apparatus, electronic device, medium, and program product
CN117422556B (en) Derivative transaction system, device and computer medium based on replication state machine
CN113761548B (en) Data transmission method and device for Shuffle process
CN113760860B (en) Data reading method and device
CN111400322B (en) Method, apparatus, electronic device and medium for storing data
US10489219B2 (en) Transitioning between data stream processors in a publish-subscribe system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination