CN115480939A

CN115480939A - Data sharing method, device and system based on message queue

Info

Publication number: CN115480939A
Application number: CN202211143612.7A
Authority: CN
Inventors: 谢心甜; 王鹏; 韩广宇
Original assignee: Hangzhou Dt Dream Technology Co Ltd
Current assignee: Hangzhou Dt Dream Technology Co Ltd
Priority date: 2022-09-20
Filing date: 2022-09-20
Publication date: 2022-12-16

Abstract

The application provides a data sharing method, a device and a system based on a message queue, wherein the method comprises the following steps: the method comprises the steps of responding to an execution request aiming at a target task, obtaining target resource data from a target message queue according to a resource identifier contained in the target task, wherein the target task further comprises a processing rule aiming at data stored in the target message queue and a notification mode aiming at a processing result, processing the obtained target resource data aiming at the processing rule to obtain a processing result, and further sharing the processing result according to the notification mode. Therefore, the acquired target resource data can be processed based on the processing rule aiming at the data stored in the target message queue included in the target task in response to the execution request aiming at the target task, so that a message consumer does not need to develop a processing process by himself, meanwhile, the message queue is effectively prevented from being directly exposed to the message consumer, and the safety of the message queue is ensured.

Description

Data sharing method, device and system based on message queue

Technical Field

The application relates to the technical field of big data, in particular to data sharing in the field of data transmission. The application provides a data sharing method, device and system based on a message queue.

Background

Message Queues (MQs) are containers that hold messages during their transmission. The message queue is essentially a queue in which individual messages are stored. A Message (Message) is a unit of data that is transmitted between two computers. The message may be very simple, e.g. containing only text strings; it may also be more complex, for example, complex object structures may be embedded. The sender of a message is called a message Producer (Producer) and the recipient of the message is called a message Consumer (Consumer).

In the related technology, on one hand, the message queue is directly exposed to the message consumer, and the message queue has a safety problem, on the other hand, the data in the message queue generally needs to be processed to have significance, and the processing processes need the message consumer to develop by oneself and are not friendly to the message consumer.

Disclosure of Invention

The present application is directed to solving, at least to some extent, one of the technical problems in the related art.

Therefore, a first objective of the present application is to provide a data sharing method based on a message queue, so as to achieve that, in response to an execution request for a target task, target resource data is obtained from the target message queue according to a resource identifier included in the target task, thereby effectively avoiding directly exposing the message queue to a message consumer, and ensuring the security of the message queue. Meanwhile, the acquired target resource data is processed based on the processing rule aiming at the data stored in the target message queue included in the target task, so that a message consumer does not need to develop a processing process by himself.

A second objective of the present application is to provide a message queue-based data sharing apparatus.

A third objective of the present application is to provide a data sharing system based on message queue.

A fourth object of the present application is to provide a server.

A fifth object of the present application is to propose a server.

A sixth object of the present application is to propose a non-transitory computer readable storage medium.

In order to achieve the above object, an embodiment of a first aspect of the present application provides a message queue-based data sharing method, where the method includes:

responding to an execution request aiming at a target task, and acquiring target resource data from a target message queue according to a resource identifier contained in the target task; the target task further comprises a processing rule aiming at data stored in the target message queue and a notification mode aiming at a processing result;

processing the acquired target resource data according to the processing rule to obtain a processing result;

and sharing the processing result according to the notification mode.

Optionally, as a first possible implementation manner of the first aspect, the obtaining target resource data from a target message queue according to a resource identifier included in the target task includes:

acquiring target resource data information according to the resource identifier, and establishing connection with a target message queue storing target resource data according to data connection information in the target resource data information;

and acquiring the target resource data through the established connection.

Optionally, as a second possible implementation manner of the first aspect, the processing rule includes an alignment rule and a reserved field reserved in an alignment result; the processing the acquired target resource data according to the processing rule to obtain a processing result includes:

screening out resource data which accord with the comparison rule from the target resource data;

and determining the data corresponding to the reserved field in the resource data meeting the comparison rule as the processing result.

Optionally, as a third possible implementation manner of the first aspect, the notification mode includes a real-time sharing mode, a periodic sharing mode, and an interactive sharing mode; the sharing the processing result according to the notification mode includes:

responding to the notification mode being a real-time sharing mode, and sharing the processing result through sharing connection in real time; the shared connection comprises a WebSocket connection;

responding to the fact that the notification mode is a periodic sharing mode, determining corresponding notification content according to whether the processing result exists in a preset period, and periodically sharing the notification content through the sharing connection;

in response to the notification mode being the interactive sharing mode, after sharing the processing result through the sharing connection, receiving a new operation command sent through the sharing connection and performing corresponding processing; the new operation command includes, but is not limited to: reset the consumption offset and view the consumption details.

Optionally, as a fourth possible implementation manner of the first aspect, after the responding to that the notification mode is a periodic sharing mode, determining corresponding notification content according to whether the processing result exists within a preset period, and periodically sharing the notification content through the sharing connection, the method further includes:

and responding to the processing result existing in a preset period, receiving an acquisition request aiming at the processing result, and sharing the processing result.

Optionally, as a fifth possible implementation manner of the first aspect, the processing the acquired target resource data according to the processing rule to obtain a processing result includes:

converting the target resource data according to the column information contained in the target task to obtain the target resource data containing the column information;

and processing the target resource data containing the column information according to the processing rule to obtain a processing result.

Optionally, as a sixth possible implementation manner of the first aspect, before the target task is executed, the method further includes:

receiving a creation request aiming at the target task and initiated based on a task interface, and creating the target task; the target task comprises condition information corresponding to the task interface; the condition information includes, but is not limited to: resource identification, comparison rules, reserved fields and notification modes.

Optionally, as a seventh possible implementation manner of the first aspect, the task interface is created by using a class structured query language SQL writing method; the task interface comprises a task creation interface, a task cancellation interface, a task state inquiry interface, a task data acquisition interface and a data change notification interface.

In order to achieve the above object, a second aspect of the present application provides a message queue-based data sharing apparatus, including:

the acquisition module is used for responding to an execution request aiming at a target task and acquiring target resource data from a target message queue according to a resource identifier contained in the target task; the target task also comprises a processing rule aiming at data stored in the target message queue and a notification mode aiming at a processing result;

the first processing module is used for processing the acquired target resource data according to the processing rule to obtain a processing result;

and the sharing module is used for sharing the processing result according to the notification mode.

Optionally, as a first possible implementation manner of the second aspect, the obtaining module is further configured to:

and acquiring the target resource data through the established connection.

Optionally, as a second possible implementation manner of the second aspect, the processing rule includes an alignment rule, and a reserved field reserved in the alignment result; the first processing module is further configured to:

and determining data corresponding to the reserved fields in the resource data which accords with the comparison rule as the processing result.

Optionally, as a third possible implementation manner of the second aspect, the notification mode includes real-time sharing, periodic sharing, and interactive sharing, and the sharing module includes:

the first sharing unit is used for responding to the notification mode as a real-time sharing mode and sharing the processing result through sharing connection in real time; the shared connection comprises a WebSocket connection;

the second sharing unit is used for responding to the fact that the notification mode is a periodic sharing mode, determining corresponding notification content according to whether the processing result exists in a preset period, and periodically sharing the notification content through the sharing connection;

the interaction unit is used for responding to the fact that the notification mode is an interaction sharing mode, receiving a new operation command sent by the sharing connection after the processing result is shared by the sharing connection, and carrying out corresponding processing; the new operation command includes, but is not limited to: reset the consumption offset and view the consumption details.

Optionally, as a fourth possible implementation manner of the second aspect, the apparatus further includes:

and the second processing module is used for responding to the processing result existing in a preset period, receiving an acquisition request aiming at the processing result and sharing the processing result.

Optionally, as a fifth possible implementation manner of the first aspect, the first processing module is further configured to:

Optionally, as a sixth possible implementation manner of the second aspect, the apparatus further includes:

the third processing module is used for receiving a creation request aiming at the target task and initiated based on a task interface and creating the target task; the creating request comprises condition information corresponding to the task interface; the condition information includes, but is not limited to: resource identification, comparison rules, reserved fields and notification modes.

Optionally, as a seventh possible implementation manner of the second aspect, the task interface is created by using a SQL-like writing method; the task interface comprises a task establishing interface, a task canceling interface, a task state inquiring interface, a task data acquiring interface and a data change informing interface.

In order to achieve the above object, an embodiment of a third aspect of the present application provides a data sharing system based on a message queue, where the data sharing system based on the message queue includes a configuration management module, a task management module, and a task processing module:

the configuration management module is used for maintaining resource data information, and the resource data information is used for describing basic information of the resource data and comprises data connection information, resource information and column information;

the task management module is used for providing task interfaces, and the task interfaces comprise a task creation interface, a task cancellation interface, a task state inquiry interface, a task data acquisition interface and a data change notification interface;

and the task processing module is used for executing the message queue-based data sharing method in the first aspect.

To achieve the above object, a fourth aspect of the present application provides a server, including:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the message queue based data sharing method of the first aspect.

To achieve the above object, a fifth embodiment of the present application provides a server, including:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

the memory stores instructions executable by the at least one processor to enable the at least one processor to run a data sharing system;

the data sharing system comprises a configuration management module, a task management module and a task processing module; the configuration management module is used for maintaining resource data information, and the resource data information is used for describing basic information of resource data and comprises data connection information, resource information and column information; the task management module is used for providing a task interface for a task issuing party, wherein the task interface comprises a task creation interface, a task cancellation interface, a task state inquiry interface, a task data acquisition interface and a data change notification interface; the task processing module is configured to execute the message queue-based data sharing method according to the first aspect.

To achieve the above object, a sixth aspect of the present application proposes a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are configured to cause the computer to execute the message queue based data sharing method of the first aspect.

The technical scheme provided by the embodiment of the application has the following beneficial effects:

the method comprises the steps of responding to an execution request aiming at a target task, obtaining target resource data from a target message queue according to a resource identifier contained in the target task, wherein the target task further comprises a processing rule aiming at data stored in the target message queue and a notification mode aiming at a processing result, processing the obtained target resource data aiming at the processing rule to obtain a processing result, and further sharing the processing result according to the notification mode. Therefore, the acquired target resource data can be processed based on the processing rule aiming at the data stored in the target message queue included in the target task in response to the execution request aiming at the target task, so that a message consumer does not need to develop a processing process by himself, meanwhile, the message queue is effectively prevented from being directly exposed to the message consumer, and the safety of the message queue is ensured.

Additional aspects and advantages of the present application will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the present application.

Drawings

The foregoing and/or additional aspects and advantages of the present application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:

fig. 1 is a schematic flowchart of a message queue-based data sharing method according to an embodiment of the present disclosure;

fig. 2 is a schematic flowchart of another message queue-based data sharing method according to an embodiment of the present disclosure;

fig. 3 is a schematic diagram of target resource data information in a scene according to an embodiment of the present disclosure;

fig. 4 is a schematic flowchart of another message queue-based data sharing method according to an embodiment of the present application;

fig. 5 is a schematic flowchart of another message queue-based data sharing method according to an embodiment of the present application;

fig. 6 is a schematic flowchart of a data sharing method based on a message queue in a scenario provided in an embodiment of the present application;

fig. 7 is a schematic block diagram of a data sharing system based on a message queue in a scenario provided in an embodiment of the present application;

fig. 8 is a schematic structural diagram of a message queue-based data sharing apparatus according to an embodiment of the present disclosure;

fig. 9 is a schematic structural diagram of another message queue-based data sharing apparatus according to an embodiment of the present application;

fig. 10 is a schematic structural diagram of another message queue-based data sharing apparatus according to an embodiment of the present application; and

fig. 11 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

Reference will now be made in detail to embodiments of the present application, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are exemplary and intended to be used for explaining the present application and should not be construed as limiting the present application.

In view of the above problems, embodiments of the present application provide a data sharing method based on a message queue, so as to implement that, in response to an execution request for a target task, target resource data is obtained from the target message queue according to a resource identifier included in the target task, thereby effectively avoiding directly exposing the message queue to a message consumer, and ensuring the security of the message queue. Meanwhile, the acquired target resource data is processed based on the processing rule aiming at the data stored in the target message queue included in the target task, so that a message consumer does not need to develop a processing process by himself.

Before describing the embodiments of the present application in detail, for the sake of understanding, common technical terms are first introduced:

a message is a unit of data that is transferred between two computers. The message may be very simple, e.g. containing only text strings; it may also be more complex, for example a complex object structure may be embedded.

A message queue refers to a container that holds messages during their transmission. When a recipient of a message is unavailable, the message queue will hold the message until successful delivery to the recipient is possible. In the current production environment, there are Kafka, activeMQ, rocktmq, etc. which are the most used message queues.

Topic, which is a concept of a message queue, corresponds to the classification of messages in the message queue.

WebSocket is a Protocol for performing full-duplex communication on a single TCP (Transmission Control Protocol) connection, so that data exchange between a client and a server is simpler, and a server is allowed to actively push data to the client. In a WebSocket API (Application Programming Interface), a browser and a server only need to complete one handshake, and a persistent connection can be directly established between the browser and the server, and bidirectional data transmission is performed.

The following describes a message queue-based data sharing method, apparatus, and system according to an embodiment of the present application with reference to the drawings.

Fig. 1 is a schematic flowchart of a message queue-based data sharing method according to an embodiment of the present disclosure.

As shown in fig. 1, the message queue-based data sharing method includes the following steps:

step 101, responding to an execution request for a target task, and according to a resource identifier included in the target task, acquiring target resource data from a target message queue, wherein the target task further includes a processing rule for data stored in the target message queue and a notification mode for a processing result.

It should be noted that the message queue based data sharing method according to the embodiment of the present application may be executed by the message queue based data sharing apparatus provided in the embodiment of the present application. The message queue based data pushing device may be a server to perform the data sharing function, or the message queue based data pushing device may be configured in the server to enable the server to perform the data sharing function.

In this embodiment, in response to an execution request for a target task, target resource data may be acquired from a target message queue according to a resource identifier included in the target task. Wherein the resource identifier is used for identifying the resource data. It should be noted that, the target task includes, in addition to the resource identifier, a processing rule for data stored in the target message queue and a notification mode for a processing result.

It should be noted that, before executing the target task, the target task needs to be created. Alternatively, the target task may be created in response to a creation request for the target task. The creation request of the target task can be initiated by the task issuing party based on the task interface. The task issuing party can be understood as any client which initiates a task request based on the task interface. Optionally, the task interface may include a create task interface, a cancel task interface, a query task status interface, a get task data interface, and a data change notification interface, and the task request may include a create request of a task, a cancel request of a task, a query request of a task, a get request for a processing result, and an establish request for a shared connection. The task interface may be created by using a Structured Query Language (SQL) writing method.

It should be noted that the task request initiated by the task issuing party based on the task interface corresponds to the task interface, that is, the task issuing party may initiate the creation request of the task based on the creation task interface; initiating a task canceling request based on a task canceling interface; initiating a query request of a task based on a query task state interface; initiating an acquisition request for a processing result based on the acquisition task data interface; the interface initiates a setup request for the shared connection based on the data change notification.

It is understood that the resource data can be used for unique identification of the resource identifier, so that the target resource data can be obtained from the target message queue based on the resource identifier included in the target task.

And 102, processing the acquired target resource data according to the processing rule to obtain a processing result.

In this embodiment, since the target task includes a processing rule for data stored in the target message queue, the obtained target resource data may be processed according to the processing rule to obtain a processing result. Optionally, the processing rule may be used to perform filtering processing on the obtained target resource data, may also be used to perform classification processing on the obtained target resource data, may also be used to perform format conversion processing on the obtained target resource data, and the like, which is not limited in this embodiment.

In a possible implementation manner of this embodiment, in response to that the processing rule is used to perform filtering processing on the acquired target resource data, the processing rule may be a filtering rule, so that resource data meeting the filtering rule may be obtained by filtering from the acquired target resource data based on the processing rule, and the resource data meeting the filtering rule is taken as a processing result. Similarly, in response to the processing rule being used for performing classification processing on the acquired target resource data, the processing rule may be a classification rule, so that the acquired target resource data may be classified based on the processing rule to obtain classified resource data, and the classified resource data is taken as a processing result. And responding to the processing rule for performing format conversion processing on the acquired target resource data, wherein the processing rule can be a format conversion rule, so that the data format of the acquired target resource data can be converted based on the processing rule to obtain resource data after format conversion, and the resource data after format conversion is used as a processing result.

And 103, sharing the processing result according to the notification mode.

In this embodiment, since the notification mode for the processing result is included in the target task, the processing result can be shared according to the notification mode. The notification mode may be understood as a sharing mode used when the processing result is shared. Optionally, the notification mode may be real-time sharing, periodic sharing, or interactive sharing, which is not limited in this embodiment.

In this embodiment, the processing result may be shared with the task issuing party, or the processing result may be shared with the data sharing party specified in the task request initiated by the task issuing party, which is not limited in this embodiment. Similarly, a data sharer may be understood as any client that receives shared data.

According to the data sharing method based on the message queue, the target resource data are obtained from the target message queue according to the resource identification contained in the target task by responding to the execution request aiming at the target task, wherein the target task further comprises a processing rule aiming at data stored in the target message queue and a notification mode aiming at the processing result, so that the obtained target resource data are processed according to the processing rule to obtain the processing result, and the processing result is shared according to the notification mode. Therefore, the acquired target resource data can be processed based on the processing rule aiming at the data stored in the target message queue included in the target task in response to the execution request aiming at the target task, so that a message consumer does not need to develop a processing process by himself, meanwhile, the message queue is effectively prevented from being directly exposed to the message consumer, and the safety of the message queue is ensured.

To clearly illustrate the previous embodiment, this embodiment provides another data sharing method based on a message queue, and fig. 2 is a schematic flow chart of the another data sharing method based on a message queue provided in this embodiment.

As shown in fig. 2, the message queue-based data sharing method may include the following steps:

step 201, receiving a creation request for a target task initiated based on a task interface, and creating the target task.

Similarly, the message queue-based data sharing method according to the embodiment of the present application can also be executed by the message queue-based data sharing apparatus provided in the embodiment of the present application.

In this embodiment, the message queue-based data sharing device may receive a creation request for a target task initiated based on the task interface, thereby creating the target task. Alternatively, a create request for a target task may be initiated by a task issuer based on a create task interface. The task issuing party can be understood as any client which initiates a task request based on the task interface. It should be noted that the creation request may include condition information corresponding to the creation task interface, where the condition information may include a resource identifier, an alignment rule, a reserved field, a notification mode, and the like, which is not limited in this embodiment.

Similarly, the task interface may include a create task interface, a cancel task interface, a query task status interface, a get task data interface, and a data change notification interface, and the task request may include a create request of a task, a cancel request of a task, a query request of a task, a get request for a processing result, and an establish request for a shared connection. The task interface may be created by using SQL (Structured Query Language) writing, so that as an example, the body for creating the task interface may be as follows:

the Condition information comprises Condition information, wherein the Condition is used for filtering the information in the information queue, and corresponds to a comparison rule in the Condition information, and the Condition indicates that data with an intCol field of 1 is expected to be acquired; the resource is used for indicating the resource identification of the data expected to be acquired, and the resource identification corresponds to the resource identification in the condition information, wherein the resource is used for indicating that the resource identification of the data expected to be acquired is xiaogu; the DataItems is used for indicating fields needing to be reserved in the filtered message, and corresponds to reserved fields in the condition information, wherein the reserved fields comprise two fields, namely an intCol field and a stringCol field, and { "Name": intCol "}; the DataNotifyMode indicates a notification mode used when sharing data, and corresponds to the notification mode in the condition information, and the "DataNotifyMode" indicates that the notification mode is real-time sharing.

It can be understood that, since the creation request may include condition information corresponding to the creation task interface, where the condition information may include a resource identifier, an alignment rule, a reserved field, and a notification mode, and the like, the created target task may include the resource identifier, a processing rule for data stored in the target message queue, and a notification mode for a processing result, where the processing rule for data stored in the target message queue includes the alignment rule and the reserved field reserved in the alignment result.

Step 202, in response to the execution request for the target task, according to the resource identifier included in the target task, obtaining target resource data information, so as to establish a connection with the message queue storing the target resource data according to the data connection information in the target resource data information.

It can be understood that, since the target task includes the resource identifier, the processing rule for the data stored in the target message queue, and the notification mode for the processing result, in this embodiment, the target resource data information may be obtained according to the resource identifier included in the target task in response to the execution request for the target task, so as to establish a connection with the message queue in which the target resource data is stored according to the data connection information in the target resource data information. It should be noted that the target resource data information is basic information for describing the target resource data, and may include data connection information, resource information, and column information. The data connection information is used to record information for establishing connection with the message queue storing the target resource data, and may include a data connection type, a data link name, a message queue server, and the like. The resource information is used to record the information of the target resource data, and may include resource identification, topic, basic information (name, description, etc.) of the target resource, and so on. The column information is used to indicate the data format of the target resource data desired to be acquired, and may include a number, a column name, a chinese name, an english name, a type, and the like.

To illustrate the target resource data information more comprehensively, the present embodiment provides a schematic diagram of the target resource data information in one scenario as shown in fig. 3. As can be seen from fig. 3, in the data connection information, the data connection type is Kafka, the data link name is Kaf, and the message queue server is xx. Wherein, "xx.xxx.xxx: xxxx" may schematically represent values, for example, 10.157.143.200. It should be noted that "x" does not indicate that the corresponding position only includes one character, and those skilled in the art know that "x" may be a single character, or may be two or more characters, which is not limited in this embodiment. In the resource information, the resource identifier is xiaogu, the Topic is xiaogu, the name is xiaogu, and the description is-, (optionally, the description may be a description of representing the target resource by using a word, or a description of representing the target resource by using a picture, which is not limited in this embodiment. In the column information, 4 pieces of column information are listed in total, the column name of the column information with the number of 1 is int-in-Kafka, the Chinese name is intColEng, the English name is intCol, and the type is int; the column name of the column information with the number of 2 is double-in-Kafka, the Chinese name is double ColEng, the English name is double Col, and the type is double; the column name of the column information with the number of 3 is bol-in-Kafka, the Chinese name is bolColEng, the English name is bolCol, and the type is bol; the column name of the column information numbered 4 is string-in-Kafka, the Chinese name is string ColEng, the English name is string Col, and the type is string. It should be noted that the class information shown in fig. 2 is only an exemplary example, and does not mean that all the column information in the resource data information includes only 4 column information, optionally, less than 4 column information may be included, or more than 4 column information may be included, which is not limited in this embodiment.

And step 203, acquiring target resource data through the established connection.

In this embodiment, since the connection may be established with the message queue storing the target resource data according to the data connection information in the target resource data information, the target resource data may be acquired through the established connection. As one possible implementation, the target resource data may be fetched from the message queue by using the msgrcv () function.

And 204, screening the resource data which accord with the comparison rule from the target resource data.

In this embodiment, the target task includes the resource identifier, the processing rule for the data stored in the target message queue, and the notification mode for the processing result, where the processing rule for the data stored in the target message queue includes the comparison rule and the reserved field reserved in the comparison result, so that the resource data meeting the comparison rule can be screened from the target resource data according to the comparison rule. The comparison rule is used for indicating the condition which needs to be met in the comparison result, and the reserved field is used for indicating the field which needs to be reserved in the comparison result.

For example, for the example of creating the body of the task interface in step 201, the comparison rule corresponds to Condition, where Condition is used to filter the messages in the message queue, "Condition": intCol =1 "indicates that it is desired to obtain data with intCol field 1, so that the resource data with intCol field 1 can be screened out from the target resource data according to the comparison rule.

Step 205, determining the data corresponding to the reserved field in the resource data meeting the comparison rule as the processing result.

In this embodiment, after the resource data meeting the comparison rule is screened from the target resource data based on the comparison rule, the data corresponding to the reserved field in the resource data meeting the comparison rule may be determined as the processing result. As a possible implementation manner, the fields of the resource data that meet the comparison rule may be filtered based on the reserved fields, so as to determine, as the processing result, data corresponding to the reserved fields in the filtered resource data that meet the comparison rule.

For example, for the resource data with the intCol field 1 screened in step 204, the reserved field corresponds to the DataItems in the body of the task interface created in step 201, where the DataItems is used to indicate the field that needs to be reserved in the filtered message, and the "DataItems" [ { "Name": intCol "}, {" Name ":" stringCol "} ] indicates that the reserved field includes two fields, i.e., the intCol field and the stringCol field, so that the field of the screened resource data with the intCol field 1 can be filtered based on the reserved field, only the intCol field and the stringCol field are reserved, and the other fields are filtered out, so that the resource data that only includes two fields, i.e., the intCol field and the stringCol field, is determined as the processing result.

And step 206, sharing the processing result according to the notification mode.

In this embodiment, after determining the processing result, the processing result may be shared according to a notification mode for the processing result included in the target task. The notification mode may be understood as a sharing mode used when the processing result is shared. Optionally, the notification mode may be real-time sharing, periodic sharing, or interactive sharing, which is not limited in this embodiment.

In this embodiment, the processing result may be shared with the task issuing party, or the processing result may be shared with the data sharing party specified in the task request initiated by the task issuing party, which is not limited in this embodiment. As a possible implementation manner, since the task issuing party may include the condition information corresponding to the task creation interface in the task creation request initiated based on the task creation interface, the processing result may be shared to the data sharing party according to the notification mode when the condition information further includes the specified data sharing party, and the processing result may be shared to the task issuing party according to the notification mode when the condition information does not include the specified data sharing party. Similarly, a data sharer may be understood as any client that receives shared data.

For example, for the example of creating the body of the task interface in step 101, the notification mode corresponds to DataNotifyMode, where DataNotifyMode is used to indicate the notification mode used when sharing data, "DataNotifyMode", and "immediatately" indicates that the notification mode is real-time sharing, so that the determined processing result can be shared in real time to the task originator or the data sharer based on that the notification mode is real-time sharing.

According to the data sharing method based on the message queue, the target task is created by receiving a creation request aiming at the target task initiated based on the task interface, so that the execution request aiming at the target task is responded, the target resource data information is obtained according to the resource identification contained in the target task, the connection is established with the message queue storing the target resource data according to the data connection information in the target resource data information, the target resource data is obtained through the established connection, after the resource data meeting the comparison rule is screened out from the target resource data, the data corresponding to the reserved field in the resource data meeting the comparison rule is determined as the processing result, and the processing result is shared according to the notification mode. Therefore, the direct exposure of the message queue to the message consumer can be avoided by externally exposing the task interface. Meanwhile, the processing rule aiming at the data stored in the target message queue comprises the comparison rule and the reserved field reserved in the comparison result, so that data comparison can be supported.

As can be seen from the above analysis, in the embodiment of the present application, the processing result may be shared according to the notification mode, and in order to clearly illustrate how the processing result is shared according to the notification mode in the present application, the embodiment provides a data sharing method based on the message queue as shown in fig. 4.

Fig. 4 is a flowchart illustrating another message queue-based data sharing method according to an embodiment of the present application. As shown in fig. 4, the message queue-based data sharing method may include the steps of:

step 401, in response to the execution request for the target task, obtaining target resource data from the target message queue according to the resource identifier included in the target task, where the target task further includes a processing rule for data stored in the target message queue and a notification mode for a processing result.

And 402, processing the acquired target resource data according to the processing rule to obtain a processing result.

It should be noted that, for specific implementation processes of steps 401 to 402, reference may be made to the description of embodiments 101 to 102, and the principle is the same, which is not described herein again.

And step 403, in response to the notification mode being the real-time sharing mode, sharing the processing result through the sharing connection in real time.

In this embodiment, when the notification mode for the processing result included in the target task is real-time sharing, the processing result may be shared in real time by sharing connection. Similarly, in this embodiment, the processing result may be shared with the task issuing party, or the processing result may be shared with the data sharing party specified in the task request initiated by the task issuing party, which is not limited in this embodiment.

For example, for the example of creating the body of the task interface in step 101 in the previous embodiment, the notification mode corresponds to DataNotifyMode, "DataNotifyMode": im mediateable "indicates that the notification mode is real-time sharing, so that in the case of" DataNotifyMode ": im mediateable", the processing result may be shared to the task issuing party or the data sharing party in real time through the shared connection.

It should be noted that, in the present embodiment, the premise that the processing result can be shared by the shared connection is that the shared connection has already been established. In this embodiment, the processing result may be shared to the task issuing party, or the processing result may be shared to the data sharing party specified in the task request initiated by the task issuing party, so that the shared connection may be established with the data sharing party or the task issuing party in response to the establishment request for the shared connection initiated by the data sharing party or the task issuing party based on the task interface, so as to actively share data to the data sharing party or the task issuing party through the shared connection after the shared connection is established. As a possible implementation, the data sharing party or the task issuing party may initiate a setup request for the shared connection based on the data change notification interface.

Alternatively, the shared connection may be a WebSocket connection. It can be understood that, because the established sharing connection is the WebSocket connection, the persistent connection can be directly established only by completing one-time interaction between the client and the server, and bidirectional data transmission is performed, so that data exchange between the client and the server becomes simpler, and the server is allowed to actively push data to the client.

And step 404, in response to that the notification mode is the periodic sharing mode, determining corresponding notification content according to whether a processing result exists in a preset period, and periodically sharing the notification content through a sharing connection.

In this embodiment, when the notification mode for the processing result included in the target task is periodic sharing, the corresponding notification content is determined according to whether the processing result exists in a preset period, and the notification content is periodically shared through the sharing connection. The specific value of the preset period is not limited in this embodiment, and optionally may be set according to manual experience, for example, the preset period may be set to 1 hour, or may also be dynamically adjusted according to an actual application requirement, which is not limited in this embodiment. Similarly, in this embodiment, the processing result may be shared with the task issuing party, or the processing result may be shared with the data sharing party specified in the task request initiated by the task issuing party, which is not limited in this embodiment.

For example, for the example of creating the body of the task interface in step 101 in the previous embodiment, the notification mode corresponds to DataNotifyMode, "DataNotifyMode": cycle "indicates that the notification mode is cycle sharing, so that in the case of" DataNotifyMode ": cycle", according to whether there is a processing result in a preset cycle, the corresponding notification content may be determined, and the notification content may be periodically shared to the data sharing party or the task issuing party through a sharing connection.

In a possible implementation manner of this embodiment, when there is a processing result in a preset period, it may be determined that the corresponding notification content is "there is a processing result in the time of one period", and when there is no processing result in the preset period, it may be determined that the corresponding notification content is "there is no processing result in the time of one period".

It should be noted that, in a case that the processing result exists in the preset period, the data sharing party or the task issuing party may initiate an obtaining request for the processing result based on the obtaining task data interface to obtain the processing result. That is to say, after responding to the notification mode being periodic sharing, determining corresponding notification content according to whether a processing result exists in a preset period, and periodically sharing the notification content through sharing connection, and also responding to the processing result existing in the preset period, receiving an acquisition request for the processing result initiated by the data sharing party or the task issuing party based on the task interface, thereby sharing the processing result to the data sharing party or the task issuing party.

Step 405, in response to the notification mode being the interactive sharing mode, after the processing result is shared through the sharing connection, receiving a new operation command sent through the sharing connection and performing corresponding processing.

In this embodiment, in the case that the notification mode for the processing result included in the target task is interactive sharing, after the processing result is shared through the sharing connection, a new operation command sent through the sharing connection may be received and corresponding processing may be performed. Wherein, the new operation command includes but is not limited to: reset the consumption offset and view the consumption details. Similarly, in this embodiment, the processing result may be shared with the task issuing party, or the processing result may be shared with the data sharing party specified in the task request initiated by the task issuing party, which is not limited in this embodiment.

It can be understood that, since the processing result may be shared to the task issuing party, or the processing result may be shared to the data sharing party specified in the task request initiated by the task issuing party, in a possible implementation manner of this embodiment, after the processing result is shared to the data sharing party or the task issuing party through the sharing connection, a new operation command sent by the data sharing party or the task issuing party through the sharing connection may be received and corresponding processing may be performed.

For example, for the example of creating the body of the task interface in step 101 in the previous embodiment, the notification mode corresponds to DataNotifyMode, "DataNotifyMode": interactive "indicates that the notification mode is interactive sharing, so that when the notification mode is" DataNotifyMode ": interactive sharing", the processing result is shared to the data sharing party or the task issuing party through the shared connection, and then a new operation command sent by the data sharing party or the task issuing party through the shared connection is received and corresponding processing is performed.

It is to be understood that since the new operation commands include, but are not limited to: the consumption offset is reset and the consumption details are checked, so that in a possible implementation manner of this embodiment, in response to a new operation command sent by the data sharing party or the task issuing party through the shared connection being the reset consumption offset, the consumption offset set when the target resource data stored in the message queue is acquired may be reset, so as to acquire the target resource data stored in the message queue from the new consumption offset. And in response to a new operation command sent by the data sharing party or the task issuing party through the sharing connection for checking the consumption details, returning the consumption information of the target resource data stored in the current acquired message queue to the data sharing party or the task issuing party. Optionally, consumption information such as Topic, consumption offset, etc. of the message queue storing the target resource data at the current time may be returned to the data sharing party or the task issuing party.

The message queue-based data sharing method provided by the embodiment of the application can respond to the notification mode as a real-time sharing mode, share the processing result through sharing connection in real time, or respond to the notification mode as a periodic sharing mode，Determining corresponding notification content according to whether a processing result exists in a preset period, and periodically sharing the notification content through sharing connection, or responding to that the notification mode is an interactive sharing mode, and sharing the processing result through sharing connection，And receiving a new operation command sent through the shared connection and performing corresponding processing. Thus, the processing results can be shared in different notification modes through the shared connection.

For clearly illustrating the above embodiments, this embodiment provides another message queue-based data sharing method, and fig. 5 is a schematic flow chart of another message queue-based data sharing method provided in this embodiment.

As shown in fig. 5, the message queue-based data sharing method may include the steps of:

step 501, in response to an execution request for a target task, obtaining target resource data from a target message queue according to a resource identifier included in the target task, where the target task further includes a processing rule for data stored in the target message queue and a notification mode for a processing result.

It should be noted that, for a specific implementation process of this step, reference may be made to the description of the above embodiment 101, and the principle is the same, which is not described herein again.

Step 502, converting the target resource data according to the column information contained in the target task to obtain the target resource data containing the column information.

In this embodiment, since the target resource data stored in the message queue may not be in accordance with the data format of the target resource data expected to be obtained and indicated by the column information in the target resource data information, the target resource data needs to be converted according to the column information in the target resource data information, and the resource data that does not comply with the column information is removed, so as to obtain the target resource data including the column information.

Step 503, processing the target resource data containing the column information according to the processing rule to obtain a processing result.

In this embodiment, after the target resource data including the column information is obtained, the target resource data including the column information may be processed according to a processing rule for data stored in the target message queue included in the target task, so as to obtain a processing result. Optionally, the processing rule may be used to perform filtering processing on the obtained target resource data including the column information, may also be used to perform classification processing on the obtained target resource data including the column information, may also be used to perform format conversion processing on the obtained target resource data including the column information, and the like, which is not limited in this embodiment.

In a possible implementation manner of this embodiment, in response to that the processing rule is used to perform filtering processing on the obtained target resource data including the column information, the processing rule may be a filtering rule, so that resource data that meets the filtering rule may be obtained by filtering the obtained target resource data including the column information based on the processing rule, and the resource data that meets the filtering rule is taken as a processing result. Similarly, in response to the processing rule being used for performing classification processing on the obtained target resource data containing the column information, the processing rule may be a classification rule, so that the obtained target resource data containing the column information may be classified based on the processing rule to obtain classified resource data, and the classified resource data is taken as a processing result. And responding to a processing rule for performing format conversion processing on the obtained target resource data containing the column information, wherein the processing rule can be a format conversion rule, so that the data format of the obtained target resource data containing the column information can be converted based on the processing rule to obtain resource data after format conversion, and the resource data after format conversion is used as a processing result.

Step 504, according to the notification mode, sharing the processing result.

It should be noted that, for a specific implementation process of this step, reference may be made to the description of the foregoing embodiment 103, and the principle is the same, which is not described herein again.

According to the data sharing method based on the message queue, after the target resource data is acquired from the target message queue according to the resource identifier included in the target task in response to the execution request for the target task and according to the resource identifier included in the target task, the target resource data is converted according to the column information included in the target task to obtain the target resource data including the column information, so that the target resource data including the column information is processed according to the processing rule to obtain the processing result, and the processing result is shared according to the notification mode.

To more clearly illustrate the above embodiments, the present embodiment provides a flowchart of a data sharing method based on a message queue in a scenario as shown in fig. 6. As shown in fig. 6, the message queue-based data sharing method may include the steps of:

step 601, establishing connection with the message queue.

Here, the target task may be created by receiving a creation request for the target task initiated by the task issuing party based on the task interface, so as to respond to the execution request for the target task and obtain the target resource data information according to the resource identifier included in the target task, so as to establish a connection with the message queue in which the target resource data is stored according to the data connection information in the target resource data information. The task issuing party can be understood as any client which initiates a task request based on the task interface.

Optionally, the task interface may include a create task interface, a cancel task interface, a query task status interface, a get task data interface, and a data change notification interface, and the task request may include a create request of a task, a cancel request of a task, a query request of a task, a get request for a processing result, and an establish request for a shared connection. The task interface may be created by using a Structured Query Language (SQL) writing method.

It should be noted that the creation request may include condition information corresponding to the creation task interface. Wherein, the condition information includes but is not limited to: resource identification and notification mode. Therefore, the created target task comprises the resource identification and the notification mode aiming at the processing result.

Step 602, obtaining target resource data stored in the message queue, and performing data format conversion on the target resource data.

Here, after the target resource data stored in the message queue is acquired, since the target resource data stored in the message queue may not be consistent with the data format of the target resource data expected to be acquired, which is indicated by the column information in the target resource data information, it is necessary to convert the target resource data according to the column information in the target resource data information, remove the resource data that does not meet the column information, and obtain the target resource data including the column information.

Step 603, determining whether there is a comparison rule and a reserved field.

It should be noted that, here, the condition information included in the creation request may include only the resource identifier and the notification mode, and the created target task may also include only the resource identifier and the notification mode for the processing result, so that it is necessary to determine whether there is a comparison rule and a reserved field, so as to perform corresponding processing for different situations.

Step 604, in response to the existence of the comparison rule and the reserved field, filtering the target resource data according to the comparison rule to obtain the resource data meeting the comparison rule, and determining the data corresponding to the reserved field in the resource data meeting the comparison rule as a processing result for storage.

Here, in the case that there are an alignment rule and a reserved field, that is, the condition information included in the creation request includes, in addition to the resource identifier and the notification mode, the alignment rule and the reserved field, that is, the created target task includes, in addition to the resource identifier and the notification mode for the processing result, a processing rule for data stored in the target message queue, where the processing rule for data stored in the target message queue includes the alignment rule and the reserved field reserved in the alignment result, the obtained target resource data including the column information may be filtered based on the alignment rule to obtain resource data that meets the alignment rule, and the resource data is stored.

Step 605, in response to the absence of the comparison rule and the reserved field, directly determining the target resource data as a processing result, and storing the processing result.

Here, in the case where there is no comparison rule and reserved field, that is, the condition information included in the creation request includes only the resource identifier and the notification pattern, and does not include the comparison rule and the reserved field, that is, the created target task includes only the resource identifier and the notification pattern for the processing result, and does not include the processing rule for the data stored in the target message queue, where the processing rule for the data stored in the target message queue includes the comparison rule and the reserved field reserved in the comparison result, the target resource data may be directly determined as the processing result, and the data in the comparison may be stored.

At step 606, a notification mode is determined.

Here, the notification pattern may be determined first so as to perform the corresponding subsequent step.

And step 607, responding to the notification mode being real-time sharing, and sharing the stored processing result externally in real time.

Here, when the data notification mode is real-time sharing, the stored processing result can be externally shared in real time. In particular, the stored processing results may be shared in real time over a shared connection. Alternatively, the shared connection may be a WebSocket connection. The processing result can be shared to the task issuing party, and can also be shared to the data sharing party specified in the task request initiated by the task issuing party.

And step 608, in response to that the notification mode is periodic sharing, periodically pushing notification content, wherein the notification content is used for indicating whether a processing result exists in a preset period.

Here, in the case where the notification mode is the periodic sharing, the notification content indicating whether or not there is a processing result within a preset period may be periodically pushed. The specific value of the preset period is not limited, and optionally may be set according to manual experience, for example, the preset period may be set to 1 hour, or may be dynamically adjusted according to actual application requirements.

As a possible implementation manner, in response to that the notification mode is periodic sharing, according to whether a processing result exists in a preset period, the corresponding notification content may be determined, and the notification content may be periodically shared through a sharing connection. Alternatively, in a case where a processing result exists in a preset period, it may be determined that the corresponding notification content is "the processing result exists within the time of one period", and in a case where the processing result does not exist in the preset period, it may be determined that the corresponding notification content is "the processing result does not exist within the time of one period". Similarly, the shared connection may be a WebSocket connection. The processing result can be shared to the task issuing party, and can also be shared to the data sharing party specified in the task request initiated by the task issuing party.

It should be noted that, in the case that a processing result exists in a preset period, after the push of the push content is completed, an acquisition request for the processing result, which is initiated by the data sharing party or the task issuing party based on the task interface, may be received, so as to share the stored processing result externally. Here, the acquisition request for the processing result is initiated based on the acquisition task data interface.

Step 609, in response to the notification mode being interactive sharing, after the stored processing result is shared externally, receiving a new operation command and performing corresponding processing.

Here, in the case that the notification mode is real-time sharing, after the stored processing result is externally shared, a new operation command may be received and corresponding processing may be performed. Wherein, the new operation command includes but is not limited to: reset the consumption offset and view the consumption details.

Optionally, in response to the new operation command being to reset the consumption offset, the consumption offset set when the target resource data stored in the message queue is acquired may be reset, so as to start acquiring the target resource data stored in the message queue from the new consumption offset. In response to the new operation command to view the consumption details, the consumption information of the target resource data stored in the message queue at the current time may be returned, and specifically, the consumption information of the message queue in which the target resource data is stored at the current time, such as Topic, consumption offset, and the like, may be returned.

As a possible implementation manner, in response to the notification mode being interactive sharing, after the processing result is shared through the sharing connection, a new operation command sent through the sharing connection is received and corresponding processing is performed. Similarly, the shared connection may be a WebSocket connection. The processing result can be shared to the task issuing party, and can also be shared to the data sharing party specified in the task request initiated by the task issuing party. It can be understood that, since the processing result may be shared to the task issuing party, or may be shared to the data sharing party specified in the task request initiated by the task issuing party, after the processing result is shared to the data sharing party or the task issuing party through the sharing connection, a new operation command sent by the data sharing party or the task issuing party through the sharing connection may be received and corresponding processing may be performed.

The data sharing method based on the message queue, provided by the embodiment of the application, includes establishing a connection through the message queue, obtaining target resource data stored in the message queue, and performing data format conversion on the target resource data, so that after judging whether a comparison rule and a reserved field exist, in response to the existence of the comparison rule and the reserved field, filtering the target resource data according to the comparison rule to obtain resource data meeting the comparison rule, determining data corresponding to the reserved field in the resource data meeting the comparison rule as a processing result, and storing the processing result, or in response to the absence of the comparison rule and the reserved field, directly determining the target resource data as the processing result, and storing the processing result, after determining a notification mode, in response to the notification mode being real-time sharing, performing external sharing on the stored processing result in real time, or in response to the notification mode being periodic sharing, periodically pushing notification content, where the notification content is used for indicating whether a processing result exists in a preset period, or in response to the notification mode being interactive sharing, and after performing the stored processing result being shared, receiving a new operation and performing corresponding processing command. Therefore, the data in the message queue can be shared outwards by adopting three different sharing modes while supporting data comparison.

In order to implement the above embodiments, the present application further provides a data sharing system based on a message queue.

Fig. 7 is a schematic block diagram of a data sharing system based on a message queue in a scenario provided by an embodiment of the present application.

As shown in fig. 7, the message queue-based data sharing system includes: a configuration management module 71, a task management module 72 and a task processing module 73.

The configuration management module 71 is configured to maintain resource data information, where the resource data information is used to describe basic information of resource data, and includes data connection information, resource information, and column information;

a task management module 72 for providing a task interface; the task interface comprises a task creation interface, a task cancellation interface, a task state inquiry interface, a task data acquisition interface and a data change notification interface;

and the task processing module 73 is configured to execute the message queue-based data sharing method according to any of the embodiments of the present application.

As can be seen from fig. 7, the configuration manager may configure and maintain the resource data information through the configuration management module; the task issuing party can issue the tasks through the task management module. The resource data information in the configuration management module and the task interface in the task management module can be stored in the database, so that the related data stored in the database is pushed to the task processing module to execute the message queue-based data sharing method provided by any of the embodiments of the present application. It can be understood that the data shared by the task processing module may be shared with the task issuing party, or may be shared with a data sharing party designated by the task issuing party. It should be noted that the task processing module needs to establish a connection with the message queue storing the target resource data to acquire the target resource data stored in the message queue.

In order to implement the above embodiments, the present application further provides a data sharing apparatus based on a message queue.

Fig. 8 is a schematic structural diagram of a message queue-based data sharing apparatus according to an embodiment of the present disclosure.

As shown in fig. 8, the message queue-based data sharing apparatus includes: an acquisition module 81, a first processing module 82 and a sharing module 83.

An obtaining module 81, configured to respond to an execution request for a target task, and obtain target resource data from a target message queue according to a resource identifier included in the target task; the target task also comprises a processing rule aiming at the data stored in the target message queue and a notification mode aiming at a processing result;

the first processing module 82 is configured to process the acquired target resource data according to the processing rule to obtain a processing result;

and a sharing module 83, configured to share the processing result according to the notification mode.

Further, in a possible implementation manner of the embodiment of the present application, the obtaining module 81 is further configured to:

and acquiring target resource data through the established connection.

Further, in a possible implementation manner of the embodiment of the present application, the processing rule includes a comparison rule and a reserved field reserved in the comparison result; a first processing module 82, further configured to:

and determining the data corresponding to the reserved fields in the resource data which accords with the comparison rule as a processing result.

Further, in a possible implementation manner of the embodiment of the present application, the notification mode includes real-time sharing, periodic sharing, and interactive sharing, and the sharing module 83 includes:

the first sharing unit is used for responding to the notification mode as a real-time sharing mode and sharing the connection sharing processing result in real time; the shared connection comprises a WebSocket connection;

the second sharing unit is used for responding to the fact that the notification mode is a periodic sharing mode, determining corresponding notification content according to whether a processing result exists in a preset period, and periodically sharing the notification content through sharing connection;

the interaction unit is used for responding to the fact that the notification mode is an interaction sharing mode, receiving a new operation command sent by the sharing connection after sharing a processing result through the sharing connection, and performing corresponding processing; the new operation commands include, but are not limited to: reset the consumption offset and view the consumption details.

Further, in a possible implementation manner of the embodiment of the present application, the first processing module 82 is further configured to:

It should be noted that the foregoing explanation on the embodiment of the data sharing method based on the message queue is also applicable to the data sharing device based on the message queue in this embodiment, and is not repeated here.

Based on the foregoing embodiment, a possible implementation manner of a data sharing device based on a message queue is further provided in the embodiments of the present application, and fig. 9 is a schematic structural diagram of another data sharing device based on a message queue provided in the embodiments of the present application, where on the basis of the foregoing embodiments, the data sharing device based on a message queue further includes: a second processing module 84.

The second processing module 84 is configured to receive, in response to the processing result existing in the preset period, an acquisition request for the processing result, and share the processing result.

Based on the foregoing embodiment, a possible implementation manner of a data sharing device based on a message queue is further provided in the embodiments of the present application, and fig. 10 is a schematic structural diagram of another data sharing device based on a message queue provided in the embodiments of the present application, where on the basis of the foregoing embodiments, the data sharing device based on a message queue further includes: a third processing module 85.

A third processing module 85, configured to receive a creation request for a target task initiated based on a task interface, and create the target task; the creating request comprises condition information corresponding to the task interface; the condition information includes, but is not limited to: resource identification, comparison rules, reserved fields and notification modes.

Further, in a possible implementation manner of the embodiment of the present application, the task interface is created by using a SQL-like writing method; the task interface comprises a task creation interface, a task cancellation interface, a task state inquiry interface, a task data acquisition interface and a data change notification interface.

According to the data sharing device based on the message queue, the target task is responded to the execution request aiming at the target task, and the target resource data are obtained from the target message queue according to the resource identification contained in the target task, wherein the target task further comprises a processing rule aiming at data stored in the target message queue and a notification mode aiming at the processing result, so that the obtained target resource data are processed according to the processing rule to obtain the processing result, and the processing result is shared according to the notification mode. Therefore, the acquired target resource data can be processed based on the processing rule aiming at the data stored in the target message queue included in the target task in response to the execution request aiming at the target task, so that a message consumer does not need to develop a processing process by himself, meanwhile, the message queue is effectively prevented from being directly exposed to the message consumer, and the safety of the message queue is ensured.

In order to implement the above embodiments, the present application also provides an electronic device, including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the message queue-based data sharing method proposed in any of the above embodiments of the present application.

Fig. 11 is a schematic structural diagram of an electronic device according to an embodiment of the present application. The electronic device shown in fig. 11 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present application.

As shown in fig. 11, the electronic device may include: the device comprises a shell 11, a processor 12, a memory 13, a circuit board 14 and a power circuit 15, wherein the circuit board 14 is arranged inside a space enclosed by the shell 11, and the processor 12 and the memory 13 are arranged on the circuit board 14; a power supply circuit 15 for supplying power to each circuit or device of the electronic apparatus; the memory 13 is used for storing executable program codes; the processor 12 executes a program corresponding to the executable program code by reading the executable program code stored in the memory 13, so as to execute the message queue-based data sharing method proposed in any of the above embodiments of the present application.

For a specific execution process of the above steps by the processor 12 and further steps executed by the processor 12 by running the executable program code, reference may be made to the description of the method embodiment shown in fig. 1 to 6 in this application, which is not described herein again.

To achieve the above embodiments, the present application further provides a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the message queue based data sharing method proposed in any of the above embodiments of the present application.

In order to implement the foregoing embodiments, the present application further provides a computer program product, which includes a computer program and when executed by a processor, the computer program implements the message queue-based data sharing method provided in any of the foregoing embodiments of the present application.

In the description herein, reference to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the application. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Moreover, various embodiments or examples and features of various embodiments or examples described in this specification can be combined and combined by one skilled in the art without being mutually inconsistent.

Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present application, "plurality" means at least two, e.g., two, three, etc., unless explicitly specified otherwise.

Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing steps of a custom logic function or process, and alternate implementations are included within the scope of the preferred embodiment of the present application in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present application.

The logic and/or steps represented in the flowcharts or otherwise described herein, e.g., an ordered listing of executable instructions that can be considered to implement logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium could even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via for instance optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.

It should be understood that portions of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are well known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.

It will be understood by those skilled in the art that all or part of the steps carried out in the method of implementing the above embodiments may be implemented by hardware related to instructions of a program, which may be stored in a computer readable storage medium, and the program, when executed, includes one or a combination of the steps of the method embodiments.

In addition, functional units in the embodiments of the present application may be integrated into one processing module, or each unit may exist alone physically, or two or more units are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module, if implemented in the form of a software functional module and sold or used as a stand-alone product, may also be stored in a computer readable storage medium.

The storage medium mentioned above may be a read-only memory, a magnetic or optical disk, etc. While embodiments of the present application have been shown and described above, it will be understood that the above embodiments are exemplary and should not be construed as limiting the present application and that changes, modifications, substitutions and alterations in the above embodiments may be made by those of ordinary skill in the art within the scope of the present application.

Claims

1. A message queue-based data sharing method, the method comprising:

responding to an execution request aiming at a target task, and acquiring target resource data from a target message queue according to a resource identifier contained in the target task; the target task also comprises a processing rule aiming at data stored in the target message queue and a notification mode aiming at a processing result;

and sharing the processing result according to the notification mode.

2. The method according to claim 1, wherein the obtaining target resource data from a target message queue according to the resource identifier included in the target task comprises:

and acquiring the target resource data through the established connection.

3. The method of claim 1, wherein the processing rules comprise alignment rules and reserved fields reserved in alignment results; the processing the acquired target resource data according to the processing rule to obtain a processing result includes:

4. The method of claim 1, wherein the notification mode comprises a real-time sharing mode, a periodic sharing mode, an interactive sharing mode; the sharing the processing result according to the notification mode comprises:

responding to the notification mode being a periodic sharing mode, determining corresponding notification content according to whether the processing result exists in a preset period, and periodically sharing the notification content through the sharing connection;

responding to the notification mode being an interactive sharing mode, after sharing the processing result through the sharing connection, receiving a new operation command sent through the sharing connection and performing corresponding processing; the new operation command includes, but is not limited to: reset the consumption offset and view the consumption details.

5. The method according to claim 4, wherein after the responding to the notification mode being a periodic sharing mode, determining corresponding notification content according to whether the processing result exists within a preset period, and periodically sharing the notification content through the sharing connection, further comprising:

6. The method according to claim 1, wherein the processing the acquired target resource data according to the processing rule to obtain a processing result includes:

7. The method of any of claims 1-6, wherein prior to performing the target task, the method further comprises:

receiving a creation request aiming at the target task and initiated based on a task interface, and creating the target task; the creation request comprises condition information corresponding to the task interface; the condition information includes, but is not limited to: resource identification, comparison rules, reserved fields and notification modes.

8. The method of claim 7, wherein the task interface is created using a class Structured Query Language (SQL) notation; the task interface comprises a task creation interface, a task cancellation interface, a task state inquiry interface, a task data acquisition interface and a data change notification interface.

9. A message queue-based data sharing apparatus, comprising:

the acquisition module is used for responding to an execution request aiming at a target task and acquiring target resource data from a target message queue according to a resource identifier contained in the target task; the target task further comprises a processing rule aiming at data stored in the target message queue and a notification mode aiming at a processing result;

the processing module is used for processing the acquired target resource data according to the processing rule to obtain a processing result;

10. A data sharing system based on message queues is characterized by comprising a configuration management module, a task management module and a task processing module:

the task management module is used for providing a task interface; the task interface comprises a task creation interface, a task cancellation interface, a task state inquiry interface, a task data acquisition interface and a data change notification interface;

a task processing module for performing the message queue based data sharing method of any one of claims 1-8.

11. A server, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the message queue-based data sharing method of any one of claims 1-8.

12. A server, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

the memory stores instructions executable by the at least one processor to enable the at least one processor to run a data push system;

the data push system comprises a configuration management module, a task management module and a task processing module; the configuration management module is used for maintaining resource data information, and the resource data information is used for describing basic information of resource data and comprises data connection information, resource information and column information; the task management module is used for providing task interfaces, and the task interfaces comprise a task creation interface, a task cancellation interface, a task state inquiry interface, a task data acquisition interface and a data change notification interface; the task processing module is used for executing the message queue-based data sharing method of any one of claims 1 to 8.

13. A non-transitory computer readable storage medium storing computer instructions for causing a computer to perform the message queue-based data sharing method of any one of claims 1-8.