CN112527528A

CN112527528A - Data transmission method, device and storage medium based on message queue

Info

Publication number: CN112527528A
Application number: CN202011506075.9A
Authority: CN
Inventors: 赵楚旋; 蒋英明; 万书武; 林琪琛; 刘微明; 覃芳
Original assignee: Ping An Technology Shenzhen Co Ltd
Current assignee: Ping An Technology Shenzhen Co Ltd
Priority date: 2020-12-18
Filing date: 2020-12-18
Publication date: 2021-03-19

Abstract

The application discloses a data transmission method, a device, computer equipment and a storage medium based on a message queue, wherein the method comprises the following steps: receiving message data sent by a message producer, wherein the message data carries a message generated by the message producer and data source information related to the message; determining a message queue associated with data source information, and generating a target message subject according to the data source information; determining a target storage area associated with a target message theme in the message queue, and storing the target message theme and the message into the target storage area; after receiving a message subscription request which is sent by a message consumer and carries data source information, pulling a message corresponding to a target message theme from a target storage area of a message queue according to the data source information and pushing the message to the message consumer. By the aid of the method and the device, operation convenience of storing data in the message queue and pulling data from the message queue can be improved, data transmission efficiency is improved, and applicability is high.

Description

Data transmission method, device and storage medium based on message queue

Technical Field

The present application relates to the field of computer technologies, and in particular, to a data transmission method and apparatus based on a message queue, and a storage medium.

Background

In the field of big data, a message queue is an important component in a distributed system, and the middleware of the message queue mainly solves the problems of application coupling, asynchronous messages, traffic peak clipping and the like. In the existing producer and consumer applications based on the message queue, an upstream producer application generates a message according to a predefined message theme and sends the message to message queue middleware, and a downstream consumer application consumes a corresponding message according to the message theme agreed in advance by the upstream producer application. However, in such an application model, when a message topic is changed in the message generated by the upstream producer, the downstream consumer application must be notified, and the developer code adjusts and redevelops the consumer application to accommodate the new message topic generated by the upstream producer. In addition, when the message queue middleware performs reselection or technology upgrade, application architectures at both ends of an upstream producer and a downstream consumer need to be reconstructed and code redeveloped to store the message into the message queue or pull data from the message queue, so that the operation is complex, the implementation cost is high, and the applicability is poor.

Disclosure of Invention

The application provides a data transmission method, a data transmission device and a data transmission medium based on a message queue, which can improve the operation convenience of storing data in the message queue and pulling data from the message queue, improve the data transmission efficiency and have high applicability.

In a first aspect, the present application provides a data transmission method based on a message queue, where the method is applied to a message manager, and the method includes:

a message manager receives message data sent by a message producer, wherein the message data carries a message generated by the message producer and data source information related to the message;

the message manager determines the message queue associated with the data source information and generates a target message theme according to the data source information;

the message manager determines a target storage area associated with the target message theme in the message queue and stores the target message theme and the message into the target storage area;

when a message subscription request sent by a message consumer is received and the data source information is carried in the message subscription request, the message manager determines the target message topic from the target storage area of the message queue according to the data source information, pulls the message corresponding to the target message topic from the target storage area and pushes the message to the message consumer.

With reference to the first aspect, in a possible implementation manner, the message queue includes at least one storage area including the target storage area, one storage area is associated with one message topic and used for storing messages of the message topic, and data source information associated with messages of the same message topic is the same.

With reference to the first aspect, in a possible implementation manner, the data source information includes a data source name of a data source to which the message belongs and an item name of an item to which the data source belongs, where the item to which the data source belongs includes at least one data source including the data source;

the generating of the target message subject according to the data source information includes:

and generating a target message theme according to the project names and the data source names, wherein one data source name corresponds to one message theme.

With reference to the first aspect, in a possible implementation manner, the message data includes a message mode identifier carried in header information of the message, where the message mode identifier is used to identify a message mode of the message, so that the message consumer consumes the message based on the message mode.

With reference to the first aspect, in a possible implementation manner, after the message manager receives message data sent by a message producer, the method further includes:

the message manager registers the message mode of the message to a message registration service center, and generates a message mode identifier corresponding to the message mode of the message through the message registration service center;

and when a message mode acquisition request sent by the message consumer is received and the message mode acquisition request carries the message mode identifier, acquiring the message mode corresponding to the message mode identifier from the message registration service center and sending the message mode to the message consumer.

In a second aspect, the present application provides a message queue-based data transmission method, which is applicable to a message consumer, and includes:

a message consumer acquires data source information of a data source created by a message producer;

when the message consumer requests to subscribe the message generated by the message producer, sending a message subscription request to a message manager, wherein the message subscription request carries data source information, and the data source information is used for determining a target message theme requested to be subscribed by the message consumer;

and the message consumer receives the message related to the target message theme from the message manager, analyzes and consumes the message.

With reference to the second aspect, in a possible implementation manner, the parsing and consuming the message includes:

the message consumer obtains message mode identification information from the header information of the message, and determines the message mode of the message according to the message mode identification information;

and analyzing the message based on the message mode of the message and then consuming the message.

With reference to the second aspect, in a possible implementation manner, the parsing and consuming the message further includes:

the message consumer acquires a message mode identifier from the head information of the message and sends a message mode acquisition request to a message manager, wherein the message mode acquisition request carries the message mode identifier;

and acquiring a message mode corresponding to the message mode identifier from the message manager, and analyzing and consuming the message based on the message mode.

In a third aspect, a data transmission method based on a message queue, the method is applied to a message producer, and the method includes:

creating a data source, and generating a message and data source information related to the message according to the data source;

and sending message data to a message manager, wherein the message data carries the message and the data source information.

With reference to the third aspect, in a possible implementation manner, after the message producer creates the data source, the method further includes:

sending the data source information of the data source to a message manager and/or a message consumer;

the data source information comprises a data source name and a project name of a project to which the data source belongs.

In a fourth aspect, the present application provides a data transmission device based on a message queue, the data transmission device is suitable for a message manager, and the data transmission device includes:

a receiving module, configured to receive message data sent by a message producer, where the message data carries a message generated by the message producer and data source information associated with the message;

a determining module, configured to determine a message queue associated with the data source information received by the message data receiving module, and generate a target message topic according to the data source information;

the determining module is further configured to determine a target storage area associated with the target message topic in the message queue, and store the target message topic and the message in the target storage area;

the receiving module is further configured to receive a message subscription request sent by a message consumer, where the message subscription request carries the data source information;

and the message pushing module is used for determining the target message theme from the target storage area of the message queue according to the data source information carried in the message subscription request received by the receiving module, pulling the message corresponding to the target message theme from the target storage area and pushing the message to the message consumer.

With reference to the fourth aspect, in a possible implementation manner, the message queue includes at least one storage area including the target storage area, one storage area is associated with one message topic and used for storing messages of the message topic, and data source information associated with messages of the same message topic is the same.

With reference to the fourth aspect, in a possible implementation manner, the data source information includes a data source name of a data source to which the message belongs and an item name of an item to which the data source belongs, and the item to which the data source belongs includes at least one data source including the data source; the determining module is configured to:

With reference to the fourth aspect, in a possible implementation manner, the message transmission apparatus based on a message queue further includes:

a registration module, configured to register the message mode in a message registration service center, and generate a message mode identifier corresponding to the message mode of the message through the message registration service center;

the message pushing module is further configured to:

In a fifth aspect, the present application provides a message queue-based data transmission device, which is suitable for a message consumer, and includes:

the acquisition module is used for acquiring data source information of a data source created by a message producer;

a sending module, configured to send a message subscription request to a message manager when subscribing to a message generated by the message producer, where the message subscription request carries data source information, and the data source information is used to determine a target message topic requested to be subscribed;

a receiving module, configured to receive a message associated with the target message topic from the message manager;

and the analysis consumption module is used for analyzing and consuming the message received by the receiving module.

With reference to the fifth aspect, in a possible implementation manner, the parsing consumption module is configured to:

acquiring message mode identification information from the header information of the message received by the receiving module, and determining the message mode of the message according to the message mode identification information;

acquiring a message mode identifier from the header information of the message received by the receiving module, and sending a message mode acquisition request to a message manager through the sending module, wherein the message mode acquisition request carries the message mode identifier;

In a sixth aspect, the present application provides a message queue-based data transmission apparatus, which is suitable for a message producer, and includes:

a creation module for creating a data source;

a message generating module for generating a message and data source information associated with the message according to the data source created by the creating module;

and the sending module is used for sending message data to a message manager, wherein the message data carries the message generated by the message generating module and the data source information.

With reference to the sixth aspect, in a possible implementation manner, the sending module is further configured to:

In a seventh aspect, the present application provides a computer device, comprising: a processor, a memory, a transceiver, a network interface;

the processor is connected to a memory for providing data communication functions, a network interface for storing a computer program, and a transceiver for invoking the computer program to perform the method as performed by any one of the above-mentioned first aspect and possible embodiments of the first aspect of the present application.

In an eighth aspect, the present application provides a computer device comprising: a processor, a memory, a transceiver, a network interface;

the processor is connected to a memory for providing data communication functions, a network interface for storing a computer program, and a transceiver for invoking the computer program to perform the method as performed by any one of the above-mentioned second aspect and possible embodiments of the second aspect of the present application.

In a ninth aspect, the present application provides a computer device comprising: a processor, a memory, a transceiver, a network interface;

the processor is connected to a memory for providing data communication functionality, a network interface for storing a computer program, and a transceiver for invoking the computer program for performing the method performed by any of the above-mentioned possible embodiments of the third aspect and the third aspect of the present application.

In a tenth aspect, the present application provides a computer-readable storage medium storing a computer program comprising program instructions that, when executed by a processor, perform the method performed by any one of the above-mentioned first aspect and possible embodiments of the first aspect of the present application.

In an eleventh aspect, the present application provides a computer-readable storage medium storing a computer program comprising program instructions that, when executed by a processor, perform the method performed by any one of the possible embodiments of the second aspect and the second aspect described above in the present application.

In a twelfth aspect, the present application provides a computer-readable storage medium storing a computer program comprising program instructions that, when executed by a processor, perform the method performed by any of the possible embodiments of the third and third aspects described above in the present application.

In the application, a message producer can create a data source by itself when generating a message, after the data source is created, a message manager can operate and bind a corresponding message queue component, and create a target storage area in a corresponding message queue, wherein the target storage area is in a binding corresponding relation with the data source, a message consumer and a message producer can repeatedly use the data source, and the message manager can store data or pull data to the target storage area corresponding to the data source, so that the operation convenience of storing data in a message queue and pulling data from the message queue by the message manager can be improved, the data transmission efficiency is improved, and the applicability is high. The message queue is transparent to both the message producer and the message consumer, the message producer does not need to care what component is the middleware of the message queue used specifically, the middleware is associated with the message producer through the data source, as long as the message producer binds a certain data source, and subsequently, if the message producer changes the bottom message queue, the message manager switches the component types of the message queue according to the data source bound by the message producer, a new target storage area is created in a new message queue, the application of a message producer can be switched seamlessly without modifying the program and configuration of the application end, the cost of learning different types of message queues by a message producer or a message consumer can be reduced, the development mode of the message queue application is simplified, the development technical difficulty is reduced, and a business party is helped to be faster and more efficient in business logic of the upper-layer application of the actual industry.

Drawings

In order to more clearly illustrate the technical solutions in the present application, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.

FIG. 1 is a schematic structural diagram of a message queue-based data transmission system architecture provided in the present application;

FIG. 2 is an interactive schematic diagram of message queue based data transmission provided herein;

FIG. 3 is a flow chart of a message queue-based data transmission method provided in the present application;

FIG. 4 is another schematic flow chart of a message queue-based data transmission method provided in the present application;

FIG. 5 is another schematic flow chart of a message queue-based data transmission method provided in the present application;

FIG. 6 is a schematic structural diagram of a message queue-based data transmission apparatus provided in the present application;

fig. 7 is another schematic structural diagram of a message queue-based data transmission apparatus provided in the present application;

fig. 8 is another schematic structural diagram of a message queue-based data transmission device provided in the present application;

FIG. 9 is a schematic diagram of a computer device provided herein;

FIG. 10 is a schematic diagram of another configuration of a computer device provided herein;

fig. 11 is another schematic structural diagram of a computer device provided in the present application.

Detailed Description

The technical solutions in the present application will be described clearly and completely with reference to the accompanying drawings in the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

Referring to fig. 1, fig. 1 is a schematic structural diagram of a data transmission system architecture based on a message queue according to the present application. As shown in fig. 1, the network architecture may include a server 20, a user terminal cluster, and the user terminal cluster may include a plurality of user terminals, as shown in fig. 1, and specifically may include a user terminal 100a, a user terminal 100b, user terminals 100c, …, and a user terminal 100 n.

It is to be understood that, in alternative embodiments of the present application, the message producer and the message consumer may be user terminals in the user terminal cluster, the message producer may be a user terminal that generates and stores messages in a queue of messages, and the message consumer may be a user terminal that subscribes and consumes messages stored in the message queue. The user terminal may include, but is not limited to, a mobile phone, a tablet computer, a notebook computer, a palm computer, a smart audio, a Mobile Internet Device (MID), a point of sale (POS), and a wearable device (e.g., a smart watch, a smart bracelet, etc.). In this embodiment of the application, the message manager, the message queue, the data source, and the message registration service center may be a part of the same server, or may be distributed in different servers, where the server may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server that provides basic cloud computing services such as a cloud service, a cloud database, cloud computing, a cloud function, cloud storage, a network service, cloud communication, a middleware service, a domain name service, a security service, a CDN, and a big data and artificial intelligence platform, and may be specifically determined according to an actual application scenario, which is not limited herein. For convenience of description, the message manager will be described as a server or a message transmission center (such as the server 20 in fig. 1) in the distributed system. The message queue may be middleware in a distributed system for storing messages generated by message producers. The message manager can receive the message from the message producer and store the message in the message queue, pull the data from the message queue and push the data to the message consumer, register the message mode such as the message type of the message to the message registration service center, or obtain the message mode requested by the message consumer from the message registration service center and send the message mode to the message consumer.

In some possible embodiments, as shown in fig. 1, the user terminal 100a, the user terminal 100b, the user terminal 100c, …, and the user terminal 100n may be respectively connected to the server 20 via a network, so that each user terminal may interact data with the server 20 via the network. For example, any user terminal (i.e., a message producer, such as the user terminal 100a) in the user terminal cluster having the message production application sends message data carrying the data source information to the server 20 (i.e., a message manager), or the server 20 sends corresponding message data to any user terminal (i.e., a message consumer, such as the user terminal 100b) in the user terminal cluster having the message consumption application when receiving a message subscription request carrying the data source information sent by the user terminal (i.e., a message consumer, such as the user terminal 100 b). The message production application and the message consumption application may be a social application or an online shopping application, and the message data may be a chat message sent by a user or an order message, and may be determined specifically according to an actual application scenario, which is not limited herein.

Referring to fig. 2, fig. 2 is an interactive schematic diagram of data transmission based on a message queue according to the present application. As shown in fig. 2, the message queue-based data transmission system provided by the present application may include a message producer 10a, a message manager 10b, a message consumer 10c, a data source 10d, a message registry 10e, and a message queue 10 f. The data source 10d in fig. 1 is a data source created by the message producer 10a or the message consumer 10c and can be used together with the message producer 10a or a data source specified by a protocol, such as a data source bound by the message producer 10a, and then notified to the message consumer 10c by means of the protocol. The data source 10d may include a relational database (e.g., Mysql, Oracle, Nosql database) and may also include a "message cache" such data source. "message cache" refers to a reusable data source, which is a logical concept. Here, the data source name may be customized by a user, and the data source name may be composed of english full lowercase letters, underlines, or numbers. Creating the data source 10d also requires items, which are the basic part of the real-time streaming computing system for differentiating the different purpose data sources created by the user, and items must be created before other operations are performed, and then the data sources, real-time computing, resources and members under each item can be managed in the item management. The source end of the data source 10d mainly interfaces with various types of databases such as a database (Mysql/PG/SQLServer/Oracle) \ acquisition log file, JQData, Kafka and the like. The data source 10d is used to interface with the message queue 10f, associate the message producer 10a with the message consumer 10c, and push data from the message queue 10f to the real-time streaming computing system for use in downstream computing tasks. Real-time computation refers primarily to real-time computation tasks that interface with the message queue 10f, such as flink or spark streaming tasks. When the message producer 10a or the message consumer 10c creates the data source 10d and reports the data source to the message manager 10b, the message manager 10b binds the message queue (e.g., the message queue 10f) corresponding to the data source 10d, and generates the target message topic corresponding to the data source according to the data source information. The corresponding relationship between the data source 10d and the message topic may be that the message topic is an item name + a data source name. It can be understood that the data source 10d can directly find the corresponding message topic in the message queue 10f, and associate the message with the data source 10d through the corresponding relationship between the message topic and the message, so that the message topic in the message queue 10f can be found through the data source 10d, and the message can be found through the message topic. Therefore, the message manager 10b can find the message topic in the message queue 10f through the data source 10d, and find the message through the message topic, thereby implementing operations such as data production and consumption in the corresponding target storage area in the corresponding message queue 10 f. The created data source 10d may be used as a source input end, a sink output end, a data source of various forms of real-time computation engines such as flink, and the like, or a place where data flow after real-time computation flows.

The message producer 10a in fig. 2 is a user terminal that produces message data. The message producer 10a may create the data source 10d by itself when producing the message data, and the message manager 10b may bind the corresponding message queue 10f according to the operation of the data source 10d created by the message producer 10a, and automatically create a target storage area in the corresponding message queue 10b, which also corresponds to the data source 10 d. The message producer 10a sends the generated message data to the message manager 10b, and the message manager 10b stores the received message data in the target storage area in the message queue 10 f. As long as the message producer 10a creates a certain data source 10d, subsequently, if the middleware type of the message queue 10b is switched, the message manager 10b creates a new target storage area in the new message queue according to the data source 10d created by the message producer 10a, for storing the message data sent to the message manager 10b according to the message producer 10 a. In this way, the application of the message producer 10a can be seamlessly switched without modifying the program and configuration of the application side.

The message queue 10f in fig. 2 is an important component in a distributed system, and mainly solves the problems of application coupling, asynchronous messages, traffic peak clipping, and the like. In the embodiment of the present application, the message queue 10f corresponds to a container for holding messages. It is characterized in that the message data through the message queue 10f can only be sent out first and then, the message manager 10b firstly delivers the message generated by the message producer 10a and the message data of the target message subject generated according to the data source information to the target storage area in the message queue 10f for storage through the general logic concept of the data source 10 d. The message consumer 10c monitors the data in the message queue 10f, and if there is data available for consumption in the message queue 10f, the message manager 10b sends a message subscription request to the message manager 10b by using the general logic concept of the data source 10d, and the message manager 10b determines a target message topic of the message subscription request from a target storage area in the message queue 10f according to the data source information, pulls a message corresponding to the target message topic from the target storage area, and pushes the message to the message consumer 10c for consumption one by one. Because the performance of the message queue 10f to receive data is very high, the performance of the message producer 10a is not affected, and the timely efficiency of the message consumer 10c is also guaranteed. In an alternative embodiment of the present application, the message queue 10f is transparent to the message producer 10a and the message consumer 10b, which are associated with each other through the data source 10d, and the message queue 10f may be activeMQ, rabbitMQ, rockattmq, zeroMQ, kafka, etc. message queue middleware.

Among them, the message consumer 10c in fig. 2 is a user terminal that processes data. The message consumer 10c sends a message subscription request carrying information of the data source to the message manager 10b through the data source shared with the message producer 10 a. The message manager 10b determines the message topic subscribed by the message subscription request from the target storage area of the message queue 10f according to the data source information, pulls out a message corresponding to the target message topic from the target storage area, and pushes the message to the message consumer 10 c. After receiving the returned subscription message, the message consumer 10c parses the message pattern identification information in the message header information, where the message pattern identification information is used to identify the message pattern of the message, and the message consumer 10c parses and consumes the message by using a corresponding parsing method according to the message pattern.

In some possible embodiments, if there is no message pattern identifier information in the message header information, the message consumer 10c may further send a message pattern obtaining request carrying the message pattern identifier to the message manager 10 b. After receiving the request, the message manager 10b obtains the message mode corresponding to the message mode identifier from the message registration service center 10e, and sends the message mode to the message consumer 10 c. The message consumer 10c parses and consumes the message through the message schema described above. In an optional embodiment of the present application, if the message consumer 10c needs to subscribe to data in a certain data source, the message consumer may introduce an SDK packet to call a data source subscription interface, and the subscription service provides a function of storing user consumption points at a server, and can implement a highly available point location storage service only by simple configuration and processing.

In an optional embodiment of the present application, the message queue-based data transmission system in fig. 2 may further include a message registration service center 10e, where the message registration service center 10e is configured to record a message mode registered by the message manager 10 b. In an optional embodiment of the present application, after the message manager 10b receives the message data sent by the message producer 10a, the message manager 10b registers the message mode of the message in the message registration service center 10e, generates a message mode identifier corresponding to the message mode of the message through the message registration service center 10e, and adds the message mode identifier information to a header message of the message to be pushed together with the message data. When the message manager 10b receives the message mode obtaining request sent by the message consumer 10c and the message mode request carries the message mode identifier, the message manager 10b obtains the message mode corresponding to the message mode identifier from the message registration service center 10e and sends the message mode to the message consumer 10 c. Optionally, the message registration service center 10e may be a Spring Cloud component, which is an ordered set of a series of frames, and includes a large amount of packages for each component in the aspect of the micro-service framework, thereby simplifying development.

The message queue-based data transmission method provided by the embodiment of the present application will be described below with reference to fig. 3 to 5. Fig. 3 is a flowchart of a data transmission method based on a message queue according to an embodiment of the present application. As shown in fig. 3, the data transmission method based on the message queue provided in the embodiment of the present application includes the following steps:

s101, message data carrying data source information is sent.

In some possible embodiments, the message manager receives message data sent by a message producer, where the message data carries a message generated by the message producer and data source information associated with the message. The message producer creates a data source before sending the message data, associates the created data source with a certain project, generates a message and data source information associated with the message according to the data source, sends the message data carrying the message and the data source information to a message manager after creating the data source, and sends the data source information to a message consumer. And the message consumer adds the received data source information into the message subscription request and sends the message subscription request to the message manager. The message manager stores and pulls messages from the message queue according to the data source. The data source information includes a data source name and a data source type. Here, the data source name may be an english full lowercase letter, an underline, a numeric composition, for example, "source _ 123". Alternatively, the created data source may be a data source such as a "message cache", which is a logical concept and refers to a data source that can be reused. The project is a data source set (including one or more data sources) which is created by a user and used for different purposes, in other words, one or more data sources can be associated with the same project, and then the data sources, real-time computation, resources and members under each project can be managed in project management. Optionally, the message data may further include a message mode identifier carried in header information of the message, where the message mode identifier is used to identify a message mode of the message, and the message mode is used for a message consumer to analyze and consume the message based on the message mode. It will be appreciated that the data source is used to establish a relationship between a message generated by a message producer and a message subject and the data source, and the data source is also used to interface with a message queue, associate the message producer with a message consumer, and push data from the message queue to a real-time streaming computing system for use in downstream computing tasks. And after the data source is created, the data source information is added into the message data and is sent to a message manager together with the header information of the message. Optionally, the Apache Avro format may be used for transmitting the message during message transmission, and the function may be uniformly encapsulated and managed in an SDK manner. Optionally, the message producer sends the message data carrying the data source information, and may be a message manager that completes persistent processing on an order process after the user places an order and sends the order information of the user to the server in a certain shopping application of the terminal.

S102, registering the message mode of the message data.

In some possible embodiments, after the message manager receives the message data sent by the message producer, the message manager sends the message mode of the message to the message registration service center, generates a message mode identifier corresponding to the message mode of the message through the message registration service center, and adds the message mode identifier to the header message of the message to be pushed together with the message data. When the message consumer sends a message mode acquisition request to the message manager, and the message mode request carries the message mode identifier, the message manager can acquire the message mode corresponding to the message mode identifier from the message registration service center and send the message mode to the message consumer. In an optional embodiment of the present application, the registering, by the message manager, the message mode of the message data to the message registration service center may be to register the message data to a spring group registration center, and generate a message mode identifier corresponding to the message mode of the message through the spring group.

S103, generating a target message theme according to the data source information, and storing the message corresponding to the target message theme to a corresponding target storage area in the message queue.

In some feasible embodiments, after receiving the message data, the message manager binds the corresponding message queue according to the data source information, and generates a corresponding target message topic according to the data source information, where the data source and the target message topic are in a one-to-one correspondence relationship. Optionally, the subject name of the target message is the item name plus the name of the data source, e.g., "program 1+ source 123". It can be understood that the corresponding target message topic can be found in the message queue through the data source, and the message and the data source are associated through the corresponding relationship between the target message topic and the message, so that the target message topic in the message queue can be found through the data source, and the message can be found through the target message topic, so that the message producer and the message consumer can find the target message topic in the bottom layer message queue through the data source, and find the message through the target message topic, thereby realizing the production and consumption operation of data in the corresponding target storage area in the message queue. The message queue comprises at least one storage area including the target storage area, one storage area is associated with one message theme, data source information associated with the message of the same message theme is the same, and the storage area is used for storing the message of the message theme. Optionally, when the message data enters the corresponding storage area in the message queue, data formats such as CSV, JSON, AVRO and the like may be selected, and a data dictionary of the message topic does not need to be set separately. After a message producer or a message consumer creates a data source, a message manager operates and binds a corresponding message queue, and operates and binds a target storage area which is also in a binding corresponding relation with the data source in the corresponding message queue and is used for storing messages of message themes corresponding to the data source. The message consumer and the message producer can repeatedly use the data source, and can also produce push data or consume pull data in the target storage area corresponding to the data source. And subsequently, if the message queue types are switched, the message manager can automatically create and bind a new target storage area in the new message queue according to the data source bound by the message producer. Therefore, the application of the message producer can be switched seamlessly, the middleware of the message queue is transparent to the message producer and the message consumer, and the message producer and the message consumer can push and pull data without being concerned about which message queue is used specifically.

And S104, sending a message subscription request carrying data source information.

In some feasible embodiments, a message consumer sends a subscription request carrying information of a data source to a message manager, and since the message consumer and the message producer share the data source, after receiving a message subscription request carrying the information of the data source sent by the message consumer, the message manager firstly parses the message subscription request, and finds a message of a corresponding message topic stored in a target storage area corresponding to the data source in a message queue according to the information of the data source carried by the message subscription request. The data source information comprises a data source name, and the name of the message topic corresponding to the data source name is found according to the data source name so as to determine the message topic subscribed by the message subscription request. In an optional embodiment of the present application, the message consumer requesting subscription to a message generated by a message producer may send a subscription request for order placing information of a user to a message manager in a server through an agreed data source for an inventory management process in a shopping application in a terminal, and when the inventory management process subscribes the order placing information of the user, the order placing information of the user is obtained in a pulling manner, and corresponding inventory operation is performed according to the order placing information of the user.

S105, returning the information in the corresponding storage area pulled according to the data source information.

In some possible embodiments, after finding the name of the message topic corresponding to the data source name according to the data source name in the data source information, the message manager pulls the message subscribed by the message consumer from the target storage area storing the message of the message topic, and pushes the message to the message consumer.

S106, obtaining the message mode identification from the header information of the received message.

In some possible embodiments, after a message consumer receives a message from a target storage area stored in a message queue sent by a message manager, a corresponding format parsing method is adopted for different data formats to parse a message pattern identifier from header information of the message, the message consumer determines a message pattern of the message received according to a message subscription request according to the obtained message pattern identifier, and the message pattern is used for the message consumer to parse the message. And if the message mode corresponding to the message generated by the message producer is changed, the message consumer directly obtains the changed message mode identifier according to the header information of the message. Optionally, if the message header information received by the message consumer does not include the message mode identification information, the message consumer may execute step S107.

S107, sending a message mode request carrying the message mode identifier.

In some possible embodiments, the message consumer sends a message pattern request carrying the message pattern identifier to the message manager according to the message pattern identifier parsed from the header information of the message.

S108, the message mode corresponding to the message mode identification is taken from the message registration service center.

In some possible embodiments, the message manager takes the corresponding message mode from the message registration service center according to the received message mode identifier carried with the message mode identifier, and sends the message mode to the message consumer.

S109, the corresponding message mode obtained from the message registration service center is sent.

In some possible embodiments, when a message consumer sends a message mode obtaining request to a message manager, and the message mode request carries the message mode identifier, the message manager obtains a message mode corresponding to the message mode identifier from a message registration service center, and sends the message mode to the message consumer.

And S110, analyzing the message based on the acquired message mode and consuming the message.

In some possible embodiments, the message consumer may consume the message after parsing the message according to different message patterns by using corresponding parsing methods. The message mode includes but is not limited to the format type of the message and the type of the message queue, and because the message consumer and the message producer share the common logical concept of the data source, the management of the message subject data dictionary is avoided, and the message consumer downstream can consume the message only after the message mode of the received message is analyzed according to the corresponding analysis method.

In the application, a message producer or a message consumer creates a data source by himself, a message manager maintains a mapping relation between the data source and a message subject in a bottom message queue, a logical relation between the message queue and the message producer and the message consumer in a unified manner, a bus data flow model which is only abstracted and presented to application sides of the message consumer and the message producer is presented, and the message consumer and the message producer can generate and consume messages only by knowing a data source name and a target storage name. Here, the data source is used to interface with the message queue, associate the message producer with the message consumer, and push data from the message queue to the real-time streaming computing system for subsequent computing tasks. The message consumer can identify the message subject corresponding to the data source according to the data source information shared by the message producer, does not need to manage the message subject data dictionary, can analyze the message by using a corresponding analysis method according to the message modes of different messages, and is simple in operation and high in applicability.

Referring to fig. 4, fig. 4 is another schematic flow chart of a message queue-based data transmission method provided in the present application. The method is suitable for a message manager side and comprises the following steps:

s201, a message manager receives message data sent by a message producer, and the message data carries a message generated by the message producer and data source information related to the message.

For a specific implementation of step S201, reference may be made to the description of step S101 in the embodiment corresponding to fig. 3, which will not be described herein again.

S202, the message manager determines the message queue associated with the data source information and generates a target message theme according to the data source information.

The specific implementation of step S202 may refer to the description of step S103 in the embodiment corresponding to fig. 3, which will not be described herein again.

S203, the message manager determines a target storage area associated with the target message theme in the message queue, and stores the target message theme and the message into the target storage area.

The specific implementation of step S203 may refer to the description of step S103 in the embodiment corresponding to fig. 3, which will not be described herein again.

And S204, when a message subscription request sent by a message consumer is received and the data source information is carried in the message subscription request, the message manager determines the target message topic from the target storage area of the message queue according to the data source information.

In some feasible embodiments, the message consumer and the message producer share the data source, so that when the message manager receives a message subscription request carrying the information of the data source sent by the message consumer, the message manager firstly parses the message subscription request, and finds a message of a corresponding message topic stored in a target storage area corresponding to the data source in the message queue according to the information of the data source carried by the message subscription request. The data source information comprises a data source name, and the name of the message topic corresponding to the data source name is found according to the data source name so as to determine the message topic subscribed by the message subscription request.

S205, the message corresponding to the target message subject is pulled from the target storage area and pushed to the message consumer.

In the application, when a message producer generates a message, a data source can be automatically created, after the data source is created, a message manager can operate and bind a corresponding message queue component, and create a target storage area in a corresponding message queue, wherein the target storage area corresponds to the binding of the data source. The development mode of the message queue application can be simplified, and the operation convenience of storing data in the message queue and pulling data from the message queue by a message manager is improved.

Referring to fig. 5, fig. 5 is a flow chart illustrating a data transmission method based on a message queue according to the present application. The method is suitable for the message consumer side, and comprises the following steps:

s301, the message consumer obtains the data source information of the data source created by the message producer.

In some possible embodiments, the data source created by the message producer may be shared by the message consumer, and optionally, the data source may also be created by the message consumer for sharing by the message producer. The data source information includes, but is not limited to, a data source name and a data source type. In an optional embodiment of the present application, the created data source is not only used as a source input end, but also used as a sink output end, and is used as a data source of a flink and other similar real-time computation engines, and may also be a place where data flows after real-time computation, where the real-time computation mainly refers to a real-time computation task such as a flink or spark streaming task that is interfaced with a message queue component.

And S302, when the message consumer requests to subscribe the message generated by the message producer, sending a message subscription request to a message manager, wherein the message subscription request carries data source information, and the data source information is used for determining a target message theme which the message consumer requests to subscribe.

In some possible embodiments, when the message consumer needs to subscribe to the message, the obtained data source information for determining the target message topic is sent to the message manager along with the message subscription request. For details, please refer to step S103, where the correspondence between the data source information and the target message topic is not described in detail.

S303, the message consumer obtains the message mode identification information from the head information of the message, and determines the message mode of the message according to the message mode identification information.

In some possible embodiments, after the message consumer receives a message from a target storage area stored in a message queue sent by a message manager, the message consumer analyzes message pattern identification information from header information of the message by using a corresponding format analysis method for different data formats, where the message pattern identification information includes, but is not limited to, a message pattern identification, a message pattern, and the like. And the message consumer determines the message mode to which the message received according to the message subscription request belongs according to the acquired message identification mode information. And if the message mode corresponding to the message generated by the message producer is changed, the message consumer directly obtains the changed message mode according to the header information of the message, and analyzes and consumes the changed message according to the received message mode.

S304, the message is analyzed and consumed based on the message mode of the message.

The specific implementation of step S304 may refer to the description of step S109 in the embodiment corresponding to fig. 3, which will not be described herein again.

In the application, a message producer or a message consumer creates a data source by himself, a message manager maintains a mapping relation between the data source and a message subject in a bottom message queue, a logical relation between the message queue and the message producer and the message consumer uniformly, a bus data flow model which is only shown to the message consumer side after abstraction is presented, and the message consumer and the message producer can generate and consume messages only by knowing a data source name and a target storage name. Here, the data source is used to interface with the message queue, associate the message producer with the message consumer, and push data from the message queue to the real-time streaming computing system for subsequent computing tasks. The message consumer can identify the message subject corresponding to the data source according to the data source information shared by the message producer, does not need to manage the message subject data dictionary, can analyze the message by using a corresponding analysis method according to the message modes of different messages, and is simple in operation and high in applicability.

Further, please refer to fig. 6, fig. 6 is a schematic structural diagram of a message queue-based data transmission apparatus provided in the present application. The message queue-based data transmission device may be a user terminal corresponding to a message manager, or may be a computer program (including program code) running in a computer device, for example, the message queue-based data transmission device is an application software; the message queue-based data transmission device can be used for executing the corresponding steps in the method provided by the application. As shown in fig. 6, the message queue based data transmission apparatus 1 may be applied to a computer device, and the message queue based data transmission apparatus 1 may include: the device comprises a receiving module 11, a determining module 12, a message pushing module 13 and a registering module 14.

A receiving module 11, configured to receive message data sent by a message producer, where the message data carries a message generated by the message producer and data source information associated with the message;

a determining module 12, configured to determine a message queue associated with the data source information received by the message data receiving module, and generate a target message topic according to the data source information;

the determining module 12 is further configured to determine a target storage area associated with the target message topic in the message queue, and store the target message topic and the message in the target storage area;

the receiving module 11 is further configured to receive a message subscription request sent by a message consumer, where the message subscription request carries the data source information;

a message pushing module 13, configured to determine the target message topic from the target storage area of the message queue according to the data source information carried in the message subscription request received by the receiving module, pull the message corresponding to the target message topic from the target storage area, and push the message to the message consumer.

In a possible implementation manner, the message queue includes at least one storage area including the target storage area, one storage area is associated with one message topic and used for storing messages of the message topic, and data source information associated with messages of the same message topic is the same.

In a possible implementation manner, the data source information includes a data source name of a data source to which the message belongs and an item name of an item to which the data source belongs, and the item to which the data source belongs includes at least one data source including the data source; the determining module is configured to:

and generating a target message theme according to the project name and the data source name, wherein one data source name corresponds to one message theme.

In a possible implementation manner, the message transmission apparatus based on a message queue further includes:

and a registering module 14, configured to register the message mode in a message registration service center, and generate a message mode identifier corresponding to the message mode of the message through the message registration service center.

In a possible implementation manner, the message pushing module 13 is further configured to:

The specific functions of the receiving module 11, the determining module 12, the pushing module 13, and the registering module 14 may be implemented in steps S101 to S105 in the embodiment corresponding to fig. 3, which is not described herein again.

In the application, when a message producer generates a message, a data source can be automatically created, after the data source is created, a message manager can operate and bind a corresponding message queue component, and create a target storage area in a corresponding message queue, wherein the target storage area corresponds to the data source in a binding manner, and a message consumer and a message producer can repeatedly use the data source, and can also produce consumption data or consumption pull data in the target storage area corresponding to the previous data source. At this time, the message queue is transparent to both the message producer and the message consumer, the message producer does not need to care what component is the middleware of the message queue used specifically, and the middleware is associated with the message producer through the data source. The cost of learning different types of message queues by a message producer or a message consumer can be reduced, the operation convenience of storing data in the message queues and pulling data from the message queues is improved, and the data transmission efficiency is improved.

Please refer to fig. 7, which is a schematic structural diagram of another data transmission apparatus based on a message queue according to an embodiment of the present invention. As shown in fig. 7, the message queue-based data transmission apparatus 2 may be a user terminal corresponding to a message consumer, the data transmission apparatus 2 including: the system comprises an acquisition module 21, a sending module 22, a receiving module 23 and an analysis consumption module 24.

An obtaining module 21, configured to obtain data source information of a data source created by a message producer;

a sending module 22, configured to send a message subscription request to a message manager when subscribing to a message generated by the message producer, where the message subscription request carries data source information, and the data source information is used to determine a target message topic requested to be subscribed;

a receiving module 23, configured to receive, by the message consumer, the message associated with the target message topic from the message manager, analyze and consume the message;

referring to fig. 7, in a possible implementation, the parsing consumption module 24 is configured to:

In a possible implementation, the parsing consumption module 24 is further configured to:

The specific functional implementation manners of the obtaining module 21, the sending module 22, the receiving module 23, and the analyzing and consuming module 24 may refer to steps S301 to S304 in the embodiment corresponding to fig. 5, which is not described herein again.

Please refer to fig. 8, which is a schematic structural diagram of another data transmission apparatus based on a message queue according to an embodiment of the present invention. As shown in fig. 8, the message queue-based data transmission apparatus 3 may be a user terminal corresponding to a message producer, the data transmission apparatus 3 including: a creation module 31, a message generation module 32, and a sending module 33.

A creation module 31 for creating a data source;

a message generating module 32, which generates a message and data source information associated with the message according to the data source created by the creating module;

a sending module 33, configured to send message data to a message manager, where the message data carries the message generated by the message generating module and the data source information.

Referring to fig. 8, in a possible implementation, the sending module 33 is further configured to:

Further, please refer to fig. 9, fig. 9 is a schematic structural diagram of the computer device provided in the present application. As shown in fig. 9, the computer device 1000 may be adapted to a user terminal of a message manager, and the computer device 1000 may include: at least one processor 1001, such as a CPU, at least one network interface 1004, a transceiver 1003, a memory 1005, at least one communication bus 1002. Wherein a communication bus 1002 is used to enable connective communication between these components. The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface), among others. The memory 1005 may be a Random Access Memory (RAM) memory or a non-volatile memory (non-volatile memory), such as at least one disk memory. The memory 1005 may optionally also be at least one storage device located remotely from the aforementioned processor 1001. As shown in fig. 9, a memory 1005, which is a kind of computer storage medium, may include therein an operating system, a network communication module, a user interface module, and a device control application program.

In the computer device 1000 shown in fig. 9, the network interface 1004 is mainly used for network communication of the user terminal, and the processor 1001 and the transceiver 1003 may be used for calling the device control application stored in the memory 1005 to perform the following operations:

the transceiver 1003 is configured to receive message data sent by a message producer, where the message data carries a message generated by the message producer and data source information associated with the message;

the processor 1001 is configured to determine a message queue associated with data source information, and generate a target message topic according to the data source information;

the processor 1001 is further configured to determine a target storage area associated with a target message topic in the message queue, and store the target message topic and the message in the target storage area;

when the transceiver 1003 receives a message subscription request sent by a message consumer and the message subscription request carries data source information, the processor 1001 is further configured to determine a target message topic from a target storage area of the message queue according to the data source information, pull a message corresponding to the target message topic from the target storage area, and push the message to the message consumer.

In some possible embodiments, the message queue includes at least one storage area including the target storage area, one storage area is associated with a message topic and used for storing messages of the message topic, and data source information associated with messages of the same message topic is the same.

In some possible embodiments, the data source information includes a data source name of a data source to which the message belongs and an item name of an item to which the data source belongs, and the item to which the data source belongs includes at least one data source including the data source; the processor 1001 is further configured to generate a target message topic according to the item name and the data source name, where one data source name corresponds to one message topic.

In some possible embodiments, the processor 1001 is further configured to register the message mode in a message registration service center, and generate a message mode identifier corresponding to the message mode of the message through the message registration service center;

the transceiver 1003 is further configured to, after receiving a message mode obtaining request sent by the message consumer and the message mode obtaining request carries the message mode identifier, obtain, from the message registration service center, the message mode corresponding to the message mode identifier, and send the message mode to the message consumer.

It should be understood that the computer device 1000 described in this application may perform the description of the message queue based data transmission method in the embodiment corresponding to fig. 4, and may also perform the description of the message queue based data transmission apparatus 1 in the embodiment corresponding to fig. 6, which is not described herein again. In addition, the beneficial effects of the same method are not described in detail.

Further, here, it is to be noted that: the present application further provides a computer-readable storage medium, and the computer-readable storage medium stores the aforementioned computer program executed by the message queue-based data transmission apparatus 1, and the computer program includes program instructions, and when the processor executes the program instructions, the description of the message queue-based data transmission method in the embodiment corresponding to fig. 4 can be executed, so that details are not repeated here.

Further, please refer to fig. 10, which is a schematic structural diagram of another computer device according to an embodiment of the present invention. As shown in fig. 10, the computer device 2000 may be adapted to a user terminal of a message consumer, and the computer device 2000 may include: at least one processor 2001, e.g., a CPU, at least one network interface 2004, a transceiver 2003, a memory 2005, at least one communication bus 2002. The communication bus 2002 is used to implement connection communication between these components. The network interface 2004 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface), among others. The memory 2005 may be a Random Access Memory (RAM) memory or a non-volatile memory (non-volatile memory), such as at least one disk memory. The memory 2005 may optionally also be at least one memory device located remotely from the aforementioned processor 2001. As shown in fig. 10, the memory 2005, which is one type of computer storage medium, may include therein an operating system, a network communication module, a user interface module, and a device control application program.

In the computer device 2000 shown in fig. 10, the network interface 2004 is mainly used for the user terminal to perform network communication; and the processor 2001 and transceiver 2003 may be used to invoke the device control application stored in memory 2005 to perform the following operations:

the transceiver 2003, which is used for obtaining the data source information of the data source created by the message producer;

the transceiver 2003 is further configured to send a message subscription request to a message manager when requesting to subscribe to a message generated by the message producer, where the message subscription request carries data source information, and the data source information is used to determine a target message topic to which the message consumer requests subscription;

when the transceiver 2003 receives the message associated with the target message topic from the message manager, the processor 2001 is configured to parse and consume the message.

In a possible implementation, the transceiver 2003 is further configured to obtain message mode identification information from header information of the message;

the processor 2001 is further configured to determine a message pattern of the message according to the message pattern identification information, and parse and consume the message based on the message pattern of the message.

In a possible implementation manner, the transceiver 2003 is further configured to obtain a message mode identifier from header information of the message, and send a message mode obtaining request to a message manager, where the message mode obtaining request carries the message mode identifier; acquiring a message mode corresponding to the message mode identifier from the message manager;

the processor 2001 is further configured to parse and consume the message based on the message pattern.

It should be understood that the computer device 2000 described in this application may perform the description of the message queue based data transmission method in the embodiment corresponding to fig. 5, and may also perform the description of the message queue based data transmission apparatus 2 in the embodiment corresponding to fig. 7, which is not described herein again. In addition, the beneficial effects of the same method are not described in detail.

Further, here, it is to be noted that: the present application further provides a computer-readable storage medium, and the computer-readable storage medium stores the aforementioned computer program executed by the message queue-based data transmission apparatus 2, and the computer program includes program instructions, and when the processor executes the program instructions, the description of the message queue-based data transmission method in the embodiment corresponding to fig. 5 can be executed, so that details are not repeated here.

Further, please refer to fig. 11, which is a schematic structural diagram of another computer device according to an embodiment of the present invention. As shown in fig. 11, the computer device 3000 may be adapted to a user terminal of a message producer, and the computer device 3000 may include: at least one processor 3001, e.g., a CPU, at least one network interface 3004, a transceiver 3003, a memory 3005, at least one communication bus 3002. The communication bus 3002 is used to realize connection communication between these components. The network interface 3004 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface), among others. The memory 3005 may be a Random Access Memory (RAM) memory or a non-volatile memory (non-volatile memory), such as at least one disk memory. The memory 3005 may optionally also be at least one storage device located remotely from the processor 3001. As shown in fig. 11, the memory 3005, which is a kind of computer storage medium, may include therein an operating system, a network communication module, a user interface module, and a device control application program.

In the computer device 3000 shown in fig. 11, the network interface 3004 is mainly used for the user terminal to perform network communication; and the processor 3001 and the transceiver 3003 may be configured to invoke a device control application stored in the memory 3005 to perform the following operations:

the processor 3001 is configured to create a data source, and generate a message and data source information associated with the message according to the data source;

the transceiver 3003 is configured to send message data to a message manager, where the message data carries the message and the data source information.

In a possible implementation, the transceiver 3003 is further configured to send data source information of the data source to a message manager and/or a message consumer after the processor 3001 creates the data source, where the data source information includes a data source name and an item name of an item to which the data source belongs.

It should be understood that the computer device 3000 described in this application may perform the description of the message queue based data transmission method in the embodiment corresponding to fig. 3, and may also perform the description of the message queue based data transmission apparatus 3 in the embodiment corresponding to fig. 8, which is not described herein again. In addition, the beneficial effects of the same method are not described in detail.

Further, here, it is to be noted that: the present application further provides a computer-readable storage medium, and the computer-readable storage medium stores the aforementioned computer program executed by the message queue-based data transmission apparatus 3, and the computer program includes program instructions, and when the processor executes the program instructions, the description of the message queue-based data transmission method in the embodiment corresponding to fig. 3 can be executed, so that details will not be repeated here.

In addition, the beneficial effects of the same method are not described in detail. For technical details not disclosed in embodiments of the computer-readable storage medium referred to in the present application, reference is made to the description of embodiments of the method of the present application. As an example, program instructions may be deployed to be executed on one computing device or on multiple computing devices at one site or distributed across multiple sites and interconnected by a communication network, which may comprise a block chain system.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The computer-readable storage medium may be the message queue-based data transmission apparatus provided in any of the foregoing embodiments or an internal storage unit of the foregoing device, such as a hard disk or a memory of an electronic device. The computer readable storage medium may also be an external storage device of the electronic device, such as a plug-in hard disk, a Smart Memory Card (SMC), a Secure Digital (SD) card, a flash card (flash card), and the like, which are provided on the electronic device. The computer readable storage medium may further include a magnetic disk, an optical disk, a read-only memory (ROM), a random access memory (ram), or the like. Further, the computer readable storage medium may also include both an internal storage unit and an external storage device of the electronic device. The computer-readable storage medium is used for storing the computer program and other programs and data required by the electronic device. The computer readable storage medium may also be used to temporarily store data that has been output or is to be output.

The terms "comprising" and "having" and any variations thereof in the claims, specification and drawings of the present invention are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or apparatus. Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the invention. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein can be combined with other embodiments. The term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

Those of ordinary skill in the art will appreciate that the elements and algorithm steps of the examples described in connection with the embodiments disclosed herein may be embodied in electronic hardware, computer software, or combinations of both, and that the components and steps of the examples have been described in a functional general in the foregoing description for the purpose of illustrating clearly the interchangeability of hardware and software. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.

The above disclosure is only for the purpose of illustrating the preferred embodiments of the present application and is not to be construed as limiting the scope of the present application, so that the present application is not limited thereto, and all equivalent variations and modifications can be made to the present application.

Claims

1. A message queue-based data transmission method, the method comprising:

the message manager determines a message queue associated with the data source information and generates a target message theme according to the data source information;

2. The method of claim 1, wherein the message queue comprises at least one storage area including the target storage area, and one storage area is associated with one message topic and used for storing messages of the message topic, and data source information associated with messages of the same message topic is the same.

3. The method according to claim 1 or 2, wherein the data source information includes a data source name of a data source to which the message belongs and an item name of an item to which the data source belongs, and the item to which the data source belongs includes at least one data source including the data source;

the generating of the target message subject according to the data source information comprises:

4. The method of any of claims 1-3, wherein after the message manager receives the message data sent by the message producer, the method further comprises:

5. A message queue-based data transmission method, the method comprising:

when the message consumer requests to subscribe the message generated by the message producer, sending a message subscription request to a message manager, wherein the message subscription request carries the data source information, and the data source information is used for determining a target message topic requested to be subscribed by the message consumer;

and the message consumer receives the message related to the target message subject from the message manager, analyzes and consumes the message.

6. The method of claim 5, wherein said parsing and consuming the message comprises:

7. The method of claim 5, wherein parsing and consuming the message comprises:

the message consumer acquires a message mode identifier from the header information of the message and sends a message mode acquisition request to a message manager, wherein the message mode acquisition request carries the message mode identifier;

and acquiring the message mode corresponding to the message mode identifier from the message manager, and analyzing and consuming the message based on the message mode.

8. A message queue-based data transmission method, the method comprising:

a message producer creates a data source and generates a message and data source information related to the message according to the data source;

and the message producer sends message data to a message manager, wherein the message data carries the message and the data source information.

9. A computer device, comprising: a transceiver, a processor, a memory, and a network interface;

the processor is coupled to the transceiver, the memory, and a network interface for providing data communication functionality, the memory is for storing program code, and the transceiver and the processor are for invoking the program code to perform the method of any of claims 1-8.

10. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program comprising program instructions which, when executed by a processor, perform the method of any of claims 1-8.