CN111464598A - Data pushing method and device - Google Patents
Data pushing method and device Download PDFInfo
- Publication number
- CN111464598A CN111464598A CN202010194937.2A CN202010194937A CN111464598A CN 111464598 A CN111464598 A CN 111464598A CN 202010194937 A CN202010194937 A CN 202010194937A CN 111464598 A CN111464598 A CN 111464598A
- Authority
- CN
- China
- Prior art keywords
- data
- pushed
- pushing
- identifier
- demander
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 61
- 238000003860 storage Methods 0.000 claims abstract description 21
- 238000012544 monitoring process Methods 0.000 claims abstract description 12
- 238000004590 computer program Methods 0.000 claims description 15
- 230000005540 biological transmission Effects 0.000 claims description 11
- 238000006243 chemical reaction Methods 0.000 claims description 4
- 238000009826 distribution Methods 0.000 claims description 4
- 238000010586 diagram Methods 0.000 description 14
- 230000008569 process Effects 0.000 description 14
- 238000005516 engineering process Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000005538 encapsulation Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000011022 operating instruction Methods 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000011282 treatment Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
- H04L67/025—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP] for remote control or remote monitoring of applications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/252—Integrating or interfacing systems involving database management systems between a Database Management System and a front-end application
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/546—Message passing systems or structures, e.g. queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q40/00—Finance; Insurance; Tax strategies; Processing of corporate or income taxes
- G06Q40/08—Insurance
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/14—Session management
- H04L67/146—Markers for unambiguous identification of a particular session, e.g. session cookie or URL-encoding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/55—Push-based network services
Abstract
The invention discloses a data pushing method, a data pushing device, electronic equipment and a computer readable storage medium. The method comprises the following steps: acquiring data generated by a data producer, and converting the data into data in a JSON format; determining data to be pushed required by different data demanders from JSON-format data according to preset first corresponding relations of the data required by different data demanders and different data demanders; adding a data demander identifier for the data to be pushed according to the first corresponding relation, and issuing the data to be pushed with the data demander identifier added to the theme of the message queue; distributing the data to be pushed to the determined queue in a mode of distributing the data to be pushed to the queue corresponding to the data demand party identifier added to the data to be pushed; and pushing the data to be pushed added with the data demander identifications to the data demanders represented by the data demander identifications corresponding to the determined queues by monitoring the queues.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a data pushing method and apparatus, an electronic device, and a computer-readable storage medium.
Background
And data pushing, namely pushing data to one or more data demand parties by a data generation party through a data transmission channel. For example, in the field of insurance technology (InsurTech), after a data generating party (e.g., an insurance system) generates business data (e.g., a policy), the business data may be pushed to a plurality of data demanding parties (e.g., a reinsurance system, a financial system, etc.), so that after the data demanding parties obtain the business data, the data demanding parties may perform further processing based on the business data, such as calculating commissions, generating reinsurance contracts, etc.
In the prior art, data pushing is usually realized by setting a timing task. For example, information such as which data are needed by each data demand party is recorded in a log table; and then, scanning the log table at regular time by setting a timed task script, and acquiring data required by the data demander from the database according to the information recorded in the log table and pushing the data to the data demander.
In practical applications, situations may occur where different data consumers need the same data, such as: the data demand side M needs data A and data B, and the data demand side N also needs data A and data B, so that the database interface needs to be called twice to acquire data A and data B twice in a timed task mode. Obviously, the working mode of data pushing is not efficient, and due to the limited performance of the database interface, when the pushed data volume is large, frequent access to the database further reduces the data pushing efficiency.
Disclosure of Invention
Embodiments of the present specification provide a data pushing method and apparatus, an electronic device, and a computer-readable storage medium, which solve the problem in the prior art that data pushing efficiency is not high.
The embodiment of the specification adopts the following technical scheme:
a method of data push, comprising:
acquiring data generated by a data producer, and converting the data into data in a JSON format;
determining data required by different data demanders from the JSON-format data as data to be pushed according to a preset first corresponding relation between the different data demanders and the data required by the different data demanders;
adding a data demand party identifier for the data to be pushed according to the first corresponding relation, and publishing the data to be pushed with the data demand party identifier to a theme of a message queue; wherein, the message queue comprises a theme and each queue; the queues subscribe to the topic together; the queues are preset according to the setting mode that a single queue corresponds to a single data demand party identifier and different queues correspond to different data demand party identifiers;
determining queues respectively corresponding to the data demander identifications added by the data to be pushed from the queues according to the second corresponding relation between the queues and the data demander identifications;
and respectively executing the following steps aiming at the data to be pushed of the added data demander identifier issued to the theme: distributing the data to be pushed to the determined queue in a mode of distributing the data to be pushed to the queue corresponding to the data demand party identifier added to the data to be pushed;
and pushing the data to be pushed added with the data demander identifications to the data demanders represented by the data demander identifications corresponding to the determined queues by monitoring the queues.
A data pushing apparatus comprising:
the conversion module is used for acquiring data generated by a data producer and converting the data into data in a JSON format;
the first determining module is used for determining data required by different data demanders from the JSON-format data according to a preset first corresponding relation between the different data demanders and the data required by the different data demanders, and the data are used as data to be pushed;
the adding module is used for adding a data demand party identifier for the data to be pushed according to the first corresponding relation and publishing the data to be pushed with the data demand party identifier to the subject of a message queue; wherein, the message queue comprises a theme and each queue; the queues subscribe to the topic together; the queues are preset according to the setting mode that a single queue corresponds to a single data demand party identifier and different queues correspond to different data demand party identifiers;
a second determining module, configured to determine, according to a second correspondence between each of the queues and the identifier of the data demander, queues corresponding to the identifier of the data demander added to the data to be pushed, respectively, from the queues;
a distribution module, configured to execute, for each to-be-pushed data issued to the added data demander identifier in the topic, respectively: distributing the data to be pushed to the determined queue in a mode of distributing the data to be pushed to the queue corresponding to the data demand party identifier added to the data to be pushed;
and the pushing module is used for pushing the data to be pushed added with the data demander identifications to the data demanders represented by the data demander identifications corresponding to the determined queues by monitoring the queues.
An electronic device, comprising: memory, a processor and a computer program stored on the memory and executable on the processor, the computer program, when executed by the processor, implementing the steps of any of the data push methods as described above.
A computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of any of the data push methods described above.
The embodiment of the specification adopts at least one technical scheme which can achieve the following beneficial effects:
the data pushing method comprises the steps of adding data demand party identifications to serve as data to be pushed according to the requirements of data demand parties by using data generated by a data production party, adding one or more data demand party identifications to single data, distributing the data to be pushed to each queue through the subjects of a message queue, wherein each queue of the message queue can correspond to each data demand party identification one by one, namely correspond to each data demand party one by one, so that the single data can be pushed to a plurality of data demand parties at the same time.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:
fig. 1 is a schematic flow chart of a data pushing method provided in an embodiment of the present specification;
fig. 2 is a schematic diagram of a publish-subscribe pattern of a message queue provided by an embodiment of the present specification;
fig. 3 is a schematic diagram of a specific implementation scenario of a data pushing method provided in an embodiment of the present specification;
fig. 4 is a schematic structural diagram of a data pushing apparatus provided in an embodiment of this specification
Fig. 5 is a schematic structural diagram of an electronic device provided in an embodiment of this specification.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the technical solutions of the present application will be described in detail and completely with reference to the following specific embodiments of the present application and the accompanying drawings. It should be apparent that the described embodiments are only some of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The technical solutions provided by the embodiments of the present application are described in detail below with reference to the accompanying drawings.
In the prior art, data pushing is usually realized by setting a timing task. However, in practical applications, situations may occur where different data consumers need the same data, such as: the data demand side M needs data A and data B, and the data demand side N also needs data A and data B, so that the database interface needs to be called twice to acquire data A and data B twice in a timed task mode. Obviously, the working mode of data pushing is not efficient, and due to the limited performance of the database interface, when the pushed data volume is large, frequent access to the database further reduces the data pushing efficiency.
In order to solve the above technical problem, an embodiment of the present specification provides a data pushing method, which is used to improve data pushing efficiency. The execution subject of the method includes, but is not limited to, a server, a personal computer, a notebook computer, a tablet computer, a smart phone, and other intelligent electronic devices that can execute a predetermined process, such as numerical calculation and/or logical calculation, by running a predetermined program or instruction. The server may be a single network server or a server group consisting of a plurality of network servers or a Cloud Computing (Cloud Computing) based Cloud consisting of a large number of computers or network servers. In the embodiments of the present specification, the execution subject of the method is not limited. The flow diagram of the method is shown in figure 1, and comprises the following steps:
step 11: and acquiring data generated by a data producer, and converting the data into data in a JSON format.
In practical applications, the data producer may be a client, a server, a business system, and the like that can produce data, for example, in the field of insurance technology, the data producer may be a client such as an insurance application and an insurance web page, or a server interacting with the insurance application and the insurance web page, or may be the entire insurance system.
When data generated by the data producer is obtained, the data generated by the data producer may be obtained through a Transmission Control Protocol/Internet Protocol (TCP/IP), an Application Programming Interface (API), or the like, and the data obtaining method is not limited in the present Application.
In practical applications, in order to facilitate later data transmission through the message queue, the data can be converted into serialized data for transmission, and specifically, the data generated by the data producer can be converted into data in JSON format.
Step 12: and determining the data required by different data demanders from the data converted into the JSON format in the step 11 as the data to be pushed according to the preset first corresponding relation between the different data demanders and the data required by the different data demanders.
In practical application, since the data producer may correspond to one or more data demanders, and the data demanded by different data demanders may be different, the first corresponding relationship between the data demanded by different data demanders and the data demanded by different data demanders may be preset, and it may be understood that which data demanders demand the data produced by the data producer and which data is demanded may be preset.
And determining data required by different data demanders from the data converted into the JSON format in the step 11 according to a preset first corresponding relation, and taking the determined data as the data to be pushed so as to add data demanders marks to the data to be pushed.
Step 13: and adding a data demander identifier for the data to be pushed according to the first corresponding relation, and publishing the data to be pushed with the data demander identifier to the subject of the message queue.
The data to be pushed may be the data required by different data demanders determined by step 12. In order to facilitate the later push of the data to be pushed to different data demanders, the data demander identifier may be added to the data to be pushed according to the first corresponding relationship.
In practical application, the data demander identifier may be a character string composed of characters such as numbers and letters and capable of representing different data demanders, and the data demander identifier in which form is adopted is not limited in the present application.
In practical applications, the inventor finds that a message queue of a publish-subscribe mode as shown in fig. 2 can implement that a piece of data is sent to multiple data consumers at the same time, and the work mode of the message queue of the publish-subscribe mode may be that a producer of the data publishes the data to a topic, one or more subscribers subscribe to the topic, and each time there is data in the topic, the topic delivers the data to the subscribers according to a preset rule.
In this embodiment of the present specification, after the data demander identifier is added, the to-be-pushed data to which the data demander identifier has been added may be published to a topic of a message queue, where the message queue may include a topic and queues, the queues subscribe to the topic together, and the queues may be preset according to a setting manner in which a single queue corresponds to a single data demander identifier and different queues correspond to different data demander identifiers, so as to distribute the to-be-pushed data to each queue according to the data demander identifier, thereby implementing data distribution and pushing.
Step 14: and determining queues respectively corresponding to the data demander identifications added by the data to be pushed from each queue according to the second corresponding relation between each queue and the data demander identifications.
As described above, each queue may be preset according to a setting manner that a single queue corresponds to a single data demander identifier and different queues correspond to different data demander identifiers, that is, a second correspondence between each queue and a data demander method identifier is preset.
In practical application, queues respectively corresponding to the data demander identifiers added to the data to be pushed are determined from the queues included in the message queue according to a preset second corresponding relationship, so that the data to be pushed is distributed to the queues according to the data demander identifiers.
Because the data demanders can be in one-to-one correspondence with the data demanders identifications, the data to be pushed contained in each queue determined according to the second correspondence can be the data required by each data demander, and the accuracy of the data pushed to each data demander is also ensured.
Step 15: for each to-be-pushed data issued to the topic by step 13 with the data demander identifier added, the following steps are respectively executed: and distributing the data to be pushed to the queue determined in the step 14 in a manner of distributing the data to be pushed to the queue corresponding to the data demander identifier added to the data to be pushed.
In practical application, for each piece of data to be pushed which is issued to the topic and added with the data demander identifier, each piece of data to be pushed is distributed to the queue determined in step 14 according to the data demander identifier, so that the data to be pushed distributed in each queue may be the data required by each data demander.
Step 16: and pushing the data to be pushed added with the data demander identifications to the data demanders represented by the data demander identifications corresponding to the determined queues by monitoring the queues.
In practical application, by monitoring each queue, to the data demander represented by the data demander identifier corresponding to each determined queue, the data to be pushed to which the data demander identifier has been added is pushed, which may specifically include:
aiming at each queue, respectively executing:
when the data to be pushed are distributed in the queue, the data to be pushed in the queue are packaged into a message according to the data format of the data demand side demand represented by each data demand side identification corresponding to the queue;
and respectively calling the data transmission interfaces corresponding to the data demanders represented by the data demander identifications, and pushing the message to the data demanders represented by the data demander identifications.
In the embodiment of the present disclosure, the data formats required by different data demanders may be written in the form of an array into the program code or stored in the form of a table in the readable storage medium, and the storage form of the data format may be set according to actual requirements, which is not limited in this application.
In practical application, when it is monitored that data to be pushed is distributed in a certain queue, a data format required by a data demand party corresponding to a data demand party identifier can be determined according to the data demand party identifier, and then the data to be pushed in the queue is packaged into a message according to the determined data format.
In this embodiment of the present specification, in order to improve the real-time performance of data pushing, for example, as shown in fig. 3, a schematic diagram of a specific implementation scenario of a data pushing method may monitor each queue in real time in a multi-process manner, and each process may monitor all queues, so as long as it is monitored that a certain queue is pushed with data, an unoccupied process may push data to be pushed, to which a data demander identifier has been added, to a data demander corresponding to the queue, and when the process finishes pushing the data to be pushed, the process may continue to monitor all queues. It will be appreciated that a process may be an occupied process when it is pushing data, and may be an unoccupied process when it is not pushing data.
In practical application, the message can be pushed to each data demand party by calling the data transmission interface corresponding to each data demand party. In practical application, the different data transmission interfaces corresponding to the different data demanders may be written in a program code in an array form or stored in a readable storage medium in a table form, and the application is not limited thereto.
In practical applications, different pushing results may occur in the process of data pushing, for example: in order to record a push result and the like to ensure accuracy of data pushing, in one or more embodiments of the present specification, a push log table may be set, and information generated in a data pushing process is recorded, for example, table 1 may be a set push log table.
Table 1: push log sheet
In the push log table shown in table 1, when data push fails, a developer can troubleshoot the problem of data push failure according to information such as error information, push times, last push time and the like returned by the push failure recorded in the push log table.
Based on the set push log table, in one or more embodiments of the present specification, after pushing the to-be-pushed data to which the data demander identifier is added, the method may further include: determining a pushing result of the data to be pushed added with the data demand party identification, and correspondingly recording the determined pushing result and the data identification of the data to be pushed into a pushing log table. Wherein the pushing result may include: push success or push failure.
In practical applications, there may be different treatments for different push results, and in this embodiment of the present specification, specifically, the method may include:
if the pushing result is the data to be pushed with failed pushing, correspondingly recording a data identifier of the data to be pushed with failed pushing and a data demander identifier corresponding to the data to be pushed with failed pushing in a pushing log table; according to a data demander identifier corresponding to the data to be pushed which is recorded in a push log table and is failed in pushing and the data identifier of the data to be pushed, the data to be pushed which is failed in pushing is pushed again;
and if the data to be pushed successfully pushed exists in the pushing result, deleting the data to be pushed successfully pushed from the queue containing the data to be pushed successfully pushed according to the data identification of the data to be pushed successfully pushed recorded in the pushing log table.
In practical application, in order to improve the efficiency of pushing data when a certain data to be pushed which fails to be pushed is pushed again, the encapsulated message can be recorded into a push log table, so that if the data to be pushed which fails to be pushed exists as a pushing result, the message which fails to be pushed can be obtained from the push log table according to the data identifier corresponding to the data to be pushed which fails to be pushed, and then the obtained message which fails to be pushed can be pushed again according to the data demander identifier corresponding to the data to be pushed which fails to be pushed. It can be understood that, when data to be pushed which fails to be pushed is pushed again, the message is directly acquired from the push log table, so that the step of repeatedly executing encapsulation of the data to be pushed into the message can be avoided, and the data pushing efficiency is improved.
In the embodiment of the present specification, data generated by a data producer is added with a data demander identifier according to a requirement of a data demander as data to be pushed, so that one or more data demander identifiers can be added to a single piece of data, and then the data to be pushed is distributed to each queue through a topic of a message queue, and each queue of the message queue can be in one-to-one correspondence with each data demander identifier, that is, in one-to-one correspondence with each data demander, so that a single piece of data can be pushed to a plurality of data demanders at the same time.
The data pushing method provided by the embodiment of the present specification is based on the same inventive concept, and the embodiment of the present specification further provides a corresponding data pushing device. As shown in fig. 4, the apparatus specifically includes:
the conversion module 101 is configured to obtain data generated by a data producer, and convert the data into data in a JSON format;
a first determining module 102, configured to determine, according to a preset first corresponding relationship between different data demanders and data required by the different data demanders, data required by the different data demanders from the JSON-formatted data, where the data is used as data to be pushed;
the adding module 103 is configured to add a data demander identifier to the data to be pushed according to the first corresponding relationship, and issue the data to be pushed to which the data demander identifier has been added to a subject of a message queue; wherein, the message queue comprises a theme and each queue; the queues subscribe to the topic together; the queues are preset according to the setting mode that a single queue corresponds to a single data demand party identifier and different queues correspond to different data demand party identifiers;
a second determining module 104, configured to determine, according to the second correspondence between each queue and the identifier of the data demander, queues that respectively correspond to the identifier of the data demander added to the data to be pushed from each queue;
a distributing module 105, configured to execute, for each to-be-pushed data issued to the added data demander identifier in the topic, respectively: distributing the data to be pushed to the determined queue in a mode of distributing the data to be pushed to the queue corresponding to the data demand party identifier added to the data to be pushed;
and the pushing module 106 is configured to push the to-be-pushed data to which the data demander identifier is added to the data demander identifier represented by the data demander identifier corresponding to each determined queue by monitoring each queue.
The specific workflow of the above device embodiment may include: a conversion module 101 for obtaining data generated by a data producer and converting the data into data in a JSON format, a first determination module 102 for determining data required by different data demanders from the data in the JSON format as data to be pushed according to a preset first corresponding relationship between the different data demanders and data required by the different data demanders, an adding module 103 for adding a data demander identifier to the data to be pushed according to the first corresponding relationship and issuing the data to be pushed with the data demander identifier added to a subject of a message queue, a second determination module 104 for determining queues corresponding to the data demander identifiers added to the data to be pushed respectively from the queues according to a second corresponding relationship between the queues and the data demander identifier, a distribution module 105 for each data to be pushed issued to the added data demander identifier in the subject, respectively executing the following steps: the data to be pushed is distributed to the determined queue according to the manner of distributing the data to be pushed to the queue corresponding to the data demander identifier added to the data to be pushed, and the pushing module 106 is configured to push the data to be pushed to the data demander represented by the data demander identifier corresponding to each determined queue by monitoring each queue.
In one embodiment, the apparatus further comprises:
the pushing result determining module is used for determining the pushing result of the data to be pushed added with the data demander identifier;
the pushing result recording module is used for correspondingly recording the pushing result and the data identifier of the data to be pushed into a pushing log table; the pushing result comprises: push success or push failure.
In one embodiment, the apparatus further comprises:
the re-pushing module is used for correspondingly recording a data identifier of the data to be pushed which fails to be pushed and a data demander identifier corresponding to the data to be pushed which fails to be pushed in the pushing log table if the pushing result is the data to be pushed which fails to be pushed; according to the data demander identifier corresponding to the data to be pushed which is recorded in the push log table and fails to push and the data identifier of the data to be pushed which fails to push, the data to be pushed which fails to push is pushed again;
and the deleting module is used for deleting the successfully pushed data to be pushed from the queue containing the successfully pushed data to be pushed according to the data identifier of the successfully pushed data to be pushed recorded in the push log table if the successfully pushed data to be pushed exists in the push result.
In an embodiment, the pushing module 106 specifically includes:
and the packaging unit is used for respectively executing the following steps aiming at each queue: when the data to be pushed are distributed in the queue, the data to be pushed in the queue are packaged into a message according to the data format of the data demand side demand represented by each data demand side identification corresponding to the queue;
a pushing unit, configured to perform, for each queue: and respectively calling the data transmission interfaces corresponding to the data demanders represented by the data demander identifications, and pushing the message to the data demanders represented by the data demander identifications.
In one embodiment, the apparatus further comprises:
and the message recording module is used for recording the message to the push log table.
In one embodiment, the pushing module 106 further comprises:
the message acquiring unit is used for acquiring a message with a push failure from the push log table according to a data identifier corresponding to the data to be pushed with the push failure if the push result is the data to be pushed with the push failure;
and the message re-pushing unit is used for re-pushing the obtained message with the pushing failure according to the data demand party identifier corresponding to the data to be pushed with the pushing failure.
In this embodiment of the specification, by using the data pushing apparatus, data generated by a data producer may be added with a data demander identifier according to a requirement of a data demander as data to be pushed, one or more data demander identifiers may be added to a single piece of data, and then the data to be pushed is distributed to each queue by using a topic of a message queue, and each queue of the message queue may correspond to each data demander identifier one to one, that is, correspond to each data demander one to one, so that a single piece of data may be pushed to multiple data demanders at the same time.
An embodiment of this specification further provides an electronic device, and referring to fig. 5, in a hardware level, the electronic device includes a processor, and optionally further includes an internal bus, a network interface, and a memory. The Memory may include a Memory, such as a Random-Access Memory (RAM), and may further include a non-volatile Memory, such as at least 1 disk Memory. Of course, the electronic device may also include hardware required for other services.
The processor, the network interface, and the memory may be connected to each other via an internal bus, which may be an ISA (Industry Standard Architecture) bus, a PCI (peripheral component Interconnect) bus, an EISA (Extended Industry Standard Architecture) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one double-headed arrow is shown in FIG. 5, but this does not indicate only one bus or one type of bus.
And the memory is used for storing programs. In particular, the program may include program code comprising computer operating instructions. The memory may include both memory and non-volatile storage and provides instructions and data to the processor.
The processor reads the corresponding computer program from the nonvolatile memory into the memory and then runs the computer program, and the application data pushing device is formed on the logic level. A processor executing the program stored in the memory and configured to perform at least the following:
acquiring data generated by a data producer, and converting the data into data in a JSON format;
determining data required by different data demanders from the JSON-format data as data to be pushed according to a preset first corresponding relation between the different data demanders and the data required by the different data demanders;
adding a data demand party identifier for the data to be pushed according to the first corresponding relation, and publishing the data to be pushed with the data demand party identifier to a theme of a message queue; wherein, the message queue comprises a theme and each queue; the queues subscribe to the topic together; the queues are preset according to the setting mode that a single queue corresponds to a single data demand party identifier and different queues correspond to different data demand party identifiers;
determining queues respectively corresponding to the data demander identifications added by the data to be pushed from the queues according to the second corresponding relation between the queues and the data demander identifications;
and respectively executing the following steps aiming at the data to be pushed of the added data demander identifier issued to the theme: distributing the data to be pushed to the determined queue in a mode of distributing the data to be pushed to the queue corresponding to the data demand party identifier added to the data to be pushed;
and pushing the data to be pushed added with the data demander identifications to the data demanders represented by the data demander identifications corresponding to the determined queues by monitoring the queues.
The method executed by the data pushing apparatus according to the embodiment shown in fig. 1 of the present application may be applied to a processor, or may be implemented by a processor. The processor may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware in a processor or instructions in the form of software. The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a network Processor (FP), and the like; but also Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The various methods, steps and logic blocks disclosed in the embodiments of the present specification may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of a method disclosed in connection with the embodiments of the present specification may be embodied directly in a hardware decoding processor, or in a combination of hardware and software modules in the decoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The storage medium is located in a memory, and a processor reads information in the memory and completes the steps of the method in combination with hardware of the processor.
The electronic device may further execute the method executed by the data pushing apparatus in fig. 1, and implement the functions of the data pushing apparatus in the embodiment shown in fig. 1, which are not described herein again in this specification.
The present specification also proposes a computer-readable storage medium storing one or more programs, where the one or more programs include instructions, which when executed by an electronic device including a plurality of application programs, enable the electronic device to perform the method performed by the data pushing apparatus in the embodiment shown in fig. 1, and at least to perform:
acquiring data generated by a data producer, and converting the data into data in a JSON format;
determining data required by different data demanders from the JSON-format data as data to be pushed according to a preset first corresponding relation between the different data demanders and the data required by the different data demanders;
adding a data demand party identifier for the data to be pushed according to the first corresponding relation, and publishing the data to be pushed with the data demand party identifier to a theme of a message queue; wherein, the message queue comprises a theme and each queue; the queues subscribe to the topic together; the queues are preset according to the setting mode that a single queue corresponds to a single data demand party identifier and different queues correspond to different data demand party identifiers;
determining queues respectively corresponding to the data demander identifications added by the data to be pushed from the queues according to the second corresponding relation between the queues and the data demander identifications;
and respectively executing the following steps aiming at the data to be pushed of the added data demander identifier issued to the theme: distributing the data to be pushed to the determined queue in a mode of distributing the data to be pushed to the queue corresponding to the data demand party identifier added to the data to be pushed;
and pushing the data to be pushed added with the data demander identifications to the data demanders represented by the data demander identifications corresponding to the determined queues by monitoring the queues.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the specification. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in the process, method, article, or apparatus that comprises the element.
The above are merely examples of the present application and are not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.
Claims (10)
1. A data pushing method, comprising:
acquiring data generated by a data producer, and converting the data into data in a JSON format;
determining data required by different data demanders from the JSON-format data as data to be pushed according to a preset first corresponding relation between the different data demanders and the data required by the different data demanders;
adding a data demand party identifier for the data to be pushed according to the first corresponding relation, and publishing the data to be pushed with the data demand party identifier to a theme of a message queue; wherein, the message queue comprises a theme and each queue; the queues subscribe to the topic together; the queues are preset according to the setting mode that a single queue corresponds to a single data demand party identifier and different queues correspond to different data demand party identifiers;
determining queues respectively corresponding to the data demander identifications added by the data to be pushed from the queues according to the second corresponding relation between the queues and the data demander identifications;
and respectively executing the following steps aiming at the data to be pushed of the added data demander identifier issued to the theme: distributing the data to be pushed to the determined queue in a mode of distributing the data to be pushed to the queue corresponding to the data demand party identifier added to the data to be pushed;
and pushing the data to be pushed added with the data demander identifications to the data demanders represented by the data demander identifications corresponding to the determined queues by monitoring the queues.
2. The method of claim 1, wherein after pushing the data to be pushed with the added data demander identification, the method further comprises:
determining a pushing result of the data to be pushed added with the data demander identifier;
correspondingly recording the push result and the data identifier of the data to be pushed into a push log table; the pushing result comprises: push success or push failure.
3. The method of claim 2, wherein the method further comprises:
if the pushing result is the data to be pushed with the pushing failure, correspondingly recording a data identifier of the data to be pushed with the pushing failure and a data demander identifier corresponding to the data to be pushed with the pushing failure in the pushing log table; according to the data demander identifier corresponding to the data to be pushed which is recorded in the push log table and fails to push and the data identifier of the data to be pushed which fails to push, the data to be pushed which fails to push is pushed again;
and if the push result is the successfully pushed data to be pushed, deleting the successfully pushed data to be pushed from the queue containing the successfully pushed data to be pushed according to the data identification of the successfully pushed data to be pushed recorded in the push log table.
4. The method according to claim 3, wherein the pushing the to-be-pushed data to which the data demander identifier has been added by monitoring the queues and pushing the to-be-pushed data to the data demander identifier indicated by the data demander identifier corresponding to each of the determined queues includes:
for each queue, respectively executing:
when the data to be pushed are distributed in the queue, the data to be pushed in the queue are packaged into a message according to the data format of the data demand side demand represented by each data demand side identification corresponding to the queue;
and respectively calling the data transmission interfaces corresponding to the data demanders represented by the data demander identifications, and pushing the message to the data demanders represented by the data demander identifications.
5. The method of claim 4, wherein after encapsulating the data to be pushed in the queue into a packet, the method further comprises:
recording the message to the push log table;
respectively calling the data transmission interfaces corresponding to the data demanders represented by the data demander identifications, and pushing the message to the data demanders represented by the data demander identifications, wherein the method further comprises the following steps:
if the pushing result is the data to be pushed with the pushing failure, acquiring a message with the pushing failure from the pushing log table according to the data identification corresponding to the data to be pushed with the pushing failure;
and re-pushing the obtained message with the pushing failure according to the data demand side identification corresponding to the data to be pushed with the pushing failure.
6. A data pushing apparatus, comprising:
the conversion module is used for acquiring data generated by a data producer and converting the data into data in a JSON format;
the first determining module is used for determining data required by different data demanders from the JSON-format data according to a preset first corresponding relation between the different data demanders and the data required by the different data demanders, and the data are used as data to be pushed;
the adding module is used for adding a data demand party identifier for the data to be pushed according to the first corresponding relation and publishing the data to be pushed with the data demand party identifier to the subject of a message queue; wherein, the message queue comprises a theme and each queue; the queues subscribe to the topic together; the queues are preset according to the setting mode that a single queue corresponds to a single data demand party identifier and different queues correspond to different data demand party identifiers;
a second determining module, configured to determine, according to a second correspondence between each of the queues and the identifier of the data demander, queues corresponding to the identifier of the data demander added to the data to be pushed, respectively, from the queues;
a distribution module, configured to execute, for each to-be-pushed data issued to the added data demander identifier in the topic, respectively: distributing the data to be pushed to the determined queue in a mode of distributing the data to be pushed to the queue corresponding to the data demand party identifier added to the data to be pushed;
and the pushing module is used for pushing the data to be pushed added with the data demander identifications to the data demanders represented by the data demander identifications corresponding to the determined queues by monitoring the queues.
7. The apparatus of claim 6, wherein the apparatus further comprises:
the pushing result determining module is used for determining the pushing result of the data to be pushed added with the data demander identifier;
the pushing result recording module is used for correspondingly recording the pushing result and the data identifier of the data to be pushed into a pushing log table; the pushing result comprises: push success or push failure.
8. The apparatus of claim 7, wherein the apparatus further comprises:
the re-pushing module is used for correspondingly recording a data identifier of the data to be pushed which fails to be pushed and a data demander identifier corresponding to the data to be pushed which fails to be pushed in the pushing log table if the pushing result is the data to be pushed which fails to be pushed; according to the data demander identifier corresponding to the data to be pushed which is recorded in the push log table and fails to push and the data identifier of the data to be pushed which fails to push, the data to be pushed which fails to push is pushed again;
and the deleting module is used for deleting the successfully pushed data to be pushed from the queue containing the successfully pushed data to be pushed according to the data identifier of the successfully pushed data to be pushed recorded in the push log table if the successfully pushed data to be pushed exists in the push result.
9. An electronic device, comprising: memory, processor and computer program stored on the memory and executable on the processor, which when executed by the processor implements the steps of the data push method according to any of claims 1 to 5.
10. A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, which computer program, when being executed by a processor, carries out the steps of the data push method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010194937.2A CN111464598A (en) | 2020-03-19 | 2020-03-19 | Data pushing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010194937.2A CN111464598A (en) | 2020-03-19 | 2020-03-19 | Data pushing method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111464598A true CN111464598A (en) | 2020-07-28 |
Family
ID=71685640
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010194937.2A Pending CN111464598A (en) | 2020-03-19 | 2020-03-19 | Data pushing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111464598A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113296960A (en) * | 2021-06-29 | 2021-08-24 | 中国平安人寿保险股份有限公司 | Data quasi-real-time distribution method, device, equipment and storage medium |
| CN113312118A (en) * | 2021-06-01 | 2021-08-27 | 中国民航信息网络股份有限公司 | Main picture pushing method, device, equipment and computer readable storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140229504A1 (en) * | 2013-02-12 | 2014-08-14 | Samsung Techwin Co., Ltd. | System and method for managing database in data distribution service |
| CN105338014A (en) * | 2014-06-26 | 2016-02-17 | 青岛海尔智能家电科技有限公司 | Subscription mechanism-based communication method and subject server |
| CN109246220A (en) * | 2018-09-19 | 2019-01-18 | 南方科技大学 | A kind of message push system and method |
| CN109756552A (en) * | 2018-08-22 | 2019-05-14 | 比亚迪股份有限公司 | A kind of passenger information system message distributing method and device and passenger information system |
| CN110012083A (en) * | 2019-03-26 | 2019-07-12 | 青岛聚好联科技有限公司 | A kind of data transmission method, server and data transmission device |
| CN110769018A (en) * | 2018-07-27 | 2020-02-07 | 北京国双科技有限公司 | Message pushing method and device |
-
2020
- 2020-03-19 CN CN202010194937.2A patent/CN111464598A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140229504A1 (en) * | 2013-02-12 | 2014-08-14 | Samsung Techwin Co., Ltd. | System and method for managing database in data distribution service |
| CN105338014A (en) * | 2014-06-26 | 2016-02-17 | 青岛海尔智能家电科技有限公司 | Subscription mechanism-based communication method and subject server |
| CN110769018A (en) * | 2018-07-27 | 2020-02-07 | 北京国双科技有限公司 | Message pushing method and device |
| CN109756552A (en) * | 2018-08-22 | 2019-05-14 | 比亚迪股份有限公司 | A kind of passenger information system message distributing method and device and passenger information system |
| CN109246220A (en) * | 2018-09-19 | 2019-01-18 | 南方科技大学 | A kind of message push system and method |
| CN110012083A (en) * | 2019-03-26 | 2019-07-12 | 青岛聚好联科技有限公司 | A kind of data transmission method, server and data transmission device |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113312118A (en) * | 2021-06-01 | 2021-08-27 | 中国民航信息网络股份有限公司 | Main picture pushing method, device, equipment and computer readable storage medium |
| CN113296960A (en) * | 2021-06-29 | 2021-08-24 | 中国平安人寿保险股份有限公司 | Data quasi-real-time distribution method, device, equipment and storage medium |
| CN113296960B (en) * | 2021-06-29 | 2023-02-03 | 中国平安人寿保险股份有限公司 | Data quasi-real-time distribution method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110032880B (en) | Screen recording evidence obtaining method and system based on block chain and electronic equipment | |
| CN106202235B (en) | Data processing method and device | |
| CN109002362B (en) | Service method, device and system and electronic equipment | |
| CN111741035B (en) | Electronic contract signing method, device and equipment | |
| CN110035105B (en) | Screen recording evidence obtaining method and system based on block chain and electronic equipment | |
| CN108173706B (en) | Service marking method, device and equipment under multi-service system | |
| CN108599973B (en) | Log association method, device and equipment | |
| CN111858727B (en) | Multi-data source data export system and method based on template configuration | |
| CN108134812B (en) | Data processing method and device | |
| CN109062906B (en) | Translation method and device for program language resources | |
| CN110780870A (en) | Service execution method, device, equipment and storage medium | |
| CN111464598A (en) | Data pushing method and device | |
| WO2020216077A1 (en) | Blockchain event deduplication method and apparatus, and computer device and storage medium | |
| CN111899008A (en) | Resource transfer method, device, equipment and system | |
| CN111859205A (en) | Analytic method and equipment for batch processing tasks | |
| CN113722114A (en) | Data service processing method and device, computing equipment and storage medium | |
| CN110908657A (en) | Product generation method and device | |
| CN107395663B (en) | Data acquisition method and device | |
| CN111198677A (en) | Equipment object generation method, device and equipment | |
| CN114429408B (en) | Method, device, equipment and system for checking configuration file of intelligent substation | |
| CN109542401B (en) | Web development method and device, storage medium and processor | |
| CN115460265A (en) | Interface calling method, device, equipment and medium | |
| CN112491943A (en) | Data request method, device, storage medium and electronic equipment | |
| CN110750271B (en) | Service aggregation, method and device for executing aggregated service and electronic equipment | |
| CN114327941A (en) | Service providing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200728 |